Add LEP for PV encryption support (#2822)
Longhorn #1859 Signed-off-by: Joshua Moody <joshua.moody@suse.com> Co-authored-by: David Ko <dko@suse.com>
This commit is contained in:
Joshua Moody
GitHub
parent
a1359e90a6
commit
143dce4f2f
183
enhancements/2021.07.26-pv-encryption.md
Normal file
183
enhancements/2021.07.26-pv-encryption.md
Normal file
@ -0,0 +1,183 @@
|
||||
# Add PV encryption support
|
||||
|
||||
## Summary
|
||||
|
||||
This enhancement adds support for user configured (storage class, secrets) encrypted volumes,
|
||||
this in return means that backups of that volume end up also being encrypted.
|
||||
|
||||
### Related Issues
|
||||
- [RWO encryption support](https://github.com/longhorn/longhorn/issues/1859)
|
||||
- [Backup encryption](https://github.com/longhorn/longhorn/issues/902)
|
||||
- [RWX encryption support]() // TODO: create issue
|
||||
- [Expansion support](https://github.com/longhorn/longhorn/issues/2794)
|
||||
- [Restore support]() // TODO: create issue
|
||||
- [VolumeMode Block support]() // TODO: create issue
|
||||
|
||||
## Motivation
|
||||
|
||||
### Goals
|
||||
- user is able to create & use an encrypted volume
|
||||
- user is able to configure the keys that are used for encryption
|
||||
- user is able to take backups from an encrypted volume
|
||||
- user is able to restore an encrypted backup to a new encrypted volume
|
||||
|
||||
### Non-goals
|
||||
- external key management support, currently keys utilize kubernetes secrets
|
||||
- rotating key support, user can do this manually though as a workaround
|
||||
- securing of secrets, user is responsible for cluster setup and security of the secrets
|
||||
|
||||
## Proposal
|
||||
|
||||
### User Stories
|
||||
All regular longhorn operations should also be supported for encrypted volumes,
|
||||
therefore the only user story that is mentioned is
|
||||
how to create and use an encrypted volume.
|
||||
|
||||
#### Create and use an encrypted volume
|
||||
- create a storage class with (secure=true) and either a global secret or a per volume secret
|
||||
- create the secret for that volume in the configured namespace
|
||||
- create a pvc that references the created storage class
|
||||
- volume will be created then encrypted during first use
|
||||
- afterwards a regular filesystem that lives on top of the encrypted volume will be exposed to the pod
|
||||
|
||||
### User Experience In Detail
|
||||
|
||||
Creation and usage of an encrypted volume requires 2 things:
|
||||
- the storage class needs to specify `secure: "true"` as part of its parameters.
|
||||
- secrets need to be created and reference for the csi operations need to be setup.
|
||||
- see below examples for different types of secret usage.
|
||||
|
||||
The kubernetes sidecars are responsible for retrieval of the secret and passing it to the csi driver.
|
||||
If the secret hasn't been created the PVC will remain in the Pending State.
|
||||
And the side cars will retry secret retrieval periodically, once it's available the sidecar container will call
|
||||
`Controller::CreateVolume` and pass the secret after which longhorn will create a volume.
|
||||
|
||||
#### Create storage class that utilizes a global secret (all volumes use the same key)
|
||||
The below storage class uses a global secret named `longhorn-secure` in the `longhorn-system` namespace.
|
||||
```yaml
|
||||
kind: StorageClass
|
||||
apiVersion: storage.k8s.io/v1
|
||||
metadata:
|
||||
name: longhorn-secure-global
|
||||
provisioner: driver.longhorn.io
|
||||
allowVolumeExpansion: true
|
||||
parameters:
|
||||
numberOfReplicas: "3"
|
||||
staleReplicaTimeout: "2880" # 48 hours in minutes
|
||||
fromBackup: ""
|
||||
secure: "true"
|
||||
csi.storage.k8s.io/provisioner-secret-name: "longhorn-secure"
|
||||
csi.storage.k8s.io/provisioner-secret-namespace: "longhorn-system"
|
||||
csi.storage.k8s.io/node-publish-secret-name: "longhorn-secure"
|
||||
csi.storage.k8s.io/node-publish-secret-namespace: "longhorn-system"
|
||||
csi.storage.k8s.io/node-stage-secret-name: "longhorn-secure"
|
||||
csi.storage.k8s.io/node-stage-secret-namespace: "longhorn-system"
|
||||
```
|
||||
|
||||
The global secret reference by the `longhorn-secure-global` storage class.
|
||||
This type of setup means that all volumes share the same encryption key.
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: longhorn-crypto
|
||||
namespace: longhorn-system
|
||||
stringData:
|
||||
CRYPTO_KEY_VALUE: "Simple secure passphrase"
|
||||
CRYPTO_KEY_PROVIDER: "secret" # this is optional we currently only support direct keys via secrets
|
||||
```
|
||||
|
||||
#### Create storage class that utilizes per volume secrets
|
||||
The below storage class uses a per volume secret, the name and namespace of the secret is based on the pvc values.
|
||||
These templates will be resolved by the external sidecars and the resolved values end up as Secret refs on the PV.
|
||||
```yaml
|
||||
kind: StorageClass
|
||||
apiVersion: storage.k8s.io/v1
|
||||
metadata:
|
||||
name: longhorn-secure-per-volume
|
||||
provisioner: driver.longhorn.io
|
||||
allowVolumeExpansion: true
|
||||
parameters:
|
||||
numberOfReplicas: "3"
|
||||
staleReplicaTimeout: "2880" # 48 hours in minutes
|
||||
fromBackup: ""
|
||||
secure: "true"
|
||||
csi.storage.k8s.io/provisioner-secret-name: ${pvc.name}
|
||||
csi.storage.k8s.io/provisioner-secret-namespace: ${pvc.namespace}
|
||||
csi.storage.k8s.io/node-publish-secret-name: ${pvc.name}
|
||||
csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace}
|
||||
csi.storage.k8s.io/node-stage-secret-name: ${pvc.name}
|
||||
csi.storage.k8s.io/node-stage-secret-namespace: ${pvc.namespace}
|
||||
```
|
||||
|
||||
|
||||
### API changes
|
||||
add a `Secure` boolean to the `Volume` struct utilized by the http client,
|
||||
this ends up being stored in `Volume.Spec.Secure` of the volume cr.
|
||||
Storing the `Secure` value is necessary to support encryption for RWX volumes.
|
||||
|
||||
## Design
|
||||
|
||||
### Implementation Overview
|
||||
Host requires `dm_crypt` kernel module as well as `cryptsetup` installed.
|
||||
We utilize the below parameters from a secret, `CRYPTO_KEY_PROVIDER` allows us in the future to add other key management systems.
|
||||
```yaml
|
||||
CRYPTO_KEY_VALUE: "Simple secure passphrase"
|
||||
CRYPTO_KEY_PROVIDER: "secret" # this is optional we currently only support direct keys via secrets
|
||||
```
|
||||
|
||||
- utilize host `dm_crypt` kernel module for device encryption
|
||||
- utilize host installed `cryptsetup` for configuration of the crypto device
|
||||
- add csi driver `NodeStageVolume` support to handle device global per node mounting,
|
||||
we skip mounting for volumes that are being used via `VolumeMode: Block`
|
||||
- refactor csi driver NodePublishVolume to bind mount the `staging_path` into the `target_path`
|
||||
we utilize a bind mount for `VolumeMode: Mount` we do a regular device file creation for `VolumeMode: Block`
|
||||
- during csi `NodeStageVolume` encrypt (first time use) / open regular longhorn device
|
||||
- this exposes a crypto mapped device (/dev/mapper/<volume-name>)
|
||||
- mount crypto device into `staging_path`
|
||||
- during csi `NodeUnstageVolume` unmount `staging_path` close crypto device
|
||||
|
||||
### Test plan
|
||||
|
||||
#### Successful Creation of an encrypted volume
|
||||
- create a storage class with (secure=true) and either a global secret or a per volume secret
|
||||
- create the secret for that volume in the configured namespace
|
||||
- create a pvc that references the created storage class
|
||||
- create a pod that uses that pvc for a volume mount
|
||||
- wait for pod up and healthy
|
||||
|
||||
#### Missing Secret for encrypted volume creation
|
||||
- create a storage class with (secure=true) and either a global secret or a per volume secret
|
||||
- create a pvc that references the created storage class
|
||||
- create a pod that uses that pvc for a volume mount
|
||||
- verify pvc remains in pending state
|
||||
- verify pod remains in creation state
|
||||
|
||||
#### Verify encryption of volume
|
||||
- create a storage class with (secure=true) and either a global secret or a per volume secret
|
||||
- create the secret for that volume in the configured namespace
|
||||
- create a pvc that references the created storage class
|
||||
- create a pod that uses that pvc for a volume mount
|
||||
- wait for pod up and healthy
|
||||
- write known test pattern into fs
|
||||
- verify absence (grep) of known test pattern after reading block device content `/dev/longhorn/<volume-name>`
|
||||
|
||||
#### Verify wrong key failure
|
||||
- create a storage class with (secure=true) and either a global secret or a per volume secret
|
||||
- create the secret for that volume in the configured namespace
|
||||
- create a pvc that references the created storage class
|
||||
- create a pod that uses that pvc for a volume mount
|
||||
- wait for pod up and healthy
|
||||
- scale down pod
|
||||
- change `CRYPTO_KEY_VALUE` of secret
|
||||
- scale up pod
|
||||
- verify pod remains in pending state (failure to mount volume)
|
||||
|
||||
### Upgrade strategy
|
||||
- requires new pvc's since encryption would overwrite the previously created filesystem. (csi driver prevents this)
|
||||
|
||||
## Note
|
||||
- Host requires `dm_crypt` kernel module as well as `cryptsetup` installed.
|
||||
- [csi sidecars and secrets](https://kubernetes-csi.github.io/docs/secrets-and-credentials.html)
|
||||
- supporting external key vaults is possible in the future with some additional implementation
|
||||
- support rotating keys is possible in the future with some additional implementation
|
||||
Loading…
Reference in New Issue
Block a user