From 158c767e558721a4cd4d740af03aaa4a4d078e08 Mon Sep 17 00:00:00 2001 From: James Oliver Date: Tue, 21 Aug 2018 20:39:53 -0700 Subject: [PATCH] Refer to check environment script where applicable --- README.md | 51 ++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 42 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 10d7ab9..7989a73 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Longhorn is 100% open source software. Project source code is spread across a nu ## Kubernetes Driver Requirements -Longhorn can be used in Kubernetes to provide persistent storage through either Longhorn Container Storage Interface (CSI) driver or Longhorn Flexvolume driver. Longhorn will automatically deploy one of the drivers, depends on user's Kubernetes cluster's setup. User can also specify the driver in the deployment yaml file. CSI is preferred. +Longhorn can be used in Kubernetes to provide persistent storage through either Longhorn Container Storage Interface (CSI) driver or Longhorn FlexVolume driver. Longhorn will automatically deploy one of the drivers, depending on the Kubernetes cluster configuration. User can also specify the driver in the deployment yaml file. CSI is preferred. ### Requirement for the CSI driver @@ -37,7 +37,7 @@ Longhorn can be used in Kubernetes to provide persistent storage through either 1. CSI is in beta release for this version of Kubernetes, and enabled by default. 2. Mount Propagation feature gate enabled. 1. It's enabled by default in Kubernetes v1.10. But some early versions of RKE may not enable it. -3. If above conditions cannot be met, Longhorn will falls back to use Flexvolume driver. +3. If above conditions cannot be met, Longhorn will fall back to the FlexVolume driver. ### Check if your setup satisfied CSI requirement 1. Use the following command to check your Kubernetes server version @@ -48,14 +48,25 @@ Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCom ``` The `Server Version` should be `v1.10` or above. -2. Use the following command on the hosts to check if the feature gate is enabled for Mount Propagation +2. Use the following script to check if `MountPropagation` feature is enabled. ``` -# ps aux|grep kube|grep MountPropagation -root 1707 3.1 12.4 1087008 503848 ? Ssl Jul12 1288:35 kube-apiserver --storage-backend=etcd3 --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --tls-cert-file=/etc/kubernetes/ssl/kube-apiserver.pem --kubelet-client-certificate=/etc/kubernetes/ssl/kube-apiserver.pem --apiserver-count=1 --secure-port=6443 --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --kubelet-client-key=/etc/kubernetes/ssl/kube-apiserver-key.pem --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --allow-privileged=true --insecure-port=0 --admission-control=ServiceAccount,NamespaceLifecycle,LimitRanger,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds --cloud-provider= --service-cluster-ip-range=10.43.0.0/16 --tls-private-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --service-account-key-file=/etc/kubernetes/ssl/kube-apiserver-key.pem --authorization-mode=Node,RBAC --bind-address=0.0.0.0 --feature-gates=MountPropagation=true --insecure-bind-address=127.0.0.1 --etcd-cafile=/etc/kubernetes/ssl/kube-ca.pem --etcd-certfile=/etc/kubernetes/ssl/kube-node.pem --etcd-keyfile=/etc/kubernetes/ssl/kube-node-key.pem --etcd-servers=https://138.197.199.191:2379 --etcd-prefix=/registry -root 1760 4.7 6.4 1508564 260724 ? Ssl Jul12 1970:59 kubelet --network-plugin=cni --resolv-conf=/etc/resolv.conf --cluster-domain=cluster.local --v=2 --enforce-node-allocatable= --cgroups-per-qos=True --cni-bin-dir=/opt/cni/bin --cluster-dns=10.43.0.10 --cloud-provider= --fail-swap-on=false --address=0.0.0.0 --cadvisor-port=0 --volume-plugin-dir=/var/lib/kubelet/volumeplugins --hostname-override=yasker-longhorn-dev-1 --client-ca-file=/etc/kubernetes/ssl/kube-ca.pem --root-dir=/var/lib/kubelet --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --feature-gates=MountPropagation=true --cni-conf-dir=/etc/cni/net.d --allow-privileged=true --pod-infra-container-image=rancher/pause-amd64:3.0 --kubeconfig=/etc/kubernetes/ssl/kubecfg-kube-node.yaml --read-only-port=0 --anonymous-auth=false --cgroup-driver=cgroupfs -``` -Both `kube-apiserver` and `kubelet` should have `--feature-gates=MountPropagation=true` +# curl -sSfL https://raw.githubusercontent.com/rancher/longhorn/v0.3-rc/scripts/environment_check.sh | bash +pod/detect-flexvol-dir created +daemonset.apps/longhorn-environment-check created +waiting for pod/detect-flexvol-dir to finish +pod/detect-flexvol-dir completed +waiting for pods to become ready (1/7) +waiting for pods to become ready (6/7) +all pods ready (7/7) + FlexVolume Path: /var/lib/kubelet/volumeplugins + + + MountPropagation is enabled! + +pod "detect-flexvol-dir" deleted +daemonset.apps "longhorn-environment-check" deleted +``` ### Requirement for the Flexvolume driver @@ -64,7 +75,29 @@ Both `kube-apiserver` and `kubelet` should have `--feature-gates=MountPropagatio 3. User need to know the volume plugin directory in order to setup the driver correctly. 1. Rancher RKE: `/var/lib/kubelet/volumeplugins` 2. Google GKE: `/home/kubernetes/flexvolume` - 3. For other distro, please find the correct directory by running `ps aux|grep kubelet` on the host and check the `--volume-plugin-dir` parameter. If there is none, it would be the default value `/usr/libexec/kubernetes/kubelet-plugins/volume/exec/` . + 3. For any other distro, please run the directory detection script in the next section. + +### Detect Volume Plugin Directory + +Use the following script to detect your volume plugin directory. +``` +# curl -sSfL https://raw.githubusercontent.com/rancher/longhorn/v0.3-rc/scripts/environment_check.sh | bash +pod/detect-flexvol-dir created +daemonset.apps/longhorn-environment-check created +waiting for pod/detect-flexvol-dir to finish +pod/detect-flexvol-dir completed +waiting for pods to become ready (1/7) +waiting for pods to become ready (6/7) +all pods ready (7/7) + + FlexVolume Path: /var/lib/kubelet/volumeplugins + + + MountPropagation is enabled! + +pod "detect-flexvol-dir" deleted +daemonset.apps "longhorn-environment-check" deleted +``` # Upgrading