# OpenShift / OKD Extra Configuration Steps - [OpenShift / OKD Extra Configuration Steps](#openshift--okd-extra-configuration-steps) - [Notes](#notes) - [Known Issues](#known-issues) - [Preparing Nodes (Optional)](#preparing-nodes-optional) - [Default /var/lib/longhorn setup](#default-varliblonghorn-setup) - [Separate /var/mnt/longhorn setup](#separate-varmntlonghorn-setup) - [Create Filesystem](#create-filesystem) - [Mounting Disk On Boot](#mounting-disk-on-boot) - [Label and Annotate Nodes](#label-and-annotate-nodes) - [Example values.yaml](#example-valuesyaml) - [Installation](#installation) - [Refs](#refs) ## Notes Main changes and tasks for OCP are: - On OCP / OKD, the Operating System is Managed by the Cluster - OCP Imposes [Security Context Constraints](https://docs.openshift.com/container-platform/4.11/authentication/managing-security-context-constraints.html) - This requires everything to run with the least privilege possible. For the moment every component has been given access to run as higher privilege. - Something to circle back on is network polices and which components can have their privileges reduced without impacting functionality. - The UI probably can be for example. - openshift/oauth-proxy for authentication to the Longhorn Ui - **⚠️** Currently Scoped to Authenticated Users that can delete a longhorn settings object. - **⚠️** Since the UI it self is not protected, network policies will need to be created to prevent namespace <--> namespace communication against the pod or service object directly. - Anyone with access to the UI Deployment can remove the route restriction. (Namespace Scoped Admin) - Option to use separate disk in /var/mnt/longhorn & MachineConfig file to mount /var/mnt/longhorn - Adding finalizers for mount propagation ## Known Issues - General Feature/Issue Thread - [[FEATURE] Deploying Longhorn on OKD/Openshift](https://github.com/longhorn/longhorn/issues/1831) - 4.10 / 1.23: - 4.10.0-0.okd-2022-03-07-131213 to 4.10.0-0.okd-2022-07-09-073606 - Tested, No Known Issues - 4.11 / 1.24: - 4.11.0-0.okd-2022-07-27-052000 to 4.11.0-0.okd-2022-11-19-050030 - Tested, No Known Issues - 4.11.0-0.okd-2022-12-02-145640, 4.11.0-0.okd-2023-01-14-152430: - Workaround: [[BUG] Volumes Stuck in Attach/Detach Loop](https://github.com/longhorn/longhorn/issues/4988) - [MachineConfig Patch](https://github.com/longhorn/longhorn/issues/4988#issuecomment-1345676772) - 4.12 / 1.25: - 4.12.0-0.okd-2022-12-05-210624 to 4.12.0-0.okd-2023-01-20-101927 - Tested, No Known Issues - 4.12.0-0.okd-2023-01-21-055900 to 4.12.0-0.okd-2023-02-18-033438: - Workaround: [[BUG] Volumes Stuck in Attach/Detach Loop](https://github.com/longhorn/longhorn/issues/4988) - [MachineConfig Patch](https://github.com/longhorn/longhorn/issues/4988#issuecomment-1345676772) - 4.12.0-0.okd-2023-03-05-022504 - 4.12.0-0.okd-2023-04-16-041331: - Tested, No Known Issues - 4.13 / 1.26: - 4.13.0-0.okd-2023-05-03-001308 - 4.13.0-0.okd-2023-08-18-135805: - Tested, No Known Issues - 4.14 / 1.27: - 4.14.0-0.okd-2023-08-12-022330 - 4.14.0-0.okd-2023-10-28-073550: - Tested, No Known Issues ## Preparing Nodes (Optional) Only required if you require additional customizations, such as storage-less nodes, or secondary disks. ### Default /var/lib/longhorn setup Label each node for storage with: ```bash oc get nodes --no-headers | awk '{print $1}' export NODE="worker-0" oc label node "${NODE}" node.longhorn.io/create-default-disk=true ``` ### Separate /var/mnt/longhorn setup #### Create Filesystem On the storage nodes create a filesystem with the label longhorn: ```bash oc get nodes --no-headers | awk '{print $1}' export NODE="worker-0" oc debug node/${NODE} -t -- chroot /host bash # Validate Target Drive is Present lsblk export DRIVE="sdb" #vdb sudo mkfs.ext4 -L longhorn /dev/${DRIVE} ``` > ⚠️ Note: If you add New Nodes After the below Machine Config is applied, you will need to also reboot the node. #### Mounting Disk On Boot The Secondary Drive needs to be mounted on every boot. Save the Concents and Apply the MachineConfig with `oc apply -f`: > ⚠️ This will trigger an machine config profile update and reboot all worker nodes on the cluster ```yaml apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker name: 71-mount-storage-worker spec: config: ignition: version: 3.2.0 systemd: units: - name: var-mnt-longhorn.mount enabled: true contents: | [Unit] Before=local-fs.target [Mount] Where=/var/mnt/longhorn What=/dev/disk/by-label/longhorn Options=rw,relatime,discard [Install] WantedBy=local-fs.target ``` #### Label and Annotate Nodes Label and annotate storage nodes like this: ```bash oc get nodes --no-headers | awk '{print $1}' export NODE="worker-0" oc annotate node ${NODE} --overwrite node.longhorn.io/default-disks-config='[{"path":"/var/mnt/longhorn","allowScheduling":true}]' oc label node ${NODE} node.longhorn.io/create-default-disk=config ``` ## Example values.yaml Minimum Adjustments Required ```yaml openshift: oauthProxy: repository: quay.io/openshift/origin-oauth-proxy tag: 4.14 # Use Your OCP/OKD 4.X Version, Current Stable is 4.14 # defaultSettings: # Preparing nodes (Optional) # createDefaultDiskLabeledNodes: true openshift: enabled: true ui: route: "longhorn-ui" port: 443 proxy: 8443 ``` ## Installation ```bash # helm template ./chart/ --namespace longhorn-system --values ./chart/values.yaml --no-hooks > longhorn.yaml # Local Testing helm template longhorn --namespace longhorn-system --values values.yaml --no-hooks > longhorn.yaml oc create namespace longhorn-system -o yaml --dry-run=client | oc apply -f - oc apply -f longhorn.yaml -n longhorn-system ``` ## Refs - - - okd 4.5: - okd 4.6: - oauth-proxy: -