How to create an NFS StorageClass(En)

After setting up the Kubernetes cluster, one of the first things I typically do is create a StorageClass for application services to use, and I consider NFS to be a relatively straightforward approach to implement.

Of course, there are also many choices, such as the previously mentioned local-path provider, which is another approach.

StorageClass: local-path provider(EN)

Access shared data over the network

I’ll break it down into a few parts to explain:

(1) What’s StorageClass ?

(2) Why do we need it?

(3) Workflow

(4) Deploy your first NFS StorageClass

Let’s get started!

1. What’s StorageClass?

Official:
A StorageClass provides a way for administrators to describe the "classes" of 
storage they offer. Different classes might map to quality-of-service levels, 
or to backup policies, or to arbitrary policies determined by the cluster 
administrators. 

Kubernetes itself is unopinionated about what classes represent. 
This concept is sometimes called "profiles" in other storage systems

Inside the StorageClass object, you will define:

(1) The properties of the PV, including storage type, volume size

(2) Which plugin the PV should use

After providing the above information, Kubernetes can use the PVC to find the corresponding StorageClass, and then Kubernetes directly invokes the plugin defined in the StorageClass to create the corresponding PV resource.

2. Why do we need it?

I don’t want manual PV creation errors to prevent my PVC from binding to a PV and potentially causing application failures. I prefer to define my PVC properly and let Kubernetes manage my PV resources automatically.

In a large-scale cluster, not using StorageClass would mean having to create 1000 PVs manually for 1000 PVCs, making storage management a difficult task to keep track of.

We categorize storage needs based on different characteristics

3. Workflow

From: https://forum.huawei.com/enterprise/en/dynamic-provisioning-with-kubernetes-storage-classes-huawei-evs/thread/694728783145353216-667213860102352896

We can further simplified for NFS provisioner as shown below:

After setting up the NFS provisioner, when PVCs are requested, they will automatically link to the StorageClass and create PV resources using the NFS protocol to provide storage.

4. Deploy your first NFS StorageClass

(1) Create nfs server

# cat >> /etc/fstab << EOF
/dev/vg00/nfs /var/nfsshare   xfs  defaults 0 0
EOF

// nfs-server install

# yum install -y nfs-utils policycoreutils-python-utils policycoreutils-python
# mkdir /var/nfsshare
# chmod 777 /var/nfsshare
# cat > /etc/exports << EOF
/var/nfsshare     *(rw,sync,no_wdelay,no_root_squash,insecure)
EOF

# systemctl start rpcbind nfs-server nfs-lock nfs-idmap
# systemctl enable rpcbind nfs-server nfs-lock nfs-idmap
# systemctl start rpcbind nfs-server nfs-lock nfs-idmap

//Firewalld rules

# export FIREWALLD_DEFAULT_ZONE=`firewall-cmd --get-default-zone`
# echo ${FIREWALLD_DEFAULT_ZONE}
public

# firewall-cmd --permanent --zone=${FIREWALLD_DEFAULT_ZONE} --add-service=rpc-bind
# firewall-cmd --permanent --zone=${FIREWALLD_DEFAULT_ZONE} --add-service=nfs
# firewall-cmd --permanent --zone=${FIREWALLD_DEFAULT_ZONE} --add-service=mountd
# firewall-cmd --reload ; firewall-cmd --list-all

//SELINUX

# setsebool -P nfs_export_all_rw 1
# setsebool -P nfs_export_all_ro 1
# semanage fcontext -a -t public_content_rw_t  "/var/nfsshare(/.*)?"
# restorecon -R /var/nfsshare

//Verify

# mount -t nfs nfs-server:/var/nfsshare /mnt
# touch /mnt/1 && rm -f /mnt/1 

(2) Deploy NFS provisioner in Kubernetes

> 20240524_Updated: change github source.

// obtain resource

# git clone https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner.git kubernetes-incubator

(deprecated)# git clone https://github.com/kubernetes-incubator/external-storage.git kubernetes-incubator 

// create namespace

# kubectl create namespace k8s-nfs-storage

// Update deployment and RBAC

# cd kubernetes-incubator/nfs-client/
# sed -i'' "s/namespace:.*/namespace: k8s-nfs-storage/g" ./deploy/rbac.yaml
# sed -i'' "s/namespace:.*/namespace: k8s-nfs-storage/g" ./deploy/deployment.yaml
# kubectl create -f rbac.yaml

// Update permissions (only for OpenShift)

# kubectl adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:k8s-nfs-storage:nfs-client-provisioner

// Edit deployment.yaml

containers:
        - name: nfs-client-provisioner
          image: groundhog2k/nfs-subdir-external-provisioner:v3.2.0
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: storage.io/nfs
            - name: NFS_SERVER
              value: 192.168.1.10  << NFS server
            - name: NFS_PATH
              value: /var/nfsshare  << NFS shared folder
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.1.10
            path: /var/nfsshare

// Edit class.yaml

# vim deploy/class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: storage.io/nfs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

# kubectl create -f deploy/class.yaml
# kubectl create -f deploy/deployment.yaml
# kubectl get all -n k8s-nfs-storage

Next, use the test-pvc.yaml file in the folder to test if it can correctly auto-generate the corresponding PV and ensure the PVC acquires the resource.

If you want to modify the default StorageClass, you can achieve it using the following method:

# kubectl patch storageclass managed-nfs-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
storageclass.storage.k8s.io/managed-nfs-storage patched

Above is the basic deployment of NFS StorageClass and an explanation of StorageClass. There will be more articles in the future to share further operations related to StorageClass.

References:

• https://www.tpisoftware.com/tpu/articleDetails/2445
• http://kubernetes.io