Build Your CI/CD Pipeline: Quickly Establishing Jenkins on Kubernetes(En)
Once you’ve set up the foundational Kubernetes cluster, one of the most common use cases is establishing a CI/CD pipeline. This comprehensive approach seamlessly connects the entire development and deployment process.
This article will guide you through the process of setting up a basic Jenkins. In terms of architecture, Jenkins Agents will be dynamically generated as needed and automatically deleted after execution, preventing an accumulation of unused Pods that could consume limited resources within the system.
Finally, we’ll create a simple pipeline on the Jenkins Web UI to validate that the Jenkins engine set up in this article is functioning correctly.
This article is structured as follows:
- Architecture Overview
- Deployment
- Installing plugins
- Jenkins Master configuration
- Testing: Executing a single job
- Testing: Executing a pipeline
- Conclusion
1. Architecture Overview
The traditional Jenkins one-to-many architecture presents the following challenges:
1. If the Master encounters issues, the entire workflow becomes non-functional.
2. Each Slave has different environment configurations, making management less convenient, especially for tasks involving entirely different programming languages.
3. Uneven resource allocation, leading to some Slave jobs waiting in a queue while others remain idle, resulting in resource wastage.
Therefore, running Jenkins master/slave within a Kubernetes cluster effectively addresses these issues.
Approach: The Master stores configuration data in a volume, and Slaves run on various nodes. Slaves are not constantly running; they are dynamically generated as needed and automatically deleted upon completion.
This approach addresses the following:
1. If the Jenkins master fails, Kubernetes automatically rebuilds a new Jenkins master, allocating the volume to the new instance, ensuring data integrity.
2. A Slave is created dynamically for each job run and automatically deleted after completion, freeing up resources. Kubernetes allocates resources to suitable idle nodes, preventing queueing issues caused by an overconcentration on specific nodes.
3. If the overall resources are insufficient, Kubernetes can easily resolve the issue by adding a new node.
2. Deployment
There are two deployment methods:
- Running everything inside Kubernetes (as used in this article).
- Running the Master outside Kubernetes, with all executing Slaves generated within Kubernetes.
#---------------------------------------------------
# S2-1. create namespace
#---------------------------------------------------
[master]# kubectl create ns jenkinspoc#---------------------------------------------------
# S2-2. create service account
#---------------------------------------------------
[master]# vim jenkinspoc-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins-admin
namespace: jenkinspoc
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: jenkins
namespace: jenkinspoc
labels:
"app.kubernetes.io/name": 'jenkins'
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: jenkins-role-binding
namespace: jenkinspoc
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:
- kind: ServiceAccount
name: jenkins-admin
namespace: jenkinspoc
[master]# kubectl create -f jennkinspoc-sa.yaml -n jenkinspoc
serviceaccount/jenkins-admin created
role.rbac.authorization.k8s.io/jenkins created
rolebinding.rbac.authorization.k8s.io/jenkins-role-binding created
[master]# kubectl get sa -n jenkinspoc#---------------------------------------------------
# S2-3. Create master data volume
#---------------------------------------------------
[master]# vim jenkinspoc-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: jenkins-pv-claim
spec:
storageClassName: managed-nfs-storage
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
[master]# kubectl create -f jenkinspoc-pvc.yaml -n jenkinspoc
[master]# kubectl get pvc -n jenkinspoc
#---------------------------------------------------
# S2-4. Deploy Jenkins master
#---------------------------------------------------
[master]# vim jenkinspoc-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins-deployment
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
serviceAccountName: jenkins-admin
securityContext:
fsGroup: 1000
runAsUser: 1000
containers:
- name: jenkins
image: jenkins/jenkins:lts
resources:
limits:
memory: "2Gi"
cpu: "1000m"
requests:
memory: "500Mi"
cpu: "500m"
ports:
- name: httpport
containerPort: 8080
- name: jnlpport
containerPort: 50000
livenessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 90
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 5
readinessProbe:
httpGet:
path: "/login"
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
volumeMounts:
- name: jenkins-data
mountPath: /var/jenkins_home
volumes:
- name: jenkins-data
persistentVolumeClaim:
claimName: jenkins-pv-claim
[master]# kubectl create -f jenkinspoc-deploy.yaml -n jenkinspoc
[master]# kubectl get pod -n jenkinspoc#---------------------------------------------------
# S2-5. create svc
#---------------------------------------------------
[master]# vim jenkinspoc-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-service
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8080'
spec:
selector:
app: jenkins
type: NodePort
ports:
- name: httpport
port: 8080
targetPort: 8080
nodePort: 32003
- name: jnlpport
port: 50000
targetPort: 50000
[master]# kubectl create -f jenkinspoc-svc.yaml -n jenkinspoc
[master]# kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/jenkins-service NodePort 10.109.168.87 <none> 8080:32003/TCP,50000:30779/TCP 3sf#---------------------------------------------------
# S2-6. Obtain admin passowrd
#---------------------------------------------------
[master]# kubectl get pod -n jenkinspoc
[master]# kubectl exec jenkins-deployment-bdf778654-br6mv -n jenkinspoc -- cat /var/jenkins_home/secrets/initialAdminPassword
b1a8117e9a364353855ebfe03b3308be
http://worker01.test.example.poc:32003
admin/P@ssw0rd3. Installing Plugins
#---------------------------------------------
# S3-1. Install Suggestion plugin
#---------------------------------------------
#---------------------------------------------
# S3-2. Install kubernetes plugin
#---------------------------------------------
Manage Jenkins > Plugins > Available plugins > "Kubernetes" > Install
4. Jenkins Master configuration
#---------------------------------------------
# S4-1. create cloud
#---------------------------------------------
Manage Jenkins > Clouds > New cloud > Name: K8s-pipeline (click Kubernetes) > Create#---------------------------------------------
# S4-2. create credentials
# everything is in the same Kubernetes cluster,
# authorization is handled through ServiceAccounts
#---------------------------------------------
Kubernetes URL : empty
Certification key: empty
Kubernetes namespace: jenkinspoc
=> test connection
#---------------------------------------------
# S4-3. Jenkins URL
#---------------------------------------------
[syntax] http://<service-name>.<namespace>.svc.cluster.local:8080
Jenkins URL: http://jenkins-service.jenknspoc.svc.cluster.local:8080
=> SAVE#---------------------------------------------
# S4-4. Pod template
#---------------------------------------------
Name: jenkins-agent
namespace: jenkinspoc
Labels: jenkinsagent <== Job uses this label to identify which Pod configuration to use for creation
Containers:
(If it can connect to DockerHub, you can use the default jenkins/inbound-agent)
(a custom jnlp is used, overwriting the default values)
(1) Name: jnlp
(2) Image: jenkins/inbound-agent:latest
(3) Remove "Sleep" , "99999" <== Otherwise, it will overwrite the default entrypoint.
(4) Service Account : jenkins-admin5. Testing: Executing a single job
#---------------------------------------------
# S5-1. Create Job
#---------------------------------------------
Dashboard > New item > name: jenkins-job-1 (free style project) > OK
Label express: jenkinsagent
Build Step > Excute Shell > echo "Test Jenkins JOB Successfully." > SAVE
> Build Now
6. Testing: Executing a pipeline
#---------------------------------------------
# S6-1. Creat Pipeline
#---------------------------------------------
Dashboard > New item > pipeline (name: pipeline-test) > OK
#---------------------------------------------
# S6-2. Pipeline script
#---------------------------------------------
node('jenkinsagent') {
stage('Clone') {
echo "1.Clone Stage"
}
stage('Test') {
echo "2.Test Stage"
}
stage('Build') {
echo "3.Build Stage"
}
stage('Deploy') {
echo "4. Deploy Stage"
}
}
> SAVE > Build Now
7. Conclusion
Many organizations utilizing Kubernetes incorporate various CI/CD tools for practical operations. Jenkins, as discussed in this article, is one widely used tool. The skill set required for many roles in these organizations often includes maintaining smooth CI/CD pipeline operations. Therefore, for those learning Kubernetes, it is recommended to grasp skills in setting up, operating, and maintaining at least one pipeline tool.
This article only demonstrated one phase of the CI/CD process. If interested, explore the complete CI/CD cycle and associated tools. For instance, the Harbor registry mentioned earlier is another phase within the cycle.
That’s it for today; see you in the next part!
References:
