Jenkins for K8s Cluster (I)

July 12, 2020 - 3 mins read

jenkins-on-kubernetes

With Jenkins for Kubernetes Cluster, you can orchestrate your build, test and deployment pipelines

Jenkins on Kubernetes Cluster (Part I)

(Total Setup Time: 40 mins)

Jenkins is the leading open source automation server. It provides hundreds of plugins for supporting the building, deploying and automating of any project.

Preparation

(25 min)

In this guide, I am going to build the docker image for Jenkins on Kubernetes Cluster on Pi. You may use the base image from Balena in any Docker environment. In addition, you can find more details about the Balena base images here.

First, create a /mnt/hdd/master1k8s/docker/Dockerfile and insert the following. If you are interested in the Jenkins war file, please visit here:

FROM balenalib/raspberrypi4-64-debian-openjdk:11-bullseye

ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000

RUN apt-get update \
  && apt-get install -y --no-install-recommends curl
  
RUN curl -fL -o /opt/jenkins.war http://updates.jenkins-ci.org/download/war/2.235.2/jenkins.war

VOLUME ${JENKINS_HOME}
WORKDIR ${JENKINS_HOME}

EXPOSE 8080 ${JENKINS_SLAVE_AGENT_PORT}

CMD ["/bin/bash","-c","java -jar /opt/jenkins.war"]

Second, build the image and tag it:

docker build -t seehiong/jenkins:1.0 .

You may verify the built status by running docker images:

REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
seehiong/jenkins                     1.0                 9f4f4930275a        1 hours ago         506MB

Last, with this Jenkins image created, we are ready to deploy Jenkins for Kubernetes Cluster.

Installing Jenkins

(15 mins)

As usual, create the required folders and kubernetes deployment file:

sudo mkdir -p /mnt/hdd/master1k8s/app/jenkins/home
sudo vi /mnt/hdd/master1k8s/app/jenkins/jenkins-deployment.yaml

First, insert the following into jenkins-deployment.yaml:

apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  ports:
    - port: 8080
      name: jenkins-http
      targetPort: 8080
      nodePort: 30080
    - port: 50000
      name: jenkins-slave
      targetPort: 50000
  selector:
    app: jenkins
  type: NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - name: jenkins
        image: seehiong/jenkins:1.0
        ports:
        - containerPort: 8080
        volumeMounts:
          - name: jenkins-home
            mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-home
          hostPath:
            path: /mnt/hdd/master1k8s/app/jenkins/home
            type: Directory
      nodeSelector:
        jenkins-data-storage: "true"

Second, apply the Jenkins deployment to the Kubernetes cluster and label the node:

kubectl apply -f /mnt/hdd/master1k8s/app/jenkins/jenkins-deployment.yaml
kubectl label nodes master1k8s jenkins-data-storage=true

Third, verify if Jenkins is deployed successfully:

kubectl describe deployment jenkins

That’s all to it! You may access Jenkins from your eth0 address, nodePort 30080. You may install the recommended plugins:

jenkins-ready-to-work

Finally, you may create the user and log in. Your Jenkins web page will look similar to this:

jenkins-welcome

You may refer to the next post on the detailed steps for setting up Jenkins Agents for Kubernetes.

Troubleshooting

exec user process caused “exec format error”

Since my Kubernetes Cluster runs on raspberry Pi ARM architecture, most of the publicly available Genkins images failed to work. As a result, I decided to build the docker image by myself.

Error looking up service account default/jenkins: serviceaccount “jenkins” not found

Because of missing serviceaccount, my Jenkins did not run:

kubectl get rs
kubectl describe rs jenkins-69f669b7c4

Create a jenkins-rolebinding.yaml and insert the following:

apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: default
  name: jenkins
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: jenkins
rules:
  - apiGroups: [""]
    resources: ["services"]
    verbs: ["get", "watch", "list"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create","delete","get","list","patch","update","watch"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get","list","watch"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get","watch","list"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get","watch","list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: default
  name: jenkins
roleRef:
  kind: Role
  name: jenkins
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
  name: jenkins