在k8s中安装jenkins并实现动态生成jenkinsslave
安装jenkins
1、创建一个命名空间
$ kubectl create namespace kube-ops
2、为jenkins创建pvc(也可以使用存储类创建)
发展壮大离不开广大客户长期以来的信赖与支持,我们将始终秉承“诚信为本、服务至上”的服务理念,坚持“二合一”的优良服务模式,真诚服务每家企业,认真做好每个细节,不断完善自我,成就企业,实现共赢。行业涉及办公窗帘等,在成都网站建设公司、成都全网营销推广、WAP手机网站、VI设计、软件开发等项目上具有丰富的设计经验。
apiVersion: v1
kind: PersistentVolume
metadata:
name: opspv
spec:
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Delete
nfs:
server: 192.168.1.244
path: /data/k8s
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: opspvc
namespace: kube-ops
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
3、创建jenkins需要的rbac权限
apiVersion: v1
kind: ServiceAccount
metadata:
name: jenkins2
namespace: kube-ops
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins2
rules:
- apiGroups: ["extensions", "apps"]
resources: ["deployments"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["services"]
verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: jenkins2
namespace: kube-ops
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: jenkins2
subjects:
- kind: ServiceAccount
name: jenkins2
namespace: kube-ops
也可以为ServiceAccoun绑定一个系统现有的 cluster-admin 集群角色权限
4、创建jenkins pod
$ docker pull docker.io/jenkins/jenkins:lts
$ docker pull cnych/jenkins:jnlp6
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: jenkins2
namespace: kube-ops
spec:
template:
metadata:
labels:
app: jenkins2
spec:
terminationGracePeriodSeconds: 10
serviceAccount: jenkins2
containers:
- name: jenkins
image: docker.io/jenkins/jenkins:lts
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
name: web
protocol: TCP
- containerPort: 50000
name: agent
protocol: TCP
resources:
limits:
cpu: 1000m
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
livenessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
readinessProbe:
httpGet:
path: /login
port: 8080
initialDelaySeconds: 60
timeoutSeconds: 5
failureThreshold: 12
volumeMounts:
- name: jenkinshome
subPath: jenkins2
mountPath: /var/jenkins_home
env:
- name: LIMITS_MEMORY
valueFrom:
resourceFieldRef:
resource: limits.memory
divisor: 1Mi
- name: JAVA_OPTS
value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvision
er.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
securityContext:
fsGroup: 1000
volumes:
- name: jenkinshome
persistentVolumeClaim:
claimName: opspvc
---
apiVersion: v1
kind: Service
metadata:
name: jenkins2
namespace: kube-ops
labels:
app: jenkins2
spec:
selector:
app: jenkins2
type: NodePort
ports:
- name: web
port: 8080
targetPort: web
nodePort: 30003
- name: agent
port: 50000
targetPort: agent
$ kubectl apply -f jenkins2.yaml
$ kubectl get pod -n kube-ops
jenkins2-76644dbc9b-llcsp 0/1 Running 0 #不能正常启动
$ kubectl describe pod jenkins2-76644dbc9b-llcsp -n kube-ops
$ kubectl logs -f jenkins2-76644dbc9b-llcsp -n kube-ops
5、在nfs服务器上修改jenkins持久目录的权限并重新创建jenkins pod
$ chown -R 1000 /data/k8s/jenkins2 #在192.168.1.244上
$ kubectl delete -f jenkins2.yaml
$ kubectl apply -f jenkins2.yaml
$ kubectl get pod -n kube-ops
jenkins2-76644dbc9b-llcsp 1/1 Running 0
为什么是1000?
上述镜像的Dockerfile文件中定义的是:user=jenkins group=jenkins uid=1000 gid=1000
Dockerfile文件的地址:
https://github.com/jenkinsci/docker/blob/master/Dockerfile
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
ARG http_port=8080
ARG agent_port=50000
ARG JENKINS_HOME=/var/jenkins_home
$ kubectl get svc -n kube-ops
jenkins2 NodePort 10.105.121.176
http://192.168.1.243:30003
初始密码在nfs服务器上
$ cat /data/k8s/jenkins2/secrets/initialAdminPassword
在jenkins上创建kubernetes云
enkins Master 和 Jenkins Slave 以 Pod 形式运行在 Kubernetes 集群的 Node 上,Master 运行在其中一个节点,并且将其配置数据存储到一个 Volume 上去,Slave 运行在各个节点上,并且它不是一直处于运行状态,它会按照需求动态的创建并自动删除
这种方式的工作流程大致为:当 Jenkins Master 接受到 Build 请求时,会根据配置的 Label 动态创建一个运行在 Pod 中的 Jenkins Slave 并注册到 Master 上,当运行完 Job 后,这个 Slave 会被注销并且这个 Pod 也会自动删除,恢复到最初状态。
1、安装插件
安装kubernetes plugin, 点击 Manage Jenkins -> Manage Plugins -> Available -> Kubernetes plugin
2、增加kubernetes云
点击 Manage Jenkins —> Configure System —> (拖到最下方)Add a new cloud —> 选择 Kubernetes,然后填写 Kubernetes 和 Jenkins 配置信息----连接测试
name:kubernetes
Kubernetes 地址:https://kubernetes.default.svc.cluster.local
Kubernetes 命名空间:kube-ops
Jenkins 地址:http://jenkins2.kube-ops.svc.cluster.local:8080(jenkins2是svc)
3、添加pod模板
添加pod模板----Kubernetes Pod Template
名称:jnlp
命名空间:kube-ops
标签列表:dongyali-jnlp
4、添加容器模板
添加容器----Container Template
名称:jnlp
Docker 镜像:cnych/jenkins:jnlp6(Jenkins 版本在 2.176.x以下的镜像名字去掉6)
工作目录:/home/jenkins/agent
运行的命令:清空
命令参数:清空
5、添加两个卷
添加卷----Host Path Volume
主机路径:/var/run/docker.sock
挂载路径:/var/run/docker.sock
主机路径:/root/.kube
挂载路径:/root/.kube
6、可能需要配置ServiceAccount
$ kubectl get sa -n kube-ops
jenkins2 1 14h
点击添加卷下面的高级----Service Account----jenkins2
7、用shell测试 Kubernetes 动态生成 jenkins slave
新建任务----名字----自由风格
通用----勾选限制项目的运行节点----标签表达式:dongyali-jnlp
构建----执行shell----输入如下内容----保存立即构建
echo "测试 Kubernetes 动态生成 jenkins slave"
echo "==============docker in docker==========="
docker info
echo "=============kubectl============="
kubectl get pods
观察 Kubernetes 集群中 Pod 的变化:
$ kubectl get pod -n kube-ops
jenkins2-76644dbc9b-llcsp 1/1 Running 0 3h59m
jnlp-tl1km 1/1 Running 0 44s
当任务运行完毕,jnlp这个slave pod就会自动消失。
8、用pipeline测试 Kubernetes 动态生成 jenkins slave
新建任务----名字----流水线
在流水线脚本中输入如下内容----保存立即构建
node('dongyali-jnlp') {
stage('Clone') {
echo "1.Clone Stage"
}
stage('Test') {
echo "2.Test Stage"
}
stage('Build') {
echo "3.Build Stage"
}
stage('Deploy') {
echo "4. Deploy Stage"
}
}
本文题目:在k8s中安装jenkins并实现动态生成jenkinsslave
转载来源:http://scyanting.com/article/jdoohj.html