kubernetes安装Jenkins(3)

将master部署到k8s中，并通过Kubernetes插件，动态生成slave节点，保证有job时动态创建slave，构建完成之后删除slave，充分利用k8s的资源，同时也节约Jenkins集群单独部署时，slave节点资源的浪费。

  1. 部署nfs网络文件系统，来持久化Jenkins的job和构建时的依赖

    1. 新建一台虚拟机，IP地址分配为:

       192.168.241.144

    2. 参照进行安装部署,注意必须全用户具有写的权限

  2. 部署安装Jenkins

    1. 安装所需要的文件可以从GitHub获取

    2. 创建na-rbac.yaml文件，用于创建namespace和serviceAcount

       apiVersion: v1
 kind: Namespace
 metadata:
   name: kube-ops
 
 ---
 
 apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: jenkins
   namespace: kube-ops
 
 ---
 
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1beta1
 metadata:
   name: jenkins
   namespace: kube-ops
 rules:
   - apiGroups: [""]
     resources: ["pods"]
     verbs: ["create","delete","get","list","patch","update","watch"]
   - apiGroups: [""]
     resources: ["pods/exec"]
     verbs: ["create","delete","get","list","patch","update","watch"]
   - apiGroups: [""]
     resources: ["pods/log"]
     verbs: ["get","list","watch"]
   - apiGroups: [""]
     resources: ["secrets"]
     verbs: ["get"]
   - apiGroups: ["apps"]
     resources: ["deployments"]  # 相对于GitHub中的文件，添加了deployments资源的权限，主要用于后期部署，不然会报错
     verbs: ["create","delete","get","list","patch","update","watch"]
 
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRoleBinding
 metadata:
   name: jenkins
   namespace: kube-ops
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: jenkins
 subjects:
   - kind: ServiceAccount
     name: jenkins
     namespace: kube-ops

    3. 创建pvc.yaml文件，主要用过nfs来进行网络存储

       apiVersion: v1
 kind: PersistentVolume          # 定义资源，创建pv(persistence volume)
 metadata:
   name: opspv
   namespace: kube-ops
 spec:
   capacity:
     storage: 10Gi
   accessModes:
     - ReadWriteMany
   persistentVolumeReclaimPolicy: Delete
   mountOptions:
     - hard
     - nfsvers=4.1
   nfs:
     server: 192.168.241.144
     path: /nfs-share        # 改文件路径必须再nfs得服务器中存在
 
 ---
 kind: PersistentVolumeClaim     # 向pv申请资源
 apiVersion: v1
 metadata:
   name: opspvc
   namespace: kube-ops
 spec:
   accessModes:
     - ReadWriteMany
   resources:
     requests:
       storage: 10Gi

    4. 创建deployment.yaml文件,部署deployment和service

       ---
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: jenkins
   namespace: kube-ops
 spec:
   selector:
     matchLabels:
       app: jenkins
   template:
     metadata:
       labels:
         app: jenkins
     spec:
       terminationGracePeriodSeconds: 10
       serviceAccountName: jenkins
       containers:
       - name: jenkins
         image: jenkins/jenkins:lts
         imagePullPolicy: IfNotPresent
         ports:
         - containerPort: 8080  # web服务暴露的端口
           name: web
           protocol: TCP
         - containerPort: 50000 # 用于和agent通信的端口
           name: agent
           protocol: TCP
         resources:                  # 使用的资源
           limits:
             cpu: 2000m
             memory: 2Gi
           requests:
             cpu: 1000m
             memory: 1Gi
         livenessProbe:
           httpGet:
             path: /login
             port: 8080
           initialDelaySeconds: 60
           timeoutSeconds: 5
           failureThreshold: 12
         readinessProbe:
           httpGet:
             path: /login
             port: 8080
           initialDelaySeconds: 60
           timeoutSeconds: 5
           failureThreshold: 12
         volumeMounts:
         - name: jenkinshome
           subPath: jenkins
           mountPath: /var/jenkins_home
         env:
         - name: LIMITS_MEMORY
           valueFrom:
             resourceFieldRef:
               resource: limits.memory
               divisor: 1Mi
         - name: JAVA_OPTS
           value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85 -Duser.timezone=Asia/Shanghai
       securityContext:
         fsGroup: 1000
       volumes:
       - name: jenkinshome
         persistentVolumeClaim:
           claimName: opspvc  # 引用前面创建的pvc
 
 ---
 apiVersion: v1
 kind: Service      # 部署service
 metadata:
   name: jenkins
   namespace: kube-ops
   labels:
     app: jenkins
 spec:
   selector:
     app: jenkins
   ports:
   - name: web
     port: 8080
     targetPort: web
   - name: agent
     port: 50000
     targetPort: agent

    5. 创建ingress.yaml文件，这里使用得是nginx-ingress-controller部署得ingress-controller，主要得目的是每个域名分配不同得IP，则必须部署一个新得ingress-controller

      1. ingress-controller配置文件

         kind: ConfigMap
 apiVersion: v1
 metadata:
   name: nginx-configuration
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 
 ---
 kind: ConfigMap
 apiVersion: v1
 metadata:
   name: tcp-services
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 
 ---
 kind: ConfigMap
 apiVersion: v1
 metadata:
   name: udp-services
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 
 ---
 apiVersion: v1
 kind: ServiceAccount
 metadata:
   name: nginx-ingress-serviceaccount
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: Role
 metadata:
   name: nginx-ingress-role
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 rules:
   - apiGroups:
       - ""
     resources:
       - configmaps
       - pods
       - secrets
       - namespaces
     verbs:
       - get
       - update
       - list
   - apiGroups:
       - ""
     resources:
       - configmaps
     resourceNames:
       # Defaults to "<election-id>-<ingress-class>"
       # Here: "<ingress-controller-leader>-<nginx>"
       # This has to be adapted if you change either parameter
       # when launching the nginx-ingress-controller.
       - "ingress-controller-leader-nginx"
     verbs:
       - get
       - update
       - list
   - apiGroups:
       - ""
     resources:
       - configmaps
     verbs:
       - create
       - update
       - list
   - apiGroups:
       - ""
     resources:
       - endpoints
     verbs:
       - get
       - list
 
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: RoleBinding
 metadata:
   name: nginx-ingress-role-nisa-devops-binding  # 注意该名字最好修改一下，保证唯一性
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: Role
   name: nginx-ingress-role
 subjects:
   - kind: ServiceAccount
     name: nginx-ingress-serviceaccount
     namespace: kube-ops
 
 ---
 apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRoleBinding
 metadata:
   name: nginx-ingress-clusterrole-nisa-devops-binding  # 注意该名字最好修改一下，保证唯一
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: nginx-ingress-clusterrole                # 这里使用的不是dashboard时创建的ingress的集群角色
 subjects:
   - kind: ServiceAccount
     name: nginx-ingress-serviceaccount
     namespace: kube-ops
 
 ---
 
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: nginx-ingress-controller
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx
     app.kubernetes.io/part-of: ingress-nginx
 spec:
   replicas: 1
   selector:
     matchLabels:
       app.kubernetes.io/name: ingress-nginx
       app.kubernetes.io/part-of: ingress-nginx
   template:
     metadata:
       labels:
         app.kubernetes.io/name: ingress-nginx
         app.kubernetes.io/part-of: ingress-nginx
       annotations:
         prometheus.io/port: "10254"
         prometheus.io/scrape: "true"
     spec:
       # wait up to five minutes for the drain of connections
       terminationGracePeriodSeconds: 300
       serviceAccountName: nginx-ingress-serviceaccount
       containers:
         - name: nginx-ingress-controller
           image: wistiaanders/nginx-ingress-controller:0.25.1
           args:
             - /nginx-ingress-controller
             - --configmap=$(POD_NAMESPACE)/nginx-configuration
             - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
             - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
             - --publish-service=$(POD_NAMESPACE)/ingress-nginx
             - --annotations-prefix=nginx.ingress.kubernetes.io
             - --ingress-class=devops-nginx-ingress   # 这里配置类别，用于安装运维的ingress，都注册到这个上面
             - --enable-ssl-passthrough
           securityContext:
             allowPrivilegeEscalation: true
             capabilities:
               drop:
                 - ALL
               add:
                 - NET_BIND_SERVICE
             # www-data -> 33
             runAsUser: 33
           env:
             - name: POD_NAME
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.name
             - name: POD_NAMESPACE
               valueFrom:
                 fieldRef:
                   fieldPath: metadata.namespace
           ports:
             - name: http
               containerPort: 80
             - name: https
               containerPort: 443
           livenessProbe:
             failureThreshold: 3
             httpGet:
               path: /healthz
               port: 10254
               scheme: HTTP
             initialDelaySeconds: 10
             periodSeconds: 10
             successThreshold: 1
             timeoutSeconds: 10
           readinessProbe:
             failureThreshold: 3
             httpGet:
               path: /healthz
               port: 10254
               scheme: HTTP
             periodSeconds: 10
             successThreshold: 1
             timeoutSeconds: 10
           lifecycle:
             preStop:
               exec:
                 command:
                   - /wait-shutdown
 
 ---


 
 ---
 kind: Service
 apiVersion: v1
 metadata:
   name: ingress-nginx-jenkins
   namespace: kube-ops
   labels:
     app.kubernetes.io/name: ingress-nginx-jenkins
     app.kubernetes.io/part-of: ingress-nginx-jenkins
 spec:
   externalTrafficPolicy: Local
   type: LoadBalancer
   selector:
     app.kubernetes.io/name: ingress-nginx-jenkins
     app.kubernetes.io/part-of: ingress-nginx-jenkins
   ports:
     - name: http
       port: 80
       targetPort: http
     - name: https
       port: 443
       targetPort: https
 
 ---

      2. ingress-controller配置文件

         apiVersion: networking.k8s.io/v1beta1
 kind: Ingress
 metadata:
   name: jenkins-ingress
   namespace: kube-ops
   annotations:
     kubernetes.io/ingress.class: "devops-nginx-ingress"  # 需要和上面创建ingress-controller时配置启动参数中的ingress-class一致
 spec:
   rules:
   - host: jenkins.tlh.com
     http:
       paths:
         - path: /
           backend:
             serviceName: jenkins
             servicePort: 8080

    6. 对上面的文件依次执行

       kuberctl apply -f *.yaml

    7. 配置csrf

      jenkins-csrf

  3. 安装Jenkins插件

    1. kubernetes插件

      1. 该插件的目的是用于动态生成slave节点，优化资源，提示构建的性能

      2. 配置

        k8s-plugin-conf

      3. 配置中的地址是通过k8s提供的服务发现来实现的

    2. Git Parameter Plug-In
    3. Extended Choice Parameter Plug-In
最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 205,132评论 6赞 478
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 87,802评论 2赞 381
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 151,566评论 0赞 338
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,858评论 1赞 277
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,867评论 5赞 368
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,695评论 1赞 282
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 38,064评论 3赞 399
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,705评论 0赞 258
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 42,915评论 1赞 300
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,677评论 2赞 323
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,796评论 1赞 333
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,432评论 4赞 322
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 39,041评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,992评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,223评论 1赞 260
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 45,185评论 2赞 352
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,535评论 2赞 343