在OpenPOWER上安装红帽OpenShift3.11教程

机器集群规划

操作系统要求

IBM POWER9: RHEL-ALT 7.5 with the "Minimal" installation option and the latest packages from the Extras channel.

IBM POWER8: RHEL 7.5 with the "Minimal" installation option and the latest packages from the Extras channel.

Master :

Minimum 4 vCPU (additional are strongly recommended).

Minimum 16 GB RAM (additional memory is strongly recommended, especially if etcd is co-located on masters).

Minimum 40 GB hard disk space for the file system containing/var/.

Minimum 1 GB hard disk space for the file system containing/usr/local/bin/.

Minimum 1 GB hard disk space for the file system containing the system’s temporary directory.

Masters with a co-located etcd require a minimum of 4 cores. Two-core systems do not work.

Nodes:

NetworkManager 1.0 or later.

1 vCPU.

Minimum 8 GB RAM.

Minimum 15 GB hard disk space for the file system containing/var/.

Minimum 1 GB hard disk space for the file system containing/usr/local/bin/.

Minimum 1 GB hard disk space for the file system containing the system’s temporary directory.

An additional minimum 15 GB unallocated space per system running containers for Docker’s storage back end; seeConfiguring Docker Storage. Additional space might be required, depending on the size and number of containers that run on the node.

实验集群

Master   172.XX.XX.175 

Node   172.XX.XX.182   

            172.XX.XX.183   

操作步骤

1 Enable Security-Enhanced Linux (SELinux) on all of the nodes

       a. vi /etc/selinux/config

                set SELINUX=enforcing and SELINUXTYPE=targeted

        b. touch /.autorelabel; reboot

2 Ensuring host access

设置master到各个Node的免密登录

2.1 Generate an SSH key on the host you run the installation playbook on:

# ssh-keygen

2.2 Distribute the key to the other cluster hosts. You can use abashloop:

# for host in master.openshift.example.com \1node1.openshift.example.com \2node2.openshift.example.com; \3do ssh-copy-id -i ~/.ssh/id_rsa.pub $host; \ done

3  更新网卡配置信息

        In /etc/sysconfig/network-scripts/ifcfg-ethxx

                a. Make sure that: NM_CONTROLLED=yes

                b. Add following entries:

                        DNS1=

                        DNS2=

                        DOMAIN=

                (You can get DNS values from: /etc/sysconfig/network-scripts/ifcfg-bootnet and /etc/resolv.conf)

如果都没有值DNS1=本机IP地址

                (You can get DOMAIN value by this command:  domainname -d)

4 每台机器设置/etc/hosts

[root@node1 network-scripts]# cat /etc/hosts

127.0.0.1  localhost localhost.localdomain localhost4 localhost4.localdomain4

::1        localhost localhost.localdomain localhost6 localhost6.localdomain6

172.xx.xx.175  master.openshift.example.com

172.xx.xx.182  node1.openshift.example.com

172.xx.xx.183  node2.openshift.example.com

5 yum 设置代理

如果机器不能直接上网，需要设置上网代理服务器

vi /etc/yum.conf

set  proxy=http://xx.xx.xx.xx:xxxx

6  Registering hosts(需要有红帽的订阅）

在每台机器执行

# subscription-manager register --username=<user_name> --password=<password>

# subscription-manager refresh

# subscription-manager list --available --matches '*OpenShift*'

# subscription-manager attach --pool=<pool_id>

6 注册yum 源

For on-premise installations on IBM POWER8 servers, run the following command

subscription-manager repos \

--enable="rhel-7-for-power-le-rpms" \

--enable="rhel-7-for-power-le-extras-rpms" \

--enable="rhel-7-for-power-le-optional-rpms" \

--enable="rhel-7-server-ansible-2.6-for-power-le-rpms" \

--enable="rhel-7-for-power-le-ose-3.11-rpms" \

--enable="rhel-7-for-power-le-fast-datapath-rpms" \

--enable="rhel-7-server-for-power-le-rhscl-rpms"

For on-premise installations on IBM POWER9 servers, run the following command:

# subscription-manager repos \

    --enable="rhel-7-for-power-9-rpms" \

    --enable="rhel-7-for-power-9-extras-rpms" \

    --enable="rhel-7-for-power-9-optional-rpms" \

    --enable="rhel-7-server-ansible-2.6-for-power-9-rpms" \

    --enable="rhel-7-server-for-power-9-rhscl-rpms" \

    --enable="rhel-7-for-power-9-ose-3.11-rpms"

7 安装基础包

7.1 每台机器都执行

# yum -y install wget git net-tools bind-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct

# yum -y update

# reboot

# yum install atomic-openshift-excluder-3.11.141*

Now install a container engine:

To install CRI-O:

# yum -y install cri-o

To install Docker:

# yum -y install docker

7.2在master执行

# yum -y install openshift-ansible

# yum install atomic-openshift atomic-openshift-clients atomic-openshift-hyperkube atomic-openshift-node flannel glusterfs-fuse  (可以不执行此命令）

# yum install cockpit-docker cockpit-kubernetes

7.3 在node执行

#  yum install atomic-openshift atomic-openshift-node flannel glusterfs-fuse   (可以不执行此命令）

8 开始安装openshift 在master节点上执行

8.1 安装前检查

$ cd /usr/share/ansible/openshift-ansible

$ ansible-playbook -i <inventory_file> playbooks/prerequisites.yml

8.2 执行安装

$ cd /usr/share/ansible/openshift-ansible

$ ansible-playbook -i <inventory_file> playbooks/deploy_cluster.yml

9 inventory_file 示例（1 master +2 node ）

[root@master openshift-ansible]# ls

ansible.cfg  host.311  inventory  playbooks  roles

[root@master openshift-ansible]# cat host.311

# Create an OSEv3 group that contains the masters, nodes, and etcd groups

[OSEv3:children]

masters

nodes

etcd

# Set variables common for all OSEv3 hosts

[OSEv3:vars]

# SSH user, this user should allow ssh based auth without requiring a password

ansible_ssh_user=root

openshift_deployment_type=openshift-enterprise

# If ansible_ssh_user is not root, ansible_become must be set to true

#ansible_become=true

openshift_master_default_subdomain=master.openshift.example.com

debug_level=2

# default selectors for router and registry services

# openshift_router_selector='node-role.kubernetes.io/infra=true'

# openshift_registry_selector='node-role.kubernetes.io/infra=true'

# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider

#openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider', 'filename': '/etc/origin/master/htpasswd'}]

openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]

openshift_master_htpasswd_users={'my-rhel-icp-admin': '$apr1$6eO/grkf$9jRafb0tw/2KQEAejT8Lc.'}

# supposedly encrypted password of: S3cure-icp-wordP*s?

openshift_disable_check=memory_availability,disk_availability,docker_image_availability

openshift_master_cluster_hostname=master.openshift.example.com

openshift_master_cluster_public_hostname=master.openshift.example.com

# false

#ansible_service_broker_install=false

#openshift_enable_service_catalog=false

#template_service_broker_install=false

#openshift_logging_install_logging=false

# registry passwd

oreg_url=registry.redhat.io/openshift3/ose-${component}:${version}

oreg_auth_user=****@xxx

oreg_auth_password=*******

openshift_http_proxy=http://xxx.xxx.xxx.xxx:3130

#openshift_https_proxy=https://xx.xxx.xxx.xxx:3130

openshift_no_proxy=".openshift.example.com"

# docker config

openshift_docker_additional_registries=registry.redhat.io

#openshift_docker_insecure_registries

#openshift_docker_blocked_registries

openshift_docker_options="--log-driver json-file --log-opt max-size=1M --log-opt max-file=3"

# openshift_cluster_monitoring_operator_install=false

# openshift_metrics_install_metrics=true

# openshift_enable_unsupported_configurations=True

#openshift_logging_es_nodeselector='node-role.kubernetes.io/infra: "true"'

#openshift_logging_kibana_nodeselector='node-role.kubernetes.io/infra: "true"'

# host group for masters

[masters]

master.openshift.example.com  openshift_public_hostname="master.openshift.example.com"

# host group for etcd

[etcd]

master.openshift.example.com  openshift_public_hostname="master.openshift.example.com"

# host group for nodes, includes region info

[nodes]

master.openshift.example.com openshift_public_hostname="master.openshift.example.com"  openshift_node_group_name='node-config-master-infra'

node[1:2].openshift.example.com openshift_public_hostname="node-[1:2].openshift.example.com" openshift_node_group_name='node-config-compute'

10 安装过程中可能出现的错误情况

10.1 如果安装openshift_cluster_monitoring_operator_install ，对master需要设置openshift_node_group_name='node-config-master-infra'

参考https://github.com/vorburger/opendaylight-coe-kubernetes-openshift/issues/5

10.2 对于代理设置，需要设置no_proxy

参考https://github.com/openshift/openshift-ansible/issues/11365

10.3   https://github.com/openshift/openshift-ansible/issues/10427

10.3.1 FAILED - RETRYING: Wait for the ServiceMonitor CRD to be created #10427

File /etc/sysconfig/network-scripts/ifcfg-eth0 (CentOS)

There is a flag NM_CONTROLLED=no

 10.3.2  FAILED - RETRYING: Wait for the ServiceMonitor CRD to be created    #10427

I have the same issue, but what I did was....

Add NM_CONTROLLED=yes to ifcfg-eth0 to all my nodes

Verify my pods with $oc get pods --all-namespaces

$oc describe [pod cluster-monitoring-operator-WXYZ-ASDF] -n openshift-monitoring ==> With this command, in last part I could see reason with my pod didn't initiate, I have this message....

Warning  FailedCreatePodSandBox  1h                  kubelet, infra-openshift-nuuptech  Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "70719b9ee2bb9c54fc1d866a6134b229b3c1c151148c9558ea0a4ef8cb66526a" network for pod "cluster-monitoring-operator-67579f5cb5-gxmwc": NetworkPlugin cni failed to set up pod "cluster-monitoring-operator-67579f5cb5-gxmwc_openshift-monitoring" network:failed to find plugin "bridge" in path [/opt/cni/bin], failed to clean up sandbox container "70719b9ee2bb9c54fc1d866a6134b229b3c1c151148c9558ea0a4ef8cb66526a" network for pod "cluster-monitoring-operator-67579f5cb5-gxmwc": NetworkPlugin cni failed to teardown pod "cluster-monitoring-operator-67579f5cb5-gxmwc_openshift-monitoring" network: failed to find plugin "bridge" in path [/opt/cni/bin]]

I searched what is in bold, and I find a next solution.....

$ls -l /etc/cni/net.d ==> Normally the only file should be 80-openshift-network.conf, and I had 3 files

$ ls -l /etc/cni/net.d

-rw-r--r--. 1 root root 294 Mar 12 16:46 100-crio-bridge.conf

-rw-r--r--. 1 root root  54 Mar 12 16:46 200-loopback.conf

-rw-r--r--. 1 root root  83 May 15 16:15 80-openshift-network.conf

Red Hat suggest delete extra files and only keep 80-openshift-network.conf, but I only move 100-crio-bridge.conf and 200-loopback.conf to other directory. After do that, I reboot all my nodes, and in master node I executeplaybooks/openshift-monitoring/config.ymlagain and it worked.


11 安装成功后登陆用户创建

由于admin无法直接登陆，需要创建用户

11.1 用htpasswd创建dev/dev的用户

htpasswd -b /etc/origin/master/htpasswd dev dev

11.2  给dev用户添加集群管理员权限，这样可以访问集群内所有项目

# oc login -u system:admin

# htpasswd -b /etc/origin/master/htpasswd dev dev

# oc adm policy add-cluster-role-to-user cluster-admin dev

[root@master openshift-ansible]# oc get clusterrolebindings |grep dev

cluster-admin-0                                                            /cluster-admin                                                          dev

11.3 访问https://master.openshift.example.com:8443 

输入用户名dev 密码dev 

12 卸载 openshift

ansible-playbook -i hosts.311 /usr/share/ansible/openshift-ansible/playbooks/adhoc/uninstall.yml

最后编辑于 ：
©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 204,921评论 6赞 478
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 87,635评论 2赞 381
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 151,393评论 0赞 338
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,836评论 1赞 277
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,833评论 5赞 368
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,685评论 1赞 281
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 38,043评论 3赞 399
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,694评论 0赞 258
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 42,671评论 1赞 300
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,670评论 2赞 321
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,779评论 1赞 332
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,424评论 4赞 321
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 39,027评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,984评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,214评论 1赞 260
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 45,108评论 2赞 351
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,517评论 2赞 343