Install Docker
Following Kubernetes's recommendation, we are choosing docker-1.12.6 as our container.
# yum install docker-1.13.1
Copy the Registry's certs to docker cert folder. Do the step for all of the nodes.
This step is not required, just only for your existing private docker registry.
# scp -r ca.crt sokm1:/etc/docker/certs.d/
Add proxy settings for Docker. As Kubernetes's images from Google are needed, so proxy is required.
Create a file called /etc/systemd/system/docker.service.d/http-proxy.conf that adds the HTTP_PROXY environment variable.
Or, if you are behind an HTTPS proxy server, create a file called /etc/systemd/system/docker.service.d/https-proxy.conf that adds the HTTPS_PROXY environment variable.
# mkdir -p /etc/systemd/system/docker.service.d/
# vi /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://196.168.0.127:8118/" "NO_PROXY=localhost,127.0.0.1,180.169.188.90,hub.docker.gemii.cc,192.168.0.169"
#
# vi /etc/systemd/system/docker.service.d/https-proxy.conf
[Service]
Environment="HTTP_PROXY=http://192.168.0.127:8118/" "NO_PROXY=localhost,127.0.0.1,180.169.188.90,hub.docker.gemii.cc,192.168.0.169"
Configure Docker storage
Here we created a disk /dev/sdb already.
# cat <<EOF > /etc/sysconfig/docker-storage-setup
STORAGE_DRIVER=overlay2
DEVS=/dev/sdb
CONTAINER_ROOT_LV_NAME=docker-lv
CONTAINER_ROOT_LV_SIZE=100%FREE
CONTAINER_ROOT_LV_MOUNT_PATH=/var/lib/docker
VG=docker-vg
EOF
# docker-storage-setup
INFO: Device node /dev/sdb1 exists.
Physical volume "/dev/sdb1" successfully created.
Volume group "docker-vg" successfully created
#
Verify the configuration. You should have a dm.thinpooldev value in the /etc/sysconfig/docker-storage file and a docker-pool logical volume:
# cat /etc/sysconfig/docker-storage
DOCKER_STORAGE_OPTIONS="--storage-driver devicemapper --storage-opt dm.fs=xfs --storage-opt dm.thinpooldev=/dev/mapper/docker--vg-docker--pool --storage-opt dm.use_deferred_removal=true --storage-opt dm.use_deferred_deletion=true "
# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
home cl -wi-ao---- <141.12g
root cl -wi-ao---- 50.00g
swap cl -wi-a----- <7.88g
docker-pool docker-vg twi-a-t--- 39.79g 0.00 0.05
Recreate Docker Storage
If you delete the docker lib files, you may encounter this issue when you try to re-start the docker.
# systemctl stop docker
# rm -rf /var/lib/docker/*
# systemctl restart docker
# systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─http-proxy.conf, https-proxy.conf
Active: failed (Result: exit-code) since Fri 2018-02-09 21:51:53 EST; 3min 41s ago
Docs: http://docs.docker.com
Process: 1909 ExecStart=/usr/bin/dockerd-current --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current --default-runtime=docker-runc --exec-opt native.cgroupdriver=systemd --userland-proxy-path=/usr/libexec/docker/docker-proxy-current $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $ADD_REGISTRY $BLOCK_REGISTRY $INSECURE_REGISTRY $REGISTRIES (code=exited, status=1/FAILURE)
Main PID: 1909 (code=exited, status=1/FAILURE)
Feb 09 21:51:52 oskw1 systemd[1]: Starting Docker Application Container Engine...
Feb 09 21:51:52 oskw1 dockerd-current[1909]: time="2018-02-09T21:51:52.716637492-05:00" level=info msg="libcontainerd: new containerd process, pid: 1916"
Feb 09 21:51:53 oskw1 dockerd-current[1909]: time="2018-02-09T21:51:53.750274954-05:00" level=fatal msg="Error starting daemon: error initializing graphdriver: devmapper: Unable to take ownership of thin-pool (docker...used data blocks"
Feb 09 21:51:53 oskw1 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Feb 09 21:51:53 oskw1 systemd[1]: Failed to start Docker Application Container Engine.
Feb 09 21:51:53 oskw1 systemd[1]: Unit docker.service entered failed state.
Feb 09 21:51:53 oskw1 systemd[1]: docker.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
So may need to Recreate the storage.
Delete the LV and PV firstly.
# lvremove docker-lv
# vgremove docker-vg
# pvs
# fdisk -l /dev/sdb
Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000626bf
Device Boot Start End Blocks Id System
/dev/sdb1 2048 209715199 104856576 8e Linux LVM
sdb1 partition exists, so we need to delete it.
# fdisk -l /dev/sdb
Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000626bf
Device Boot Start End Blocks Id System
/dev/sdb1 2048 209715199 104856576 8e Linux LVM
[root@oskw1 docker]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): m
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
g create a new empty GPT partition table
G create an IRIX (SGI) partition table
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)
Command (m for help): p
Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000626bf
Device Boot Start End Blocks Id System
/dev/sdb1 2048 209715199 104856576 8e Linux LVM
Command (m for help): d
Selected partition 1
Partition 1 is deleted
Command (m for help): v
Remaining 209715199 unallocated 512-byte sectors
Command (m for help): p
Disk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000626bf
Device Boot Start End Blocks Id System
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Then try to recreate the docker storage, we still failed. so need to erase the dos.
# docker-storage-setup
WARNING: Device for PV c1fD2e-ke13-OXvg-XPLO-SJ1f-a2RV-UwdTTO not found or rejected by a filter.
ERROR: Found dos signature on device /dev/sdb at offset 0x1fe. Wipe signatures using wipefs or use WIPE_SIGNATURES=true and retry.
#
# wipefs /dev/sdb
offset type
----------------------------------------------------------------
0x1fe dos [partition table]
#
# wipefs --all --force /dev/sdb
/dev/sdb: 2 bytes were erased at offset 0x000001fe (dos): 55 aa
/dev/sdb: calling ioclt to re-read partition table: Success
# wipefs /dev/sdb
#
# docker-storage-setup --reset
# docker-storage-setup
INFO: Device node /dev/sdb1 exists.
Physical volume "/dev/sdb1" successfully created.
Volume group "docker-vg" successfully created
Using default stripesize 64.00 KiB.
Rounding up size to full physical extent 104.00 MiB
Thin pool volume with chunk size 512.00 KiB can address at most 126.50 TiB of data.
Logical volume "docker-pool" created.
Logical volume docker-vg/docker-pool changed.
# systemctl start docker
#
Check Docker Status
[root@oskm1 ~]# docker info
Containers: 15
Running: 15
Paused: 0
Stopped: 0
Images: 10
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc docker-runc
Default Runtime: docker-runc
Init Binary: docker-init
containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
seccomp
WARNING: You're not using the default seccomp profile
Profile: /etc/docker/seccomp.json
selinux
Kernel Version: 4.15.1-1.el7.elrepo.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 4
Total Memory: 15.67 GiB
Name: oskm1.os.gemii.tech
ID: ZNMN:H5NQ:LKFW:W4EM:PDWJ:RJGS:NVV5:NTU3:QRSB:FJOY:V2D7:J4T2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://192.168.0.127:8118/
Https Proxy: http://192.168.0.127:8118/
No Proxy: localhost,127.0.0.1,180.169.188.90,hub.docker.gemii.cc,192.168.0.169
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)