【ELK实践&开发★壹】基于Docker纯手工构建Elasticsearch高可用集群环境

jasonruan 2018.12.03

[TOC]

0 前言

Elasticsearch——You Know，For Search：），既然如此，笔者也不过多介绍，直接进入正题——基于Docker纯手工构建Elasticsearch高可用集群。

1 Elasticsearch环境搭建

1.1 构建开发环境基础镜像

关于开发基础镜像构建，请参看笔者另外一篇文章：《基于以太坊为女儿发一枚数字货币》(https://www.jianshu.com/p/455f17f4143b)

1.2 构建Elasticsearch基础镜像

基于开发环境基础镜像安装Elasticsearch当前较新版本6.5.1(November 20, 2018)

PS：之所以用较新版本，因笔者准备发文前一看，才发现ES已经更新到6.5.2（December 05, 2018），ES的版本发布速度也是太快了：（

1.2.1 编写Dockerfile

# js.es.Dockerfile 
FROM js/ubuntu:18.04
ENV ES_VERSION "6.5.1"
ENV JAVA_HOME /opt/java/jdk1.8.0_181
ENV PATH $PATH:$JAVA_HOME/bin
RUN mkdir -p /rzexin/soft && \
        wget https://artifacts.elastic.co/downloads/Elasticsearch/Elasticsearch-$ES_VERSION.tar.gz -P /rzexin/soft && \
        tar -zxvf /rzexin/soft/Elasticsearch-$ES_VERSION.tar.gz -C /rzexin/soft 
WORKDIR /rzexin/soft/
RUN ln -s Elasticsearch-$ES_VERSION es
WORKDIR /rzexin/soft/es
 
RUN groupadd jses && useradd jses -g jses && chown -R jses:jses /rzexin/soft
USER jses 
 
RUN sed -i 's%#network.host.*$%network.host: 0.0.0.0%g' config/Elasticsearch.yml 
RUN mkdir /rzexin/soft/es/data
 
ENTRYPOINT ["./bin/Elasticsearch"] 

1.2.2 构建镜像

$ sudo docker build -t rzexin/es:6.5.1 -f js.es.Dockerfile .

1.3 单节点环境搭建

1.3.1 启动镜像

$ sudo docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 --name JS01 rzexin/es:6.5.1
a4941ef67aae56e394a7d6ada511d026fad4cba29879c3fe86f37e8120c7fe5a

1.3.2 测试验证

在宿主机执行命令，如输出如下信息，表明单节点的Elasticsearch已经启动成功：）

$ http http://localhost:9200
HTTP/1.1 200 OK
content-encoding: gzip
content-length: 297
content-type: application/json; charset=UTF-8

{
    "cluster_name": "Elasticsearch",
    "cluster_uuid": "V5-76-zfQmKv-JgQkjlK4Q",
    "name": "RDwpJOV",
    "tagline": "You Know, for Search",
    "version": {
        "build_date": "2018-11-16T02:22:42.182257Z",
        "build_flavor": "default",
        "build_hash": "8c58350",
        "build_snapshot": false,
        "build_type": "tar",
        "lucene_version": "7.5.0",
        "minimum_index_compatibility_version": "5.0.0",
        "minimum_wire_compatibility_version": "5.6.0",
        "number": "6.5.1"
    }
}

1.4 高可用集群环境搭建

以下以搭建3个Elasticsearch节点的集群环境为例介绍

1.4.1 构建Elasticsearch集群镜像

1.4.1.1 编写Dockerfile

在Elasticsearch基础镜像之上构建集群镜像

主要是设置集群名称、节点名称、集群节点列表等

FROM rzexin/es:6.5.1          
 
RUN sed -i 's%#cluster.name.*$%cluster.name: jses%g' config/elasticsearch.yml && \
        sed -i 's%#discovery.zen.minimum_master_nodes:.*$%discovery.zen.minimum_master_nodes: 1%g' config/elasticsearch.yml
 
RUN echo "[ ! -z $""NODE_NAME"" ] && sed -i 's%#node.name.*$%node.name: '$""NODE_NAME""'%g' config/elasticsearch.yml" > /rzexin/soft/es/start.sh && \
    echo "[ ! -z $""DISCOVERY_HOSTS"" ] && sed -i 's%#discovery.zen.ping.unicast.hosts:.*$%discovery.zen.ping.unicast.hosts:                     ['$""DISCOVERY_HOSTS""']%g' config/elasticsearch.yml" >> /rzexin/soft/es/start.sh && \
    echo "./bin/elasticsearch" >> /rzexin/soft/es/start.sh && \
    chmod a+x /rzexin/soft/es/start.sh
 
ENTRYPOINT ["sh", "/rzexin/soft/es/start.sh"]

1.4.1.2 构建镜像

$ sudo docker build -t rzexin/escluster:6.5.1 -f js.es.cluster.Dockerfile .

1.4.2 创建数据文件挂载目录

分别为3个节点创建数据文件挂载目录

$ sudo docker volume create --name=data-js-es-1
data-js-es-1
$ sudo docker volume create --name=data-js-es-2
data-js-es-2
$ sudo docker volume create --name=data-js-es-3
data-js-es-3

1.4.3 编写docker-compose编排文件

# cat js.es.docker-compose.yml 
version: "3"
services:
  js01:
    build:
      context: .
      dockerfile: js.es.cluster.Dockerfile
    image: rzexin/escluster:6.5.1 
    container_name: js01
    hostname: js01
    privileged: true
    ports:
      - "9200:9200"
      - "9300:9300"
    expose:
      - 9200
      - 9300
    environment:
      NODE_NAME: "js-es-1"
      DISCOVERY_HOSTS: "\"js01\",\"js02\",\"js03\""
      ES_JAVA_OPTS: "-Xms256m -Xmx256m"
    volumes:
        - data-js-es-1:/rzexin/soft/es/data
  js02:
    build:
      context: .
      dockerfile: js.es.cluster.Dockerfile
    image: rzexin/escluster:6.5.1 
    container_name: js02
    hostname: js02
    ports:
      - "29200:9200"
      - "29300:9300"
    expose:
      - 29200
      - 29300
    environment:
      NODE_NAME: "js-es-2"
      DISCOVERY_HOSTS: "\"js01\",\"js02\",\"js03\""
      ES_JAVA_OPTS: "-Xms256m -Xmx256m"
    volumes:
      - data-js-es-2:/rzexin/soft/es/data
  js03:
    build:
      context: .
      dockerfile: js.es.cluster.Dockerfile
    image: rzexin/escluster:6.5.1 
    container_name: js03
    hostname: js03
    ports:
      - "39200:9200"
      - "39300:9300"
    expose:
      - 39200
      - 39300
    environment:
      NODE_NAME: "js-es-3"
      DISCOVERY_HOSTS: "\"js01\",\"js02\",\"js03\""
      ES_JAVA_OPTS: "-Xms256m -Xmx256m"
    volumes:
      - data-js-es-3:/rzexin/soft/es/data
volumes:
  data-js-es-1:
    external: true
  data-js-es-2:
    external: true
  data-js-es-3:
    external: true

1.4.4 启动集群节点

$ sudo docker-compose -f js.es.docker-compose.yml up -d
Creating network "dockerescluster_default" with the default driver
Creating js01 ... done
Creating js03 ... done
Creating js02 ... done

1.4.5 查看集群节点状态

$ sudo docker-compose -f js.es.docker-compose.yml ps
Name             Command             State                                   Ports     
-------------------------------------------------------------------------------------------------------------------
js01   sh /rzexin/soft/es/start.sh   Up      0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                        
js02   sh /rzexin/soft/es/start.sh   Up      29200/tcp, 29300/tcp, 0.0.0.0:29200->9200/tcp, 0.0.0.0:29300->9300/tcp
js03   sh /rzexin/soft/es/start.sh   Up      39200/tcp, 39300/tcp, 0.0.0.0:39200->9200/tcp, 0.0.0.0:39300->9300/tcp

1.4.6 测试验证

在宿主机执行命令，查看集群节点情况，打星号*的是自动选举出的主节点

$ curl http://localhost:9200/_cat/nodes?pretty
172.19.0.4 58 98 32 2.31 3.63 2.19 mdi - js-es-1
172.19.0.3 60 98 33 2.31 3.63 2.19 mdi * js-es-2
172.19.0.2 57 98 33 2.31 3.63 2.19 mdi - js-es-3

1.5 官方docker镜像搭建

要是不愿意折腾手工编译安装搭建，也可考虑直接使用Elasticsearch官方的docker镜像，具体步骤如下：

1.5.1 获取官方镜像

• 搜索镜像

$ sudo docker search Elasticsearch:6.5.1
NAME             DESCRIPTION                                     STARS OFFICIAL  
Elasticsearch    Elasticsearch is a powerful open source sear…   3270  [OK]      
kibana           Kibana gives shape to any kind of data — str…   1332  [OK]    

• 下载镜像

$ sudo docker pull Elasticsearch:6.5.1
6.5.1: Pulling from library/Elasticsearch
Digest: sha256:0db7521b9cd9f3906aae70fd146b78ce204b1c7c13fc32e4a2ed449e6576af53
Status: Image is up to date for Elasticsearch:6.5.1

• 查看镜像

$ sudo docker images elasticsearch
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
elasticsearch       6.5.1               32f93c89076d        2 weeks ago         773MB

1.5.2 运行官方镜像

• 运行镜像

运行该镜像默认会占用2G内存，受限于机器性能，故通过ES_JAVA_OPTS参数限制内存使用

$ sudo docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 --name ES01 32f93c89076d
29ff9574492ce0f3c008664fc1044851402e3594fbada62d91b4da725615fbbf

• 检查镜像

$ sudo docker ps 
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                            NAMES
29ff9574492c        32f93c89076d        "/usr/local/bin/dock…"   23 seconds ago      Up 22 seconds       0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   ES01

1.5.3 测试验证

看到如下输出，表示Elasticsearch启动成功

$ http 'http://localhost:9200' 
HTTP/1.1 200 OK
content-encoding: gzip
content-length: 298
content-type: application/json; charset=UTF-8

{
    "cluster_name": "docker-cluster",
    "cluster_uuid": "djDC-rsqQ9KD2hOXRP3Rew",
    "name": "jzqf9Oy",
    "tagline": "You Know, for Search",
    "version": {
        "build_date": "2018-11-16T02:22:42.182257Z",
        "build_flavor": "default",
        "build_hash": "8c58350",
        "build_snapshot": false,
        "build_type": "tar",
        "lucene_version": "7.5.0",
        "minimum_index_compatibility_version": "5.0.0",
        "minimum_wire_compatibility_version": "5.6.0",
        "number": "6.5.1"
    }
}

2 容灾验证

• 查看当前集群节点情况

  打星号*的是自动选举出的主节点

$ curl http://localhost:9200/_cat/nodes?pretty
172.19.0.4 58 98 32 2.31 3.63 2.19 mdi - js-es-1
172.19.0.3 60 98 33 2.31 3.63 2.19 mdi * js-es-2
172.19.0.2 57 98 33 2.31 3.63 2.19 mdi - js-es-3

• 停掉2号主节点（js-es-2）

  $ sudo docker-compose -f js.es.docker-compose.yml stop js02
  Stopping js02 ... done
  

• 查看集群节点情况

  可见2号主节点（js-es-2）已被剔掉，并重新选举出了新的主节点

  $ curl http://localhost:9200/_cat/nodes?pretty
  172.23.0.4 48 90 22 1.01 2.86 2.64 mdi * js-es-3
  172.23.0.3 56 90 22 1.01 2.86 2.64 mdi - js-es-1
  

• 启动2号节点（js-es-2）

  $ sudo docker-compose -f js.es.docker-compose.yml start js02
  Starting js02 ... done
  

• 再次查看集群节点情况

  可见2号节点又重新加入集群，但这时已经不是主节点了

  $ curl http://localhost:9200/_cat/nodes?pretty
  172.23.0.2 43 98 31 1.58 2.25 2.43 mdi - js-es-2
  172.23.0.4 51 98 31 1.58 2.25 2.43 mdi * js-es-3
  172.23.0.3 58 98 31 1.58 2.25 2.43 mdi - js-es-1
  

3 常见错误

3.1 错误：can not run elasticsearch as root

• 错误描述

  执行Elasticsearch启动命令报错

org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root

• 错误原因

  出于系统安全考虑，不可使用root用户启动，因Elasticsearch可以接收用户输入的脚本并且执行，为了系统安全考虑，建议创建一个单独的用户用来运行Elasticsearch。

• 解决方案

  创建用户和分组，并使用该用户启动es

groupadd jses
useradd jses -g jses
chown -R jses:jses /rzexin/soft/es
su jses
./bin/elasticsearch

3.2 错误：vm.max_map_count is too low

[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

（1）临时修改

• 在宿主机上执行命令

# 查看当前值
$ sudo sysctl -a | grep "vm.max_map_count"
vm.max_map_count = 65530

# 修改成最低要求
$ sudo sysctl -w vm.max_map_count=262144
vm.max_map_count = 262144

# 查看修改后的值
$ sudo sysctl -a | grep "vm.max_map_count"
vm.max_map_count = 262144

• 重启docker镜像

# 先删除容器，否则报错
$ sudo docker run -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -d -p 9200:9200 -p 9300:9300 --name ES01 32f93c89076d
docker: Error response from daemon: Conflict. The container name "/ES01" is already in use by container "a9e2daa0227bb60a7ce3fffd0881eb7d4caa2b032c3933f52fb6a5bcd55def12". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

$ sudo docker ps -l
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
a9e2daa0227b        32f93c89076d        "/usr/local/bin/dock…"   6 minutes ago       Exited (78) 6 minutes ago                       ES01

$ sudo docker rm a9e2daa0227b
a9e2daa0227b

# 重启镜像
$ sudo docker run -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -d -p 9200:9200 -p 9300:9300 --name ES01 32f93c89076d
6749988a01723a1ba9079066274676fbdd302866011e8158a70729b0b98de11b

（2）永久修改

修改配置文件：sudo vi /etc/sysctl.conf

加入：

vm.max_map_count=262144

保存即可。

4 后记

完成了Elasticsearch集群环境搭建，下一篇将介绍Elasticsearch与SpringBoot和Golang的整合。