Docker部署Elasticsearch8集群

这里我在虚拟机里面安装的，Ubuntu22.04，4核8G内存，要安装Elasticsearch集群+Kibana可视化工具。Elasticsearch的部署官网都是给了教程的。我们可以直接参考。

单节点安装的：https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_start_a_single_node_cluster
集群安装：https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file
官网给的集群安装是在一个compose文件里面的，也就是在一个机器里面安装的集群，没有分在不同的节点。官网这么给的，我们就这么用。机器多给点内存和硬盘。

1. 修改系统内核文件

首先，需要修改vm.max_map_count的值，不然可能报错：max virtual memory areas vm.max_map_count [65530] is too low。

# 1. 修改
vim /etc/sysctl.conf
# 在最后添加如下内容
vm.max_map_count=262144
# 2. 使改动生效
sysctl -p

2. 配置文件

官网给了 2个。注意这2个配置文件要在同一目录下。
.env和docker-compose.yml

2.1 env

# Password for the 'elastic' user (at least 6 characters)
ELASTIC_PASSWORD=123456

# Password for the 'kibana_system' user (at least 6 characters)
KIBANA_PASSWORD=123abc

# Version of Elastic products
STACK_VERSION=8.14.3

# Set the cluster name
CLUSTER_NAME=docker-cluster

# Set to 'basic' or 'trial' to automatically start the 30-day trial
LICENSE=basic
#LICENSE=trial

# Port to expose Elasticsearch HTTP API to the host
ES_PORT=9200
#ES_PORT=127.0.0.1:9200

# Port to expose Kibana to the host
KIBANA_PORT=5601
#KIBANA_PORT=80

# Increase or decrease based on the available host memory (in bytes)
MEM_LIMIT=1073741824

# Project namespace (defaults to the current folder name if not set)
#COMPOSE_PROJECT_NAME=myproject

2.2 docker-compose.yml

这个文件建了5个service，因为elasticsearch要求要证书什么的，第一个是为了生成统一证书，好像还在修改了es01中kibana_system账号的密码，以后就没用了；2、3、4都是elasticsearch服务，都是用的同一个证书；最后是kibana服务。
映射目录稍作了改动，加了网络和ip(注意它生成证书的时候指定了ip，那里也要改成指定的IP)，其他基本和官网一样。

#version: '3.8'
services:
  es-certs:
    env_file:
      - .env
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    container_name: es-certs
    privileged: true
    volumes:
      - /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
    user: "0"
    command: >
      bash -c '
        if [ x${ELASTIC_PASSWORD} == x ]; then
          echo "Set the ELASTIC_PASSWORD environment variable in the .env file";
          exit 1;
        elif [ x${KIBANA_PASSWORD} == x ]; then
          echo "Set the KIBANA_PASSWORD environment variable in the .env file";
          exit 1;
        fi;
        if [ ! -f config/certs/ca.zip ]; then
          echo "Creating CA";
          bin/elasticsearch-certutil ca --silent --pem -out config/certs/ca.zip;
          unzip config/certs/ca.zip -d config/certs;
        fi;
        if [ ! -f config/certs/certs.zip ]; then
          echo "Creating certs";
          echo -ne \
          "instances:\n"\
          "  - name: es01\n"\
          "    dns:\n"\
          "      - es01\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 172.18.0.11\n"\
          "  - name: es02\n"\
          "    dns:\n"\
          "      - es02\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 172.18.0.12\n"\
          "  - name: es03\n"\
          "    dns:\n"\
          "      - es03\n"\
          "      - localhost\n"\
          "    ip:\n"\
          "      - 172.18.0.13\n"\
          > config/certs/instances.yml;
          bin/elasticsearch-certutil cert --silent --pem -out config/certs/certs.zip --in config/certs/instances.yml --ca-cert config/certs/ca/ca.crt --ca-key config/certs/ca/ca.key;
          unzip config/certs/certs.zip -d config/certs;
        fi;
        echo "Setting file permissions"
        chown -R root:root config/certs;
        find . -type d -exec chmod 750 \{\} \;;
        find . -type f -exec chmod 640 \{\} \;;
        echo "Waiting for Elasticsearch availability";
        until curl -s --cacert config/certs/ca/ca.crt https://es01:9200 | grep -q "missing authentication credentials"; do sleep 30; done;
        echo "Setting kibana_system password";
        until curl -s -X POST --cacert config/certs/ca/ca.crt -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" https://es01:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}"; do sleep 10; done;
        echo "All done!";
      '
    healthcheck:
      test: [ "CMD-SHELL", "[ -f config/certs/es01/es01.crt ]" ]
      interval: 1s
      timeout: 5s
      retries: 120
    networks:
      elastic:
        ipv4_address: 172.18.0.10
    
  es01:
    env_file:
      - .env
    depends_on:
      es-certs:
        condition: service_healthy
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    container_name: es01
    hostname: es01
    restart: always
    privileged: true
    volumes:
      - /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
      - '/opt/soft/elasticsearch/es01/plugins:/usr/share/elasticsearch/plugins'
      - '/opt/soft/elasticsearch/es01/data:/usr/share/elasticsearch/data'
      - '/opt/soft/elasticsearch/es01/logs:/usr/share/elasticsearch/logs'
    ports:
      - ${ES_PORT}:9200
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "TZ=Asia/Shanghai"
      - "http.host=0.0.0.0"
      - node.name=es01
      - cluster.name=${CLUSTER_NAME}
      # 选举主节点master资格的节点
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es02,es03
      - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
      - bootstrap.memory_lock=true
      # 默认为true，表示启用 Elasticsearch 安全功能
      - xpack.security.enabled=true
      # 用于在 Elasticsearch 用于与其他客户端通信的 HTTP 网络层上启用或禁用 TLS/SSL。默认值为false：
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es01/es01.key
      - xpack.security.http.ssl.certificate=certs/es01/es01.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      # 用于在传输网络层上启用或禁用 TLS/SSL，节点间相互通信。默认值为false
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es01/es01.key
      - xpack.security.transport.ssl.certificate=certs/es01/es01.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    deploy:
      resources:
        limits:
          memory: ${MEM_LIMIT}
    # 句柄数配置
    ulimits:
      memlock:
        soft: -1
        hard: -1
      #nofile:
       # soft: 65536
        #hard: 65536
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120
    networks:
      elastic:
        ipv4_address: 172.18.0.11

  es02:
    env_file:
      - .env
    depends_on:
      - es01
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    privileged: true
    container_name: es02
    hostname: es02
    restart: always
    volumes:
      - /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
      - '/opt/soft/elasticsearch/es02/plugins:/usr/share/elasticsearch/plugins'
      - '/opt/soft/elasticsearch/es02/data:/usr/share/elasticsearch/data'
      - '/opt/soft/elasticsearch/es02/logs:/usr/share/elasticsearch/logs'
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "TZ=Asia/Shanghai"
      - node.name=es02
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es03
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es02/es02.key
      - xpack.security.http.ssl.certificate=certs/es02/es02.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es02/es02.key
      - xpack.security.transport.ssl.certificate=certs/es02/es02.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    deploy:
      resources:
        limits:
          memory: ${MEM_LIMIT}
    # 句柄数配置
    ulimits:
      memlock:
        soft: -1
        hard: -1
     # nofile:
      #  soft: 65536
       # hard: 65536
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120
    networks:
      elastic:
        ipv4_address: 172.18.0.12
  
  es03:
    env_file:
      - .env
    depends_on:
      - es02
    image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
    container_name: es03
    hostname: es03
    restart: always
    privileged: true
    volumes:
      - /opt/soft/elasticsearch/config/certs:/usr/share/elasticsearch/config/certs
      - '/opt/soft/elasticsearch/es03/plugins:/usr/share/elasticsearch/plugins'
      - '/opt/soft/elasticsearch/es03/data:/usr/share/elasticsearch/data'
      - '/opt/soft/elasticsearch/es03/logs:/usr/share/elasticsearch/logs'
    environment:
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - "TZ=Asia/Shanghai"
      - node.name=es03
      - cluster.name=${CLUSTER_NAME}
      - cluster.initial_master_nodes=es01,es02,es03
      - discovery.seed_hosts=es01,es02
      - bootstrap.memory_lock=true
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=certs/es03/es03.key
      - xpack.security.http.ssl.certificate=certs/es03/es03.crt
      - xpack.security.http.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.key=certs/es03/es03.key
      - xpack.security.transport.ssl.certificate=certs/es03/es03.crt
      - xpack.security.transport.ssl.certificate_authorities=certs/ca/ca.crt
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.license.self_generated.type=${LICENSE}
    deploy:
      resources:
        limits:
          memory: ${MEM_LIMIT}
    # 句柄数配置
    ulimits:
      memlock:
        soft: -1
        hard: -1
      #nofile:
       # soft: 65536
       # hard: 65536
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s --cacert config/certs/ca/ca.crt https://localhost:9200 | grep -q 'missing authentication credentials'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120
    networks:
      elastic:
        ipv4_address: 172.18.0.13
      
  kibana:
    env_file:
      - .env
    depends_on:
      es01:
        condition: service_healthy
      es02:
        condition: service_healthy
      es03:
        condition: service_healthy
    image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
    container_name: kibana
    hostname: kibana
    restart: always
    privileged: true
    volumes:
      - /opt/soft/elasticsearch/config/certs:/usr/share/kibana/config/certs
      - /opt/soft/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml
      - '/opt/soft/kibana/data:/usr/share/kibana/data'
    ports:
      - ${KIBANA_PORT}:5601
    environment:
      - SERVERNAME=kibana
      - ELASTICSEARCH_HOSTS=https://es01:9200
      - ELASTICSEARCH_USERNAME=kibana_system
      - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
      - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
    deploy:
      resources:
        limits:
          memory: ${MEM_LIMIT}
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "curl -s -I http://localhost:5601 | grep -q 'HTTP/1.1 302 Found'",
        ]
      interval: 10s
      timeout: 10s
      retries: 120
    networks:
      elastic:
        ipv4_address: 172.18.0.14

# 自定义网络 elastic
networks:
  elastic:
    # 启动时不自动创建，需要提前手动创建 docker network create -d bridge elastic
    external: true
    driver: bridge

# https://www.w3cschool.cn/doc_docker_1_11/docker_1_11-engine-reference-commandline-volume_create-index.html
# 创建的 volume 将存储到 /var/lib/docker/volumes/ 路径下
#volumes:
  # CA 证书 挂载
#  certs:
#    driver: local

3. 运行compose文件

注意：

由于在国内，用到的镜像自己提前想办法下载下来，不然构建不成功的
文件中映射的目录和文件要提前创建好，并给予读写权限
上面用到了kibana.yml配置文件，主要来设置中文的，原来写在 environment底下了，但是没生效。

kibana.yml

server.host: "0.0.0.0"
server.shutdownTimeout: "5s"
# 连接es集群配置多个地址，单机一个地址
elasticsearch.hosts: ["http://10.10.1.31:9200"]
#elasticsearch.username: "test"
#elasticsearch.password: "zrb123"
# 设置kibana中文
i18n.locale: "zh-CN"

3.1 运行命令

# 1. 检查文件格式有没有问题
docker compose -f docker-compose-elastic.yml config -q
# 2. 运行
docker compose -f docker-compose-elastic.yml up -d

elastic.png

容器

可以看出es还是挺占内存的。

3.2 查看

先看看elasticsearch：https://10.10.1.31:9200，账号 elastic，密码就是配置文件中设置的。

elasticsearch

再看看kibana：http://10.10.1.31:5601，账号密码都用上面elasticsearch的。

kibana

4. 补充

其实，如果只是一主2从节点的话，官网的这个后期并不一定好维护，好多东西都写在了compose文件中，不好找了。

1. 关于证书文件，我觉得自己的证书文件可以移到自己节点目录下，虽然都是一样的，生成完以后，复制到自己节点目录下，这样目录映射也是映射自己节点目录
1. 关于一些配置，我更喜欢放到配置文件中，什么账号、密码、节点名称、集群名称，放到es的配置文件(应该是config/elasticsearch.yml)中，就像kibana.yml那个文件一样，然后做映射。这样后期维护感觉更好用。
1. elasticsearch设置时常用的一些工具
  
  bin目录

我们先来看一下它自带的账号有哪些：

自带账号

可以看到有好几个，连接kibana和logstash的都有。其中elastic应该是管理员账号，密码是自己写在配置文件中的。上面第一个服务中也给了通过接口修改其他账号密码的命令，可以参考。

添加账号：

# 添加test用户
./elasticsearch-users useradd test
# 授予超级管理员角色
./elasticsearch-users roles -a superuser test 
# 授予kibana的用户角色
./elasticsearch-users roles -a kibana_system test

添加账号

最后编辑于：2024.08.16 08:39:35

人面猴
序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
沈念sama阅读 194,242评论 5赞 459
死咒
序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
沈念sama阅读 81,769评论 2赞 371
救了他两次的神仙让他今天三更去死
文/潘晓璐我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
开封第一讲书人阅读 141,484评论 0赞 319
道士缉凶录：失踪的卖姜人
文/不坏的土叔我叫张陵，是天一观的道长。经常有香客问我，道长，这世上最难降的妖魔是什么？我笑而不...
开封第一讲书人阅读 52,133评论 1赞 263
港岛之恋（遗憾婚礼）
正文为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
茶点故事阅读 61,007评论 4赞 355
恶毒庶女顶嫁案：这布局不是一般人想出来的
文/花漫我一把揭开白布。她就那样静静地躺着，像睡着了一般。火红的嫁衣衬着肌肤如雪。梳的纹丝不乱的头发上，一...
开封第一讲书人阅读 46,080评论 1赞 272
城市分裂传说
那天，我揣着相机与录音，去河边找鬼。笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。我是一名探鬼主播，决...
沈念sama阅读 36,496评论 3赞 381
双鸳鸯连环套：你想象不到人心有多黑
文/苍兰香墨我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
开封第一讲书人阅读 35,190评论 0赞 253
万荣杀人案实录
序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
沈念sama阅读 39,464评论 1赞 290
护林员之死
正文独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋以下内容为张勋视角年9月15日...
茶点故事阅读 34,549评论 2赞 309
白月光启示录
正文我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
茶点故事阅读 36,330评论 1赞 326
活死人
序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
沈念sama阅读 32,205评论 3赞 312
日本核电站爆炸内幕
正文年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
茶点故事阅读 37,567评论 3赞 298
男人毒药：我在死后第九天来索命
文/蒙蒙一、第九天我趴在偏房一处隐蔽的房顶上张望。院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
开封第一讲书人阅读 28,889评论 0赞 17
一桩弑父案，背后竟有这般阴谋
文/苍兰香墨我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。一阵脚步声响...
开封第一讲书人阅读 30,160评论 1赞 250
情欲美人皮
我被黑心中介骗来泰国打工，没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。一个月前我还...
沈念sama阅读 41,475评论 2赞 341
代替公主和亲
正文我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
茶点故事阅读 40,650评论 2赞 335