linux安装Elasticsearch5.6.X详细步骤
第一步:环境
linux 系统 Java 1.8.0_151 elasticsearch-5.6.8
第二步:下载安装
-
1,下载JDK,安装JDK环境
JDK的下载可以去官网上直接下载:http://www.oracle.com/technetwork/java/javase/downloads/index.html
-
直接将下载好的jdk-8u151-linux-x64.rpm 安装包 ;上传到自己创建好的JAVA文件下;cd 命令进入到JAVA文件下使用rpm 命令进行安装 rpm -ivh jdk-8u131-linux-x64.rpm 安装完成后执行 java -version 命令查看安装是否成功。
配置环境变量,执行命令 vi /etc/profile;然后进入编辑模式,在文件的最后添加下面的配置,如图
JAVA_HOME=/usr/javajdk1.8.0_151
JRE_HOME=/usr/java/jdk1.8.0_151/jre
CLASSPATH=JRE_HOME/lib:JAVA_HOME/bin:PATH
执行命令 source /etc/profile 使用环境变量生效
验证环境变量是否生效:echo $JAVA_HOME
- 2,下载安装ES
去官网直接下载,本人使用的是 5.6.8版本;因为版本的不同安装head插件的时候安装步骤不同;好像是从5.0 以后的版本安装head 插件的步骤就不一样了;下面会详细介绍。
ES官网:https://www.elastic.co/下载ES安装包elasticsearch-5.6.8.zip 在usr 目录下创建了es文件夹,将安装包上传到里面,解压 unzip elasticsearch-5.6.8.zip cd 进入 elasticsearch-5.6.8 文件里面。
创建ES用户和组(创建elsearch用户组及elsearch用户),因为使用root用户执行ES程序,将会出现错误;所以这里需要创建单独的用户去执行ES 文件;命令如下:
命令一:groupadd elsearch
命令二:useradd elsearch -g elsearch
命令三:chown -R elsearch:elsearch-
创建ES数据文件和日志文件,直接在root用户根目录一下创建就可以了
执行命令:mkdir /data
命令二:chown -R elsearch:elsearch /data/
命令三:su - elsearch 切换用户
命令四:mkdir -p es/data
命令五:mkdir -p es/logs 修改ES配置文件,使用cd命令进入到config 文件下,执行 vi elasticsearch.yml 命令,如图所示,本人没有配置集群,只是简单的配置了一下;详细说明可以参考官网;
执行ES文件,进入到bin 目录下执行 ./elasticsearch 命令就可以了,执行 ./elasticesrarch -d 是后台运行。
如果没有什么问题话,就可以安全生成了;然后执行curl 'http://自己配置的IP地址:9200/' 命令,就出现下面的结果
后台启动es命令:./bin/elasticsearch -d
可以看到es在后台启动了
第四步:问题
- 1,[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] 意思是说你的进程不够用了
解决方案: 切到root 用户:进入到security目录下的limits.conf;执行命令 vim /etc/security/limits.conf 在文件的末尾添加下面的参数值:
- soft nofile 65536
- hard nofile 131072
- soft nproc 2048
hard nproc 4096
前面的*符号必须带上,然后重新启动就可以了。执行完成后可以使用命令 ulimit -n 查看进程数 。
root@data-hadoop-112-15:~ $ ssh 192.168.129.94
Last login: Mon Dec 9 20:23:29 2019 from data-hadoop-112-15.bjrs.zybang.com
[root@owl-nginx-129-94 ~]# ulimit -n
655350
用户退出后重新登录生效
- 2,[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] 需要修改系统变量的最大值
解决方案:切换到root用户修改配置sysctl.conf 增加配置值: vm.max_map_count=655360
执行命令 sysctl -p 这样就可以了,然后重新启动ES服务 就可以了。
第五步:安装head插件
-
1,下载head安装包,下载地址:https://github.com/mobz/elasticsearch-head/archive/master.zip 这是接从git 上下载下来 ,然后上传到虚拟机上的;由于head 插件不能放在elasticsearch-5.6.8 文件夹里,head 插件需要单独放,单独去执行; 所 以在elasticsearch-5.6.8 同级目录下解压了 head 插件;解压出来的文件名字,如图:
-
2,安装head 插件,需要node.js 的支持,所以,下面先安装一node.js
- 执行命令一:curl -sL https://rpm.nodesource.com/setup_8.x | bash -
- 命令二:yum install -y nodejs-----用brew来安装
➜ softeare brew install -y node.js
Updating Homebrew...
==> Installing dependencies for node: icu4c
==> Installing node dependency: icu4c
==> Downloading https://homebrew.bintray.com/bottles/icu4c-61.1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
执行完成后,可以使用命令 node -v 验证是否安装成功,同时npm 也安装成功了;执行命令 npm -v 也是可以验证的。
- 3,安装grunt ,由于head 插件的执行文件是有grunt 命令来执行的,所以这个命令必须安装
- npm install grunt --save-dev
---》brew ➜ softeare brew install grunt --save-dev
Updating Homebrew...
Warning: grunt-cli: this formula has no --save-dev option so it will be ignored!
==> Downloading https://homebrew.bintray.com/bottles/grunt-cli-1.2.0.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring grunt-cli-1.2.0.high_sierra.bottle.tar.gz
🍺 /usr/local/Cellar/grunt-cli/1.2.0: 154 files, 251.3KB
如果出现以下错误:
- npm install grunt --save-dev
-
则安装缺少的包:
参考:https://www.jianshu.com/p/a73d95030edb
npm install grunt-contrib-clean
npm install grunt-contrib-concat
npm install grunt-contrib-watch
npm install grunt-contrib-connect (安装了太久)
npm install grunt-contrib-copy
npm install grunt-contrib-jasmine-
最后一个比较难安装,要联网到一个国外网站下载一个东西,而且貌似 有个引擎要求的node比我这里的node版本高,会有一个warning,安装到Receiving...就杀了进程,杀掉之后再运行grunt可以看到一切都正常了,有条件的可以等待Receiving...后面的内容。
-
命令二:npm install
修改配置文件,cd 进入elasticsearch-head-master 文件夹下,执行命令vim Gruntfile.js文件:增加hostname属性,设置为*;如图:
路径:elasticsearch-head-master/_site/app.js
修改 localhost:9200 为实际 Elasticsearch 的地址
修改前:this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";
修改后:
this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://172.16.X.X:9200";
-
- 修改 vim _site/app.js 文件,修改head的连接地址,如图所示:
- 最后一个命令: grunt server & 执行完成后就OK了。
- 涉及到的问题,在网页上无法正常访问;查看防火墙是否关闭
执行命令service iptables status 查看状态 ;直接将防火墙关闭就好了 执行命令service iptables stop。
- 最后执行的结果是这样的,我没有配置集群: 注意下面使用的端口号,不在是9200 了 而是head 插件中的 9100 了。
- 看到上面的出现的健康值了吗,说明的连接还是有问题的,解决方案是修改 cd 命令进入到elasticsearch-5.6.3 /config 文件中 vi elasticsearch.yml
文件下添加 :
http.cors.enabled: true
http.cors.allow-origin: "*"
然后重新执行ES ./elasticsearch 成功起来就可以了,执行结果就是这样的
- 未写入数据前:
ES集群安装遇到的问题
报错:
root@hue:~/elasticsearch-5.6.8/bin# [2018-04-03T16:28:51,641][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [master1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.8.jar:5.6.8]
Caused by: java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:106) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) ~[elasticsearch-5.6.8.jar:5.6.8]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.8.jar:5.6.8]
... 6 more
切换到es用户
注意将data目录的用户和用户组设置为es
Master启动日志如下:
es@hue:/usr/local/es/elasticsearch-5.6.8/bin$ ./elasticsearch &
[1] 25086
es@hue:/usr/local/es/elasticsearch-5.6.8/bin$ [2018-04-03T17:44:54,347][INFO ][o.e.n.Node ] [master1] initializing ...
[2018-04-03T17:44:54,552][INFO ][o.e.e.NodeEnvironment ] [master1] using [1] data paths, mounts [[/ (/dev/vda1)]], net usable_space [28.2gb], net total_space [39.2gb], spins? [possibly], types [ext4]
[2018-04-03T17:44:54,553][INFO ][o.e.e.NodeEnvironment ] [master1] heap size [1.9gb], compressed ordinary object pointers [true]
[2018-04-03T17:44:54,554][INFO ][o.e.n.Node ] [master1] node name [master1], node ID [T73gSpfdTS-34qIYEwFPYA]
[2018-04-03T17:44:54,554][INFO ][o.e.n.Node ] [master1] version[5.6.8], pid[25086], build[688ecce/2018-02-16T16:46:30.010Z], OS[Linux/4.4.0-62-generic/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12]
[2018-04-03T17:44:54,555][INFO ][o.e.n.Node ] [master1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/es/elasticsearch-5.6.8]
[2018-04-03T17:44:55,482][INFO ][o.e.p.PluginsService ] [master1] loaded module [aggs-matrix-stats]
[2018-04-03T17:44:55,482][INFO ][o.e.p.PluginsService ] [master1] loaded module [ingest-common]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [lang-expression]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [lang-groovy]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [lang-mustache]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [lang-painless]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [parent-join]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [percolator]
[2018-04-03T17:44:55,483][INFO ][o.e.p.PluginsService ] [master1] loaded module [reindex]
[2018-04-03T17:44:55,484][INFO ][o.e.p.PluginsService ] [master1] loaded module [transport-netty3]
[2018-04-03T17:44:55,484][INFO ][o.e.p.PluginsService ] [master1] loaded module [transport-netty4]
[2018-04-03T17:44:55,484][INFO ][o.e.p.PluginsService ] [master1] no plugins loaded
[2018-04-03T17:44:56,800][INFO ][o.e.d.DiscoveryModule ] [master1] using discovery type [zen]
[2018-04-03T17:44:57,287][INFO ][o.e.n.Node ] [master1] initialized
[2018-04-03T17:44:57,288][INFO ][o.e.n.Node ] [master1] starting ...
[2018-04-03T17:44:57,592][INFO ][o.e.t.TransportService ] [master1] publish_address {10.1.72.136:9300}, bound_addresses {10.1.72.136:9300}
[2018-04-03T17:44:57,605][INFO ][o.e.b.BootstrapChecks ] [master1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-04-03T17:45:00,686][INFO ][o.e.c.s.ClusterService ] [master1] new_master {master1}{T73gSpfdTS-34qIYEwFPYA}{ofDnSPHlSfe8kLZZx4NCaQ}{10.1.72.136}{10.1.72.136:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2018-04-03T17:45:00,745][INFO ][o.e.h.n.Netty4HttpServerTransport] [master1] publish_address {10.1.72.136:9200}, bound_addresses {10.1.72.136:9200}
[2018-04-03T17:45:00,746][INFO ][o.e.n.Node ] [master1] started
[2018-04-03T17:45:00,765][INFO ][o.e.g.GatewayService ] [master1] recovered [0] indices into cluster_state
访问master:
注意要启动slave!!!
可以看到slave节点也启动了
附es配置文件:
- master
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
cluster.name: bingo-es
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
node.name: master1
node.master: true
node.data: true
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
path.data: /usr/local/data/es/data
# Path to log files:
#
#path.logs: /path/to/logs
path.logs: /usr/local/data/es/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
network.host: host1
#
# Set a custom port for HTTP:
http.port: 9200
discovery.zen.ping.unicast.hosts: ["host1","host2"]
discovery.zen.minimum_master_nodes: 2
#head
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
#xpack.security.enabled: false
#xpack.reporting.enabled: false
#xpack.monitoring.enabled: false
#xpack.monitoring.exporters.auth.username: "elastic"
#xpack.monitoring.exporters.auth.password: "changeme"
#xpack.security.audit.enabled: true
#action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*
#elasticsearch.username: "elastic"
#elasticsearch.password: "changeme"
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
- data节点
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
cluster.name: bingo-es
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
node.name: data1
#node.master: false
node.master: true
node.data: true
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
#path.data: /path/to/data
path.data: /usr/local/data/es/data
# Path to log files:
#
#path.logs: /path/to/logs
path.logs: /usr/local/data/es/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
network.host: host2
# Set a custom port for HTTP:
#
#http.port: 9200
http.port: 9200
discovery.zen.ping.unicast.hosts: ["host1","host2"]
discovery.zen.minimum_master_nodes: 2
#discovery.zen.ping.multicast.enabled: false
#
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
#
#xpack.security.enabled: false
#xpack.reporting.enabled: false
#xpack.monitoring.enabled: false
#action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*
#xpack.monitoring.elasticsearch.username: "elastic"
#xpack.monitoring.elasticsearch.password: "changeme"
#elsticsearch.username: "elastic"
#elasticsearch.password: "changeme"
#For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true