一、前期准备工作
环境准备一台机器，ip分别为：192.168.2.128
安装jdk1.8
去官网获取最新Elasticsearch包
curl -O -L https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.5.1.zip
unzip elasticsearch-5.5.1.zip
cd elasticsearch-5.5.1
./bin/elasticsearch 启动
出现三个错误
2017-08-16T01:27:32,668][INFO ][o.e.b.BootstrapChecks ] [node-1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: [2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
[2]max number of threads [1024] for user [biligeci] is too low, increase to at least [2048]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
[2017-08-16T01:27:32,726][INFO ][o.e.n.Node ] [node-1] stopping ...
[2017-08-16T01:27:35,707][INFO ][o.e.n.Node ] [node-1] stopped
[2017-08-16T01:27:35,707][INFO ][o.e.n.Node ] [node-1] closing ...
[2017-08-16T01:27:35,727][INFO ][o.e.n.Node ] [node-1] closed
问题一：
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
原因：无法创建本地文件问题,用户最大可创建文件数太小
解决方法：
vim /etc/security/limits.conf
添加如下内容:
elastic soft nofile 65536
elastic hard nofile 65536
elastic soft nproc 2048
elastic hard nproc 2048
elastic soft memlock unlimited
elastic hard memlock unlimited
重新登录才可生效
问题二：
[2]: max number of threads [1024] for user [biligeci] is too low, increase to at least [2048]
原因：无法创建本地线程问题,用户最大可创建线程数太小
解决方法：
vim /etc/security/limits.d/90-nproc.conf
找到如下内容：
soft nproc 1024修改为soft nproc 2048
问题三：
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
原因：最大虚拟内存太小
解决方法：切换到root用户下，修改配置文件sysctl.conf
vim /etc/sysctl.conf
添加下面配置：
vm.max_map_count=655360
并执行命令：
sysctl -p
另外还需注意一个问题（在日志发现如下内容，这样也会导致启动失败，这一问题困扰了很久）
[2017-06-14T19:19:01,641][INFO][o.e.b.BootstrapChecks] [elk-1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2017-06-14T19:19:01,658][ERROR][o.e.b.Bootstrap] [elk-1] node validation exception
[1] bootstrap checks failed
[1]:system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk
解决：修改配置文件，在配置文件添加一项参数
vim /etc/elasticsearch/elasticsearch.yml
bootstrap.system_call_filter: false
==========================================================
二、logstash
去官网获取最新logstash包
curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-5.5.2.zip
unzip logstash-5.5.2.zip
cd logstash-5.5.2
新建启动配置文件
vi logstash-simple.conf
文件内容：
input {
kafka {
bootstrap_servers => "198.218.1.24:9092,198.218.1.26:9092"
topics => ["logStash"]
}
}
filter {
json {
source => "message"
}
}
output {
stdout {
codec => rubydebug { metadata => true }
}
elasticsearch {
hosts => ["198.218.35.24:9200","198.218.35.25:9200","198.218.35.26:9200"]
index => "system_log-%{+YYYY.MM.dd}"
codec => rubydebug { metadata => true}
}
}
上面输入使用kafka 也可换成redis或者file 都是可以的。
如：
input {
file {
path => "/var/log/bootstrap.log"
type => "system"
start_position => "beginning"
}
file {
path => "/home/elasticsearch/testlog/logstash-*.log"
type => "program"
start_position => "beginning"
}
}
output {
elasticsearch {
hosts =>["198.218.35.24:9200","198.218.35.25:9200","198.218.35.26:9200"]
index => "system-%{+YYYY.MM.dd}"
}
elasticsearch {
hosts =>["198.218.35.24:9200","198.218.35.25:9200","198.218.35.26:9200"]
index => "program-%{+YYYY.MM.dd}"
}
}
输出日志到es里
启动：./bin/logstash -f logstash-simple.conf
