Elasticsearch
Elasticsearch是一个基于lucene的全文搜索引擎
1.安装
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.0.tar.gz
tar -xzvf elasticsearch-5.3.0.tar.gz
mv elasticsearch-5.3.0 /usr/share/elasticsearch-5.3.0
2. 配置
修改配置文件
path.data: /var/lib/elasticsearch //数据存放的路径
path.logs: /var/log/elasticsearch //log的路径
node.name: wddlc //节点的名字
network.host: _网卡编号_
http.port: 9206 //监听的端口号
配置文件：/usr/share/elasticsearch-5.3.0/config/elasticsearch.yml
如果需要的话修改jvm的内存参数
-Xms1g 初始内存大小
-Xmx1g 最大内存大小
配置文件：/usr/share/elasticsearch-5.3.0/config/jvm.options
3. 启动
nohup /usr/share/elasticsearch-5.3.0/bin/elasticsearch -Epath.conf=/usr/share/elasticsearch-5.3.0/config/ > /dev/null 2>&1 &
ps -ef | grep elas //关闭
kill pid
检测启动是否成功.png
4.一个小问题
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
这是因为系统进程最大虚拟内存映射的数量不足，用下面的命令设置一下就可以了
sysctl -w vm.max_map_count=262144
5.监控插件
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
npm run start
open
[http://localhost:9100/](http://localhost:9100/)
注意：使用这个插件的时候，要打开es的跨域
http.cors.enabled: true
http.cors.allow-origin: "*"
logstash
logstash是一个开源的数据收集引擎，负责从多个数据源汇总数据、过滤、并输出到指定的输出中
1.安装
wget wget https://artifacts.elastic.co/downloads/logstash/logstash-5.3.0.tar.gz
tar -xzvf logstash-5.3.0.tar.gz
mv logstash-5.3.0 /usr/share/logstash-5.3.0
2.配置
input {
beats {
# 监听filebeat的输入
port => 5043
}
}
filter {
# 删除无用的key
if ([type] == "log") {
ruby {
code => "
event.to_hash.each { |k,v|
if (!['json', 'type'].include?(k))
event.remove(k)
end
}
"
}
# 根据ip，补充地理位置信息
geoip {
source => "[json][ip]"
}
}
}
output {
if ([type] == "log") {
# 输出到标准输出
stdout { codec => rubydebug}
# 输出到刚刚配置好的es里
elasticsearch {
hosts => ["172.16.3.2:9206"]
index => "%{[json][model]}"
}
}
}
2.启动
nohup /usr/share/logstash-5.3.0/bin/logstash -f /usr/share/logstash-5.3.0/config/first-pipeline.conf > /etc/logstash.log 2>&1 &
filebeat
1.安装
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.3.0-x86_64.rpm
sudo rpm -vi filebeat-5.3.0-x86_64.rpm
2.配置
配置文件在：/etc/filebeat/filebeat.yml
# 设置输入
input_type: log
# 表示数据的是json
json.key_under_root: true
# 监控这个路径下的log
paths:
- /var/www/prometheus-library/logs/*.log
# 设置控制台输出，方便调试，正式环境可以关掉
output.console:
pretty: true
# 输出到logstash
output.logstash:
hosts: ["172.16.3.2:5043"]
3.启动
sudo /etc/init.d/filebeat start
kibana
1.安装
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.0-linux-x86_64.tar.gz
tar -xzvf kibana-5.3.0-linux-x86_64.tar.gz
mv kibana-5.3.0-linux-x86_64 /usr/share/kibana-5.3.0-linux-x86_64
2.配置
配置文件：/usr/share/kibana-5.3.0-linux-x86_64/config/kibana.yml
# 设置es的地址
server.host: "0.0.0.0"
# es请求的路径
elasticsearch.url: "http://172.16.3.2:9206"
2.启动
nohup /usr/share/kibana-5.3.0-linux-x86_64/bin/kibana > kibana.log 2>&1 &
