- Logstash安装：
1.下载安装包：wget https://artifacts.elastic.co/downloads/logstash/logstash-6.6.2.tar.gz
2.下载mysql的连接库jar包(版本库)：wget http://central.maven.org/maven2/mysql/mysql-connector-java/6.0.6/mysql-connector-java-6.0.6.jar
2.解压安装包：tar -xvf logstash-6.6.2.tar.gz
4.安装jdbc、elasticsearch插件:
logstash插件安装命令
cd logstash-6.6.2/bin
./logstash-plugin install logstash-input-jdbc
./logstash-plugin install logstash-output-elasticsearch
- 创建配置文件
在config目录下，创建配置文件（logstash-mysql-es.conf）：
input {
# stdin { }
jdbc {
jdbc_connection_string => "jdbc:mysql://localhost:30012/data_acquisition?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull&allowMultiQueries=true"
jdbc_user => "root"
jdbc_password => "123456"
jdbc_driver_library => "/path/mysql-connector-java-6.0.6.jar" #全路径
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_paging_enabled => "true"
jdbc_page_size => "50000"
statement => "select * from result_comment where create_time >= '2019-03-20'"
#schedule => "* * * * *"
}
}
output {
# 这里输出调试，正式运行时可以注释掉
stdout {
codec => json_lines
}
elasticsearch {
hosts => "127.0.0.1:9200"
index => "wl_travel_analyze"
document_type => "travel"
document_id => "%{id}"
#ssl => true
#ssl_certificate_verification => true
#truststore => "/opt/elk/search-guard-tlstool/search-guard-certificates/truststore.jks"
#truststore_password => "286a7b7a8970af4e8467"
user => "admin"
password => "admin"
}
}
注意添加ElasticSearch用户名和密码
- 启动改脚本
nohup ./bin/logstash -f config/logstash-mysql-es.conf > logs/logstash.out &
- 查看日志
tail -f logs/logstash.out
借鉴文章如下:
Elasticsearch - Logstash实现mysql同步数据到elasticsearch
