原始学习资料，请参考官方文档：https://www.elastic.co/guide/en/logstash/current/output-plugins.html
Logstash的output模块,相比于input模块来说是一个输出模块,output模块集成了大量的输出插件,可以输出到指定文件,也可输出到指定的网络端口,当然也可以输出数据到ES.
在这里我只介绍如何输出到ES,至于如何输出到端口和指定文件,有很多的文档资料可查找.
elasticsearch{
hosts=>["172.132.12.3:9200"]
action=>"index"
index=>"indextemplate-logstash"
#document_type=>"%{@type}"
document_id=>"ignore"
template=>"/opt/logstash-conf/es-template.json"
template_name=>"es-template.json"
template_overwrite=>true
}
action=>”index” #es要执行的动作 index, delete, create, update
l index:将logstash.时间索引到一个文档
l delete:根据id删除一个document(这个动作需要一个id)
l create:建立一个索引document，如果id存在 动作失败.
l update:根据id更新一个document，有一种特殊情况可以upsert--如果document不是已经存在的情况更新document 。参见upsert选项。
l A sprintf style string to change the action based on the content of the event. The value %{[foo]} would use the foo field for the action
document_id=>” ” 为索引提供document id ，对重写elasticsearch中相同id词目很有用
document_type=>” ”事件要被写入的document type，一般要将相似事件写入同一type，可用%{}引用事件type，默认type=log
index=>”logstash-%{+YYYY,MM.dd}” 事件要被写进的索引，可是动态的用%{foo}语句
hosts=>[“127.0.0.0”] ["127.0.0.1:9200","127.0.0.2:9200"] "https://127.0.0.1:9200"
manage_template=>true 一个默认的es mapping 模板将启用（除非设置为false 用自己的template）
template=>”” 有效的filepath 设置自己的template文件路径，不设置就用已有的
template_name=>”logstash” 在es内部模板的名字
这里需要十分注意的一个问题是,document_id尽量保证值得唯一,这样会解决你面即将面临的ES数据重复问题,切记切记!
输出到文件
input{
beats{
codec => plain{charset => "UTF-8"}
port => "5044"
type => "routing-inspection-log-tofile"
}
}
filter{
mutate{
remove_field => "@version"
remove_field => "offset"
remove_field => "input_type"
remove_field => "beat"
remove_field => "tags"
}
ruby{
code => "event.timestamp.time.localtime"
}
}
output{
if [type] != "routing-inspection-log-tofile" { //根据type类型过滤，排除不需要保存的输入
file{
path => "/home/app/logbak/%{+YYYY.MM.dd}-file.txt"
codec => line {format => "%{[collectValue]}"}//设置根据原始数据格式保存，不会带Json格式
}
}
}
开始设置的path => "/tmp/%{+YYYY.MM.dd}-%{host}-file.txt"，由于没有%{host} 这个字段，导致出现无法创建文件错误，删除这个之后恢复正常。
[2019-04-13T01:17:44,206][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<Errno::EACCES: Permission denied - /home/app/logbak/2019.04.12-{"hostname":"WIN-NN7AS1GRN2F","os":{"build":"9200.0","kernel":"6.2.9200.16384 (win8_rtm.120725-1247)","name":"Windows Server 2012 Standard","family":"windows","version":"6.2","platform":"windows"},"name":"WIN-NN7AS1GRN2F","id":"22067c43-882c-43e7-8a37-7c929f621593","architecture":"x86_64"}-file.txt>, :backtrace=>["org/jruby/RubyFile.java:370:in `initialize'", "org/jruby/RubyIO.java:871:in `new'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:280:in `open'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:132:in `multi_receive_encoded'", "org/jruby/RubyHash.java:1342:in `each'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:131:in `multi_receive_encoded'", "org/jruby/ext/thread/Mutex.java:149:in `synchronize'", "/home/app/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-file-4.0.2/lib/logstash/outputs/file.rb:130:in `multi_receive_encoded'", "/home/app/logstash/logstash-core/lib/logstash/outputs/base.rb:90:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/output_delegator_strategies/shared.rb:13:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/output_delegator.rb:47:in `multi_receive'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:420:in `output_batch'", "org/jruby/RubyHash.java:1342:in `each'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:419:in `output_batch'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:365:in `worker_loop'", "/home/app/logstash/logstash-core/lib/logstash/pipeline.rb:330:in `start_workers'"]}
[elkuser@localhost logstash]$ cd ..
修改之后恢复正常
---------------------
作者：xcl119xxcl
来源：CSDN
原文：https://blog.csdn.net/xcl119xcl/article/details/89244563
版权声明：本文为博主原创文章，转载请附上博文链接！
