本文 Tomcat 版本为 8.0.36。
首先要搞清楚一点,开发打的业务日志要么让开发打成 json 格式,要么使用 Logstash 进行解析,不可能有通用的产品可以完成,因为每个公司开发打印的日志的标准都不一样。这里改变的只是 tomcat 本身的日志格式。
tomcat 自身的日志有:
catalina.2018-01-05.log
host-manager.2018-01-05.log
localhost.2018-01-05.log
manager.2018-01-05.log
localhost_access_log.2018-01-05.log
catalina.out 包括 tomcat 本身和开发打印的应用日志,不做讨论。
Tomcat 日志的访问日志是在 server.xml 中配置的,配置成 json 格式也十分方便,而对于其他日志想要配置成 json 格式就十分不便了。对于这些日志,tomcat 提供了两种记录的方式,一种是默认的 java.util.logging
,另一种就是 log4j
。默认的方式无法记录成 json(至少我没找到方式),而通过 log4j 可以,因为 Logstash 提供了一个插件。
首先我们要做的就是将 tomcat 日志的记录的方式改为 log4j,修改的方式在此。这是官方文档的翻译,你也可以直接看官方文档。
需要注意的是:
- log4j 的 jar 包一定要是 1.2.X 版本的,不要下载最新版;
- 示例的
log4j.properties
中指定的日志文件是 catalina,而非 catalina.out。
所有 jar 包都可以在 https://mvnrepository.com/ 这里下载。
如果没有改变官方给出的 log4j.properties 的日志记录格式,那么配置成功后,tomcat/logs/catalina 的输出和之前的 catalina.out 一样。下一步就是使用 Logstash 提供的插件,这个怎么用呢?很简单,首先去上面给出的 maven 仓库中下载 jar 包(最好使用 1.6 版),然后放入 tomcat/lib 目录下,接着修改 log4j.properties:
log4j.rootCategory=WARN, RollingLog
log4j.appender.RollingLog=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RollingLog.Threshold=TRACE
log4j.appender.RollingLog.File=api.log
log4j.appender.RollingLog.DatePattern=.yyyy-MM-dd
log4j.appender.RollingLog.layout=net.logstash.log4j.JSONEventLayoutV1
为了可以直接效果,可以将日志级别改为 INFO
。这个时候如果直接使用的话,会有下面的报错:
Using CATALINA_BASE: /tmp/tomcat
Using CATALINA_HOME: /tmp/tomcat
Using CATALINA_TMPDIR: /tmp/tomcat/temp
Using JRE_HOME: /usr/local/jdk1.7.0_79
Using CLASSPATH: /tmp/tomcat/bin/bootstrap.jar:/tmp/tomcat/bin/tomcat-juli.jar
log4j:WARN Error during default initialization
java.lang.NoClassDefFoundError: org/apache/commons/lang/time/FastDateFormat
at net.logstash.log4j.JSONEventLayoutV0.<clinit>(JSONEventLayoutV0.java:36)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:379)
at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:336)
at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:123)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:764)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.juli.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262)
at org.apache.juli.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.juli.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025)
at org.apache.juli.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:844)
at org.apache.juli.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541)
at org.apache.juli.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292)
at org.apache.juli.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269)
at org.apache.juli.logging.LogFactory.getLog(LogFactory.java:657)
at org.apache.catalina.startup.Catalina.<clinit>(Catalina.java:820)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:379)
at org.apache.catalina.startup.Bootstrap.init(Bootstrap.java:268)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:455)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.time.FastDateFormat
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 36 more
需要下载以下包,放入 tomcat/lib 目录下:
commons-lang-2.4
json-smart-1.1.1
junit-4.8.1
然后就可以在 api.log 中看到效果了:
{"@fields":{"level":"INFO","threadName":"localhost-startStop-1","mdc":{},"file":"HostConfig.java","class":"org.apache.catalina.startup.HostConfig","line_number":"1143","method":"deployDirectory","loggerName":"org.apache.catalina.startup.HostConfig"},"@timestamp":"2018-01-05T01:36:38.263Z","@message":"Deployment of web application directory \/tmp\/tomcat\/webapps\/manager has finished in 64 ms","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"AbstractProtocol.java","class":"org.apache.coyote.AbstractProtocol","line_number":"471","method":"start","loggerName":"org.apache.coyote.http11.Http11NioProtocol"},"@timestamp":"2018-01-05T01:36:38.265Z","@message":"Starting ProtocolHandler [\"http-nio-9021\"]","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"AbstractProtocol.java","class":"org.apache.coyote.AbstractProtocol","line_number":"471","method":"start","loggerName":"org.apache.coyote.ajp.AjpNioProtocol"},"@timestamp":"2018-01-05T01:36:38.270Z","@message":"Starting ProtocolHandler [\"ajp-nio-8009\"]","@source_host":"SUC-IAS-JOB-02.UAT"}
{"@fields":{"level":"INFO","threadName":"main","mdc":{},"file":"Catalina.java","class":"org.apache.catalina.startup.Catalina","line_number":"642","method":"start","loggerName":"org.apache.catalina.startup.Catalina"},"@timestamp":"2018-01-05T01:36:38.272Z","@message":"Server startup in 695 ms","@source_host":"SUC-IAS-JOB-02.UAT"}