公司需要将应用日志上报到kafka,以供分析与查看。
结合logback可以完成此功能,大致配置如下:
xml
<appender name="KafkaAppender" class="com.github.danielwegener.logback.kafka.KafkaAppender">
<encoder class="com.github.danielwegener.logback.kafka.encoding.LayoutKafkaMessageEncoder">
<layout class="net.logstash.logback.layout.LogstashLayout" >
<includeContext>true</includeContext>
<includeCallerData>true</includeCallerData>
<fieldNames class="net.logstash.logback.fieldnames.ShortenedFieldNames"/>
</layout>
<charset>UTF-8</charset>
</encoder>
<!--kafka topic 需要与配置文件里面的topic一致 否则kafka会沉默并鄙视你-->
<topic>${topic}</topic>
<keyingStrategy class="com.github.danielwegener.logback.kafka.keying.HostNameKeyingStrategy" />
<deliveryStrategy class="com.github.danielwegener.logback.kafka.delivery.AsynchronousDeliveryStrategy" />
<producerConfig>bootstrap.servers=${BOOT_SERVER}</producerConfig>
</appender>
上述配置有个问题:
当kafka服务异常时,会阻塞业务系统请求。
此时,引入AsyncAppender可以处理上面的问题,让日志系统异步执行,当kafka出现问题时,不阻塞业务系统,大致配置如下:
xml
<appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="KafkaAppender"/>
<queueSize>1000</queueSize>
<discardingThreshold>0</discardingThreshold>
<!--这里一定要配置-->
<neverBlock>true</neverBlock>
</appender>
实际使用时,本地启动,一切正常,当将jar放到服务器上时,日志系统无法启动,最终导致服务启动失败,部分堆栈如下:
txt
Caused by: java.lang.IllegalStateException: Logback configuration error detected:
ERROR in com.github.danielwegener.logback.kafka.KafkaAppender[KafkaAppender] - Appender [KafkaAppender] failed to append. org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
ERROR in com.github.danielwegener.logback.kafka.KafkaAppender[KafkaAppender] - Appender [KafkaAppender] failed to append. org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
ERROR in com.github.danielwegener.logback.kafka.KafkaAppender[KafkaAppender] - Appender [KafkaAppender] failed to append. org.apache.kafka.common.errors.InterruptException: java.lang.InterruptedException
at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadConfiguration(LogbackLoggingSystem.java:169)
at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithConventions(AbstractLoggingSystem.java:80)
at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:60)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.initialize(LogbackLoggingSystem.java:118)
at org.springframework.boot.context.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:313)
at org.springframework.boot.context.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:288)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingApplicationListener.java:246)
at org.springframework.boot.context.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.java:223)
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:127)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:76)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:53)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:345)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at cn.com.bdo.gateway.BizBdoGatewayApplication.main(BizBdoGatewayApplication.java:16)
经过不断尝试,日志系统添加以下配置即可处理上述问题:
xml
<logger name="org.springframework.boot.context.logging" level="DEBUG"/>
由于只有服务器上有问题,debug比较困难,所以没有继续分析原因。
特此记录。