一、graylog介绍
GrayLog是一个轻量型的分布式日志管理平台,一个开源的日志聚合、分析、审计、展示和预警工具。 在功能上来说,和ELK类似,但又比ELK要简单轻量许多。 依靠着更加简洁,高效,部署使用简单的优势很快受到许多公司的青睐。 GrayLog包含Elasticsearch、MongoDb 和Graylog三个模块
二、graylog环境搭建
这里采用docker-compose来搭建graylog测试环境,具体配置如下
yaml
# docker-compose配置参考:https://docs.graylog.org/en/3.3/pages/installation/docker.html
version: '3'
services:
mongo:
image: mongo:3
container_name: graylog_demo_mongo
restart: unless-stopped
# volumes:
# - ./graylog/mongo_data:/data/db
networks:
- graylog
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch-oss:6.8.10
container_name: graylog_demo_elasticsearch
restart: unless-stopped
# volumes:
# - ./graylog/es_data:/usr/share/elasticsearch/data
environment:
- http.host=0.0.0.0
- transport.host=localhost
- network.host=0.0.0.0
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
deploy:
resources:
limits:
memory: 1g
networks:
- graylog
graylog:
image: graylog/graylog:3.3
container_name: graylog_demo_graylog
restart: unless-stopped
# volumes:
# - ./graylog/graylog_data_journal:/usr/share/graylog/data/journal
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper # CHANGE ME (must be at least 16 characters)!
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918 # Password: admin
- GRAYLOG_HTTP_EXTERNAL_URI=http://127.0.0.1:9000/ # TODO 这里填写自己的ip地址
ports:
# Graylog web interface and REST API
- 9000:9000
# Syslog TCP
- 1514:1514
# Syslog UDP
- 1514:1514/udp
# GELF TCP
- 12201:12201
# GELF UDP
- 12201:12201/udp
networks:
- graylog
depends_on:
- mongo
- elasticsearch
# 网桥graylog -> 方便相互通讯
networks:
graylog:
driver: bridge
启动服务
css
docker-compose -f docker-compose-graylog.yml -p graylog_demo up -d
访问地址:http://ip地址:9001
默认登录账号密码:admin/admin
2.1、新建input http input
测试是否正常发送
makefile
curl -XPOST http://localhost:12201/gelf -p0 -d '{"message":"hello这是一条消息", "host":"127.0.0.1", "facility":"test", "topic": "meme"}'
2.2、创建 gref udp input
由于http发送效率比较低,所以后面springboot会采用UDP方法来采集日志
三、代码工程
pom.xml
xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>springboot-demo</artifactId>
<groupId>com.et</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>grayLog</artifactId>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<dependency>
<groupId>biz.paluch.logging</groupId>
<artifactId>logstash-gelf</artifactId>
<version>1.11.1</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
</dependencies>
</project>
HelloWorldController.java
typescript
package com.et.graylog.controller;
import com.et.graylog.service.DemoService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import java.util.HashMap;
import java.util.Map;
@Slf4j
@Controller
public class HelloWorldController {
@Autowired
DemoService demoService;
@RequestMapping("/hello")
@ResponseBody
public Map<String, Object> showHelloWorld(){
Map<String, Object> map = new HashMap<>();
map.put("msg", "HelloWorld");
log.info("my name is HelloWorldController");
demoService.hello();
return map;
}
@GetMapping("log")
public String log(){
log.error("error");
log.warn("warn");
log.info("info");
log.debug("debug");
int i = 1/0;
return "success";
}
}
DemoApplication.java
typescript
package com.et.graylog;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
}
application.yaml
yaml
server:
port: 8088
groylog:
ip: 127.0.0.1
port: 12201
maximumMessageSize: 8192
logback.xml
xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="GELF" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
<host>udp:127.0.0.1</host><!--graylog 服务器ip-->
<port>12201</port><!--graylog udp端口-->
<version>1.1</version>
<facility>springboot-with-graylog</facility>
<extractStackTrace>true</extractStackTrace>
<filterStackTrace>true</filterStackTrace>
<mdcProfiling>true</mdcProfiling>
<timestampPattern>yyyy-MM-dd HH:mm:ss,SSSS</timestampPattern>
<maximumMessageSize>8192</maximumMessageSize>
<!-- This are fields using MDC -->
<mdcFields>mdcField1,mdcField2</mdcFields>
<dynamicMdcFields>mdc.*,(mdc|MDC)fields</dynamicMdcFields>
<includeFullMdc>true</includeFullMdc>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="GELF"/>
<appender-ref ref="CONSOLE"/>
</root>
</configuration>
四、测试
启动服务
css
"C:\Program Files\Java\jdk1.8.0_251\bin\java.exe" -Dvisualvm.id=505582826333400 "-javaagent:D:\huiyida\develop\IntelliJ IDEA Community Edition 2022.1.3\lib\idea_rt.jar=57262:D:\huiyida\develop\IntelliJ IDEA Community Edition 2022.1.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_251\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_251\jre\lib\rt.jar;D:\IdeaProjects\ETFramework\grayLog\target\classes;D:\jar_repository\org\springframework\boot\spring-boot-starter-web\2.2.5.RELEASE\spring-boot-starter-web-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter\2.2.5.RELEASE\spring-boot-starter-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-logging\2.2.5.RELEASE\spring-boot-starter-logging-2.2.5.RELEASE.jar;D:\jar_repository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\jar_repository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\jar_repository\org\slf4j\slf4j-api\1.7.30\slf4j-api-1.7.30.jar;D:\jar_repository\org\apache\logging\log4j\log4j-to-slf4j\2.12.1\log4j-to-slf4j-2.12.1.jar;D:\jar_repository\org\apache\logging\log4j\log4j-api\2.12.1\log4j-api-2.12.1.jar;D:\jar_repository\org\slf4j\jul-to-slf4j\1.7.30\jul-to-slf4j-1.7.30.jar;D:\jar_repository\jakarta\annotation\jakarta.annotation-api\1.3.5\jakarta.annotation-api-1.3.5.jar;D:\jar_repository\org\springframework\spring-core\5.2.4.RELEASE\spring-core-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-jcl\5.2.4.RELEASE\spring-jcl-5.2.4.RELEASE.jar;D:\jar_repository\org\yaml\snakeyaml\1.25\snakeyaml-1.25.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-json\2.2.5.RELEASE\spring-boot-starter-json-2.2.5.RELEASE.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-databind\2.10.2\jackson-databind-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-annotations\2.10.2\jackson-annotations-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\core\jackson-core\2.10.2\jackson-core-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\datatype\jackson-datatype-jdk8\2.10.2\jackson-datatype-jdk8-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.10.2\jackson-datatype-jsr310-2.10.2.jar;D:\jar_repository\com\fasterxml\jackson\module\jackson-module-parameter-names\2.10.2\jackson-module-parameter-names-2.10.2.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-tomcat\2.2.5.RELEASE\spring-boot-starter-tomcat-2.2.5.RELEASE.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-core\9.0.31\tomcat-embed-core-9.0.31.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-el\9.0.31\tomcat-embed-el-9.0.31.jar;D:\jar_repository\org\apache\tomcat\embed\tomcat-embed-websocket\9.0.31\tomcat-embed-websocket-9.0.31.jar;D:\jar_repository\org\springframework\boot\spring-boot-starter-validation\2.2.5.RELEASE\spring-boot-starter-validation-2.2.5.RELEASE.jar;D:\jar_repository\jakarta\validation\jakarta.validation-api\2.0.2\jakarta.validation-api-2.0.2.jar;D:\jar_repository\org\hibernate\validator\hibernate-validator\6.0.18.Final\hibernate-validator-6.0.18.Final.jar;D:\jar_repository\org\jboss\logging\jboss-logging\3.4.1.Final\jboss-logging-3.4.1.Final.jar;D:\jar_repository\com\fasterxml\classmate\1.5.1\classmate-1.5.1.jar;D:\jar_repository\org\springframework\spring-web\5.2.4.RELEASE\spring-web-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-beans\5.2.4.RELEASE\spring-beans-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-webmvc\5.2.4.RELEASE\spring-webmvc-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-aop\5.2.4.RELEASE\spring-aop-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-context\5.2.4.RELEASE\spring-context-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\spring-expression\5.2.4.RELEASE\spring-expression-5.2.4.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot-autoconfigure\2.2.5.RELEASE\spring-boot-autoconfigure-2.2.5.RELEASE.jar;D:\jar_repository\org\springframework\boot\spring-boot\2.2.5.RELEASE\spring-boot-2.2.5.RELEASE.jar;D:\jar_repository\biz\paluch\logging\logstash-gelf\1.11.1\logstash-gelf-1.11.1.jar;D:\jar_repository\org\projectlombok\lombok\1.18.12\lombok-1.18.12.jar" com.et.graylog.DemoApplication
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.2.5.RELEASE)
10:07:22.906 INFO com.et.graylog.DemoApplication - Starting DemoApplication on BLHHUAPC with PID 30200 (D:\IdeaProjects\ETFramework\grayLog\target\classes started by Dell in D:\IdeaProjects\ETFramework)
10:07:22.918 INFO com.et.graylog.DemoApplication - No active profile set, falling back to default profiles: default
10:07:23.801 INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat initialized with port(s): 8088 (http)
10:07:23.811 INFO o.a.coyote.http11.Http11NioProtocol - Initializing ProtocolHandler ["http-nio-8088"]
10:07:23.813 INFO o.a.catalina.core.StandardService - Starting service [Tomcat]
10:07:23.813 INFO o.a.catalina.core.StandardEngine - Starting Servlet engine: [Apache Tomcat/9.0.31]
10:07:23.885 INFO o.a.c.c.C.[Tomcat].[localhost].[/] - Initializing Spring embedded WebApplicationContext
10:07:23.887 INFO o.s.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 930 ms
10:07:24.025 INFO o.s.s.c.ThreadPoolTaskExecutor - Initializing ExecutorService 'applicationTaskExecutor'
10:07:24.124 INFO o.a.coyote.http11.Http11NioProtocol - Starting ProtocolHandler ["http-nio-8088"]
10:07:24.143 INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat started on port(s): 8088 (http) with context path ''
10:07:24.147 INFO com.et.graylog.DemoApplication - Started DemoApplication in 1.564 seconds (JVM running for 2.09)
登录graylog,查看日志是否已经发送过去