写在最前
如果这个项目让你有所收获,记得 Star 关注哦,这对我是非常不错的鼓励与支持。
源码地址(后端):gitee.com/csps/mingyu...
源码地址(前端):gitee.com/csps/mingyu...
分布式任务调度框架
Java 常见的分布式任务调度框架主要是 Quartz 与 Xxl-Job,还有一股新势力 PowerJob。这里简单介绍一下他们三个,并横向对比一下。
Quartz
- 特点:Quartz是一个功能强大的任务调度框架,适用于定时任务、计划任务等。它支持多种触发器类型,如SimpleTrigger和CronTrigger,以及任务状态管理、监听器、集群部署等功能。
- 适用场景:Quartz适合一般的任务调度需求,特别是那些需要高度可配置性和定制性的场景。它在单机和小规模分布式环境中表现出色。
XXL-Job
- 特点:XXL-Job是一个专注于分布式任务调度的平台,提供任务管理中心、执行器节点、任务依赖、分布式执行等特性。它具有简单的Web界面和管理功能,适合处理大规模分布式任务。
- 适用场景:XXL-Job适合大规模分布式任务的定时调度和管理,如数据同步、报表生成等。它具有高可用性、伸缩性和任务管理功能。
PowerJob
- 特点:PowerJob(Hydrogen Job)是一个开源的分布式任务调度框架,旨在提供高性能、高可用性、易用性和可扩展性。它支持分布式任务的执行、任务定义、任务依赖、分布式锁等功能。
- 适用场景:PowerJob适用于需要高性能和可扩展性的大规模分布式任务。它提供了任务调度中心、执行器节点、任务依赖等特性,可用于各种数据处理、ETL和任务调度场景。
如何抉择
适合的才是最好的~
- 如果你需要一个通用的任务调度框架,Quartz 是一个不错的选择,特别是在单机或小规模分布式环境中。
- 如果你需要在大规模分布式环境中管理任务调度,XXL-Job 或 PowerJob 是更合适的,它们提供了分布式任务执行和管理的特性,适合处理大量任务。
- XXL-Job 更加关注易用性和任务管理,PowerJob 注重性能和扩展性,具有更高的性能和弹性。
Quartz 比较老了,对分布式的支持不太友好,对于微服务项目来说不太建议。PowerJob 没用过,基于目前的场景来说暂时用不到,后续可以考虑。本项目先使用 XXL-Job 作为分布式管理任务调度框架。
mingyue-common-job
新建模块引入依赖
核心依赖 xxl-job-core
xml
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<!-- 服务发现组件 -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-commons</artifactId>
</dependency>
<!-- xxl-job-core -->
<dependency>
<groupId>com.xuxueli</groupId>
<artifactId>xxl-job-core</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-core</artifactId>
</dependency>
</dependencies>
mingyue-xxl-job-admin
新建模块引入依赖
核心依赖 mingyue-common-job
xml
<dependencies>
<!-- SpringCloud Alibaba Nacos -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
<!-- SpringCloud Alibaba Nacos Config -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
<!-- SpringCloud Alibaba Sentinel -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
</dependency>
<!-- starter-test:junit + spring-test + mockito -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<!-- freemarker-starter -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-freemarker</artifactId>
</dependency>
<!-- mail-starter -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
</dependency>
<!-- starter-actuator -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- mybatis-starter:mybatis + mybatis-spring + hikari(default) -->
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
</dependency>
<!-- mysql -->
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
</dependency>
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-web</artifactId>
</dependency>
<!-- xxl-job-core -->
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-job</artifactId>
</dependency>
</dependencies>
复制代码
Gitee: gitee.com/xuxueli0323...
选择需要的分支,复制 xxl-job-admin
代码到 mingyue-xxl-job-admin
项目
mingyue-xxl-job-admin Nacos 配置
yaml
# 任务调度配置
# server 配置
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://mingyue-mysql:3306/mingyue_job?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true
username: root
password: mingyue
hikari:
auto-commit: true
connection-test-query: SELECT 1
connection-timeout: 10000
idle-timeout: 30000
max-lifetime: 900000
maximum-pool-size: 30
minimum-idle: 10
pool-name: HikariCP
validation-timeout: 1000
mail:
from: xxx@qq.com
host: smtp.qq.com
username: xxx@qq.com
password: xxx
port: 25
properties:
mail:
smtp:
auth: true
socketFactory:
class: javax.net.ssl.SSLSocketFactory
starttls:
enable: true
required: true
# mybatis 配置
mybatis:
mapper-locations: classpath:/mybatis-mapper/*Mapper.xml
# Actuator 监控端点的配置项
management:
health:
mail:
enabled: false
endpoints:
web:
exposure:
include: '*'
endpoint:
health:
show-details: ALWAYS
logfile:
external-file: ./logs/${spring.application.name}/console.log
# xxljob系统配置
xxl:
job:
# 鉴权token
accessToken: xxl-job
# 国际化
i18n: zh_CN
# 日志清理
logretentiondays: 30
triggerpool:
fast:
max: 200
slow:
max: 100
启动项目
用户名:admin
密码:123456
登陆成功后,可以看到如下页面,并且后台启动日志没有异常信息打印即可!
csharp
2023-10-26 09:49:10 [xxl-job, admin JobScheduleHelper#scheduleThread] INFO c.x.j.a.c.thread.JobScheduleHelper
- >>>>>>>>> init xxl-job admin scheduler success.
2023-10-26 09:49:10 [xxl-job, admin JobFailMonitorHelper] INFO com.zaxxer.hikari.HikariDataSource
- HikariCP - Start completed.
mingyue-job
新建模块引入依赖
xml
<dependencies>
<!-- SpringCloud Alibaba Nacos -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
</dependency>
<!-- SpringCloud Alibaba Nacos Config -->
<dependency>
<groupId>com.alibaba.cloud</groupId>
<artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId>
</dependency>
<!-- web容器 -->
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-web</artifactId>
</dependency>
<!-- 接口文档 -->
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-doc</artifactId>
</dependency>
<!-- 认证工具 -->
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-security</artifactId>
</dependency>
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-mybatis</artifactId>
</dependency>
<dependency>
<groupId>com.csp.mingyue</groupId>
<artifactId>mingyue-common-job</artifactId>
</dependency>
</dependencies>
XxlJob开发示例
less
@Slf4j
@Service
public class SampleService {
/**
* 1、简单任务示例(Bean模式)
*/
@XxlJob("demoJobHandler")
public void demoJobHandler() throws Exception {
XxlJobHelper.log("XXL-JOB, Hello World.");
for (int i = 0; i < 5; i++) {
XxlJobHelper.log("beat at:" + i);
}
// default success
}
}
mingyue-job Nacos 配置
yaml
# 定时服务配置
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://mingyue-mysql:3306/mingyue_job?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=true&serverTimezone=GMT%2B8&rewriteBatchedStatements=true&allowPublicKeyRetrieval=true
username: root
password: mingyue
xxl:
job:
# 执行器开关
enabled: true
# 调度中心地址:如调度中心集群部署存在多个地址则用逗号分隔。
# admin-addresses: http://localhost:9900
# 调度中心应用名 通过服务名连接调度中心(启用admin-appname会导致admin-addresses不生效)
admin-appName: mingyue-xxl-job-admin
# 执行器通讯TOKEN:非空时启用
access-token: xxl-job
# 执行器配置
executor:
# 执行器AppName:执行器心跳注册分组依据;为空则关闭自动注册
appName: ${spring.application.name}-executor
# 29203 端口 随着主应用端口飘逸 避免集群冲突
port: 2${server.port}
# 执行器注册:默认IP:PORT
address:
# 执行器IP:默认自动获取IP
ip:
# 执行器运行日志文件存储磁盘路径
logpath: ./logs/${spring.application.name}/xxl-job
# 执行器日志文件保存天数:大于3生效
logretentiondays: 30
启动测试
选择 测试任务1执行一次
执行成功后查看执行日志
ini
2023-10-26 15:25:59 [com.xxl.job.core.thread.JobThread#run]-[133]-[xxl-job, JobThread-1-1698305159363]
----------- xxl-job job execute start -----------
----------- Param:
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[36]-[xxl-job, JobThread-1-1698305159363] XXL-JOB, Hello World.
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[39]-[xxl-job, JobThread-1-1698305159363] beat at:0
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[39]-[xxl-job, JobThread-1-1698305159363] beat at:1
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[39]-[xxl-job, JobThread-1-1698305159363] beat at:2
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[39]-[xxl-job, JobThread-1-1698305159363] beat at:3
2023-10-26 15:25:59 [com.csp.mingyue.job.service.SampleService#demoJobHandler]-[39]-[xxl-job, JobThread-1-1698305159363] beat at:4
2023-10-26 15:25:59 [com.xxl.job.core.thread.JobThread#run]-[179]-[xxl-job, JobThread-1-1698305159363]
----------- xxl-job job execute end(finish) -----------
----------- Result: handleCode=200, handleMsg = null
2023-10-26 15:25:59 [com.xxl.job.core.thread.TriggerCallbackThread#callbackLog]-[197]-[xxl-job, executor TriggerCallbackThread]
----------- xxl-job job callback finish.
[Load Log Finish]