前面写了一篇动态多任务，又写了一篇简单任务调度，寻思主流任务调度应该也是这样的思路，于是，下载了xxl-job的源码跑了一下，感觉确实思路差不太多。
上一篇链接
1、启动任务
1.1、修改任务状态
启动任务的接口在 com.xxl.job.admin.controller.biz.JobInfoController 中：
java
public Response<String> start(HttpServletRequest request, @RequestParam("ids[]") List<Integer> ids) {
// valid 校验，不用管
if (CollectionTool.isEmpty(ids) || ids.size()!=1) {
return Response.ofFail(I18nUtil.getString("system_please_choose") + I18nUtil.getString("system_one") + I18nUtil.getString("system_data"));
}
// invoke 获取登录信息，不用管
Response<LoginInfo> loginInfoResponse = XxlSsoHelper.loginCheckWithAttr(request);
//调用XxlJobService的start方法
return xxlJobService.start(ids.get(0), loginInfoResponse.getData());
}controller的逻辑调用了service的start方法
service的start方法在 com.xxl.job.admin.service.impl.XxlJobServiceImpl 中：
java
public Response<String> start(int id, LoginInfo loginInfo) {
// load and valid 获取任务信息
XxlJobInfo xxlJobInfo = xxlJobInfoMapper.loadById(id);
if (xxlJobInfo == null) {
return Response.ofFail(I18nUtil.getString("jobinfo_glue_jobid_invalid"));
}
// valid jobGroup permission 不用管
if (!JobGroupPermissionUtil.hasJobGroupPermission(loginInfo, xxlJobInfo.getJobGroup())) {
return Response.ofFail(I18nUtil.getString("system_permission_limit"));
}
// valid ScheduleType: can not be none 校验调度类型，不用管
ScheduleTypeEnum scheduleTypeEnum = ScheduleTypeEnum.match(xxlJobInfo.getScheduleType(), ScheduleTypeEnum.NONE);
if (ScheduleTypeEnum.NONE == scheduleTypeEnum) {
return Response.ofFail(I18nUtil.getString("schedule_type_none_limit_start"));
}
// next trigger time (5s后生效，避开预读周期) 后面的定时任务有一个5s中延迟，不用管
long nextTriggerTime = 0;
try {
// generate next trigger time 这里是生成触发时间，不用管
Date nextValidTime = scheduleTypeEnum.getScheduleType().generateNextTriggerTime(xxlJobInfo, new Date(System.currentTimeMillis() + JobScheduleHelper.PRE_READ_MS));
if (nextValidTime == null) {
return Response.ofFail ( (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_invalid")) );
}
nextTriggerTime = nextValidTime.getTime();
} catch (Exception e) {
logger.error(e.getMessage(), e);
return Response.ofFail ( (I18nUtil.getString("schedule_type")+I18nUtil.getString("system_invalid")) );
}
//修改任务为RUNNING 状态，可以被后续的线程查出来
xxlJobInfo.setTriggerStatus(TriggerStatus.RUNNING.getValue());
xxlJobInfo.setTriggerLastTime(0);
xxlJobInfo.setTriggerNextTime(nextTriggerTime);
xxlJobInfo.setUpdateTime(new Date());
xxlJobInfoMapper.update(xxlJobInfo);
// write operation log
logger.info(">>>>>>>>>>> xxl-job operation log: operator = {}, type = {}, content = {}",
loginInfo.getUserName(), "jobinfo-start", id);
return Response.ofSuccess();
}上面的代码指示将存储的任务的状态修改为RUNNING
1.2、触发任务
com.xxl.job.admin.scheduler.config.XxlJobAdminBootstrap 中有一个springboot启动时执行的方法，会开启一个线程
java
@Override
public void afterPropertiesSet() throws Exception {
// init instance
adminConfig = this;
// start
doStart();
}上面的启动方法中调用了doStart() 方法来执行实际启动逻辑
java
private void doStart() throws Exception {
// trigger-pool start
jobTriggerPoolHelper = new JobTriggerPoolHelper();
jobTriggerPoolHelper.start();
// registry monitor start
jobRegistryHelper = new JobRegistryHelper();
jobRegistryHelper.start();
// fail-alarm monitor start
jobFailAlarmMonitorHelper = new JobFailAlarmMonitorHelper();
jobFailAlarmMonitorHelper.start();
// job complate start ( depend on JobTriggerPoolHelper ) for callback and result-lost
jobCompleteHelper = new JobCompleteHelper();
jobCompleteHelper.start();
// log-report start
jobLogReportHelper = new JobLogReportHelper();
jobLogReportHelper.start();
// job-schedule start ( depend on JobTriggerPoolHelper )
jobScheduleHelper = new JobScheduleHelper();
//看这里
jobScheduleHelper.start();
logger.info(">>>>>>>>> xxl-job admin start success.");
}上面的方法最后生成了com.xxl.job.admin.scheduler.thread.JobScheduleHelper并且调用了他的start方法
java
public void start(){
// schedule thread 这里生成了一个线程，查出来任务然后刷新任务的下次执行时间，不用管
scheduleThread = new Thread(new Runnable() {
@Override
public void run() {
// align time
try {
TimeUnit.MILLISECONDS.sleep(5000 - System.currentTimeMillis()%1000 );
} catch (Throwable e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
logger.info(">>>>>>>>> init xxl-job admin scheduler success.");
// pre-read count: treadpool-size * 10 (trigger-qps: 1000ms / 100ms each trigger cost)
int preReadCount = (XxlJobAdminBootstrap.getInstance().getTriggerPoolFastMax() + XxlJobAdminBootstrap.getInstance().getTriggerPoolSlowMax()) * 10;
// do schedule
while (!scheduleThreadToStop) {
// param
long start = System.currentTimeMillis();
boolean preReadSuc = true;
// transaction start
TransactionStatus transactionStatus = null;
try {
transactionStatus = XxlJobAdminBootstrap.getInstance().getTransactionManager().getTransaction(new DefaultTransactionDefinition());
// 1、job lock
String lockedRecord = XxlJobAdminBootstrap.getInstance().getXxlJobLockMapper().scheduleLock();
long nowTime = System.currentTimeMillis();
// scan and process job
List<XxlJobInfo> scheduleList = XxlJobAdminBootstrap.getInstance().getXxlJobInfoMapper().scheduleJobQuery(nowTime + PRE_READ_MS, preReadCount);
if (CollectionTool.isNotEmpty(scheduleList)) {
// 2、push time-ring
for (XxlJobInfo jobInfo: scheduleList) {
// time-ring jump
if (nowTime > jobInfo.getTriggerNextTime() + PRE_READ_MS) {
// 2.1、trigger-expire > 5s：pass && make next-trigger-time
// 1、misfire handle
MisfireStrategyEnum misfireStrategyEnum = MisfireStrategyEnum.match(jobInfo.getMisfireStrategy(), MisfireStrategyEnum.DO_NOTHING);
misfireStrategyEnum.getMisfireHandler().handle(jobInfo.getId());
// 2、fresh next 刷新任务下次执行时间
refreshNextTriggerTime(jobInfo, new Date());
} else if (nowTime >= jobInfo.getTriggerNextTime()) {
// 2.2、trigger-expire < 5s：direct-trigger && make next-trigger-time
// 1、trigger direct
XxlJobAdminBootstrap.getInstance().getJobTriggerPoolHelper().trigger(jobInfo.getId(), TriggerTypeEnum.CRON, -1, null, null, null);
logger.debug(">>>>>>>>>>> xxl-job, schedule expire, direct trigger : jobId = " + jobInfo.getId() );
// 2、fresh next
refreshNextTriggerTime(jobInfo, new Date());
// next-trigger-time in 5s, pre-read again
if (jobInfo.getTriggerStatus()== TriggerStatus.RUNNING.getValue() && nowTime + PRE_READ_MS > jobInfo.getTriggerNextTime()) {
// 1、make ring second
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
// 2、push time ring (pre read)
pushTimeRing(ringSecond, jobInfo.getId());
logger.debug(">>>>>>>>>>> xxl-job, schedule pre-read, push trigger : jobId = " + jobInfo.getId() );
// 3、fresh next
refreshNextTriggerTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
} else {
// 2.3、trigger-pre-read：time-ring trigger && make next-trigger-time
// 1、make ring second
int ringSecond = (int)((jobInfo.getTriggerNextTime()/1000)%60);
// 2、push time ring
pushTimeRing(ringSecond, jobInfo.getId());
logger.debug(">>>>>>>>>>> xxl-job, schedule normal, push trigger : jobId = " + jobInfo.getId() );
// 3、fresh next
refreshNextTriggerTime(jobInfo, new Date(jobInfo.getTriggerNextTime()));
}
}
// 3、update trigger info
/*for (XxlJobInfo jobInfo: scheduleList) {
XxlJobAdminBootstrap.getInstance().getXxlJobInfoMapper().scheduleUpdate(jobInfo);
}*/
int batchSize = XxlJobAdminBootstrap.getInstance().getScheduleBatchSize();
List<List<XxlJobInfo>> scheduleListBatches = CollectionTool.split(scheduleList, batchSize);
for (List<XxlJobInfo> scheduleListBatch : scheduleListBatches) {
int totalAffected = XxlJobAdminBootstrap.getInstance().getXxlJobInfoMapper().scheduleBatchUpdate(scheduleListBatch);
logger.debug(">>>>>>>>>>> xxl-job, JobScheduleHelper scheduleBatchUpdate records:" + totalAffected);
}
} else {
preReadSuc = false;
}
} catch (Throwable e) {
if (!scheduleThreadToStop) {
logger.error(">>>>>>>>>>> xxl-job, JobScheduleHelper#scheduleThread error:{}", e.getMessage(), e);
}
} finally {
// transaction commit
try {
if (transactionStatus != null) {
XxlJobAdminBootstrap.getInstance().getTransactionManager().commit(transactionStatus); // avlid schedule repeat
}
} catch (Throwable e) {
logger.error(">>>>>>>>>>> xxl-job, JobScheduleHelper#scheduleThread transaction commit error:{}", e.getMessage(), e);
}
}
// transaction end
long cost = System.currentTimeMillis()-start;
// Wait seconds, align second
if (cost < 1000) { // scan-overtime, not wait
try {
// pre-read period: success > scan each second; fail > skip this period;
TimeUnit.MILLISECONDS.sleep((preReadSuc?1000:PRE_READ_MS) - System.currentTimeMillis()%1000);
} catch (Throwable e) {
if (!scheduleThreadToStop) {
logger.error(e.getMessage(), e);
}
}
}
}
logger.info(">>>>>>>>>>> xxl-job, JobScheduleHelper#scheduleThread stop");
}
});
scheduleThread.setDaemon(true);
scheduleThread.setName("xxl-job, admin JobScheduleHelper#scheduleThread");
scheduleThread.start();
// ring thread 这里又有一个线程，这个是触发任务的线程
ringThread = new Thread(new Runnable() {
@Override
public void run() {
while (!ringThreadToStop) {
// align second
try {
TimeUnit.MILLISECONDS.sleep(1000 - System.currentTimeMillis() % 1000);
} catch (Throwable e) {
if (!ringThreadToStop) {
logger.error(e.getMessage(), e);
}
}
try {
// second data
List<Integer> ringItemData = new ArrayList<>();
// collect rind data, by second
int nowSecond = Calendar.getInstance().get(Calendar.SECOND);
for (int i = 0; i <= 2; i++) { // 避免调度遗漏：处理耗时太长、跨过刻度，除当前刻度外 + 向前校验2个刻度；
List<Integer> ringItemList = ringData.remove( (nowSecond+60-i)%60 );
if (CollectionTool.isNotEmpty(ringItemList)) {
// distinct for each second
List<Integer> ringItemListDistinct = ringItemList.stream().distinct().toList(); // 避免调度重复：重复推送时间轮刻度，去重只保留一个；；
if (ringItemListDistinct.size() < ringItemList.size()) {
logger.warn(">>>>>>>>>>> xxl-job, time-ring found job repeat beat : " + nowSecond + " = " + ringItemData);
}
// collect ring item
ringItemData.addAll(ringItemListDistinct);
}
}
// ring trigger
logger.debug(">>>>>>>>>>> xxl-job, time-ring beat : " + nowSecond + " = " + ringItemData);
if (CollectionTool.isNotEmpty(ringItemData)) {
// do trigger
for (int jobId: ringItemData) {
// do trigger 触发任务
XxlJobAdminBootstrap.getInstance().getJobTriggerPoolHelper().trigger(jobId, TriggerTypeEnum.CRON, -1, null, null, null);
}
// clear
ringItemData.clear();
}
} catch (Throwable e) {
if (!ringThreadToStop) {
logger.error(">>>>>>>>>>> xxl-job, JobScheduleHelper#ringThread error:{}", e.getMessage(), e);
}
}
}
logger.info(">>>>>>>>>>> xxl-job, JobScheduleHelper#ringThread stop");
}
});
ringThread.setDaemon(true);
ringThread.setName("xxl-job, admin JobScheduleHelper#ringThread");
ringThread.start();
}这个方法中启动了两个线程，两个线程里面都是死循环，一个用来刷新任务的上次执行时间和下次执行时间，一个用来触发任务
触发任务在 com.xxl.job.admin.scheduler.thread.JobTriggerPoolHelper 的 trigger方法中
java
public void trigger(final int jobId,
final TriggerTypeEnum triggerType,
final int failRetryCount,
final String executorShardingParam,
final String executorParam,
final String addressList) {
// choose thread pool
ThreadPoolExecutor triggerPool_ = fastTriggerPool;
AtomicInteger jobTimeoutCount = jobTimeoutCountMap.get(jobId);
if (jobTimeoutCount!=null && jobTimeoutCount.get() > 10) { // job-timeout 10 times in 1 min
triggerPool_ = slowTriggerPool;
}
// trigger 由线程池去触发
triggerPool_.execute(new Runnable() {
@Override
public void run() {
long start = System.currentTimeMillis();
try {
// do trigger 触发
XxlJobAdminBootstrap.getInstance().getJobTrigger().trigger(jobId, triggerType, failRetryCount, executorShardingParam, executorParam, addressList);
} catch (Throwable e) {
logger.error(e.getMessage(), e);
} finally {
// check timeout-count-map
long minTim_now = System.currentTimeMillis()/60000;
if (minTim != minTim_now) {
minTim = minTim_now;
jobTimeoutCountMap.clear();
}
// incr timeout-count-map
long cost = System.currentTimeMillis()-start;
if (cost > 500) { // ob-timeout threshold 500ms
AtomicInteger timeoutCount = jobTimeoutCountMap.putIfAbsent(jobId, new AtomicInteger(1));
if (timeoutCount != null) {
timeoutCount.incrementAndGet();
}
}
}
}
@Override
public String toString() {
return "Job Runnable, jobId:"+jobId;
}
});
}实际触发任务在com.xxl.job.admin.scheduler.trigger.JobTrigger#trigger
java
public void trigger(int jobId,
TriggerTypeEnum triggerType,
int failRetryCount,
String executorShardingParam,
String executorParam,
String addressList) {
// load data
XxlJobInfo jobInfo = xxlJobInfoMapper.loadById(jobId);
if (jobInfo == null) {
logger.warn(">>>>>>>>>>>> trigger fail, jobId invalid，jobId={}", jobId);
return;
}
if (executorParam != null) {
jobInfo.setExecutorParam(executorParam);
}
int finalFailRetryCount = failRetryCount>=0?failRetryCount:jobInfo.getExecutorFailRetryCount();
XxlJobGroup group = xxlJobGroupMapper.load(jobInfo.getJobGroup());
// cover addressList
if (StringTool.isNotBlank(addressList)) {
group.setAddressType(1);
group.setAddressList(addressList.trim());
}
// sharding param
int[] shardingParam = null;
Date triggerTime = new Date();
if (executorShardingParam!=null){
String[] shardingArr = executorShardingParam.split("/");
if (shardingArr.length==2 && StringTool.isNumeric(shardingArr[0]) && StringTool.isNumeric(shardingArr[1])) {
shardingParam = new int[2];
shardingParam[0] = Integer.parseInt(shardingArr[0]);
shardingParam[1] = Integer.parseInt(shardingArr[1]);
}
}
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null)
&& group.getRegistryList()!=null && !group.getRegistryList().isEmpty()
&& shardingParam==null) {
for (int i = 0; i < group.getRegistryList().size(); i++) {
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, triggerTime, i, group.getRegistryList().size());
}
} else {
if (shardingParam == null) {
shardingParam = new int[]{0, 1};
}
processTrigger(group, jobInfo, finalFailRetryCount, triggerType, triggerTime, shardingParam[0], shardingParam[1]);
}
}上面的方法获取了group等信息然后交由processTrigger执行
java
private void processTrigger(XxlJobGroup group,
XxlJobInfo jobInfo,
int finalFailRetryCount,
TriggerTypeEnum triggerType,
Date triggerTime,
int index,
int total){
// param
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(jobInfo.getExecutorBlockStrategy(), ExecutorBlockStrategyEnum.SERIAL_EXECUTION); // block strategy
ExecutorRouteStrategyEnum executorRouteStrategyEnum = ExecutorRouteStrategyEnum.match(jobInfo.getExecutorRouteStrategy(), null); // route strategy
String shardingParam = (ExecutorRouteStrategyEnum.SHARDING_BROADCAST==executorRouteStrategyEnum)?String.valueOf(index).concat("/").concat(String.valueOf(total)):null;
// 1、save log-id
XxlJobLog jobLog = new XxlJobLog();
jobLog.setJobGroup(jobInfo.getJobGroup());
jobLog.setJobId(jobInfo.getId());
jobLog.setTriggerTime(triggerTime);
xxlJobLogMapper.save(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger start, jobId:{}", jobLog.getJobId());
// 2、init trigger-param
TriggerRequest triggerParam = new TriggerRequest();
triggerParam.setJobId(jobInfo.getId());
triggerParam.setExecutorHandler(jobInfo.getExecutorHandler());
triggerParam.setExecutorParams(jobInfo.getExecutorParam());
triggerParam.setExecutorBlockStrategy(jobInfo.getExecutorBlockStrategy());
triggerParam.setExecutorTimeout(jobInfo.getExecutorTimeout());
triggerParam.setLogId(jobLog.getId());
triggerParam.setLogDateTime(jobLog.getTriggerTime().getTime());
triggerParam.setGlueType(jobInfo.getGlueType());
triggerParam.setGlueSource(jobInfo.getGlueSource());
triggerParam.setGlueUpdatetime(jobInfo.getGlueUpdatetime().getTime());
triggerParam.setBroadcastIndex(index);
triggerParam.setBroadcastTotal(total);
// 3、init address
String address = null;
Response<String> routeAddressResult = null;
if (group.getRegistryList()!=null && !group.getRegistryList().isEmpty()) {
if (ExecutorRouteStrategyEnum.SHARDING_BROADCAST == executorRouteStrategyEnum) {
if (index < group.getRegistryList().size()) {
address = group.getRegistryList().get(index);
} else {
address = group.getRegistryList().get(0);
}
} else {
routeAddressResult = executorRouteStrategyEnum.getRouter().route(triggerParam, group.getRegistryList());
if (routeAddressResult.isSuccess()) {
address = routeAddressResult.getData();
}
}
} else {
routeAddressResult = Response.of(XxlJobContext.HANDLE_CODE_FAIL, I18nUtil.getString("jobconf_trigger_address_empty"));
}
// 4、trigger remote executor
Response<String> triggerResult = null;
if (address != null) {
triggerResult = doTrigger(triggerParam, address);
} else {
triggerResult = Response.of(XxlJobContext.HANDLE_CODE_FAIL, "Address Router Fail.");
}
// 5、collection trigger info
// trigger config
StringBuilder triggerMsgSb = new StringBuilder();
triggerMsgSb.append(I18nUtil.getString("jobconf_trigger_type")).append("：").append(triggerType.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_admin_adress")).append("：").append(IPTool.getIp());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regtype")).append("：")
.append( (group.getAddressType() == 0)?I18nUtil.getString("jobgroup_field_addressType_0"):I18nUtil.getString("jobgroup_field_addressType_1") );
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobconf_trigger_exe_regaddress")).append("：").append(group.getRegistryList());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorRouteStrategy")).append("：").append(executorRouteStrategyEnum.getTitle());
if (shardingParam != null) {
triggerMsgSb.append("(").append(shardingParam).append(")");
}
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorBlockStrategy")).append("：").append(blockStrategy.getTitle());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_timeout")).append("：").append(jobInfo.getExecutorTimeout());
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorFailRetryCount")).append("：").append(finalFailRetryCount);
// trigger data
triggerMsgSb.append("<br><br><span style=\"color:#00c0ef;\" > >>>>>>>>>>>").append(I18nUtil.getString("jobconf_trigger_run")).append("<<<<<<<<<<< </span><br>");
triggerMsgSb.append("<br>").append(I18nUtil.getString("joblog_field_executorAddress")).append("：");
if (StringTool.isNotBlank(address)) {
triggerMsgSb.append(address);
} else if (routeAddressResult!=null && !routeAddressResult.isSuccess() && routeAddressResult.getMsg()!=null) {
triggerMsgSb.append("address route fail, ").append(routeAddressResult.getMsg());
} else {
triggerMsgSb.append("address route fail.");
}
if (StringTool.isNotBlank(jobInfo.getExecutorHandler())) {
triggerMsgSb.append("<br>").append("JobHandler").append("：").append(jobInfo.getExecutorHandler());
}
triggerMsgSb.append("<br>").append(I18nUtil.getString("jobinfo_field_executorparam")).append("：").append(jobInfo.getExecutorParam());
triggerMsgSb.append("<br>").append(I18nUtil.getString("joblog_field_triggerMsg")).append("：");
if (triggerResult.isSuccess()) {
triggerMsgSb.append("success");
} else if (triggerResult.getMsg()!=null) {
triggerMsgSb.append("error, ").append(triggerResult.getMsg());
} else {
triggerMsgSb.append("fail");
}
// 6、save log trigger-info
jobLog.setExecutorAddress(address);
jobLog.setExecutorHandler(jobInfo.getExecutorHandler());
jobLog.setExecutorParam(jobInfo.getExecutorParam());
jobLog.setExecutorShardingParam(shardingParam);
jobLog.setExecutorFailRetryCount(finalFailRetryCount);
//jobLog.setTriggerTime();
jobLog.setTriggerCode(triggerResult.getCode());
jobLog.setTriggerMsg(triggerMsgSb.toString());
xxlJobLogMapper.updateTriggerInfo(jobLog);
logger.debug(">>>>>>>>>>> xxl-job trigger end, jobId:{}", jobLog.getJobId());
}上面的代码主要是组装请求参数 TriggerRequest，获取实际请求地址 address，就是从注册上来的服务中选择一个作为实际执行任务的机器，地址的uri为/run，比如http://127.0.0.1:9999/run
将请求参数和地址交由doTrigger
java
private Response<String> doTrigger(TriggerRequest triggerParam, String address){
try {
// build client 获取调用客户端，是一个http工具类
ExecutorBiz executorBiz = XxlJobAdminBootstrap.getExecutorBiz(address);
// invoke 发送请求并获取响应结果
Response<String> runResult = executorBiz.run(triggerParam);
// build result
StringBuffer runResultSB = new StringBuffer(I18nUtil.getString("jobconf_trigger_run") + "：");
runResultSB.append("<br>address：").append(address);
runResultSB.append("<br>code：").append(runResult.getCode());
runResultSB.append("<br>msg：").append(runResult.getMsg());
// return
runResult.setMsg(runResultSB.toString());
return runResult;
} catch (Exception e) {
logger.error(">>>>>>>>>>> xxl-job trigger error, please check if the executor[{}] is running.", address, e);
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, ThrowableTool.toString(e));
}
}上面的代码根据请求地址组装了一个http请求工具，然后请求 /run 并获得任务执行的响应
2、任务执行
2.1 将请求参数放入执行队列
查看客户端/run的执行，在这里
com.xxl.job.core.server.EmbedServer.EmbedHttpServerHandler#dispatchRequest
java
private Object dispatchRequest(HttpMethod httpMethod, String uri, String requestData, String accessTokenReq) {
// valid
if (HttpMethod.POST != httpMethod) {
return Response.ofFail("invalid request, HttpMethod not support.");
}
if (uri == null || uri.trim().isEmpty()) {
return Response.ofFail( "invalid request, uri-mapping empty.");
}
if (accessToken != null
&& !accessToken.trim().isEmpty()
&& !accessToken.equals(accessTokenReq)) {
return Response.ofFail("The access token is wrong.");
}
// services mapping
try {
switch (uri) {
case "/beat":
return executorBiz.beat();
case "/idleBeat":
IdleBeatRequest idleBeatParam = GsonTool.fromJson(requestData, IdleBeatRequest.class);
return executorBiz.idleBeat(idleBeatParam);
case "/run":
TriggerRequest triggerParam = GsonTool.fromJson(requestData, TriggerRequest.class);
return executorBiz.run(triggerParam);
case "/kill":
KillRequest killParam = GsonTool.fromJson(requestData, KillRequest.class);
return executorBiz.kill(killParam);
case "/log":
LogRequest logParam = GsonTool.fromJson(requestData, LogRequest.class);
return executorBiz.log(logParam);
default:
return Response.ofFail( "invalid request, uri-mapping(" + uri + ") not found.");
}
} catch (Throwable e) {
logger.error(e.getMessage(), e);
return Response.ofFail("request error:" + ThrowableTool.toString(e));
}
}上面方法中根据实际的url分配执行方法/run的方法是：com.xxl.job.core.openapi.impl.ExecutorBizImpl#run
java
@Override
public Response<String> run(TriggerRequest triggerRequest) {
// load old：jobHandler + jobThread
JobThread jobThread = XxlJobExecutor.loadJobThread(triggerRequest.getJobId());
IJobHandler jobHandler = jobThread!=null?jobThread.getHandler():null;
String removeOldReason = null;
// valid：jobHandler + jobThread
GlueTypeEnum glueTypeEnum = GlueTypeEnum.match(triggerRequest.getGlueType());
if (GlueTypeEnum.BEAN == glueTypeEnum) {
// new jobhandler
IJobHandler newJobHandler = XxlJobExecutor.loadJobHandler(triggerRequest.getExecutorHandler());
// valid old jobThread
if (jobThread!=null && jobHandler != newJobHandler) {
// change handler, need kill old thread
removeOldReason = "change jobhandler or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
jobHandler = newJobHandler;
if (jobHandler == null) {
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, "job handler [" + triggerRequest.getExecutorHandler() + "] not found.");
}
}
} else if (GlueTypeEnum.GLUE_GROOVY == glueTypeEnum) {
// valid old jobThread
if (jobThread != null &&
!(jobThread.getHandler() instanceof GlueJobHandler
&& ((GlueJobHandler) jobThread.getHandler()).getGlueUpdatetime()== triggerRequest.getGlueUpdatetime() )) {
// change handler or gluesource updated, need kill old thread
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
try {
IJobHandler originJobHandler = GlueFactory.getInstance().loadNewInstance(triggerRequest.getGlueSource());
jobHandler = new GlueJobHandler(originJobHandler, triggerRequest.getGlueUpdatetime());
} catch (Exception e) {
logger.error(e.getMessage(), e);
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, e.getMessage());
}
}
} else if (glueTypeEnum!=null && glueTypeEnum.isScript()) {
// valid old jobThread
if (jobThread != null &&
!(jobThread.getHandler() instanceof ScriptJobHandler
&& ((ScriptJobHandler) jobThread.getHandler()).getGlueUpdatetime()== triggerRequest.getGlueUpdatetime() )) {
// change script or gluesource updated, need kill old thread
removeOldReason = "change job source or glue type, and terminate the old job thread.";
jobThread = null;
jobHandler = null;
}
// valid handler
if (jobHandler == null) {
jobHandler = new ScriptJobHandler(triggerRequest.getJobId(), triggerRequest.getGlueUpdatetime(), triggerRequest.getGlueSource(), GlueTypeEnum.match(triggerRequest.getGlueType()));
}
} else {
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, "glueType[" + triggerRequest.getGlueType() + "] is not valid.");
}
// executor block strategy
if (jobThread != null) {
ExecutorBlockStrategyEnum blockStrategy = ExecutorBlockStrategyEnum.match(triggerRequest.getExecutorBlockStrategy(), null);
if (ExecutorBlockStrategyEnum.DISCARD_LATER == blockStrategy) {
// discard when running
if (jobThread.isRunningOrHasQueue()) {
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, "block strategy effect："+ExecutorBlockStrategyEnum.DISCARD_LATER.getTitle());
}
} else if (ExecutorBlockStrategyEnum.COVER_EARLY == blockStrategy) {
// kill running jobThread
if (jobThread.isRunningOrHasQueue()) {
removeOldReason = "block strategy effect：" + ExecutorBlockStrategyEnum.COVER_EARLY.getTitle();
jobThread = null;
}
} else {
// just queue trigger
}
}
// replace thread (new or exists invalid)
if (jobThread == null) {
jobThread = XxlJobExecutor.registJobThread(triggerRequest.getJobId(), jobHandler, removeOldReason);
}
// push data to queue
return jobThread.pushTriggerQueue(triggerRequest);
}上面的方法获取了jobThread，线程
java
public Response<String> pushTriggerQueue(TriggerRequest triggerParam) {
// avoid repeat
if (!triggerLogIdSet.add(triggerParam.getLogId())) {
logger.info(">>>>>>>>>>> repeate trigger job, logId:{}", triggerParam.getLogId());
return Response.of(XxlJobContext.HANDLE_CODE_FAIL, "repeate trigger job, logId:" + triggerParam.getLogId());
}
// push trigger queue
triggerQueue.add(triggerParam);
return Response.ofSuccess();
}上面的方法将触发任务的请求参数放入了队列
2.2 从任务队列获取任务触发
com.xxl.job.core.thread.JobThread#run
java
@Override
public void run() {
// init 初始化任务执行器
try {
handler.init();
} catch (Throwable e) {
logger.error(e.getMessage(), e);
}
// execute
while(!toStop){
running = false;
idleTimes++;
TriggerRequest triggerParam = null;
try {
// to check toStop signal, we need cycle, so we cannot use queue.take(), instead of poll(timeout) 从执行队列里面取出任务
triggerParam = triggerQueue.poll(3L, TimeUnit.SECONDS);
if (triggerParam!=null) {
running = true;
idleTimes = 0;
triggerLogIdSet.remove(triggerParam.getLogId());
// log filename, like "logPath/yyyy-MM-dd/9999.log"
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(triggerParam.getLogDateTime()), triggerParam.getLogId());
XxlJobContext xxlJobContext = new XxlJobContext(
triggerParam.getJobId(),
triggerParam.getExecutorParams(),
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
logFileName,
triggerParam.getBroadcastIndex(),
triggerParam.getBroadcastTotal());
// init job context
XxlJobContext.setXxlJobContext(xxlJobContext);
// execute
XxlJobHelper.log("<br>----------- xxl-job job execute start -----------<br>----------- Param:" + xxlJobContext.getJobParam());
if (triggerParam.getExecutorTimeout() > 0) {
// limit timeout
Thread futureThread = null;
try {//异步执行任务
FutureTask<Boolean> futureTask = new FutureTask<Boolean>(new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
// init job context
XxlJobContext.setXxlJobContext(xxlJobContext);
//任务执行器执行任务
handler.execute();
return true;
}
});
futureThread = new Thread(futureTask);
futureThread.setName("xxl-job, JobThread-future-"+jobId+"-"+System.currentTimeMillis());
futureThread.start();
Boolean tempResult = futureTask.get(triggerParam.getExecutorTimeout(), TimeUnit.SECONDS);
} catch (TimeoutException e) {
XxlJobHelper.log("<br>----------- xxl-job job execute timeout");
XxlJobHelper.log(e);
// handle result
XxlJobHelper.handleTimeout("job execute timeout ");
} finally {
futureThread.interrupt();
}
} else {
// just execute
handler.execute();
}
// valid execute handle data
if (XxlJobContext.getXxlJobContext().getHandleCode() <= 0) {
XxlJobHelper.handleFail("job handle result lost.");
} else {
String tempHandleMsg = XxlJobContext.getXxlJobContext().getHandleMsg();
tempHandleMsg = (tempHandleMsg!=null&&tempHandleMsg.length()>50000)
?tempHandleMsg.substring(0, 50000).concat("...")
:tempHandleMsg;
XxlJobContext.getXxlJobContext().setHandleMsg(tempHandleMsg);
}
XxlJobHelper.log("<br>----------- xxl-job job execute end(finish) -----------<br>----------- Result: handleCode="
+ XxlJobContext.getXxlJobContext().getHandleCode()
+ ", handleMsg = "
+ XxlJobContext.getXxlJobContext().getHandleMsg()
);
} else {
if (idleTimes > 30) {
if(triggerQueue.isEmpty()) { // avoid concurrent trigger causes jobId-lost
XxlJobExecutor.removeJobThread(jobId, "excutor idle times over limit.");
}
}
}
} catch (Throwable e) {
if (toStop) {
XxlJobHelper.log("<br>----------- JobThread toStop, stopReason:" + stopReason);
}
// handle result
StringWriter stringWriter = new StringWriter();
e.printStackTrace(new PrintWriter(stringWriter));
String errorMsg = stringWriter.toString();
XxlJobHelper.handleFail(errorMsg);
XxlJobHelper.log("<br>----------- JobThread Exception:" + errorMsg + "<br>----------- xxl-job job execute end(error) -----------");
} finally {
if(triggerParam != null) {
// callback handler info
if (!toStop) {
// common
TriggerCallbackThread.pushCallBack(new CallbackRequest(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.getXxlJobContext().getHandleCode(),
XxlJobContext.getXxlJobContext().getHandleMsg() )
);
} else {
// is killed
TriggerCallbackThread.pushCallBack(new CallbackRequest(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.HANDLE_CODE_FAIL,
stopReason + " [job running, killed]" )
);
}
}
}
}
// callback trigger request in queue
while(triggerQueue !=null && !triggerQueue.isEmpty()){
TriggerRequest triggerParam = triggerQueue.poll();
if (triggerParam!=null) {
// is killed
TriggerCallbackThread.pushCallBack(new CallbackRequest(
triggerParam.getLogId(),
triggerParam.getLogDateTime(),
XxlJobContext.HANDLE_CODE_FAIL,
stopReason + " [job not executed, in the job queue, killed.]")
);
}
}
// destroy
try {
handler.destroy();
} catch (Throwable e) {
logger.error(e.getMessage(), e);
}
logger.info(">>>>>>>>>>> xxl-job JobThread stoped, hashCode:{}", Thread.currentThread());
}上面的代码调用 IJobHandler 执行任务
比如我用的是Bean类型的任务，就用反射去执行：com.xxl.job.core.handler.impl.MethodJobHandler#execute
method:public void com.xxl.job.executor.jobhandler.SampleXxlJob.demoJobHandler() throws java.lang.Exception
java
@Override
public void execute() throws Exception {
Class<?>[] paramTypes = method.getParameterTypes();
if (paramTypes.length > 0) {
method.invoke(target, new Object[paramTypes.length]); // method-param can not be primitive-types
} else {
method.invoke(target);
}
}反射就执行到了目标方法：com.xxl.job.executor.jobhandler.SampleXxlJob#demoJobHandler
java
@XxlJob("demoJobHandler")
public void demoJobHandler() throws Exception {
XxlJobHelper.log("XXL-JOB, Hello World.");
for (int i = 0; i < 5; i++) {
XxlJobHelper.log("beat at:" + i);
TimeUnit.SECONDS.sleep(2);
}
// default success
}3、总结
实际上xxl-job的任务调度和我之前写的那一篇还是有区别的。
我的思路是：springboot生成对应的定时任务 --> springboot的定时任务触发相应的任务 --> 利用反射执行对应的代码。
xxl-job的逻辑是: 项目启动后利用死循环线程去刷新任务的当前执行时间和下一次执行时间 --> 再利用另一个死循环线程查出来应该要执行的任务去触发任务 --> 利用反射执行对应的代码。
相同点：都是根据全方法名利用反射去执行对应的代码。
不同点：任务触发不同，我利用了springboot的定时任务，xxl-job采用了死循环线程。