问题情境:
项目中存在web页面点击一键导出,导出所有数据对应的pdf文件,由于有些pdf文件是实时生成的,之前最简答的写法for循环处理速度太慢,超过了nginx配置的最大响应时间了,且对用户交互体验上很不友好,所以进一步进行优化。
解决方案:
1.对大量的数据进行分段处理;(向上取余)
代码示例:
public List<List<FrApplyGuaranteeInfo>> segmentList(List<FrApplyGuaranteeInfo> list ,int segmentSize) {
int limit = (list.size() + segmentSize - 1) / segmentSize;
List<List<FrApplyGuaranteeInfo>> segmentList =
Stream.iterate(0, n -> n + 1).limit(limit).parallel().map(a -> list.stream().
skip(a * segmentSize).limit(segmentSize).parallel().collect(Collectors.toList())).collect(Collectors.toList());
return segmentList;
}
2.通过threadPoolTaskExcetor.submit()方法进行多线程任务处理
示例代码:
第一步分段处理后的list
lists.forEach({list->{
threadPoolExecutor.submit(new xxxxTask());
})
第二步:具体的task需要实现callalbe
示例代码如下:
@Slf4j
public class DownloadLetterZipTask implements Callable {
// 需要预审的记录
private List<FrApplyGuaranteeInfo> dataList;
private ZipOutputStream zipOutputStream;
private CountDownLatch countDownLatch;
private IFrApplyGuaranteeInfoService applyGuaranteeInfoService;
public DownloadLetterZipTask(List<FrApplyGuaranteeInfo> dataList, ZipOutputStream zipOutputStream, CountDownLatch countDownLatch, IFrApplyGuaranteeInfoService applyGuaranteeInfoService)
{ this.dataList = dataList;
this.zipOutputStream = zipOutputStream;
this.countDownLatch = countDownLatch;
this.applyGuaranteeInfoService = applyGuaranteeInfoService; }
@Override
public Object call() throws Exception {
log.info("start--------------" + Thread.currentThread().getName());
try {
for (FrApplyGuaranteeInfo applyGuaranteeInfo : dataList) {
if (!StringUtils.isEmpty(applyGuaranteeInfo.getAcceptNo())) {
try {
// 这一步为具体的将文件转为字节数组输出流
ByteArrayOutputStream waterOutputStream = applyGuaranteeInfoService.getLetterPdfByteStream(applyGuaranteeInfo.getAcceptNo());
byte[] xmpMetadata = waterOutputStream.toByteArray();
synchronized (zipOutputStream) {
zipOutputStream.putNextEntry(new ZipEntry(applyGuaranteeInfo.getGenerateeLetterNo() + ".pdf"));
zipOutputStream.write(xmpMetadata);
zipOutputStream.closeEntry();
}
} catch (Exception e) {
log.error("[一键导出]---acceptNo为{}生成pdf失败", applyGuaranteeInfo.getAcceptNo());
}
}
}
} catch (Exception e) {
e.printStackTrace();
log.error("[xxxx]-批量下载zip失败");
} finally {
countDownLatch.countDown();
}
return null;
}
}