EasyExcel 写Excel超过限制自动切换sheet
查询数据库超多表超百W数据量写Excel,这里有一个小常识,Excel 2007后版本,单个工作表的记录条数最多为1048576行,最大列数为16384列。如果多于单个sheet后自动切换新sheet
步骤描述
- 查数据:查询待写入Excel的数据,并将待写的数据放在一个List对象中
- 实例化sheet:判断总记录数,超过单sheet最大限制1048576行,计算sheet个数,否则写在一个sheet中
- 写Excel: EasyExcel.write方法写数据到Excel中
- 关闭: 写完后调用写相关操作 excelWriter.finsh
参考代码信息
1. 写入Excel中的对象
package com.test.zhanglu
import com.alibaba.excel.annotation.ExcelProperty;
import com.alibaba.excel.annotation.write.style.ColumnWidth;
import com.alibaba.excel.annotation.write.style.HeadRowHeight;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
@Data
@AllArgsConstructor
@NoArgsConstructor
@ColumnWidth(30)
@HeadRowHeight(30)
public class ExcelBean {
@ExcelProperty("测点类型")
private String deviceNameStr;
@ExcelProperty("测点编码")
private String deviceNo;
@ExcelProperty("实时值")
private double value;
@ExcelProperty("安装地址")
private String deviceAddr;
@ExcelProperty("实时时间")
private String realTime;
2 Excel 写入方法逻辑
package com.test.zhanglu
import com.alibaba.excel.EasyExcel;
import com.alibaba.excel.ExcelWriter;
import com.alibaba.excel.util.DateUtils;
import com.alibaba.excel.write.metadata.WriteSheet;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.test.context.junit4.SpringRunner;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
@SpringBootTest
@RunWith(SpringRunner.class)
public class WriteExcelTest {
@Autowired
private MongoTemplate mongoTemplate;
@Test
public void getList() {
String path = "D:\\001document\\04\\01\\09\\数据样本集\\" + "data_" + DateUtils.format(new Date(), "yyyyMMdd") + ".xlsx";
//单表最多存储数
int sheetMaxNum = 1048576;
//跑去表头,单页数据最大行数为限制减一条
int pageSize = 1048575;
//获取查询的数据库表名
List<String> tableNames = getTableName(2023, 11);
//创建ExceWriter对象
ExcelWriter excelWriter = EasyExcel.write(path).build();
//所有查询出来的数据放在一个对象中
List<ExcelBean> result = new ArrayList<>(10000000);
for (String tableName : tableNames) {
List<ExcelBean> list = new ArrayList<>();
boolean isExists = mongoTemplate.collectionExists(tableName);
if (!isExists) {
continue;
}
Query query = new Query();
//mongoTemplate 模糊查询
query.addCriteria(Criteria.where("deviceAddr").regex("zhanglu"));
query.addCriteria(Criteria.where("deviceNo").regex("A"));
query.fields().include("deviceNo").include("deviceNameStr").include("deviceAddr").include("realTime").include("value").exclude("_id");
list = mongoTemplate.find(query, ExcelBean.class, tableName);
System.out.println(tableName + ":" + list.size());
result.addAll(list);
}
int sheetCount = 0;
int total = result.size();
System.out.println("===数据总量==" + total);
//根据总数计算需要多少个sheet页
if (total > sheetMaxNum) {
sheetCount = total % pageSize == 0 ? total / pageSize : (total / pageSize) + 1;
} else {
sheetCount = 1;
}
int currentIndex = 0;
//按sheet页写数据
for (int num = 1; num <= sheetCount; num++) {
String stName = "测试数据-" + num;
//实例化sheet对象,写个sheetNo,sheetName。这里敲黑板,需要写sheetNo,不然只有一个sheet没切换
WriteSheet writeSheet = EasyExcel.writerSheet(num,stName).head(ExcelBean.class).build();
System.out.println(stName);
//计算当前sheet从第几条记录开始写。和分页是一个道理
int fromIndex = pageSize * num;
if (fromIndex >= total) {
fromIndex = total;
}
//调用写的方法
excelWriter.write(result.subList(currentIndex, fromIndex), writeSheet);
//赋值给下一个sheet页的开始索引
currentIndex = fromIndex;
}
//关闭写操作
excelWriter.finish();
}
/**以下是查询表名的方法,查询多个数据表,表名规则是日期为后缀**/
public static List<String> getTableName(int year, int month) {
List<String> dates = new ArrayList<>();
Calendar calendar = Calendar.getInstance();
calendar.set(year, month - 1, 1); // 设置为当前月的第一天
int maxDay = calendar.getActualMaximum(Calendar.DAY_OF_MONTH); // 获取当前月的最大天数
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyyMMdd");
for (int i = 1; i <= maxDay; i++) {
calendar.set(Calendar.DAY_OF_MONTH, i);
String date = dateFormat.format(calendar.getTime());
String talbeName = "zhanglu.test_" + date + "_his_value";
dates.add(talbeName);
}
return dates;
}
知识小总结
1. sheet没切换,每次都写一个sheet中,超限报错
原因: WriteSheet writeSheet = EasyExcel.writerSheet(num,stName).head(ExcelBean.class).build();
这行代码中 EasyExcel.writerSheet(num,stName).只给了一个sheetName参数,没给sheetNo。 如下所示:EasyExcel.writerSheet(stName)
2. Bean中常用注解说明
可参考官网:
https://easyexcel.opensource.alibaba.com/docs/current/api/write#注解
注解 | 说明 | 常用属性 |
---|---|---|
ExcelProperty | 用于匹配excel和实体类的匹配(可为表头显示的表名) | value,order,index,converter |
ExcelIgnore | 默认所有字段都会和excel去匹配,加了这个注解会忽略该字段 | - |
DateTimeFormat | 日期转换,用String去接收excel日期格式的数据会调用这个注解 | value,use1904windowing |
NumberFormat | 数字转换,用String去接收excel数字格式的数据会调用这个注解。 | value,roundingMode |