导出100w条数据(分100次,每次模拟返回1w条数据,10线程并发3次,20线程并发2次)内存控制在1G。
代码:
easy-poi:
Date start = new Date();
ExportParams params = new ExportParams("大数据测试", "测试");
Random random=new Random();
Workbook workbook = null;
List<MsgClient> list=new ArrayList();
for (int i = 0; i < 100; i++) {
int s=10000;
for (int j = 0; j < s; j++) { //一百万数据量
MsgClient client = new MsgClient();
client.setBirthday(new Date());
client.setClientName("小明" +(i*10000+j));
client.setClientPhone("18797" + (i*10000+j));
client.setCreateBy("JueYue");
client.setId("1" + (i*10000+j));
client.setRemark("测试" + (i*10000+j));
MsgClientGroup group = new MsgClientGroup();
group.setGroupName("测试" + (i*10000+j));
list.add(client);
}
workbook=ExcelExportUtil.exportBigExcel(params, MsgClient.class, list);
list.clear();
}
ExcelExportUtil.closeExportBigExcel();
File savefile = new File("D:/excel/");
if (!savefile.exists()) {
savefile.mkdirs();
}
FileOutputStream fos = new FileOutputStream("D:/excel/excel.bigDataExport"+random.nextInt()+".xlsx");
workbook.write(fos);
fos.close();
System.out.println("bigExcelExport-获取所有list完成时间:"+(new Date().getTime() - start.getTime()));
easy-excel:
Date start = new Date();
Random random=new Random();
FileOutputStream fos = new FileOutputStream("D:/excel/alibaba.bigDataExport"+random.nextInt()+".xlsx");
ExcelWriter writer = new ExcelWriterBuilder()
.autoCloseStream(true)
.excelType(ExcelTypeEnum.XLSX)
.file(fos)
.head(MsgClientAlibaba.class)
.build();
WriteSheet writeSheet = new WriteSheet();
writeSheet.setSheetName("target");
List<MsgClientAlibaba> list=new ArrayList();
for (int i = 0; i < 100; i++) {
int s=10000;
for (int j = 0; j < s; j++) { //一百万数据量
MsgClientAlibaba client = new MsgClientAlibaba();
client.setBirthday(new Date());
client.setClientName("小明" +(i*10000+j));
client.setClientPhone("18797" + (i*10000+j));
client.setCreateBy("JueYue");
client.setId("1" + (i*10000+j));
client.setRemark("测试" + (i*10000+j));
MsgClientGroup group = new MsgClientGroup();
group.setGroupName("测试" + (i*10000+j));
// client.setGroup(group);
list.add(client);
}
writer.write(list, writeSheet);
list.clear();
}
writer.finish();
fos.close();
System.out.println("doExport-获取所有list完成时间:"+(new Date().getTime() - start.getTime()));
看下性能对比
10线程并发3次(前3次为poi,后3次为easyexcel)
从cpu和内存使用情况来看,两者相差不大。
耗时:
easyexcel耗时大概在55s左右,poi的耗时在84s左右。
20线程并发两次
内存被cpu差距不大,响应时间是10线程的两倍。
demo地址:https://gitee.com/luncene/excel-test.git
参考:https://www.yuque.com/easyexcel/doc/about