Springboot下如何将CSV文件数据导入H2、Mysql数据库表

爱码少年2026-11-13 21:25

文章目录

一、简要概述

不同数据源之间周期性地执行数据同步是一个很常见的需求。为了保险起见，一般情况下处理流程是先将原始数据导出为中间文件（如 Excel、CSV 等），数据库管理工具如Navicat等几乎都支持。

问题就转换成了：如何将 CSV 文件数据导入数据库表？

二、处理流程

下面我们以 CSV 文件导入 H2 数据库为例进行说明。

1, 准备文件

CSV 文件本质上是普通的文本文件，默认以逗号分割。Spring Boot 工程中，文件路径为 src/test/resources/csv/data.csv，内容如下：

bash 复制代码 
mid,name,slug,type,description,sort,parent
7,程序,,category,,0,0
8,诗词,,category,,0,0
9,散文,,category,,0,0
10,kidylee,https://kidylee.com,link,"",6,0
12,我的码云,https://gitee.com/00fly,link,"",5,0
13,日常,,category,,0,0
14,对联,对联,tag,,0,0
15,恭贺新年,恭贺新年,tag,,0,0
16,吉祥如意,吉祥如意,tag,,0,0
17,故乡,故乡,tag,,0,0
18,祝福,祝福,tag,,0,0
19,祈愿,祈愿,tag,,0,0
20,默认分类,默认分类,category,,0,0
21,鸡汤,,category,,0,0
22,哲理,哲理,tag,,0,0
23,凯利公式,凯利公式,tag,,0,0
25,成长,,category,,0,0
26,教育,教育,tag,,0,0
27,成长,成长,tag,,0,0
28,百科,,category,,0,0
29,国士无双,国士无双,tag,,0,0
30,脊梁,,category,,0,0
31,幽默,,category,,0,0
32,诗词,诗词,tag,,0,0
33,哈哈哈,哈哈哈,tag,,0,0
34,天才,天才,tag,,0,0
35,叶企孙,叶企孙,tag,,0,0
36,科普,科普,tag,,0,0
37,王勃,王勃,tag,,0,0
38,视角,视角,tag,,0,0
39,梅,梅,tag,,0,0
40,束星北,束星北,tag,,0,0
41,李白,李白,tag,,0,0
44,数据库管理,https://soft.00fly.online/treesoft/index,link,"",3,0
45,电脑,电脑,tag,,0,0
46,趣闻,,category,,0,0
47,历史,历史,tag,,0,0
48,联想,联想,tag,,0,0
49,吐槽,,category,,0,0
50,持续集成,https://jks.00fly.online/,link,"",2,0
52,My CSDN,https://blog.csdn.net/qq_16127313,link,"",0,0
53,新版博客,http://1.117.164.5,link,"",0,0
54,春节,春节,tag,,0,0
55,鼠年,鼠年,tag,,0,0

2, 核心代码

java 复制代码 
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.atomic.AtomicLong;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.ArrayUtils;
import org.apache.commons.lang3.StringUtils;
import org.junit.jupiter.api.Test;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;
import org.springframework.util.StopWatch;

import lombok.extern.slf4j.Slf4j;

@Slf4j
public class CsvToH2Test
{
    long batch = 10;
    
    Path path = Paths.get("src/test/resources/csv/data.csv");
    
    JdbcTemplate jdbcTemplate = new JdbcTemplate(new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2).setName("test").setScriptEncoding("UTF-8").build());
    
    @Test
    public void test1()
        throws IOException, InterruptedException
    {
        File file = path.toFile();
        log.info("{}", FileUtils.byteCountToDisplaySize(FileUtils.sizeOf(file)));
        
        StopWatch clock = new StopWatch();
        clock.start();
        String title;
        try (Stream<String> lines = Files.lines(path))
        {
            title = lines.findFirst().orElse("");
            log.info("标题行：{}", title);
            createTable(title);
        }
        int length = StringUtils.split(title, ",").length;
        List<Object[]> params = new ArrayList<>();
        String batchInsert = createBatchInsert(title);
        try (Stream<String> dataLines = Files.lines(path))
        {
            long total = dataLines.count(); // 总行数
            log.info(">>>>> total: {}", total);
            for (long i = 0; i < total; i += batch)
            {
                params.clear();
                try (Stream<String> lines = Files.lines(path))
                {
                    // 跳过表头行(注意：不能用StringUtils.split,此方法会忽略空值)
                    lines.skip(i + 1).limit(batch).forEach(line -> params.add(ArrayUtils.subarray(line.split(","), 0, length)));
                    jdbcTemplate.batchUpdate(batchInsert, params);
                }
                log.info(">>>>> params size: {}", params.size());
            }
        }
        clock.stop();
        log.info("{}", clock.prettyPrint());
        // 打印数据表数据(注意大数据量OOM风险)
        AtomicLong count = new AtomicLong(0);
        long total = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM `data_csv`", Long.class);
        for (long i = 0; i < total; i += batch)
        {
            log.info("##### start: {}", i);
            jdbcTemplate.queryForList("SELECT * FROM `data_csv` LIMIT ?, ?", i, batch).forEach(map -> log.info("{}. {}", count.incrementAndGet(), map));
        }
    }
    
    /**
     * 执行建表
     * 
     * @param title
     * @see [类、类#方法、类#成员]
     */
    private void createTable(String title)
    {
        StringBuffer sql = new StringBuffer("CREATE TABLE IF NOT EXISTS `data_csv` (\n");
        String script = Stream.of(StringUtils.split(title, ",")).map(column -> StringUtils.join("`", column, "` varchar(50)")).collect(Collectors.joining(", \n"));
        sql.append(script).append(");");
        jdbcTemplate.execute(sql.toString());
        log.info("####\n{}", sql);
    }
    
    private String createBatchInsert(String title)
    {
        String[] columns = StringUtils.split(title, ",");
        StringBuffer sql = new StringBuffer("INSERT INTO `data_csv` (");
        String script = Stream.of(columns).map(column -> StringUtils.join("`", column, "`")).collect(Collectors.joining(", "));
        sql.append(script).append(") VALUES (").append(Stream.of(columns).map(column -> "?").collect(Collectors.joining(", "))).append(")");
        log.info("#### {}", sql);
        return sql.toString();
    }
}

3, 运行结果

bash 复制代码 
2026-11-12 12:42:12.556 [main] INFO  .j.d.e.EmbeddedDatabaseFactory - Starting embedded database: url='jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false', username='sa'
2026-11-12 12:42:13.233 [main] INFO  c.f.c.CsvToH2Test              - 1 KB
2026-11-12 12:42:13.236 [main] INFO  c.f.c.CsvToH2Test              - 标题行：mid,name,slug,type,description,sort,parent
2026-11-12 12:42:13.257 [main] INFO  c.f.c.CsvToH2Test              - ####
CREATE TABLE IF NOT EXISTS `data_csv` (
`mid` varchar(50), 
`name` varchar(50), 
`slug` varchar(50), 
`type` varchar(50), 
`description` varchar(50), 
`sort` varchar(50), 
`parent` varchar(50));
2026-11-12 12:42:13.259 [main] INFO  c.f.c.CsvToH2Test              - #### INSERT INTO `data_csv` (`mid`, `name`, `slug`, `type`, `description`, `sort`, `parent`) VALUES (?, ?, ?, ?, ?, ?, ?)
2026-11-12 12:42:13.260 [main] INFO  c.f.c.CsvToH2Test              - >>>>> total: 45
2026-11-12 12:42:13.297 [main] INFO  c.f.c.CsvToH2Test              - >>>>> params size: 10
2026-11-12 12:42:13.302 [main] INFO  c.f.c.CsvToH2Test              - >>>>> params size: 10
2026-11-12 12:42:13.316 [main] INFO  c.f.c.CsvToH2Test              - >>>>> params size: 10
2026-11-12 12:42:13.320 [main] INFO  c.f.c.CsvToH2Test              - >>>>> params size: 10
2026-11-12 12:42:13.323 [main] INFO  c.f.c.CsvToH2Test              - >>>>> params size: 4
2026-11-12 12:42:13.325 [main] INFO  c.f.c.CsvToH2Test              - StopWatch '': running time = 88300798 ns
---------------------------------------------
ns         %     Task name
---------------------------------------------
088300798  100%  

2026-11-12 12:42:13.445 [main] INFO  c.f.c.CsvToH2Test              - ##### start: 0
2026-11-12 12:42:13.454 [main] INFO  c.f.c.CsvToH2Test              - 1. {MID=7, NAME=程序, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.457 [main] INFO  c.f.c.CsvToH2Test              - 2. {MID=8, NAME=诗词, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.457 [main] INFO  c.f.c.CsvToH2Test              - 3. {MID=9, NAME=散文, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.457 [main] INFO  c.f.c.CsvToH2Test              - 4. {MID=10, NAME=kidylee, SLUG=https://kidylee.com, TYPE=link, DESCRIPTION="", SORT=6, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 5. {MID=12, NAME=我的码云, SLUG=https://gitee.com/00fly, TYPE=link, DESCRIPTION="", SORT=5, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 6. {MID=13, NAME=日常, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 7. {MID=14, NAME=对联, SLUG=对联, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 8. {MID=15, NAME=恭贺新年, SLUG=恭贺新年, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 9. {MID=16, NAME=吉祥如意, SLUG=吉祥如意, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.458 [main] INFO  c.f.c.CsvToH2Test              - 10. {MID=17, NAME=故乡, SLUG=故乡, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.459 [main] INFO  c.f.c.CsvToH2Test              - ##### start: 10
2026-11-12 12:42:13.462 [main] INFO  c.f.c.CsvToH2Test              - 11. {MID=18, NAME=祝福, SLUG=祝福, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.462 [main] INFO  c.f.c.CsvToH2Test              - 12. {MID=19, NAME=祈愿, SLUG=祈愿, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.462 [main] INFO  c.f.c.CsvToH2Test              - 13. {MID=20, NAME=默认分类, SLUG=默认分类, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.463 [main] INFO  c.f.c.CsvToH2Test              - 14. {MID=21, NAME=鸡汤, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.463 [main] INFO  c.f.c.CsvToH2Test              - 15. {MID=22, NAME=哲理, SLUG=哲理, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.464 [main] INFO  c.f.c.CsvToH2Test              - 16. {MID=23, NAME=凯利公式, SLUG=凯利公式, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.464 [main] INFO  c.f.c.CsvToH2Test              - 17. {MID=25, NAME=成长, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.465 [main] INFO  c.f.c.CsvToH2Test              - 18. {MID=26, NAME=教育, SLUG=教育, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.465 [main] INFO  c.f.c.CsvToH2Test              - 19. {MID=27, NAME=成长, SLUG=成长, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.465 [main] INFO  c.f.c.CsvToH2Test              - 20. {MID=28, NAME=百科, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.465 [main] INFO  c.f.c.CsvToH2Test              - ##### start: 20
2026-11-12 12:42:13.468 [main] INFO  c.f.c.CsvToH2Test              - 21. {MID=29, NAME=国士无双, SLUG=国士无双, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.468 [main] INFO  c.f.c.CsvToH2Test              - 22. {MID=30, NAME=脊梁, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.468 [main] INFO  c.f.c.CsvToH2Test              - 23. {MID=31, NAME=幽默, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.468 [main] INFO  c.f.c.CsvToH2Test              - 24. {MID=32, NAME=诗词, SLUG=诗词, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 25. {MID=33, NAME=哈哈哈, SLUG=哈哈哈, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 26. {MID=34, NAME=天才, SLUG=天才, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 27. {MID=35, NAME=叶企孙, SLUG=叶企孙, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 28. {MID=36, NAME=科普, SLUG=科普, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 29. {MID=37, NAME=王勃, SLUG=王勃, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.469 [main] INFO  c.f.c.CsvToH2Test              - 30. {MID=38, NAME=视角, SLUG=视角, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.470 [main] INFO  c.f.c.CsvToH2Test              - ##### start: 30
2026-11-12 12:42:13.473 [main] INFO  c.f.c.CsvToH2Test              - 31. {MID=39, NAME=梅, SLUG=梅, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.473 [main] INFO  c.f.c.CsvToH2Test              - 32. {MID=40, NAME=束星北, SLUG=束星北, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.473 [main] INFO  c.f.c.CsvToH2Test              - 33. {MID=41, NAME=李白, SLUG=李白, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 34. {MID=44, NAME=数据库管理, SLUG=https://soft.00fly.online/treesoft/index, TYPE=link, DESCRIPTION="", SORT=3, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 35. {MID=45, NAME=电脑, SLUG=电脑, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 36. {MID=46, NAME=趣闻, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 37. {MID=47, NAME=历史, SLUG=历史, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 38. {MID=48, NAME=联想, SLUG=联想, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.474 [main] INFO  c.f.c.CsvToH2Test              - 39. {MID=49, NAME=吐槽, SLUG=, TYPE=category, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.475 [main] INFO  c.f.c.CsvToH2Test              - 40. {MID=50, NAME=持续集成, SLUG=https://jks.00fly.online/, TYPE=link, DESCRIPTION="", SORT=2, PARENT=0}
2026-11-12 12:42:13.475 [main] INFO  c.f.c.CsvToH2Test              - ##### start: 40
2026-11-12 12:42:13.477 [main] INFO  c.f.c.CsvToH2Test              - 41. {MID=52, NAME=My CSDN, SLUG=https://blog.csdn.net/qq_16127313, TYPE=link, DESCRIPTION="", SORT=0, PARENT=0}
2026-11-12 12:42:13.477 [main] INFO  c.f.c.CsvToH2Test              - 42. {MID=53, NAME=新版博客, SLUG=http://1.117.164.5, TYPE=link, DESCRIPTION="", SORT=0, PARENT=0}
2026-11-12 12:42:13.478 [main] INFO  c.f.c.CsvToH2Test              - 43. {MID=54, NAME=春节, SLUG=春节, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}
2026-11-12 12:42:13.479 [main] INFO  c.f.c.CsvToH2Test              - 44. {MID=55, NAME=鼠年, SLUG=鼠年, TYPE=tag, DESCRIPTION=, SORT=0, PARENT=0}

4, 切换Ｍysql

如果目标数据库为Mysql 可在springboot工程中配置系统数据源，上面的代码修改为

yaml 复制代码 
spring:
  datasource:
    url: ${jdbc.url}
    username: ${jdbc.username}
    password: ${jdbc.password}
    driverClassName: ${jdbc.driverClassName}
    type: com.zaxxer.hikari.HikariDataSource
    hikari:
      auto-commit: true
      connection-test-query: SELECT 1
      connection-timeout: 30000
      idle-timeout: 30000
      max-lifetime: 1800000
      maximum-pool-size: 15
      minimum-idle: 5
      pool-name: DatebookHikariCP

jdbc-mysql.properties

yaml 复制代码 
jdbc.url=jdbc:mysql://127.0.0.1:3306/test?useSSL=false&serverTimezone=Asia/Shanghai&rewriteBatchedStatements=true&createDatabaseIfNotExist=true
jdbc.driverClassName=com.mysql.cj.jdbc.Driver
jdbc.username=root
jdbc.password=root123
java 复制代码 
@SpringBootTest(webEnvironment = WebEnvironment.NONE)
public class Csv2MysqlTest
{　
    @Resource
    JdbcTemplate jdbcTemplate;
    
    ......

三、小结

核心步骤为三步：读取表头建表、按批构造 INSERT 批量插入、分批读取校验。

数据量增大时需注意：改用流式读取避免全量载入内存，控制批次大小并显式管理事务提交，同时根据样本推断字段类型，避免一律 varchar(50) 导致截断或精度丢失。

朋友们，你学会了吗？

--over--

相关推荐
2603_969694851 小时前
【JavaWeb学习记录】MyBatis核心知识点与实战笔记：从入门到动态SQL
笔记·sql·学习·mysql·mybatis·javaweb·学习方法
可乐ea1 小时前
智能体记忆该拆成几层：笔记、记忆表示与注入上下文
服务器·数据库·笔记·ai智能体·提示词工程·上下文工程·记忆系统
阿义有问题2 小时前
MySQL 迁移达梦全流程实战：DTS 七步图解 + 对象名全大写适配（附踩坑清单）
数据库
猫猫不是喵喵.5 小时前
从0到1创建一个微服务项目
java·数据库·微服务·1024程序员节
lsc_blog6 小时前
听到优化风声，先准备能由自己掌控的事
数据库·分布式·中间件·职场和发展·重构
墨痕诉清风6 小时前
SSRF漏洞实战记录+OSS桶命令执行+MCP漏洞
linux·服务器·数据库
Ashe8139 小时前
MySQL 慢日志先别急着加索引：三个指标能拦掉大半误判
数据库
ClouGence9 小时前
最基础的异构数据库 DDL 同步，我们踩过哪些坑？
数据库·mysql·postgresql
羑悻的小杀马特11 小时前
Redis持久化三剑客：RDB二进制快照、AOF文本日志、混合模式如何守护你的数据安全？
数据库·redis·缓存·aof·持久化存储·rdb
热门推荐
01国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）02历年考研数学一、数学二、数学三真题试卷及答案PDF03如何新建文件夹？ 电脑新建文件夹的4种方法04几个好用的ip纯净度检测网站05微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新06昇腾910B系列芯片完全指南：四款型号对比、服务器选型与大模型部署实战07GitHub 镜像站点08蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接09苹果iPA游戏软件资源下载网站10Dify：一站式AI应用开发平台全解析与Windows部署指南