MySQL运维实战(5.5) 数据导入导出时的字符集问题

作者：俊达

引言

当涉及到在MySQL中进行数据导入和导出时，字符集相关的问题成为关键因素。为了确保数据的准确导入和导出，我们可以借助诸如LOAD DATA、SELECT INTO OUTFILE或mysqldump等工具。在以下内容中，我们将详细分析与数据导入和导出相关的字符集问题，提供解决方案以确保数据的完整性和一致性。

准备测试数据

创建测试表，2个字段分别使用gbk和utf8字符集

powershell 复制代码

mysql> create table test_load(c_gbk varchar(100) character set gbk, c_utf8 varchar(100) character set utf8);
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> show create table test_load\G
*************************** 1. row ***************************
       Table: test_load
Create Table: CREATE TABLE `test_load` (
  `c_gbk` varchar(100) CHARACTER SET gbk COLLATE gbk_chinese_ci DEFAULT NULL,
  `c_utf8` varchar(100) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci
1 row in set (0.00 sec)

写入测试数据

powershell 复制代码

mysql> insert into test_load values('列列列列列AAA', '列列列列列AAA');
Query OK, 1 row affected (0.00 sec)


mysql> select c_gbk, hex(c_gbk), c_utf8, hex(c_utf8) from test_load;
+--------------------+----------------------------+--------------------+--------------------------------------+
| c_gbk              | hex(c_gbk)                 | c_utf8             | hex(c_utf8)                          |
+--------------------+----------------------------+--------------------+--------------------------------------+
| 列列列列列AAA      | C1D0C1D0C1D0C1D0C1D0414141 | 列列列列列AAA      | E58897E58897E58897E58897E58897414141 |
+--------------------+----------------------------+--------------------+--------------------------------------+

使用select into outfile导出数据

powershell 复制代码

mysql> select * into outfile 'test_load.txt' from test_load;
Query OK, 1 row affected (0.00 sec)

查看数据文件：

powershell 复制代码

$ cat /usr/local/var/mysql/test/test_load.txt
����������AAA	列列列列列AAA

$ od -t x1 /usr/local/var/mysql/test/test_load.txt
0000000    c1  d0  c1  d0  c1  d0  c1  d0  c1  d0  41  41  41  09  e5  88
0000020    97  e5  88  97  e5  88  97  e5  88  97  e5  88  97  41  41  41
0000040    0a
0000041

使用select into outfile时，默认不做字符集转换。

指定导出文件字符集

powershell 复制代码

mysql> select * into outfile 'test_load_x.txt' character set utf8 from test_load;
Query OK, 1 row affected, 1 warning (0.00 sec)

lazybug:~ lazybug$ cat /usr/local/var/mysql/test/test_load_x.txt
列列列列列AAA	列列列列列AAA

lazybug:~ lazybug$ od -t x1 /usr/local/var/mysql/test/test_load_x.txt
0000000    e5  88  97  e5  88  97  e5  88  97  e5  88  97  e5  88  97  41
0000020    41  41  09  e5  88  97  e5  88  97  e5  88  97  e5  88  97  e5
0000040    88  97  41  41  41  0a
0000046

如果指定select into outfile字符集，则会进行字符集的转换。

load data

powershell 复制代码

mysql> create table test_load2 like test_load;
Query OK, 0 rows affected (0.01 sec)


mysql> load data local infile '/usr/local/var/mysql/test/test_load.txt' into table test_load2;
ERROR 1300 (HY000): Invalid utf8mb4 character string: ''


mysql> load data local infile '/usr/local/var/mysql/test/test_load.txt' into table test_load2 character set gbk;
Query OK, 1 row affected (0.00 sec)
Records: 1  Deleted: 0  Skipped: 0  Warnings: 0

mysql> select * from test_load2;
+--------------------+----------------------------+
| c_gbk              | c_utf8                     |
+--------------------+----------------------------+
| 列列列列列AAA      | 鍒楀垪鍒楀垪鍒桝AA         |
+--------------------+----------------------------+
1 row in set (0.00 sec)

由于文件中存在2种编码的数据，可以使用binary字符集导入数据：

powershell 复制代码

mysql> truncate table test_load2;
Query OK, 0 rows affected (0.01 sec)

mysql> load data local infile '/usr/local/var/mysql/test/test_load.txt' 
  into table test_load2。
  character set binary;

Query OK, 1 row affected (0.00 sec)
Records: 1  Deleted: 0  Skipped: 0  Warnings: 0

mysql> select c_gbk, hex(c_gbk), c_utf8, hex(c_utf8) from test_load2;
+--------------------+----------------------------+--------------------+--------------------------------------+
| c_gbk              | hex(c_gbk)                 | c_utf8             | hex(c_utf8)                          |
+--------------------+----------------------------+--------------------+--------------------------------------+
| 列列列列列AAA      | C1D0C1D0C1D0C1D0C1D0414141 | 列列列列列AAA      | E58897E58897E58897E58897E58897414141 |
+--------------------+----------------------------+--------------------+--------------------------------------+
1 row in set (0.00 sec)

总结

MySQL允许一个表的多个字段使用不同的字符集。然而，在处理具有不同字符集的多个字段时，必须特别注意数据导入和导出的过程。为避免乱码问题，有两种主要的解决方案：一是在导出数据时将所有字段数据转换为相同的字符集，确保一致性；二是在导入时使用binary字符集，以保持数据的原始形式。这样，可以有效地处理多字符集表格的数据导入导出，确保数据的准确性和完整性。

更多技术信息请查看云掣官网https://yunche.pro/?t=yrgw