Hive输出表信息中文乱码解决方案

無森~2026-01-19 9:40

输出表信息的时候，如果含有中文可能会出现乱码。

检查hive_remote数据库创建,使用的utf8：

mysql> show create database hive_remote;
+-------------+------------------------------------------------------------------
----+
| Database | Create Database
|
+-------------+------------------------------------------------------------------
----+
| hive_remote | CREATE DATABASE `hive_remote` /*!40100 DEFAULT CHARACTER SET
utf8 */ |
+-------------+------------------------------------------------------------------
----+
1 row in set (0.00 sec)

检查表COLUMNS_V2的创建语句，发现表和COMMENT使用的latin1编码格式，所以出现中文乱码：

mysql> show create table COLUMNS_V2;
| Table | Create Table
| COLUMNS_V2 | CREATE TABLE `COLUMNS_V2` (
`CD_ID` bigint(20) NOT NULL,
`COMMENT` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
`COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`TYPE_NAME` mediumtext,
`INTEGER_IDX` int(11) NOT NULL,
PRIMARY KEY (`CD_ID`,`COLUMN_NAME`),
KEY `COLUMNS_V2_N49` (`CD_ID`),
CONSTRAINT `COLUMNS_V2_FK1` FOREIGN KEY (`CD_ID`) REFERENCES `CDS` (`CD_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 |

导致字符集编码使用latin1的罪魁祸首在哪里：

文件是/opt/hive-3.1.2/scripts/metastore/upgrade/mysql/hive-schema-3.1.0.mysql.sql文件中。

/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE IF NOT EXISTS `COLUMNS_V2` (
`CD_ID` bigint(20) NOT NULL,
`COMMENT` varchar(256) CHARACTER SET latin1 COLLATE latin1_bin DEFAULT NULL,
`COLUMN_NAME` varchar(767) CHARACTER SET latin1 COLLATE latin1_bin NOT NULL,
`TYPE_NAME` MEDIUMTEXT DEFAULT NULL,
`INTEGER_IDX` int(11) NOT NULL,
PRIMARY KEY (`CD_ID`,`COLUMN_NAME`),
KEY `COLUMNS_V2_N49` (`CD_ID`),
CONSTRAINT `COLUMNS_V2_FK1` FOREIGN KEY (`CD_ID`) REFERENCES `CDS` (`CD_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;

解决方法：

修改表字段注解和表注解（在mysql上执行）

sql 复制代码

alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
alter table TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set
utf8;

修改分区字段注解

sql 复制代码

alter table PARTITION_PARAMS modify column PARAM_VALUE varchar(4000) character
set utf8;
alter table PARTITION_KEYS modify column PKEY_COMMENT varchar(4000) character
set utf8;

修改索引注解

sql 复制代码

alter table INDEX_PARAMS modify column PARAM_VALUE varchar(4000) character
set utf8;