Hive数据仓库行转列

查了很多资料发现网上很多文章都是转发和抄袭,有些问题。这里分享一个自己项目中使用的行转列例子,供大家参考。代码如下:

sql 复制代码
SELECT
  my_id,
  nm_cd_map['A'] AS my_cd_a,
  nm_cd_map['B'] AS my_cd_b,
  nm_cd_map['C'] AS my_cd_c,
  nm_num_map['A'] AS my_num_a,
  nm_num_map['B'] AS my_num_b,
  nm_num_map['C'] AS my_num_c
FROM
  (
    SELECT
      t.my_id,
      STR_TO_MAP(my_nm_cds,';',':') AS nm_cd_map,
      STR_TO_MAP(my_nm_nums,';',':') AS nm_num_map
    FROM
      (
        SELECT
          my_id,
          CONCAT_WS(';',COLLECT_LIST(CONCAT(my_nm,':',my_cd))) AS my_nm_cds,
          CONCAT_WS(';',COLLECT_LIST(CONCAT(my_nm,':',my_num))) AS my_nm_nums
        FROM
          (
            SELECT '1' AS my_id,'A' AS my_nm,'D01' AS my_cd,19 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'B' AS my_nm,'D04' AS my_cd,18 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'C' AS my_nm,'D02' AS my_cd,17 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'A' AS my_nm,'D03' AS my_cd,16 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'B' AS my_nm,'D05' AS my_cd,15 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'C' AS my_nm,'D06' AS my_cd,14 AS my_num
          )
        GROUP BY my_id
      ) t
  ) t
WHERE 1=1;

如果是在SparkSQL或Presto平台,或者阿里云的MaxCompute平台,还可使用如下方式:

sql 复制代码
-- 其实也可使用CONCAT然后STR_TO_MAP的方式,或者用MAP_FROM_ARRAYS,再或者用数组排序后ARRAY[n] AS的方式
SELECT
  my_id,
  nm_cd_map['A'] AS my_cd_a,
  nm_cd_map['B'] AS my_cd_b,
  nm_cd_map['C'] AS my_cd_c,
  nm_num_map['A'] AS my_num_a,
  nm_num_map['B'] AS my_num_b,
  nm_num_map['C'] AS my_num_c
FROM
  (
    SELECT
      t.my_id,
      MAP_FROM_ENTRIES(COLLECT_LIST(nm_cd)) AS nm_cd_map,
      MAP_FROM_ENTRIES(COLLECT_LIST(nm_num)) AS nm_num_map
    FROM
      (
        SELECT
          my_id,
          my_nm,
          my_cd,
          my_num,
          STRUCT(my_nm,my_cd) AS nm_cd,
          STRUCT(my_nm,my_num) AS nm_num
        FROM
          (
            SELECT '1' AS my_id,'A' AS my_nm,'D01' AS my_cd,19 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'B' AS my_nm,'D04' AS my_cd,18 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'C' AS my_nm,'D02' AS my_cd,17 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'A' AS my_nm,'D03' AS my_cd,16 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'B' AS my_nm,'D05' AS my_cd,15 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'C' AS my_nm,'D06' AS my_cd,14 AS my_num
          )
      ) t
    GROUP BY my_id
  ) t
WHERE 1=1;
相关推荐
赤土 炙焱11 小时前
hiveserver2启动失败,磁盘满了
hive·hadoop
Francek Chen1 天前
【大数据处理与分析】数据仓库Hive:02 数据湖
大数据·数据仓库·hive·hadoop·分布式·数据分析
Waay4 天前
什么是HBase?它和Hive有什么区别?
大数据·数据库·hive·hadoop·hdfs·hbase
fastjson_5 天前
Hive的介绍和使用
数据仓库·hive·hadoop
William Dawson10 天前
【踩坑实录|Hive1\.2\.1数据服务接口5大疑难问题调试与全方位优化方案】
java·hive·spring boot
牛奶咖啡1317 天前
大数据Hadoop运维应用实践——HIVE与Hadoop实现整合_Hive的配置安装与使用
大数据·hive·hive的配置与安装·metastore服务的配置·hiveserver2服务配置·hive的常用sql操作·beeline的使用
向夏威夷 梦断明暄19 天前
基于Tez引擎的 Hive SQL 性能优化
hive·sql·性能优化
humbinal1 个月前
同时支持 gui & cli 的 parquet 文件查看工具,高性能小清新!
hive·python·rust·spark·开源·github·parquet
吾AI科技1 个月前
基于Tez引擎的 Hive SQL 性能优化
大数据·hive·性能优化·tez
abcy0712131 个月前
hive 执行器原理
hive