Hive数据仓库行转列

查了很多资料发现网上很多文章都是转发和抄袭,有些问题。这里分享一个自己项目中使用的行转列例子,供大家参考。代码如下:

sql 复制代码
SELECT
  my_id,
  nm_cd_map['A'] AS my_cd_a,
  nm_cd_map['B'] AS my_cd_b,
  nm_cd_map['C'] AS my_cd_c,
  nm_num_map['A'] AS my_num_a,
  nm_num_map['B'] AS my_num_b,
  nm_num_map['C'] AS my_num_c
FROM
  (
    SELECT
      t.my_id,
      STR_TO_MAP(my_nm_cds,';',':') AS nm_cd_map,
      STR_TO_MAP(my_nm_nums,';',':') AS nm_num_map
    FROM
      (
        SELECT
          my_id,
          CONCAT_WS(';',COLLECT_LIST(CONCAT(my_nm,':',my_cd))) AS my_nm_cds,
          CONCAT_WS(';',COLLECT_LIST(CONCAT(my_nm,':',my_num))) AS my_nm_nums
        FROM
          (
            SELECT '1' AS my_id,'A' AS my_nm,'D01' AS my_cd,19 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'B' AS my_nm,'D04' AS my_cd,18 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'C' AS my_nm,'D02' AS my_cd,17 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'A' AS my_nm,'D03' AS my_cd,16 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'B' AS my_nm,'D05' AS my_cd,15 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'C' AS my_nm,'D06' AS my_cd,14 AS my_num
          )
        GROUP BY my_id
      ) t
  ) t
WHERE 1=1;

如果是在SparkSQL或Presto平台,或者阿里云的MaxCompute平台,还可使用如下方式:

sql 复制代码
-- 其实也可使用CONCAT然后STR_TO_MAP的方式,或者用MAP_FROM_ARRAYS,再或者用数组排序后ARRAY[n] AS的方式
SELECT
  my_id,
  nm_cd_map['A'] AS my_cd_a,
  nm_cd_map['B'] AS my_cd_b,
  nm_cd_map['C'] AS my_cd_c,
  nm_num_map['A'] AS my_num_a,
  nm_num_map['B'] AS my_num_b,
  nm_num_map['C'] AS my_num_c
FROM
  (
    SELECT
      t.my_id,
      MAP_FROM_ENTRIES(COLLECT_LIST(nm_cd)) AS nm_cd_map,
      MAP_FROM_ENTRIES(COLLECT_LIST(nm_num)) AS nm_num_map
    FROM
      (
        SELECT
          my_id,
          my_nm,
          my_cd,
          my_num,
          STRUCT(my_nm,my_cd) AS nm_cd,
          STRUCT(my_nm,my_num) AS nm_num
        FROM
          (
            SELECT '1' AS my_id,'A' AS my_nm,'D01' AS my_cd,19 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'B' AS my_nm,'D04' AS my_cd,18 AS my_num
            UNION ALL
            SELECT '1' AS my_id,'C' AS my_nm,'D02' AS my_cd,17 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'A' AS my_nm,'D03' AS my_cd,16 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'B' AS my_nm,'D05' AS my_cd,15 AS my_num
            UNION ALL
            SELECT '2' AS my_id,'C' AS my_nm,'D06' AS my_cd,14 AS my_num
          )
      ) t
    GROUP BY my_id
  ) t
WHERE 1=1;
相关推荐
王小王-1231 分钟前
基于Hadoop的服装电商数据分析系统【Hdfs、flume、HIve、sqoop、MySQL、echarts】
hive·hadoop·hdfs·服装电商分析·服装大数据
青春万岁!!3 小时前
hiveSQL脚本性能优化-减少表扫描
大数据·hive·sql
阿坤带你走近大数据1 天前
HiveSQL常见性能调优策略与经验
hive·sql·调优
It's Q2 天前
Hive序列函数&&排名函数
数据仓库·hive·hadoop
Irene19912 天前
外部表(EXTERNAL_TABLE)Hive 借用数据,删表不删数据
hive·内部表·外部表
Irene19913 天前
Windows 11 WSL Ubuntu 环境:实际安装 Hive 踩坑实录
hive·windows·ubuntu
Irene19913 天前
(课堂笔记)Hive 分区、分桶与数据倾斜
hive·hadoop
Irene19914 天前
在 WSL Ubuntu 上安装和使用 Hive
linux·hive·ubuntu
Irene19914 天前
(课堂笔记)Hive 基础
hive·hadoop
水火既济__5 天前
加快hive效率
数据仓库·hive·hadoop