Hive collect_set()、collect_list()列转行,并对转换后的行值排序

Hive collect_set()、collect_list()列转行,和concat_ws()使用,并对转换后的行值排序

1、需求描述

对列值分组,并按一定顺序排序,最后多行合并一行,合并值左到右逆序排列。

2、考点:

  • sort_array(e: column, asc: boolean)将array中元素排序(自然排序),默认asc为true,即默认排升序
  • collect_set() 和 collect_list()的区别是前者去重,后者不去重

3.1、直接上collect_list()代码实现:

sql 复制代码
select st_name
      ,concat_ws(",",sort_array(collect_list(class),false)) 
      ,concat_ws(",",sort_array(collect_list(class),true))
      ,concat_ws(",",sort_array(collect_list(class))) 
from
(
  select "jack" as st_name, '3' as class
  union all
  select "jack" as st_name, '1' as class
  union all
  select "jack" as st_name, '2' as class
  union all
  select "jack" as st_name, '3' as class
  union all
  select "jack" as st_name, '5' as class
)tb_mid
group by st_name;

结果如下:

st_name concat_ws(,, sort_array(collect_list(class), false)) concat_ws(,, sort_array(collect_list(class), true)) concat_ws(,, sort_array(collect_list(class), true))

jack 5,3,3,2,1 1,2,3,3,5 1,2,3,3,5

Time taken: 0.16 seconds, Fetched 1 row(s)

3.2、直接上collect_set()代码实现:

sql 复制代码
select st_name
      ,concat_ws(",",sort_array(collect_set(class),false)) 
      ,concat_ws(",",sort_array(collect_set(class),true))
      ,concat_ws(",",sort_array(collect_set(class))) 
from
(
  select "jack" as st_name, '3' as class
  union all
  select "jack" as st_name, '1' as class
  union all
  select "jack" as st_name, '2' as class
  union all
  select "jack" as st_name, '3' as class
  union all
  select "jack" as st_name, '5' as class
)tb_mid
group by st_name;

结果如下:

st_name concat_ws(,, sort_array(collect_set(class), false)) concat_ws(,, sort_array(collect_set(class), true)) concat_ws(,, sort_array(collect_set(class), true))

jack 5,3,2,1 1,2,3,5 1,2,3,5

Time taken: 0.152 seconds, Fetched 1 row(s)

相关推荐
十月南城2 天前
数据湖技术对比——Iceberg、Hudi、Delta的表格格式与维护策略
大数据·数据库·数据仓库·hive·hadoop·spark
王九思2 天前
Hive Thrift Server 介绍
数据仓库·hive·hadoop
土拨鼠烧电路2 天前
笔记11:数据中台:不是数据仓库,是业务能力复用的引擎
数据仓库·笔记
Asher05093 天前
Hive核心知识:从基础到实战全解析
数据仓库·hive·hadoop
xhaoDream3 天前
Hive3.1.3 配置 Tez 引擎
大数据·hive·tez
weixin_449310844 天前
使用轻易云平台实现数据ETL转换与写入金蝶云星辰V2
java·数据仓库·etl
一号IT男5 天前
Hive中GROUPING SETS功能详解
数据仓库·hive·hadoop
500佰5 天前
Hive常见故障多案例FAQ宝典 --项目总结(宝典一)
大数据·linux·数据仓库·hive·hadoop·云计算·运维开发