Hive 行列转换

行列转换
列转行

使用 lateral view + explode(array|map)lateral view + inline(array_struct) 可以将列转换为行。

  • 单列转多行,降维(单列数组或键值对)

示例1:explode(array(...))

sql 复制代码
select ..., A
from T
lateral view explode(ARRAY_FIELD) as A;
sql 复制代码
select explode(`array`(88.2,98.3,67.1)) AS (price);

示例2:explode(map(...))

sql 复制代码
select ..., K, V
from T
lateral view explode(MAP_FIELD) as K, V;
sql 复制代码
select explode(`map`("java",56,"mysql",88,"javascript",66)) AS (subject, score);

示例3:inline(array_struct)

sql 复制代码
select ..., 
from T
lateral view inline(STRUCT_ARRAY_FIELD)V as F1,...,FN;
sql 复制代码
with tmp as (
select array(
	named_struct('name','henry','age',22,'is_member','true'),
	named_struct('name','pola','age',20,'is_member','true'),
	named_struct('name','ariel','age',19,'is_member','true')
   ) AS array_struct
)
select name,age,is_member
from tmp
lateral view inline(array_struct)V as name,age,is_member;

lateral view inline(array_struct)将结构体数组的每个元素都转化为一行,每一行都包含结构体字段的值.

前:

后:

  • 多列转多行
sql 复制代码
select ..., A
from T
lateral view explode(array|map(F1,...,FN))V as A;

示例:

sql 复制代码
SELECT name, class, Scores.subject, Scores.score
FROM Students
LATERAL VIEW EXPLODE(ARRAY(
	named_struct('subject','math','score',math_score),
	named_struct('subject','science','score',science_score)
	)
) V AS Scores;

前:

后:

行转列
  • 多行转多列
    条件聚合,通常用于将多行数据中满足条件的某个值聚合到单个行中。
sql 复制代码
select
		F1,...,
		sum(if(C1,0,V1)) as A1,
		sum(if(C2,0,V2)) as A2,
		sum(if(C3,0,V3)) as A3
	from TABLE_NAME
	group by F1,...
	
	drop table if exists lateral_view_stack_test1w;
	create table lateral_view_stack_test1w as
	select year,
		   sum(if(month(order_time)=1,order_amount,0)) as sum_jan,
		   sum(if(month(order_time)=2,order_amount,0)) as sum_feb,
		   sum(if(month(order_time)=3,order_amount,0)) as sum_mar,
		   sum(if(month(order_time)=4,order_amount,0)) as sum_apr,
		   sum(if(month(order_time)=5,order_amount,0)) as sum_may,
		   sum(if(month(order_time)=6,order_amount,0)) as sum_jun,
		   sum(if(month(order_time)=7,order_amount,0)) as sum_jul,
		   sum(if(month(order_time)=8,order_amount,0)) as sum_aug,
		   sum(if(month(order_time)=9,order_amount,0)) as sum_sep,
		   sum(if(month(order_time)=10,order_amount,0)) as sum_oct,
		   sum(if(month(order_time)=11,order_amount,0)) as sum_nov,
		   sum(if(month(order_time)=12,order_amount,0)) as sum_dec
	from hive_internal_par_regex_test1w
	where year>=2014
	group by year;
相关推荐
SirLancelot114 小时前
StarRocks-基本介绍(一)基本概念、特点、适用场景
大数据·数据库·数据仓库·sql·数据分析·database·数据库架构
随心............17 小时前
在开发过程中遇到问题如何解决,以及两个经典问题
hive·hadoop·spark
yumgpkpm1 天前
CMP (类ClouderaCDP7.3(404次编译) )华为鲲鹏Aarch64(ARM)信创环境 查询2100w行 hive 查询策略
数据库·数据仓库·hive·hadoop·flink·mapreduce·big data
CoookeCola2 天前
MovieNet(A holistic dataset for movie understanding) :面向电影理解的多模态综合数据集与工具链
数据仓库·人工智能·目标检测·计算机视觉·数据挖掘
想ai抽3 天前
深入starrocks-多列联合统计一致性探查与策略(YY一下)
java·数据库·数据仓库
starfalling10243 天前
【hive】一种高效增量表的实现
hive
D明明就是我3 天前
Hive 拉链表
数据仓库·hive·hadoop
嘉禾望岗5033 天前
hive join优化和数据倾斜处理
数据仓库·hive·hadoop
yumgpkpm3 天前
华为鲲鹏 Aarch64 环境下多 Oracle 数据库汇聚操作指南 CMP(类 Cloudera CDP 7.3)
大数据·hive·hadoop·elasticsearch·zookeeper·big data·cloudera
忧郁火龙果3 天前
六、Hive的基本使用
数据仓库·hive·hadoop