Hive多维分析函数——With cube、Grouping sets、With rollup

有些指标涉及【多维度】的聚合,大的汇总维度,小的明细维度,需要精细化的下钻。

  • grouping sets: 多维度组合,组合维度自定义;
  • with cube: 多维度组合,程序自由组合,组合为各个维度的笛卡尔积;
  • **with rollup:**是 with cube的子集,以左侧维度为主,即不允许左侧为NULL,右侧为非NULL的情况出现

一、grouping sets

0 数据准备

sql 复制代码
with test1 as
     (select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
      union all
      select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
      union all
      select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
      union all
      select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
      union all
      select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
      union all
      select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)
      select 
        month_date,
        day_date,
        sum(pv) as pv
     from test1
      group by month_date,day_date
      grouping sets
       (
          (),
          (month_date),
          (month_date,day_date),
          (day_date)
        )
      order by month_date,day_date;

1 结果分析

二、with cube

0 数据准备

sql 复制代码
with test1 as
(select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
 union all
 select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
 union all
 select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
 union all
 select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)
 
 select 
   month_date,
   day_date,
   sum(pv) as pv
 from test1
 group by month_date,day_date
 with cube
 order by month_date,day_date;

1 结果分析

2 总结

  • 从with cube和grouping sets的案例可以看出,两个结果是一样的;
  • with cube的维度组合(各个维度的笛卡尔积)就是groupingsets里面手动添加的维度,即为month_date,day_date两个维度的笛卡尔积。维度组合即为:()、 (month_date)、 (month_date,day_date)、 (day_date)

三、 with rollup

0 数据准备

sql 复制代码
with test1 as
(select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
 union all
 select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
 union all
 select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
 union all
 select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)

select 
  month_date,
  day_date,
  sum(pv) as pv
from test1
group by month_date,day_date
with rollup
order by month_date,day_date;

1 结果分析

2 总结

从结果上可以看出,with rollup 和with cube的区别是,少了day_date这个单独维度的聚合,因为with rollup是以左侧维度为主,当左侧month_date维度为NULL时,右侧day_date维度必须为NULL。

参考文章 :

https://zhuanlan.zhihu.com/p/631268351

https://blog.51cto.com/u_14555/6696007

相关推荐
阿里云大数据AI技术18 分钟前
StarRocks 助力数禾科技构建实时数仓:从数据孤岛到智能决策
大数据
Lx3525 小时前
Hadoop数据处理优化:减少Shuffle阶段的性能损耗
大数据·hadoop
武子康9 小时前
大数据-99 Spark Streaming 数据源全面总结:原理、应用 文件流、Socket、RDD队列流
大数据·后端·spark
阿里云大数据AI技术1 天前
大数据公有云市场第一,阿里云占比47%!
大数据
Lx3521 天前
Hadoop容错机制深度解析:保障作业稳定运行
大数据·hadoop
T06205141 天前
工具变量-5G试点城市DID数据(2014-2025年
大数据
向往鹰的翱翔1 天前
BKY莱德因:5大黑科技逆转时光
大数据·人工智能·科技·生活·健康医疗
鸿乃江边鸟1 天前
向量化和列式存储
大数据·sql·向量化
IT毕设梦工厂1 天前
大数据毕业设计选题推荐-基于大数据的客户购物订单数据分析与可视化系统-Hadoop-Spark-数据可视化-BigData
大数据·hadoop·数据分析·spark·毕业设计·源码·bigdata
java水泥工1 天前
基于Echarts+HTML5可视化数据大屏展示-白茶大数据溯源平台V2
大数据·echarts·html5