Hive多维分析函数——With cube、Grouping sets、With rollup

有些指标涉及【多维度】的聚合,大的汇总维度,小的明细维度,需要精细化的下钻。

  • grouping sets: 多维度组合,组合维度自定义;
  • with cube: 多维度组合,程序自由组合,组合为各个维度的笛卡尔积;
  • **with rollup:**是 with cube的子集,以左侧维度为主,即不允许左侧为NULL,右侧为非NULL的情况出现

一、grouping sets

0 数据准备

sql 复制代码
with test1 as
     (select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
      union all
      select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
      union all
      select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
      union all
      select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
      union all
      select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
      union all
      select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)
      select 
        month_date,
        day_date,
        sum(pv) as pv
     from test1
      group by month_date,day_date
      grouping sets
       (
          (),
          (month_date),
          (month_date,day_date),
          (day_date)
        )
      order by month_date,day_date;

1 结果分析

二、with cube

0 数据准备

sql 复制代码
with test1 as
(select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
 union all
 select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
 union all
 select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
 union all
 select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)
 
 select 
   month_date,
   day_date,
   sum(pv) as pv
 from test1
 group by month_date,day_date
 with cube
 order by month_date,day_date;

1 结果分析

2 总结

  • 从with cube和grouping sets的案例可以看出,两个结果是一样的;
  • with cube的维度组合(各个维度的笛卡尔积)就是groupingsets里面手动添加的维度,即为month_date,day_date两个维度的笛卡尔积。维度组合即为:()、 (month_date)、 (month_date,day_date)、 (day_date)

三、 with rollup

0 数据准备

sql 复制代码
with test1 as
(select '2021-08' as month_date,'2021-08-11' as day_date,10 as pv
 union all
 select '2021-08' as month_date,'2021-08-10' as day_date,15 as pv
 union all
 select '2021-08' as month_date,'2021-08-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-08' as day_date,35 as pv
 union all
 select '2021-07' as month_date,'2021-07-06' as day_date,25 as pv
 union all
 select '2021-07' as month_date,'2021-07-03' as day_date,15 as pv)

select 
  month_date,
  day_date,
  sum(pv) as pv
from test1
group by month_date,day_date
with rollup
order by month_date,day_date;

1 结果分析

2 总结

从结果上可以看出,with rollup 和with cube的区别是,少了day_date这个单独维度的聚合,因为with rollup是以左侧维度为主,当左侧month_date维度为NULL时,右侧day_date维度必须为NULL。

参考文章 :

https://zhuanlan.zhihu.com/p/631268351

https://blog.51cto.com/u_14555/6696007

相关推荐
xufwind43 分钟前
spark standlone 集群离线安装
大数据·分布式·spark
AI数据皮皮侠2 小时前
中国区域10m空间分辨率楼高数据集(全国/分省/分市/免费数据)
大数据·人工智能·机器学习·分类·业界资讯
DeepSeek大模型官方教程3 小时前
NLP之文本纠错开源大模型:兼看语音大模型总结
大数据·人工智能·ai·自然语言处理·大模型·产品经理·大模型学习
大数据CLUB4 小时前
基于spark的奥运会奖牌变化数据分析
大数据·hadoop·数据分析·spark
Edingbrugh.南空4 小时前
Hadoop高可用集群搭建
大数据·hadoop·分布式
智慧化智能化数字化方案4 小时前
69页全面预算管理体系的框架与落地【附全文阅读】
大数据·人工智能·全面预算管理·智慧财务·智慧预算
武子康5 小时前
大数据-33 HBase 整体架构 HMaster HRegion
大数据·后端·hbase
诗旸的技术记录与分享18 小时前
Flink-1.19.0源码详解-番外补充3-StreamGraph图
大数据·flink
资讯分享周18 小时前
Alpha系统联结大数据、GPT两大功能,助力律所管理降本增效
大数据·gpt
G皮T20 小时前
【Elasticsearch】深度分页及其替代方案
大数据·elasticsearch·搜索引擎·scroll·检索·深度分页·search_after