presto高级用法(grouping、grouping sets)

目录

准备工作:

在hive中建表

在presto中计算

分解式

[按照城市分组 统计人数](#按照城市分组 统计人数)

[按照性别分组 统计人数](#按照性别分组 统计人数)

​编辑

[按照爱好分组 统计人数](#按照爱好分组 统计人数)

​编辑

[按照城市和性别分组 统计人数](#按照城市和性别分组 统计人数)

[按照城市和爱好分组 统计人数](#按照城市和爱好分组 统计人数)

[按照性别和爱好分组 统计人数](#按照性别和爱好分组 统计人数)

[按照城市和性别还有爱好分组 统计人数](#按照城市和性别还有爱好分组 统计人数)

统计人数

合并式

presto使用grouping

[presto使用grouping sets](#presto使用grouping sets)

grouping作用例子展示

[高级用法: cube](#高级用法: cube)

[rollup 用法](#rollup 用法)


准备工作:

在hive中建表
sql 复制代码
drop database if exists db_test cascade;

create database db_test;

create table db_test.tb_student(
    name string,
    score   int,
    city    string,
    sex string,
    hobby string
)
row format delimited fields terminated by '\t';

load data local inpath '/test/student.txt' into table db_test.tb_student;

select * from db_test.tb_student;

student.txt数据

张三 10 北京 男 喝酒

李四 20 北京 男 抽烟

王五 30 北京 女 烫头

赵六 40 上海 男 抽烟

麻七 50 上海 女 烫头

在presto中计算

分解式
按照城市分组 统计人数
sql 复制代码
select city,count(1) as cnt from hive.db_test.tb_student group by city;
按照性别分组 统计人数
sql 复制代码
select hobby,count(1) as cnt from hive.db_test.tb_student group by hobby;
按照爱好分组 统计人数
sql 复制代码
select hobby,count(1) as cnt from hive.db_test.tb_student group by hobby;
按照城市和性别分组 统计人数
sql 复制代码
select city, sex, count(1) as cnt from hive.db_test.tb_student group by city, sex;
按照城市和爱好分组 统计人数
sql 复制代码
select city, hobby, count(1) as cnt from hive.db_test.tb_student group by city, hobby;
按照性别和爱好分组 统计人数
sql 复制代码
select sex, hobby, count(1) as cnt from hive.db_test.tb_student group by sex, hobby;
按照城市和性别还有爱好分组 统计人数
sql 复制代码
select city, sex, hobby, count(1) as cnt from hive.db_test.tb_student group by city, sex, hobby;
统计人数
sql 复制代码
select count(1) as cnt from hive.db_test.tb_student group by ();
合并式
sql 复制代码
with t1 as (
    select city, null as sex, null as hobby, count(1) as cnt, 1 as o from hive.db_test.tb_student group by city
    union all
    select null as city, sex, null as hobby, count(1) as cnt, 2 as o from hive.db_test.tb_student group by sex
    union all
    select null, null, hobby,count(1) as cnt, 3 as o from hive.db_test.tb_student group by hobby
    union all
    select city, sex, null, count(1) as cnt, 4 as o from hive.db_test.tb_student group by city, sex
    union all
    select city, null, hobby, count(1) as cnt, 5 as o from hive.db_test.tb_student group by city, hobby
    union all
    select null, sex, hobby, count(1) as cnt, 6 as o from hive.db_test.tb_student group by sex, hobby
    union all
    select city, sex, hobby, count(1) as cnt, 7 as o from hive.db_test.tb_student group by city, sex, hobby
    union all
    select null, null, null, count(1) as cnt, 8 as o from hive.db_test.tb_student group by ()
)
select * from t1
order by o, city, sex, hobby
;

presto使用grouping

sql 复制代码
select
    city,
    sex,
    count(1) as cnt,
    grouping(city, sex) as g
from hive.db_test.tb_student
group by city, sex
;

presto使用grouping sets

sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    grouping(city, sex, hobby)
from hive.db_test.tb_student
group by grouping sets (city, sex, hobby)
;
sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    grouping(city, sex, hobby)
from hive.db_test.tb_student
group by grouping sets (city, sex, hobby, (city, sex), (city, hobby), (sex, hobby), (city, sex, hobby), ())
;
sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    case
        when grouping(city, sex, hobby)=3 then 1
        when grouping(city, sex, hobby)=5 then 2
        when grouping(city, sex, hobby)=6 then 3
        when grouping(city, sex, hobby)=1 then 4
        when grouping(city, sex, hobby)=2 then 5
        when grouping(city, sex, hobby)=4 then 6
        when grouping(city, sex, hobby)=0 then 7
        when grouping(city, sex, hobby)=7 then 8
        else 100
    end as o
from hive.db_test.tb_student
group by grouping sets (city, sex, hobby, (city, sex), (city, hobby), (sex, hobby), (city, sex, hobby), ())
order by o, city, sex, hobby
;

grouping作用例子展示

sql 复制代码
with t1 as (
    select '北京' as city, '男' as sex
    union all
    select '北京' as city, '男' as sex
    union all
    select '北京' as city, '女' as sex
    union all
    select '北京' as city, null as sex
)
select
    city,
    sex,
    count(1) as cnt
from t1
group by grouping sets (city, (city, sex))
复制代码
问题:
    city=北京, sex=null, cnt=4
    city=北京, sex=null, cnt=1
    为什么 city 和 sex 的值一样, 但是结果不同?
原因:
    一个null 表示跟这一列没有关系
    另一个null 表示 这一列的值 为null, 根据 列值统计的结果
    怎么区分
解决方案:
    grouping(city, sex)
        0,0     两个都有关
        0,1     只跟city有关
        1,0     只跟sex有关
        1,1     都这两列都无关
sql 复制代码
with t1 as (
    select '北京' as city, '男' as sex
    union all
    select '北京' as city, '男' as sex
    union all
    select '北京' as city, '女' as sex
    union all
    select '北京' as city, null as sex
)
select
    city,
    sex,
    count(1) as cnt,
    grouping(city, sex) g
from t1
group by grouping sets (city, (city, sex))
sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    case
        when grouping(city, sex, hobby)=3 then 1
        when grouping(city, sex, hobby)=5 then 2
        when grouping(city, sex, hobby)=6 then 3
        when grouping(city, sex, hobby)=1 then 4
        when grouping(city, sex, hobby)=2 then 5
        when grouping(city, sex, hobby)=4 then 6
        when grouping(city, sex, hobby)=0 then 7
        when grouping(city, sex, hobby)=7 then 8
        else 100
    end as o
from hive.db_test.tb_student
group by grouping sets (city, sex, hobby, (city, sex), (city, hobby), (sex, hobby), (city, sex, hobby), ())
order by o, city, sex, hobby

高级用法: cube

sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    case
        when grouping(city, sex, hobby)=3 then 1
        when grouping(city, sex, hobby)=5 then 2
        when grouping(city, sex, hobby)=6 then 3
        when grouping(city, sex, hobby)=1 then 4
        when grouping(city, sex, hobby)=2 then 5
        when grouping(city, sex, hobby)=4 then 6
        when grouping(city, sex, hobby)=0 then 7
        when grouping(city, sex, hobby)=7 then 8
        else 100
    end as o
from hive.db_test.tb_student
group by cube(city, sex, hobby)
order by o, city, sex, hobby

rollup 用法

sql 复制代码
select
    city,
    sex,
    hobby,
    count(1) as cnt,
    case
        when grouping(city, sex, hobby)=3 then 1
        when grouping(city, sex, hobby)=5 then 2
        when grouping(city, sex, hobby)=6 then 3
        when grouping(city, sex, hobby)=1 then 4
        when grouping(city, sex, hobby)=2 then 5
        when grouping(city, sex, hobby)=4 then 6
        when grouping(city, sex, hobby)=0 then 7
        when grouping(city, sex, hobby)=7 then 8
        else 100
    end as o
from hive.db_test.tb_student
group by rollup(city, sex, hobby)
order by o, city, sex, hobby
;

总结:

presto时间函数:

date()类型 表示 年月日

timestamp类型表示 年月日时分秒

eg:timestamp('2024-08-18 22:13:10','%Y-%m-%d %H%i%s')

date_add(unit, value,timestamp)

grouping sets()相当于一个集合 都能根据括号里的内容分组查询到相应的数据

grouping 根据8421码 0表示与该列有关系1表示无关 通过计算数值 查看与列之间分组的关系

cube(city, sex, hobby) 等价于 grouping sets (city, sex, hobby, (city, sex), (city, hobby), (sex, hobby), (city, sex, hobby), ())

rollup (city, sex, name) 等价于 grouping set((city, sex, name), (city, sex), city, ())

相关推荐
直有两条腿34 分钟前
【数据迁移】HBase Bulkload批量加载原理
大数据·数据库·hbase
言之。1 小时前
ClickHouse 数据更新策略深度解析:突变操作与最佳实践
服务器·数据库·clickhouse
白衣鸽子2 小时前
数据库高可用设计的灵魂抉择:CAP权衡
数据库·后端
DokiDoki之父3 小时前
Mybatis—入门 & (配置)SQL提示和日志输出
数据库·sql·mybatis
TDengine (老段)4 小时前
TDengine 数据函数 LN 用户手册
大数据·数据库·物联网·时序数据库·tdengine·涛思数据
机灵猫4 小时前
Redis 在订单系统中的实战应用:防重、限流与库存扣减
数据库·redis·缓存
木易2.04 小时前
从零构建RAG知识库管理系统(二)
数据库·oracle
程序新视界4 小时前
什么是MySQL JOIN查询的驱动表和被驱动表?
数据库·后端·mysql
lingggggaaaa5 小时前
小迪安全v2023学习笔记(一百三十四讲)—— Windows权限提升篇&数据库篇&MySQL&MSSQL&Oracle&自动化项目
java·数据库·windows·笔记·学习·安全·网络安全
小光学长5 小时前
基于Vue的保护动物信息管理系统r7zl6b88 (程序+源码+数据库+调试部署+开发环境)带论文文档1万字以上,文末可获取,系统界面在最后面。
前端·数据库·vue.js