hive和presto的求数组长度函数区别及注意事项

1、任务

获取邮箱字符串'@'后字符串 ,求长度

2、hive & spark-sql 求数组长度的函数 size

sql 复制代码
hive & spark-sql 求数组长度的函数 size


select size(split(email, '@')),split(email, '@'),split(email, '@')[0],split(email, '@')[1]
FROM 
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;

select size(split(email, '@')),split(email, '@'),split(email, '@')[0],split(email, '@')[1]
FROM 
(select 'jack@126.com' as email union select 'tom@126.com.cn' as email) tb_mid;


2	["tom","126.com.cn"]	tom	126.com.cn
2	["jack","126.com"]	jack	126.com
Time taken: 0.723 seconds, Fetched 2 row(s)

3、presto 求数组长度的函数 cardinality

sql 复制代码
presto  求数组长度的函数 cardinality

select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM 
(select 'jack@126.com' as email union select 'tom@126.com.cn' as email) tb_mid;

_col0 |       _col1       | _col2 |   _col3    
-------+-------------------+-------+------------
     2 | [tom, 126.com.cn] | tom   | 126.com.cn 
     2 | [jack, 126.com]   | jack  | 126.com    
(2 rows)


select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM 
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;


Query 20231019_070945_20009_n9u2s failed: line 3:9: Column 'jack@126.com' cannot be resolved
select cardinality(split(email, '@')),split(email, '@'),split(email, '@')[1],split(email, '@')[2]
FROM
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid

4、注意事项

1)、在计算数组长度的时候,hive和presto的函数不同

其中hive的size函数默认数组的下标从0开始

presto的cardinality函数默认数组的下标从1开始

2)、presto 不支持双引号 ,而hive 既支持单引号,也支持双引号

sql 复制代码
presto> SELECT 
     -> email,
     -> (case when cardinality(split(email, '@')) = 2 then split(email, '@')[1] else '' end ) as email_suffix
     -> FROM 
     -> (select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid;
Query 20231016_070153_17958_p9f2s failed: line 5:9: Column 'jack@126.com' cannot be resolved
SELECT
email,
(case when cardinality(split(email, '@')) = 2 then split(email, '@')[1] else '' end ) as email_suffix
FROM
(select "jack@126.com" as email union select "tom@126.com.cn" as email) tb_mid
相关推荐
秦JaccLink2 天前
Hive语句执行顺序详解
数据仓库·hive·hadoop
AI算力网络与通信2 天前
大数据领域 Hive 数据仓库搭建实战
大数据·数据仓库·hive·ai
工业互联网专业4 天前
基于大数据hive的银行信用卡用户的数仓系统的设计与实现_django
大数据·hive·django·毕业设计·源码·课程设计·数仓系统
顧棟5 天前
【Hive实战】hive-testbench的调研
数据仓库·hive·hadoop
emmm...5 天前
hive连不上,报错9000拒绝连接
数据仓库·hive·hadoop
野老杂谈5 天前
Hive SQL 中的时间戳转换详解
hive·hadoop·sql
大数据CLUB5 天前
基于hive和mapreduce的地铁数据分析及可视化_hive作为数据库
大数据·hive·hadoop·分布式·数据分析·mapreduce
嘉禾望岗5035 天前
hive SQL查询与函数
hive·hadoop·sql
不吃饭的猪5 天前
hive表元数据修复脚本
数据仓库·hive·hadoop
白日与明月5 天前
对Hive表进行归档,减少小文件的影响
数据仓库·hive·hadoop