hive中map相关函数总结

目录

hive官方函数解释

hive官网函数大全地址:hive官网函数大全地址

Return Type Name Description
map map(key1, value1, key2, value2, ...) Creates a map with the given key/value pairs.
array map_values(Map<K.V>) Returns an unordered array containing the values of the input map.
array map_keys(Map<K.V>) Returns an unordered array containing the keys of the input map.
map<string,string> str_to_map(text[, delimiter1, delimiter2]) Splits text into key-value pairs using two delimiters. Delimiter1 separates text into K-V pairs, and Delimiter2 splits each K-V pair. Default delimiters are ',' for delimiter1 and ':' for delimiter2.
Tkey,Tvalue explode(MAP<Tkey,Tvalue> m) Explodes a map to multiple rows. Returns a row-set with a two columns (key,value) , one row for each key-value pair from the input map. (As of Hive 0.8.0.).

示例

1、map(key1, value1, key2, value2, ...)

sql 复制代码
SELECT map('name', '张三', 'age', 20, 'gender', '男') AS student;
---结果:
student	
{"age":"20","gender":"男","name":"张三"}

2、map_values(Map<K.V>)

sql 复制代码
SELECT map_keys(map('name', '张三', 'age', 20, 'gender', '男')) AS keys;
---结果:
keys
["name","age","gender"]

3、map_values(Map<K.V>)

sql 复制代码
SELECT map_values(map('name', '张三', 'age', 20, 'gender', '男')) AS values;
---结果:
values	
["张三","20","男"]

4、str_to_map(str, delimiter1, delimiter2)

str_to_map 函数用于将一个字符串转换为 Map 对象。具体来说,str_to_map 函数会将一个由键值对组成的字符串解析成一个 Map 对象,其中键和值之间使用指定的分隔符进行分隔。其中,str 是要转换的字符串,delimiter1 是键值对之间的分隔符,delimiter2 是键和值之间的分隔符。默认情况下,delimiter1 的值是 ',',delimiter2 的值是 ':'。

sql 复制代码
SELECT str_to_map('name:张三,age:20,gender:男', ',', ':') AS student;
---结果:
student	
{"age":"20","gender":"男","name":"张三"}

SELECT str_to_map('name=张三,age=20,gender=男', ',', '=') AS student;
---结果:
student	
{"age":"20","gender":"男","name":"张三"}

5、explode (map)

sql 复制代码
select explode(map('A',10,'B',20,'C',30));
select explode(map('A',10,'B',20,'C',30)) as (key,value);
select tf.* from (select 0) t lateral view explode(map('A',10,'B',20,'C',30)) tf;
select tf.* from (select 0) t lateral view explode(map('A',10,'B',20,'C',30)) tf as key,value;
---上述四个结果均为:
key     value
A       10	
B       20	
C       30

实战

给出一组学生数据,有名字,课程,等级,分数等字段,现在求每门课的情况,包含平均成绩,及这门课包含哪些学生及学生的等级

sql 复制代码
with stud as
( select  'zhang3' as name ,'优' as grade  ,'math' as course ,'88' as score  
  union all 
  select  'li4' as name ,'良' as grade  ,'math' as course ,'72' as score
    union all 
  select  'zhao6' as name ,'差' as grade  ,'math' as course ,'44' as score
    union all 
  select  'wang5' as name ,'优' as grade  ,'chinese' as course ,'80' as score
    union all 
  select  'zhao6' as name ,'优' as grade  ,'chinese' as course ,'55' as score
    union all 
  select  'tian7' as name ,'优' as grade  ,'chinese' as course ,'75' as score
)

--sql1
select course, collect_set(concat(name,':',grade)) as collect , avg(score) from stud group by course;
---结果:
course             collect                                             avg(score)	
math        ["li4:良","zhao6:差","zhang3:优"]                           68.0
chinese     ["wang5:优","tian7:优","zhao6:优"]                          70.0
----sql2
select course, concat_ws(',',collect_set(concat(name,':',grade))) as strings , avg(score) from stud group by course;
---结果:
course                      strings                                        avg(score)
math             li4:良,zhao6:差,zhang3:优                                  68.0
chinese          wang5:优,tian7:优,zhao6:优                                 70.0
----sql3
select course, str_to_map(concat_ws(',',collect_set(concat(name,':',grade))),',',':') as maps , avg(score) from stud group by course;
---结果:
course                               maps                              avg(score)	
math                 {"li4":"良","zhang3":"优","zhao6":"差"}             68.0
chinese              {"tian7":"优","wang5":"优","zhao6":"优"}            70.0

注意:

第一种sql,collect 字段的类型是array;第二种sql,strings字段的类型是string;第三种sql,maps字段的类型是map;

问题来了,能否在第二种的基础上,实现第一种和第三种的结果,且字段类型是string;

下面实现第二种转化为第三种,实际上就是map格式转换成json字符串;

sql 复制代码
with stud as
( select  'zhang3' as name ,'优' as grade  ,'math' as course ,'88' as score  
  union all 
  select  'li4' as name ,'良' as grade  ,'math' as course ,'72' as score
    union all 
  select  'zhao6' as name ,'差' as grade  ,'math' as course ,'44' as score
    union all 
  select  'wang5' as name ,'优' as grade  ,'chinese' as course ,'80' as score
    union all 
  select  'zhao6' as name ,'优' as grade  ,'chinese' as course ,'55' as score
    union all 
  select  'tian7' as name ,'优' as grade  ,'chinese' as course ,'75' as score
)

select 
course
,concat('{"',string2,'"}') as string3
from  
(select 
course
,regexp_replace(string1,'\\,','\\"\\,\\"') as string2
from  
(
select 
    course,
    concat_ws(',', collect_list(concat_ws('":"', k,v) ) ) as string1
from (
select course, str_to_map(concat_ws(',',collect_set(concat(name,':',grade))),',',':') as maps , avg(score) 
from stud group by course
)test_map_1
lateral view outer explode(maps) kv as k,v
group by course
) tt 
) tm 

---结果:
course                               string3                            	
math                 {"li4":"良","zhang3":"优","zhao6":"差"}           
chinese              {"tian7":"优","wang5":"优","zhao6":"优"}        
相关推荐
IT毕设梦工厂1 小时前
大数据毕业设计选题推荐-基于大数据的国家基站整点数据分析系统-Hadoop-Spark-数据可视化-BigData
大数据·hadoop·spark·毕业设计·源码·数据可视化
项目題供诗2 小时前
Hadoop(八)
大数据·hadoop·分布式
计算机毕业设计木哥4 小时前
Python毕业设计推荐:基于Django的饮食计划推荐与交流分享平台 饮食健康系统 健康食谱计划系统
开发语言·hadoop·spring boot·后端·python·django·课程设计
元媛媛11 小时前
数据仓库概要
数据仓库
cg.family11 小时前
Doris 数据仓库例子
数据仓库·doris
TDengine (老段)11 小时前
从 ETL 到 Agentic AI:工业数据管理变革与 TDengine IDMP 的治理之道
数据库·数据仓库·人工智能·物联网·时序数据库·etl·tdengine
BYSJMG16 小时前
计算机毕设大数据方向:基于Spark+Hadoop的餐饮外卖平台数据分析系统【源码+文档+调试】
大数据·hadoop·分布式·python·spark·django·课程设计
IT研究室1 天前
大数据毕业设计选题推荐-基于大数据的宫颈癌风险因素分析与可视化系统-Spark-Hadoop-Bigdata
大数据·hadoop·spark·毕业设计·源码·数据可视化·bigdata
镜舟科技1 天前
告别 Hadoop,拥抱 StarRocks!政采云数据平台升级之路
大数据·starrocks·数据仓库·hadoop·存算分离
毕设源码-赖学姐1 天前
【开题答辩全过程】以 基于Hadoop电商数据的可视化分析为例,包含答辩的问题和答案
大数据·hadoop·分布式