openEuler24.03 LTS下安装Hive3

Hadoop_Liang2025-03-23 16:13

目录

前提条件

安装MySQL

卸载原有mysql及mariadb

下载mysql

解压mysql

安装mysql

启动mysql服务

开机自启动mysql服务

登录mysql

修改mysql密码

远程连接mysql

安装Hive

下载安装包

解压

设置环境变量

解决日志包冲突

将mysql驱动拷贝到lib目录

配置Hive

创建hive-site.xml

调整输出日志级别

初始化Hive元数据库

修改元数据库字符集

进入Hive命令行

简单使用Hive

退出Hive命令行

前提条件

Hive依赖于Hadoop，所以需要先安装好Hadoop，可参考：openEuler24.03 LTS下安装Hadoop3完全分布式

安装MySQL

Hive的元数据存储在关系型数据库中，这里使用的关系型数据库为MySQL，所以需要先安装好MySQL。

在node2安装MySQL。

卸载原有mysql及mariadb

复制代码 
sudo systemctl stop mysql mysqld 2>/dev/null
sudo rpm -qa | grep -i 'mysql\|mariadb' | xargs -n1 sudo rpm -e --nodeps 2>/dev/null
sudo rm -rf /var/lib/mysql /var/log/mysqld.log /usr/lib64/mysql /etc/my.cnf /usr/my.cnf

下载mysql

这里下载的mysql版本为mysql8.4.2，如果下载较旧的mysql版本可能与openEuler24.03不兼容。

复制代码 
cd /opt/software
wget https://downloads.mysql.com/archives/get/p/23/file/mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar

注意：如果命令行下载较慢，可以直接使用浏览器访问https链接地址下载，再上传安装文件到Linux的安装包存放目录，例如：/opt/software。

解压mysql

复制代码 
# 创建mysql目录
[liang@node2 software]$ mkdir mysql
​
#解压
[liang@node2 software]$ tar -xvf mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar -C mysql

删除不必要的rpm包

复制代码 
[liang@node2 software]$ cd mysql
[liang@node2 mysql]$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm
mysql-community-client-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm
mysql-community-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-debugsource-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-test-debuginfo-8.4.2-1.el9.x86_64.rpm
[liang@node2 mysql]$ rm -f *debug*
[liang@node2 mysql]$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm          mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm  mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm          mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm           mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm

安装mysql

安装命令

复制代码 
[liang@node2 mysql]$ sudo rpm -ivh *.rpm --force --nodeps

安装过程

复制代码 
警告：mysql-community-client-8.4.2-1.el9.x86_64.rpm: 头 V4 RSA/SHA256 Signature, 密钥 ID a8d3785c: NOKEY
Verifying...                          ################################# [100%]
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-common-8.4.2-1.el################################# [ 11%]
   2:mysql-community-client-plugins-8.################################# [ 22%]
   3:mysql-community-libs-8.4.2-1.el9 ################################# [ 33%]
   4:mysql-community-client-8.4.2-1.el################################# [ 44%]
   5:mysql-community-icu-data-files-8.################################# [ 56%]
   6:mysql-community-server-8.4.2-1.el################################# [ 67%]
   7:mysql-community-test-8.4.2-1.el9 ################################# [ 78%]
   8:mysql-community-devel-8.4.2-1.el9################################# [ 89%]
   9:mysql-community-libs-compat-8.4.2################################# [100%]
/usr/lib/tmpfiles.d/dbus.conf:13: Line references path below legacy directory /var/run/, updating /var/run/dbus/containers → /run/dbus/containers; please update the tmpfiles.d/ drop-in file accordingly.
[liang@node2 mysql]$

启动mysql服务

复制代码 
[liang@node2 mysql]$ sudo systemctl start mysqld

开机自启动mysql服务

复制代码 
[liang@node2 mysql]$ sudo systemctl enable mysqld

登录mysql

查看临时密码

复制代码 
[liang@node2 mysql]$ sudo grep 'temporary password' /var/log/mysqld.log

输出如下

复制代码 
2025-03-19T06:43:36.958044Z 6 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: tFggwGvsx8=j

这里查看到的临时密码为：tFggwGvsx8=j

注意：查看到的临时密码可能不一样，请使用实际查到的临时密码登录mysql。

登录mysql

复制代码 
[liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.4.2
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> exit;
Bye
[liang@node2 mysql]$

修改mysql密码

修改密码策略

若在内网环境使用MySQL，想使用简单密码，则需要设置密码策略，否则，不需要设置密码策略。

复制代码 
[liang@node2 mysql]$ sudo vim /etc/my.cnf

在[mysqld]下面添加如下语句

复制代码 
validate_password.length=4
validate_password.policy=0

重启mysql

复制代码 
[liang@node2 mysql]$ sudo systemctl restart mysqld

登录mysql并修改密码

复制代码 
[liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
​
mysql> set password='000000';
Query OK, 0 rows affected (0.00 sec)
​
mysql> update mysql.user set host='%' where user='root';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0
​
mysql> alter user 'root'@'%' identified by '000000';
Query OK, 0 rows affected (0.01 sec)
​
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
​
mysql> exit;
Bye

使用新密码登录

复制代码 
[liang@node2 mysql]$ mysql -uroot -p000000
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.4.2 MySQL Community Server - GPL
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> exit;
Bye
[liang@node2 mysql]$

远程连接mysql

使用navicat等工具，通过ip及端口号远程连接

安装Hive

在node2安装Hive

下载安装包

浏览器访问以下链接下载Hive安装包

复制代码 
https://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz

上传Hive安装包到Linux /opt/software目录

复制代码 
[liang@node2 software]$ ls | grep hive
apache-hive-3.1.3-bin.tar.gz

解压

解压

复制代码 
[liang@node2 software]$ tar -zxvf /opt/software/apache-hive-3.1.3-bin.tar.gz -C /opt/module/

重命名

复制代码 
[liang@node2 software]$ ls /opt/module/
apache-hive-3.1.3-bin  hadoop-3.3.4  jdk1.8.0_271
​
[liang@node2 software]$ mv /opt/module/apache-hive-3.1.3-bin /opt/module/hive-3.1.3

设置环境变量

复制代码 
[liang@node2 software]$ sudo vim /etc/profile.d/my_env.sh

末尾添加如下内容

bash 复制代码 
#HIVE_HOME
export HIVE_HOME=/opt/module/hive-3.1.3
export PATH=$PATH:$HIVE_HOME/bin

让环境变量生效

复制代码 
[liang@node2 software]$ source /etc/profile

解决日志包冲突

解决日志Jar包依赖与Hadoop日志Jar版本不同冲突，重命名hive的日志包为其他名字，这样就只有hadoop的日志Jar包生效，冲突解决。

复制代码 
[liang@node2 software]$ cd $HIVE_HOME/lib/
​
[liang@node2 lib]$ ls | grep slf4j
log4j-slf4j-impl-2.17.1.jar
​
[liang@node2 lib]$ mv log4j-slf4j-impl-2.17.1.jar log4j-slf4j-impl-2.17.1.jar.bak
​

将mysql驱动拷贝到lib目录

浏览器访问如下地址，下载mysql驱动包

复制代码 
https://mvnrepository.com/artifact/com.mysql/mysql-connector-j/8.4.0

点击jar下载，并上传jar包文件到Linux /opt/software目录

复制代码 
[liang@node2 lib]$ ls /opt/software | grep connect
mysql-connector-j-8.4.0.jar

将jar文件复制到HIVE_HOME的lib目录

复制代码 
[liang@node2 lib]$ cp /opt/software/mysql-connector-j-8.4.0.jar $HIVE_HOME/lib/
[liang@node2 lib]$ ls | grep connect
mysql-connector-j-8.4.0.jar

配置Hive

切换到Hive配置目录，查看配置文件

复制代码 
[liang@node2 lib]$ cd $HIVE_HOME/conf
[liang@node2 conf]$ ls
beeline-log4j2.properties.template    ivysettings.xml
hive-default.xml.template             llap-cli-log4j2.properties.template
hive-env.sh.template                  llap-daemon-log4j2.properties.template
hive-exec-log4j2.properties.template  parquet-logging.properties
hive-log4j2.properties.template
创建hive-site.xml
复制代码 
[liang@node2 conf]$ vim hive-site.xml

配置内容

XML 复制代码 
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <!--配置Hive保存元数据信息所需的 MySQL URL地址-->
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://node2:3306/metastore?useSSL=false&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;allowPublicKeyRetrieval=true</value>
    </property>

    <!--配置Hive连接MySQL的驱动全类名-->
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.cj.jdbc.Driver</value>
    </property>

    <!--配置Hive连接MySQL的用户名 -->
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>

    <!--配置Hive连接MySQL的密码 -->
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>000000</value>
    </property>

    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
    </property>

    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>

    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>

    <property>
        <name>hive.server2.thrift.bind.host</name>
        <value>node2</value>
    </property>

    <property>
        <name>hive.metastore.event.db.notification.api.auth</name>
        <value>false</value>
    </property>
    
    <property>
        <name>hive.cli.print.header</name>
        <value>true</value>
    </property>

    <property>
        <name>hive.cli.print.current.db</name>
        <value>true</value>
    </property>
</configuration>

配置文件中连接nod2机器mysql的metastore数据库，所以需要先把metastore数据库创建出来。

复制代码 
[liang@node2 conf]$ mysql -uroot -p000000
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 8.4.2 MySQL Community Server - GPL
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> create database metastore;
Query OK, 1 row affected (0.01 sec)
​
mysql> exit;
Bye
[liang@node2 conf]$
调整输出日志级别

默认日志级别为INFO，执行HQL语句过程会输出太多日志，将Hive输出日志级别改为WARN减少日志输出。

复制代码 
cd $HIVE_HOME/conf

执行如下命令新建log4j.properties

XML 复制代码 
cat > log4j.properties <<EOL
log4j.rootLogger=WARN, CA
 
log4j.appender.CA=org.apache.log4j.ConsoleAppender
 
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
EOL

初始化Hive元数据库

复制代码 
[liang@node2 conf]$ schematool -initSchema -dbType mysql -verbose
复制代码 
末尾部分输出内容如下
​
0: jdbc:mysql://node2:3306/metastore> !closeall
Closing: 0: jdbc:mysql://node2:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
beeline>
beeline> Initialization script completed
schemaTool completed

看到schemaTool completed输出，说明初始化成功。

初始化本质是在元数据库里创建出相关的表及初始化相关表数据，可以到mysql的metastore数据库查看生成的表数据。

修改元数据库字符集

复制代码 
[liang@node2 conf]$ mysql -uroot -p000000
...
mysql> use metastore;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
​
Database changed
mysql> alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
Query OK, 0 rows affected, 1 warning (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 1
​
mysql> alter table TABLE_PARAMS modify column PARAM_VALUE mediumtext character set utf8;
Query OK, 0 rows affected, 1 warning (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 1
mysql> quit;
Bye
[liang@node2 conf]$
​

进入Hive命令行

进入Hive命令行之前，需要启动hadoop，如果还没启动，需要先启动hadoop

复制代码 
# 分别启动
[liang@node2 conf]$ start-dfs.sh
[liang@node3 conf]$ start-yarn.sh
​
# 或者使用脚本启动
[liang@node2 conf]$ hdp.sh start

进入hive命令行

复制代码 
[liang@node2 conf]$ hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/hive-3.1.3/bin)
Hive Session ID = a2b0c24d-7c45-4e95-9c86-1108b2606f3b
​
Logging initialized using configuration in jar:file:/opt/module/hive-3.1.3/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 5d3c533a-79e3-4690-b54c-1006d18d7aca
hive (default)>
​

简单使用Hive

查看数据库

复制代码 
hive (default)> show databases;
OK
database_name
default
Time taken: 0.609 seconds, Fetched: 1 row(s)

查看数据表

复制代码 
hive (default)> show tables;
OK
tab_name
Time taken: 0.051 seconds

创建表

复制代码 
hive (default)> CREATE TABLE IF NOT EXISTS test_students (
    id INT,
    name STRING,
    age INT
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';

插入数据

复制代码 
hive (default)> INSERT INTO TABLE test_students VALUES (103, 'Bob', 21);

查看表数据

复制代码 
hive (default)> select * from test_students;

退出Hive命令行

复制代码 
hive (default)> quit;
[liang@node2 conf]$

完成！enjoy it！

相关推荐
述雾学java2 分钟前
数据库分类、存储引擎、介绍、Mysql、SQL分类
sql·mysql·java核心基础
Wo3Shi4七4 分钟前
MySQL SELECT 语句和使用 WHERE 子句过滤数据
数据库·mysql
鹿九丸7 分钟前
多线程编程
linux·服务器·c语言·网络·c++
华唐10 分钟前
大话MySQL原理
mysql·面试
GalaxyPokemon18 分钟前
LINUX基础IO [七] - 文件缓冲区的深入理解
linux·运维·算法
什么半岛铁盒29 分钟前
Linux进程替身术：深入解析进程程序替换机制
linux·运维·服务器
脑子慢且灵31 分钟前
计算机操作系统处理机调度（1）
linux·运维·服务器·开发语言·windows·centos
Mr_liu_66641 分钟前
Hostapd2.11解析笔记_nl80211接口交互流程_消息收发细节解析
java·linux·笔记
J-v-a-v1 小时前
MySQL中的锁（全局锁、表锁和行锁）
数据库·mysql
热门推荐
01LLaMA-Factory DeepSeek-R1 模型 微调基础教程02RAG 实践- Ollama+RagFlow 部署本地知识库03DeepSeek各版本说明与优缺点分析04本地化部署AI知识库：基于Ollama+DeepSeek+AnythingLLM保姆级教程05如何在WPS和Word/Excel中直接使用DeepSeek功能06从零安装 LLaMA-Factory 微调 Qwen 大模型成功及所有的坑07FreeCAD傻瓜教程之创建参数化几何图形-螺旋体、平面、球体、椭球体、圆柱体、圆锥体、棱柱、椭圆08本地部署DeepSeek教程（Mac版本）09DeepSeek R1本地化部署 Ollama + Chatbox 打造最强 AI 工具10visual studio code 开发STM32步骤