openEuler24.03 LTS下安装Hive3

Hadoop_Liang2025-03-23 16:13

目录

前提条件

安装MySQL

卸载原有mysql及mariadb

下载mysql

解压mysql

安装mysql

启动mysql服务

开机自启动mysql服务

登录mysql

修改mysql密码

远程连接mysql

安装Hive

下载安装包

解压

设置环境变量

解决日志包冲突

将mysql驱动拷贝到lib目录

配置Hive

创建hive-site.xml

调整输出日志级别

初始化Hive元数据库

修改元数据库字符集

进入Hive命令行

简单使用Hive

退出Hive命令行

前提条件

Hive依赖于Hadoop，所以需要先安装好Hadoop，可参考：openEuler24.03 LTS下安装Hadoop3完全分布式

安装MySQL

Hive的元数据存储在关系型数据库中，这里使用的关系型数据库为MySQL，所以需要先安装好MySQL。

在node2安装MySQL。

卸载原有mysql及mariadb

复制代码 
sudo systemctl stop mysql mysqld 2>/dev/null
sudo rpm -qa | grep -i 'mysql\|mariadb' | xargs -n1 sudo rpm -e --nodeps 2>/dev/null
sudo rm -rf /var/lib/mysql /var/log/mysqld.log /usr/lib64/mysql /etc/my.cnf /usr/my.cnf

下载mysql

这里下载的mysql版本为mysql8.4.2，如果下载较旧的mysql版本可能与openEuler24.03不兼容。

复制代码 
cd /opt/software
wget https://downloads.mysql.com/archives/get/p/23/file/mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar

注意：如果命令行下载较慢，可以直接使用浏览器访问https链接地址下载，再上传安装文件到Linux的安装包存放目录，例如：/opt/software。

解压mysql

复制代码 
# 创建mysql目录
[liang@node2 software]$ mkdir mysql
​
#解压
[liang@node2 software]$ tar -xvf mysql-8.4.2-1.el9.x86_64.rpm-bundle.tar -C mysql

删除不必要的rpm包

复制代码 
[liang@node2 software]$ cd mysql
[liang@node2 mysql]$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm
mysql-community-client-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm
mysql-community-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-debugsource-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-compat-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-libs-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debug-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-server-debuginfo-8.4.2-1.el9.x86_64.rpm
mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-test-debuginfo-8.4.2-1.el9.x86_64.rpm
[liang@node2 mysql]$ rm -f *debug*
[liang@node2 mysql]$ ls
mysql-community-client-8.4.2-1.el9.x86_64.rpm          mysql-community-libs-8.4.2-1.el9.x86_64.rpm
mysql-community-client-plugins-8.4.2-1.el9.x86_64.rpm  mysql-community-libs-compat-8.4.2-1.el9.x86_64.rpm
mysql-community-common-8.4.2-1.el9.x86_64.rpm          mysql-community-server-8.4.2-1.el9.x86_64.rpm
mysql-community-devel-8.4.2-1.el9.x86_64.rpm           mysql-community-test-8.4.2-1.el9.x86_64.rpm
mysql-community-icu-data-files-8.4.2-1.el9.x86_64.rpm

安装mysql

安装命令

复制代码 
[liang@node2 mysql]$ sudo rpm -ivh *.rpm --force --nodeps

安装过程

复制代码 
警告：mysql-community-client-8.4.2-1.el9.x86_64.rpm: 头 V4 RSA/SHA256 Signature, 密钥 ID a8d3785c: NOKEY
Verifying...                          ################################# [100%]
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-common-8.4.2-1.el################################# [ 11%]
   2:mysql-community-client-plugins-8.################################# [ 22%]
   3:mysql-community-libs-8.4.2-1.el9 ################################# [ 33%]
   4:mysql-community-client-8.4.2-1.el################################# [ 44%]
   5:mysql-community-icu-data-files-8.################################# [ 56%]
   6:mysql-community-server-8.4.2-1.el################################# [ 67%]
   7:mysql-community-test-8.4.2-1.el9 ################################# [ 78%]
   8:mysql-community-devel-8.4.2-1.el9################################# [ 89%]
   9:mysql-community-libs-compat-8.4.2################################# [100%]
/usr/lib/tmpfiles.d/dbus.conf:13: Line references path below legacy directory /var/run/, updating /var/run/dbus/containers → /run/dbus/containers; please update the tmpfiles.d/ drop-in file accordingly.
[liang@node2 mysql]$

启动mysql服务

复制代码 
[liang@node2 mysql]$ sudo systemctl start mysqld

开机自启动mysql服务

复制代码 
[liang@node2 mysql]$ sudo systemctl enable mysqld

登录mysql

查看临时密码

复制代码 
[liang@node2 mysql]$ sudo grep 'temporary password' /var/log/mysqld.log

输出如下

复制代码 
2025-03-19T06:43:36.958044Z 6 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: tFggwGvsx8=j

这里查看到的临时密码为：tFggwGvsx8=j

注意：查看到的临时密码可能不一样，请使用实际查到的临时密码登录mysql。

登录mysql

复制代码 
[liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 8
Server version: 8.4.2
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> exit;
Bye
[liang@node2 mysql]$

修改mysql密码

修改密码策略

若在内网环境使用MySQL，想使用简单密码，则需要设置密码策略，否则，不需要设置密码策略。

复制代码 
[liang@node2 mysql]$ sudo vim /etc/my.cnf

在[mysqld]下面添加如下语句

复制代码 
validate_password.length=4
validate_password.policy=0

重启mysql

复制代码 
[liang@node2 mysql]$ sudo systemctl restart mysqld

登录mysql并修改密码

复制代码 
[liang@node2 mysql]$ mysql -uroot -p'tFggwGvsx8=j'
​
mysql> set password='000000';
Query OK, 0 rows affected (0.00 sec)
​
mysql> update mysql.user set host='%' where user='root';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0
​
mysql> alter user 'root'@'%' identified by '000000';
Query OK, 0 rows affected (0.01 sec)
​
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
​
mysql> exit;
Bye

使用新密码登录

复制代码 
[liang@node2 mysql]$ mysql -uroot -p000000
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9
Server version: 8.4.2 MySQL Community Server - GPL
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> exit;
Bye
[liang@node2 mysql]$

远程连接mysql

使用navicat等工具，通过ip及端口号远程连接

安装Hive

在node2安装Hive

下载安装包

浏览器访问以下链接下载Hive安装包

复制代码 
https://archive.apache.org/dist/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz

上传Hive安装包到Linux /opt/software目录

复制代码 
[liang@node2 software]$ ls | grep hive
apache-hive-3.1.3-bin.tar.gz

解压

解压

复制代码 
[liang@node2 software]$ tar -zxvf /opt/software/apache-hive-3.1.3-bin.tar.gz -C /opt/module/

重命名

复制代码 
[liang@node2 software]$ ls /opt/module/
apache-hive-3.1.3-bin  hadoop-3.3.4  jdk1.8.0_271
​
[liang@node2 software]$ mv /opt/module/apache-hive-3.1.3-bin /opt/module/hive-3.1.3

设置环境变量

复制代码 
[liang@node2 software]$ sudo vim /etc/profile.d/my_env.sh

末尾添加如下内容

bash 复制代码 
#HIVE_HOME
export HIVE_HOME=/opt/module/hive-3.1.3
export PATH=$PATH:$HIVE_HOME/bin

让环境变量生效

复制代码 
[liang@node2 software]$ source /etc/profile

解决日志包冲突

解决日志Jar包依赖与Hadoop日志Jar版本不同冲突，重命名hive的日志包为其他名字，这样就只有hadoop的日志Jar包生效，冲突解决。

复制代码 
[liang@node2 software]$ cd $HIVE_HOME/lib/
​
[liang@node2 lib]$ ls | grep slf4j
log4j-slf4j-impl-2.17.1.jar
​
[liang@node2 lib]$ mv log4j-slf4j-impl-2.17.1.jar log4j-slf4j-impl-2.17.1.jar.bak
​

将mysql驱动拷贝到lib目录

浏览器访问如下地址，下载mysql驱动包

复制代码 
https://mvnrepository.com/artifact/com.mysql/mysql-connector-j/8.4.0

点击jar下载，并上传jar包文件到Linux /opt/software目录

复制代码 
[liang@node2 lib]$ ls /opt/software | grep connect
mysql-connector-j-8.4.0.jar

将jar文件复制到HIVE_HOME的lib目录

复制代码 
[liang@node2 lib]$ cp /opt/software/mysql-connector-j-8.4.0.jar $HIVE_HOME/lib/
[liang@node2 lib]$ ls | grep connect
mysql-connector-j-8.4.0.jar

配置Hive

切换到Hive配置目录，查看配置文件

复制代码 
[liang@node2 lib]$ cd $HIVE_HOME/conf
[liang@node2 conf]$ ls
beeline-log4j2.properties.template    ivysettings.xml
hive-default.xml.template             llap-cli-log4j2.properties.template
hive-env.sh.template                  llap-daemon-log4j2.properties.template
hive-exec-log4j2.properties.template  parquet-logging.properties
hive-log4j2.properties.template
创建hive-site.xml
复制代码 
[liang@node2 conf]$ vim hive-site.xml

配置内容

XML 复制代码 
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <!--配置Hive保存元数据信息所需的 MySQL URL地址-->
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://node2:3306/metastore?useSSL=false&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;allowPublicKeyRetrieval=true</value>
    </property>

    <!--配置Hive连接MySQL的驱动全类名-->
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.cj.jdbc.Driver</value>
    </property>

    <!--配置Hive连接MySQL的用户名 -->
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>

    <!--配置Hive连接MySQL的密码 -->
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>000000</value>
    </property>

    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
    </property>

    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>

    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>

    <property>
        <name>hive.server2.thrift.bind.host</name>
        <value>node2</value>
    </property>

    <property>
        <name>hive.metastore.event.db.notification.api.auth</name>
        <value>false</value>
    </property>
    
    <property>
        <name>hive.cli.print.header</name>
        <value>true</value>
    </property>

    <property>
        <name>hive.cli.print.current.db</name>
        <value>true</value>
    </property>
</configuration>

配置文件中连接nod2机器mysql的metastore数据库，所以需要先把metastore数据库创建出来。

复制代码 
[liang@node2 conf]$ mysql -uroot -p000000
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 13
Server version: 8.4.2 MySQL Community Server - GPL
​
Copyright (c) 2000, 2024, Oracle and/or its affiliates.
​
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
​
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
​
mysql> create database metastore;
Query OK, 1 row affected (0.01 sec)
​
mysql> exit;
Bye
[liang@node2 conf]$
调整输出日志级别

默认日志级别为INFO，执行HQL语句过程会输出太多日志，将Hive输出日志级别改为WARN减少日志输出。

复制代码 
cd $HIVE_HOME/conf

执行如下命令新建log4j.properties

XML 复制代码 
cat > log4j.properties <<EOL
log4j.rootLogger=WARN, CA
 
log4j.appender.CA=org.apache.log4j.ConsoleAppender
 
log4j.appender.CA.layout=org.apache.log4j.PatternLayout
log4j.appender.CA.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
EOL

初始化Hive元数据库

复制代码 
[liang@node2 conf]$ schematool -initSchema -dbType mysql -verbose
复制代码 
末尾部分输出内容如下
​
0: jdbc:mysql://node2:3306/metastore> !closeall
Closing: 0: jdbc:mysql://node2:3306/metastore?useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
beeline>
beeline> Initialization script completed
schemaTool completed

看到schemaTool completed输出，说明初始化成功。

初始化本质是在元数据库里创建出相关的表及初始化相关表数据，可以到mysql的metastore数据库查看生成的表数据。

修改元数据库字符集

复制代码 
[liang@node2 conf]$ mysql -uroot -p000000
...
mysql> use metastore;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
​
Database changed
mysql> alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;
Query OK, 0 rows affected, 1 warning (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 1
​
mysql> alter table TABLE_PARAMS modify column PARAM_VALUE mediumtext character set utf8;
Query OK, 0 rows affected, 1 warning (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 1
mysql> quit;
Bye
[liang@node2 conf]$
​

进入Hive命令行

进入Hive命令行之前，需要启动hadoop，如果还没启动，需要先启动hadoop

复制代码 
# 分别启动
[liang@node2 conf]$ start-dfs.sh
[liang@node3 conf]$ start-yarn.sh
​
# 或者使用脚本启动
[liang@node2 conf]$ hdp.sh start

进入hive命令行

复制代码 
[liang@node2 conf]$ hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/liang/bin:/opt/module/jdk1.8.0_271/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/hive-3.1.3/bin)
Hive Session ID = a2b0c24d-7c45-4e95-9c86-1108b2606f3b
​
Logging initialized using configuration in jar:file:/opt/module/hive-3.1.3/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 5d3c533a-79e3-4690-b54c-1006d18d7aca
hive (default)>
​

简单使用Hive

查看数据库

复制代码 
hive (default)> show databases;
OK
database_name
default
Time taken: 0.609 seconds, Fetched: 1 row(s)

查看数据表

复制代码 
hive (default)> show tables;
OK
tab_name
Time taken: 0.051 seconds

创建表

复制代码 
hive (default)> CREATE TABLE IF NOT EXISTS test_students (
    id INT,
    name STRING,
    age INT
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',';

插入数据

复制代码 
hive (default)> INSERT INTO TABLE test_students VALUES (103, 'Bob', 21);

查看表数据

复制代码 
hive (default)> select * from test_students;

退出Hive命令行

复制代码 
hive (default)> quit;
[liang@node2 conf]$

完成！enjoy it！

相关推荐
嵩山小老虎9 分钟前
Windows 10/11 安装 WSL2 并配置 VSCode 开发环境（C 语言 / Linux API 适用）
linux·windows·vscode
Fleshy数模33 分钟前
CentOS7 安装配置 MySQL5.7 完整教程（本地虚拟机学习版）
linux·mysql·centos
a413244738 分钟前
ubuntu 25 安装vllm
linux·服务器·ubuntu·vllm
az44yao1 小时前
mysql 创建事件 每天17点执行一个存储过程
mysql
一只自律的鸡2 小时前
【Linux驱动】bug处理 ens33找不到IP
linux·运维·bug
17（无规则自律）3 小时前
【CSAPP 读书笔记】第二章：信息的表示和处理
linux·嵌入式硬件·考研·高考
!chen3 小时前
linux服务器静默安装Oracle26ai
linux·运维·服务器
秦老师Q3 小时前
php入门教程(超详细，一篇就够了!!!)
开发语言·mysql·php·db
REDcker3 小时前
Linux 文件描述符与 Socket 选项操作详解
linux·运维·网络
蒹葭玉树3 小时前
【C++上岸】C++常见面试题目--操作系统篇（第二十八期）
linux·c++·面试
热门推荐
01GitHub 镜像站点02Clawdbot 中文汉化版 接入微信、飞书032026美赛A题智能手机电池续航时间预测的连续时间数学模型04OpenClaw部署与配置教程：在Mac mini上接入国产大模型与飞书05OpenCode 入门教程：介绍 · 安装 · 配置第三方 API (如 Claude)062025 年大语言模型发展回顾：关键突破、意外转折与 2026 年展望07UV安装并设置国内源08Linux下V2Ray安装配置指南09Claude Code + GLM4.7 避坑指南：解决 Unable to connect to Anthropic services10【Milvus】向量数据库pymilvus使用教程