【已解决】Atlas 导入 Hive 元数据,执行 import-hive.sh 报错

部署完 Atlas 之后,尝试导入 Hive 元数据,遇到了一些错误,特此记录一下,方便你我他。

执行 import-hive.sh 报错

复制代码
[omc@hadoop102 apache-atlas-2.2.0]$ hook-bin/import-hive.sh
Using Hive configuration directory [/opt/module/hive/conf]
Log file for import is /opt/module/apache-atlas-2.2.0/logs/import-hive.log
2023-12-17T15:48:37,827 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - delete non existing flag : false
2023-12-17T15:48:37,846 INFO [main] org.apache.atlas.ApplicationProperties - Looking for atlas-application.properties in classpath
2023-12-17T15:48:37,847 INFO [main] org.apache.atlas.ApplicationProperties - Loading atlas-application.properties from file:/opt/module/hive/conf/atlas-application.properties
2023-12-17T15:48:37,883 INFO [main] org.apache.atlas.ApplicationProperties - Using graphdb backend 'janus'
2023-12-17T15:48:37,883 INFO [main] org.apache.atlas.ApplicationProperties - Using storage backend 'hbase2'
2023-12-17T15:48:37,883 INFO [main] org.apache.atlas.ApplicationProperties - Using index backend 'solr'
2023-12-17T15:48:37,883 INFO [main] org.apache.atlas.ApplicationProperties - Atlas is running in MODE: PROD.
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Setting solr.wait-searcher property 'false'
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Setting index.search.map-name property 'false'
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Setting atlas.graph.index.search.max-result-set-size = 150
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Setting atlas.graph.index.search.solr.wait-searcher = false
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache = true
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-clean-wait = 20
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.db-cache-size = 0.5
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-cache-size = 15000
2023-12-17T15:48:37,887 INFO [main] org.apache.atlas.ApplicationProperties - Property (set to default) atlas.graph.cache.tx-dirty-size = 120
Enter username for atlas :- admin
Enter password for atlas :-
2023-12-17T15:48:44,649 INFO [main] org.apache.atlas.AtlasBaseClient - Client has only one service URL, will use that for all actions: http://192.168.10.102:21000
2023-12-17T15:48:44,690 INFO [main] org.apache.hadoop.hive.conf.HiveConf - Found configuration file file:/opt/module/hive/conf/hive-site.xml
2023-12-17T15:48:45,786 WARN [main] org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-12-17T15:48:45,884 INFO [main] org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Trying to connect to metastore with URI thrift://hadoop102:9083
2023-12-17T15:48:45,921 INFO [main] org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Opened a connection to metastore, current connections: 1
2023-12-17T15:48:46,013 INFO [main] org.apache.hadoop.hive.metastore.HiveMetaStoreClient - Connected to metastore.
2023-12-17T15:48:46,013 INFO [main] org.apache.hadoop.hive.metastore.RetryingMetaStoreClient - RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=omc (auth:SIMPLE) retries=1 delay=1 lifetime=0
2023-12-17T15:48:46,291 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Importing Hive metadata
2023-12-17T15:48:46,310 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Found 7 databases
2023-12-17T15:48:46,439 INFO [main] org.apache.atlas.AtlasBaseClient - method=GET path=api/atlas/v2/entity/uniqueAttribute/type/ contentType=application/json; charset=UTF-8 accept=application/json status=200
2023-12-17T15:48:46,646 INFO [main] org.apache.atlas.hive.bridge.HiveMetaStoreBridge - Database default is already registered - id=eb2c0a75-1984-404c-8cdb-0dcbfc3e74ae. Updating it.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/HdfsConfiguration
        at org.apache.atlas.utils.HdfsNameServiceResolver.init(HdfsNameServiceResolver.java:139)
        at org.apache.atlas.utils.HdfsNameServiceResolver.<clinit>(HdfsNameServiceResolver.java:47)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.toDbEntity(HiveMetaStoreBridge.java:606)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:461)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:322)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:292)
        at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:192)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.HdfsConfiguration
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ... 7 more
Failed to import Hive Meta Data!!!
[omc@hadoop102 apache-atlas-2.2.0]$

在 $HIVE_HOME/lib 目录下添加 hadoop-hdfs-client-3.1.3.jar 即可。

说明:版本号要和自己的 Hadoop 版本号一致。

相关推荐
weixin_462446231 天前
Hive Metastore 使用 MySQL 8(CJ 驱动)完整配置实战(含完整 Shell 脚本)
hive·hadoop·mysql
zgl_200537791 天前
源代码:ZGLanguage 解析SQL数据血缘 之 显示 MERGE SQL 结构图
数据库·数据仓库·hive·数据治理·etl·sql解析·数据血缘
Gain_chance1 天前
24-学习笔记尚硅谷数仓搭建-DIM层的维度表建表思路及商品表维度表的具体建表解析
数据仓库·hive·笔记·学习·datagrip
talle20211 天前
Hive | 分区与分桶
大数据·数据仓库·hive
Gain_chance2 天前
23-学习笔记尚硅谷数仓搭建-ODS层业务全量表、增量表结构设计及数据装载脚本
数据仓库·hive·笔记·学习
zhangxl-jc2 天前
Hive基本操作日记
数据仓库·hive·hadoop
叮铃铃上课了3 天前
Hive实战:精准拆分中英文混合字符串(含重音/空格场景)
数据仓库·hive·hadoop
走遍西兰花.jpg3 天前
hive怎么实现连续登录
数据仓库·hive·hadoop
像豆芽一样优秀3 天前
深入理解与应用SQL递归CTE处理层级数据
大数据·hive·sql
B站计算机毕业设计超人3 天前
计算机毕业设计Python+Django考研院校推荐系统 考研分数线预测系统 大数据毕业设计 (代码+LW文档+PPT+讲解视频)
大数据·人工智能·hive·python·django·毕业设计·课程设计