APACHE-ATLAS-2.1.0 - 基础运维

cloneme012024-09-11 13:31

（一）SOLR相关

1. 如何创建/删除集合？
java 复制代码 
# 1. 删除
solr/bin/solr delete -c vertex_index
solr/bin/solr delete -c edge_index
solr/bin/solr delete -c fulltext_index

# 2. 创建
solr/bin/solr create -c vertex_index -force -d conf/solr/
solr/bin/solr create -c edge_index -force -d conf/solr/
solr/bin/solr create -c fulltext_index -force -d conf/solr/


# 删除
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c vertex_index
INFO  - 2023-06-15 00:19:40.690; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{
    "status":0,
    "QTime":835}}
Deleted collection 'vertex_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=vertex_index
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c edge_index
INFO  - 2023-06-15 00:19:44.906; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{
    "status":0,
    "QTime":634}}
Deleted collection 'edge_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=edge_index
[root@master apache-atlas-2.1.0]# solr/bin/solr delete -c fulltext_index
INFO  - 2023-06-15 00:19:52.255; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
{"responseHeader":{
    "status":0,
    "QTime":707}}
Deleted collection 'fulltext_index' using command:
http://192.168.118.174:8983/solr/admin/collections?action=DELETE&name=fulltext_index
[root@master apache-atlas-2.1.0]# 
# 创建
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c vertex_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:04.079; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'vertex_index' with 1 shard(s), 1 replica(s) with config-set 'vertex_index'
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c edge_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:13.817; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'edge_index' with 1 shard(s), 1 replica(s) with config-set 'edge_index'
[root@master apache-atlas-2.1.0]# solr/bin/solr create -c fulltext_index -force -d conf/solr/
INFO  - 2023-06-15 00:20:21.018; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop
Created collection 'fulltext_index' with 1 shard(s), 1 replica(s) with config-set 'fulltext_index'
[root@master apache-atlas-2.1.0]#
2. SOLR的日志在哪里？
java 复制代码 
/opt/rills/software/apache-atlas-2.1.0/solr/server/logs/solr.log

（二）HBASE相关

1. 内置的HBASE无法启动

错误信息如下：
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures

java 复制代码 
2023-06-15 09:06:28,071 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /0:0:0:0:0:0:0:1:55222
2023-06-15 09:06:28,076 INFO  [SyncThread:0] server.ZooKeeperServer: Established session 0x188bc96d4a90002 with negotiated timeout 10000 for client /0:0:0:0:0:0:0:1:55222
2023-06-15 09:06:28,076 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x188bc96d4a90002, negotiated timeout = 10000
2023-06-15 09:06:28,207 INFO  [Thread-22] procedure2.ProcedureExecutor: Starting 16 core workers (bigger of cpus/4 or 16) with max (burst) worker count=160
2023-06-15 09:06:28,214 WARN  [Thread-22] wal.WALProcedureStore: Remove uninitialized log: DeprecatedRawLocalFileStatus{path=file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/MasterProcWALs/pv2-00000000000000000002.log; isDirectory=false; length=0; replication=1; blocksize=33554432; modification_time=1686790826000; access_time=0; owner=; group=; permission=rw-rw-rw-; isSymlink=false}
2023-06-15 09:06:28,215 INFO  [Thread-22] wal.ProcedureWALFile: Archiving file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/MasterProcWALs/pv2-00000000000000000002.log to file:/opt/rills/software/apache-atlas-2.1.0/data/hbase-root/oldWALs/pv2-00000000000000000002.log
2023-06-15 09:06:28,275 ERROR [Thread-22] master.HMaster: Failed to become active master
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1044)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:383)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:649)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1282)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:842)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2086)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:553)
	at java.lang.Thread.run(Thread.java:748)
2023-06-15 09:06:28,276 ERROR [Thread-22] master.HMaster: ***** ABORTING master master,61500,1686791181740: Unhandled exception. Starting shutdown. *****
java.lang.IllegalStateException: The procedure WAL relies on the ability to hsync for proper operation during component failures, but the underlying filesystem does not support doing so. Please check the config value of 'hbase.procedure.store.wal.use.hsync' to set the desired level of robustness and ensure the config value of 'hbase.wal.dir' points to a FileSystem mount that can provide it.
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.rollWriter(WALProcedureStore.java:1044)
	at org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.recoverLease(WALProcedureStore.java:383)
	at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.init(ProcedureExecutor.java:649)
	at org.apache.hadoop.hbase.master.HMaster.createProcedureExecutor(HMaster.java:1282)
	at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:842)
	at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2086)
	at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:553)
	at java.lang.Thread.run(Thread.java:748)
2023-06-15 09:06:28,277 INFO  [Thread-22] regionserver.HRegionServer: ***** STOPPING region server 'master,61500,1686791181740' *****
2023-06-15 09:06:28,277 INFO  [Thread-22] regionserver.HRegionServer: STOPPED: Stopped by Thread-22
2023-06-15 09:06:28,841 INFO  [master/master:61500.splitLogManager..Chore.1] hbase.ScheduledChore: Chore: SplitLogManager Timeout Monitor was stopped
2023-06-15 09:06:30,484 INFO  [M:0;master:61500] ipc.NettyRpcServer: Stopping server on /192.168.118.174:61500
2023-06-15 09:06:30,497 INFO  [M:0;master:61500] regionserver.HRegionServer: Stopping infoServer
2023-06-15 09:06:30,538 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.w.WebAppContext@67440de6{/,null,UNAVAILABLE}{file:/opt/rills/software/apache-atlas-2.1.0/hbase/hbase-webapps/master}
2023-06-15 09:06:30,545 INFO  [M:0;master:61500] server.AbstractConnector: Stopped ServerConnector@3910fe11{HTTP/1.1,[http/1.1]}{0.0.0.0:61510}
2023-06-15 09:06:30,547 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@5af97169{/static,file:///opt/rills/software/apache-atlas-2.1.0/hbase/hbase-webapps/static/,UNAVAILABLE}
2023-06-15 09:06:30,547 INFO  [M:0;master:61500] handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@57adfab0{/logs,file:///opt/rills/software/apache-atlas-2.1.0/hbase/logs/,UNAVAILABLE}
2023-06-15 09:06:30,549 INFO  [M:0;master:61500] regionserver.HRegionServer: stopping server master,61500,1686791181740
2023-06-15 09:06:30,549 INFO  [M:0;master:61500] zookeeper.ReadOnlyZKClient: Close zookeeper connection 0x79f2feba to localhost:2181
2023-06-15 09:06:30,551 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x188bc96d4a90002
2023-06-15 09:06:30,553 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1:55222 which had sessionid 0x188bc96d4a90002
2023-06-15 09:06:30,554 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba] zookeeper.ZooKeeper: Session: 0x188bc96d4a90002 closed
2023-06-15 09:06:30,554 INFO  [ReadOnlyZKClient-localhost:2181@0x79f2feba-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x188bc96d4a90002
2023-06-15 09:06:30,558 INFO  [M:0;master:61500] regionserver.HRegionServer: stopping server master,61500,1686791181740; all regions closed.
2023-06-15 09:06:30,558 INFO  [M:0;master:61500] hbase.ChoreService: Chore service for: master/master:61500 had [] on shutdown
2023-06-15 09:06:30,559 WARN  [M:0;master:61500] master.ActiveMasterManager: Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null
2023-06-15 09:06:30,561 INFO  [M:0;master:61500] wal.WALProcedureStore: Stopping the WAL Procedure Store, isAbort=false
2023-06-15 09:06:30,562 INFO  [M:0;master:61500] hbase.ChoreService: Chore service for: master/master:61500.splitLogManager. had [] on shutdown
2023-06-15 09:06:30,563 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x188bc96d4a90000 type:delete cxid:0x1a zxid:0x2a txntype:-1 reqpath:n/a Error Path:/hbase/rs/master,61500,1686791181740 Error:KeeperErrorCode = NoNode for /hbase/rs/master,61500,1686791181740
2023-06-15 09:06:30,565 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x188bc96d4a90000
2023-06-15 09:06:30,567 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /0:0:0:0:0:0:0:1:55218 which had sessionid 0x188bc96d4a90000
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] zookeeper.ZooKeeper: Session: 0x188bc96d4a90000 closed
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] regionserver.HRegionServer: Exiting; stopping=master,61500,1686791181740; zookeeper connection closed.
2023-06-15 09:06:30,568 INFO  [M:0;master:61500] server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:47812 which had sessionid 0x188bc96d4a90001
2023-06-15 09:06:30,568 INFO  [main-EventThread] zookeeper.ClientCnxn: EventThread shut down for session: 0x188bc96d4a90000
2023-06-15 09:06:30,568 INFO  [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x188bc96d4a90001, likely server has closed socket, closing socket connection and attempting reconnect
2023-06-15 09:06:30,569 INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: NIOServerCnxn factory exited run method
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.ZooKeeperServer: shutting down
2023-06-15 09:06:30,572 ERROR [M:0;master:61500] server.ZooKeeperServer: ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.SessionTrackerImpl: Shutting down
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.PrepRequestProcessor: Shutting down
2023-06-15 09:06:30,572 INFO  [M:0;master:61500] server.SyncRequestProcessor: Shutting down
2023-06-15 09:06:30,572 INFO  [ProcessThread(sid:0 cport:2181):] server.PrepRequestProcessor: PrepRequestProcessor exited loop!
2023-06-15 09:06:30,572 INFO  [SyncThread:0] server.SyncRequestProcessor: SyncRequestProcessor exited!
2023-06-15 09:06:30,573 INFO  [M:0;master:61500] server.FinalRequestProcessor: shutdown of request processor complete
2023-06-15 09:06:30,574 INFO  [M:0;master:61500] zookeeper.MiniZooKeeperCluster: Shutdown MiniZK cluster with all ZK servers

在hbase-site.xml配置文件，添加如下配置：

java 复制代码 
<property>
	<name>hbase.unsafe.stream.capability.enforce</name>
	<value>false</value>
</property>

（三）ATLAS相关

1. ATLAS的日志在哪里？
java 复制代码 
/opt/rills/software/apache-atlas-2.1.0/logs/application.log
2. 内置HIVE的日志在哪里？
java 复制代码 
首先，日志存储在HIVE服务运行的机器上，在hive/conf/hive-log4j2.properties文件中进行了配置。
该文件如果不存在，就会使用默认存储路径，HIVE日志默认的存储路径是：
hive.root.logger=WARN,DRFAhive.log.dir=/tmp/${user.name}/${hive.log.file}（默认的文件名为：hive.log）
3. ATLAS启动非常慢？
java 复制代码 
ATLAS在第一次启动非常慢，一般都在10分钟左右，具体时间根据主机的性能而定。
如果ATLAS命令行启动显示成功 且 HBASE正常 且 SOLR正常，
那就只能等着ATLAS完全启动完毕后，UI才可以正常访问。
相关推荐
Alluxio16 小时前
Alluxio + OCI：为 AI 工作负载实现亚毫秒访问延迟与可扩展吞吐
运维·数据库·人工智能·分布式·缓存·ai·语言模型
Sayai20 小时前
Neo4j 图数据质量检查实战：环路检测、孤立节点与连通分量，两条 Cypher 引出的数据体检清单（含变长路径性能陷阱）
运维·neo4j·图数据库·数据质量·cypher
Elastic 中国社区官方博客21 小时前
更少的位数，相同的召回率：DiskBBQ 更保守的自动校准扫描
大数据·运维·开发语言·人工智能·elasticsearch·搜索引擎
方便面不加香菜21 小时前
Docker Volume -- 管理卷Volume详解
运维·docker·容器
沫璃染墨1 天前
从零入门计算机网络系列（十八）：TCP连接管理详解——从三次握手与四次挥手到TIME_WAIT机制
运维·服务器·网络·c++·网络协议·tcp/ip·tcp
Felven1 天前
华为鲲鹏920s芯片serdestool工具使用说明
linux·运维·服务器·华为·鲲鹏920·serdestool
江湖有缘1 天前
Docker实战 | 使用Docker部署lemonitor镜像站点列表程序
运维·docker·容器
莫晓燚1 天前
mall 电商项目完整部署---实战
linux·运维·docker·容器
爱和冰阔落1 天前
【Linux网络】JsonCpp 怎么实现序列化？从 Request/Response 到网络计算器消息设计
linux·运维·网络·php
Decay__1 天前
01_SQLServer 体系架构全解 —— 实例进程、关系引擎、SQLOS 与事务日志深入
运维·数据库·sql·mysql·架构
热门推荐
01国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）02如何新建文件夹？ 电脑新建文件夹的4种方法03历年考研数学一、数学二、数学三真题试卷及答案PDF04几个好用的ip纯净度检测网站05GitHub 镜像站点06微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新07昇腾910B系列芯片完全指南：四款型号对比、服务器选型与大模型部署实战08苹果iPA游戏软件资源下载网站09蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接10Dify：一站式AI应用开发平台全解析与Windows部署指南