问题描述
查看集群中的租户时无租户反馈，警告oceanbase-ce is not running，但是查询集群状态明明是running
--查看集群中的租户
root@localhost \~# obd cluster tenant show dxjobsingle -g
Get local repositories ok
WARN 192.168.40.110 oceanbase-ce is not running
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: 20060750-8ec0-11f0-91da-000c29e13c26
If you want to view detailed obd logs, please run: obd display-trace 20060750-8ec0-11f0-91da-000c29e13c26
问题原因
虚拟机关机再开机后未启动集群
obd 运维的数据库，启停需要使用obd进行操作，停机时候没有使用obd会导致集群状态running但是oceanbase-ce is not running。
分析过程
查看集群状态
dxjobsingle集群状态是ruuning
obd cluster list
输出如下：
root@localhost \~# obd cluster list
±---------------------------------------------------------------+
| Cluster List |
±------------±-------------------------------±----------------+
| Name | Configuration Path | Status (Cached) |
±------------±-------------------------------±----------------+
| dxjobsingle | /root/.obd/cluster/dxjobsingle | running |
±------------±-------------------------------±----------------+
Trace ID: 5bc2bd22-8ebd-11f0-810e-000c29e13c26
If you want to view detailed obd logs, please run: obd display-trace 5bc2bd22-8ebd-11f0-810e-000c29e13c26
查看集群中的租户
无返回租户却返回了WARN 192.168.40.110 oceanbase-ce is not running
查看集群中的租户
root@localhost \~# obd cluster tenant show dxjobsingle -g
Get local repositories ok
WARN 192.168.40.110 oceanbase-ce is not running
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: 20060750-8ec0-11f0-91da-000c29e13c26
If you want to view detailed obd logs, please run: obd display-trace 20060750-8ec0-11f0-91da-000c29e13c26
查看当前obd 内指定集群的状态
参数 deploy name 为部署名，可以理解为配置文件的别名。自 V3.2.0 起，若 obd 设置了加密密钥，执行 display 命令输出的集群信息中，密码相关内容为加密状态。若需输出明文密码，可在命令中添加 --epk/--encryption-passkey 指定密钥用于验证。
通过ob-configserver is not runing可推测出数据库集群未启动，集群启动前有以下工作：
1.组件检查顺序：
ob-configserver
obagent
prometheus
grafana
2.启动顺序：
ob-configserver
observer
obshell
obproxy
obagent
prometheus
grafana
root@localhost \~# obd cluster display dxjobsingle
Get local repositories and plugins ok
Open ssh connection ok
WARN 192.168.40.110 ob-configserver is not running
See https://www.oceanbase.com/product/ob-deployer/error-codes .
Trace ID: 0b992440-8ec1-11f0-a576-000c29e13c26
If you want to view detailed obd logs, please run: obd display-trace 0b992440-8ec1-11f0-a576-000c29e13c26
解决办法
启动集群后再次查看租户正常
启动集群
root@localhost \~# obd cluster start dxjobsingle -S
Get local repositories ok
Load cluster param plugin ok
Cluster status check ok
Check before start ob-configserver ok
Check before start obagent ok
Check before start prometheus ok
Check before start grafana ok
Start ob-configserver ok
ob-configserver program health check ok
cluster scenario: htap
Start observer ok
observer program health check ok
Connect to observer 192.168.40.110:2881 ok
obshell start ok
obshell program health check ok
start obproxy ok
obproxy program health check ok
Connect to obproxy ok
Start obagent ok
...
查看租户
root@localhost \~# obd cluster tenant show dxjobsingle -g
Get local repositories ok
Get deployment connections ok
Connect to observer 192.168.40.110:2881 ok
Select tenant ok
±----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tenant basic info |
±------------±------------±-------------------±-------------±--------±--------±------------±--------------------±--------------------±--------------±------------±------------+
| tenant_name | tenant_type | compatibility_mode | primary_zone | max_cpu | min_cpu | memory_size | max_iops | min_iops | log_disk_size | iops_weight | tenant_role |
±------------±------------±-------------------±-------------±--------±--------±------------±--------------------±--------------------±--------------±------------±------------+
| sys | SYS | MYSQL | RANDOM | 2.0 | 2.0 | 2G | 9223372036854775807 | 9223372036854775807 | 2G | 2 | PRIMARY |
| dxj | USER | MYSQL | RANDOM | 1.0 | 1.0 | 2G | 9223372036854775807 | 9223372036854775807 | 6G | 1 | PRIMARY |
±------------±------------±-------------------±-------------±--------±--------±------------±--------------------±--------------------±--------------±------------±------------+
Trace ID: 9d04fd46-8ec1-11f0-be08-000c29e13c26
If you want to view detailed obd logs, please run: obd display-trace 9d04fd46-8ec1-11f0-be08-000c29e13c26