Ambari 统信 UOS 适配征途:国产化环境下的集群管理破壁与实践总结

Ambari 统信 UOS 适配征途:国产化环境下的集群管理破壁与实践总结

随着国产化替代进程的加速,将 Apache Ambari 这一成熟的 Hadoop 集群管理工具部署到统信 UOS 操作系统环境的需求日益迫切。然而,由于底层操作系统、依赖库、环境配置等方面的显著差异,适配过程面临一系列独特的挑战。本文档系统总结了在 Ambari 适配统信 UOS 过程中遇到的关键问题、技术难点以及相应的解决方案与实践经验。旨在为后续在国产化统信 UOS 平台上顺利部署和管理 Ambari 及 Hadoop 生态组件提供详实参考,加速国产化大数据平台的落地进程。

问题

shell 复制代码
INFO 2025-03-13 15:47:21,825 HeartbeatThread.py:132 - Registration response received
ERROR 2025-03-13 15:47:21,825 HeartbeatThread.py:108 - Exception in HeartbeatThread. Re-running the registration
Traceback (most recent call last):
  File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 95, in run
    self.register()
  File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 135, in register
    self.handle_registration_response(response)
  File "/usr/lib/ambari-agent/lib/ambari_agent/HeartbeatThread.py", line 209, in handle_registration_response
    raise Exception(error_message)
Exception: Registration failed due to: Cannot register host with not supported os type, hostname=hadoop-02, serverOsType=kylin20, agentOsType=uos server20
INFO 2025-03-13 15:47:21,825 transport.py:358 - Receiver loop ended
', None)

解决

/usr/lib/ambari-agent/lib/ambari_commons/resources/os_family.json /usr/lib/ambari-agent/lib/ambari_commons/os_check.py

shell 复制代码
/usr/lib/ambari-agent/lib/ambari_commons/resources/os_family.json
"kylin": {
        "extends" : "redhat",
        "distro": [
          "kylin"
        ],
        "versions": [
          10,
          #加上20
          20
        ]
      },

问题

shell 复制代码
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/hook.py", line 37, in <module>
    BeforeInstallHook().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/hook.py", line 33, in hook
    install_packages()
  File "/var/lib/ambari-agent/cache/stack-hooks/before-INSTALL/scripts/shared_initialization.py", line 37, in install_packages
    retry_count=params.agent_stack_retry_count)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 125, in __new__
    cls(names_list.pop(0), env, provider, **kwargs)
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/packaging.py", line 30, in action_install
    self._pkg_manager.install_package(package_name, self.__create_context())
  File "/usr/lib/ambari-agent/lib/ambari_commons/repo_manager/yum_manager.py", line 242, in install_package
    elif context.is_upgrade or context.use_repos or not self._check_existence(name):
  File "/usr/lib/ambari-agent/lib/ambari_commons/repo_manager/yum_manager.py", line 312, in _check_existence
    return self.rpm_check_package_available(name)
  File "/usr/lib/ambari-agent/lib/ambari_commons/repo_manager/yum_manager.py", line 397, in rpm_check_package_available
    import rpm # this is faster then calling 'rpm'-binary externally.
ImportError: No module named rpm

usr/lib/ambari-agent/lib/ambari_commons/repo_manager/yum_manager.py

python 复制代码
def rpm_check_package_available(self, name):
    import rpm # this is faster then calling 'rpm'-binary externally.
    ts = rpm.TransactionSet()
    packages = ts.dbMatch()

    name_regex = re.escape(name).replace("\\?", ".").replace("\\*", ".*") + '$'
    regex = re.compile(name_regex)

    for package in packages:
      if regex.match(package['name']):
        return True
    return False

修改后

python 复制代码
def rpm_check_package_available(self, name):
    try:
        result = subprocess.run(
            ['rpm', '-q', name],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            check=False
        )
        return result.returncode == 0
    except Exception:
        return False

或者

python 复制代码
def rpm_check_package_available(self, name):
    import rpm # this is faster then calling 'rpm'-binary externally.
    ts = rpm.TransactionSet()
    packages = ts.dbMatch()

    name_regex = re.escape(name).replace("\\?", ".").replace("\\*", ".*") + '$'
    regex = re.compile(name_regex)

    for package in packages:
      if regex.match(package['name']):
        return True
    return False
   # command = ['rpm', '-qa']
   # process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
   # output, error = process.communicate()

   # if process.returncode == 0:
   #     packages = output.decode().splitlines()
   # else:
   #     return False

   # name_regex = re.escape(name).replace("\\?", ".").replace("\\*", ".*")
   # regex = re.compile(name_regex)

   # for package in packages:
   #     if regex.match(package):
   #         return True
   # return False

更多问题参考

征服国产生态!Ambari 完美适配银河麒麟 V20 实战指南:打通大数据管理"最后一公里"

相关推荐
鸭鸭鸭进京赶烤5 小时前
大学专业科普 | 云计算、大数据
大数据·云计算
G皮T9 小时前
【Elasticsearch】自定义评分检索
大数据·elasticsearch·搜索引擎·查询·检索·自定义评分·_score
搞笑的秀儿12 小时前
信息新技术
大数据·人工智能·物联网·云计算·区块链
SelectDB12 小时前
SelectDB 在 AWS Graviton ARM 架构下相比 x86 实现 36% 性价比提升
大数据·架构·aws
二二孚日12 小时前
自用华为ICT云赛道Big Data第五章知识点-Flume海量日志聚合
大数据·华为
二二孚日14 小时前
自用华为ICT云赛道Big Data第四章知识点-Flink流批一体分布式实时处理引擎
大数据·华为
xufwind15 小时前
spark standlone 集群离线安装
大数据·分布式·spark
AI数据皮皮侠16 小时前
中国区域10m空间分辨率楼高数据集(全国/分省/分市/免费数据)
大数据·人工智能·机器学习·分类·业界资讯
昱禹17 小时前
Flutter 3.29+使用isar构建失败
大数据·flutter
DeepSeek大模型官方教程17 小时前
NLP之文本纠错开源大模型:兼看语音大模型总结
大数据·人工智能·ai·自然语言处理·大模型·产品经理·大模型学习