大数据-146 Apache Kudu 安装运行 Dockerfile 模拟集群启动测试

点一下关注吧！！！非常感谢！！持续更新！！！

目前已经更新到了：

Hadoop（已更完）
HDFS（已更完）
MapReduce（已更完）
Hive（已更完）
Flume（已更完）
Sqoop（已更完）
Zookeeper（已更完）
HBase（已更完）
Redis （已更完）
Kafka（已更完）
Spark（已更完）
Flink（已更完）
ClickHouse（已更完）
Kudu（正在更新...）

章节内容

上节我们完成了如下的内容：

Kudu 的基本介绍
Kudu 的架构概述

安装准备

硬件准备

一台或者多台跑kudu-master。建议一个Master（无容错机制）、三个Master（允许一个节点出错）、五个Master（允许两个节点出错）
一台或者多台机器跑kudu-server。当需要使用副本，至少需要三个节点运行kudu-server服务。

操作系统

Ubuntu云服务器（我的环境是三台小云服务器）
内核和文件系统支持 Hole Punching 选项。
NTP 服务
xfs or ext4 formatterd drivers

存储要求

尽量使用固态存储，显著提供kudu性能

环境说明

如果你使用的是CDH，需要ClouderaManager 5.4.3 及以上的版本。
三台云服务器，h121启动Master，h122和123启动TabletServer。

Dockerfile

shell 复制代码

https://github.com/apache/kudu/blob/master/docker/quickstart.yml

官方的配置文件为：

3个Master
4个TabletServer

yml 复制代码

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
version: "3"
services:
  kudu-master-1:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    ports:
      - "7051:7051"
      - "8051:8051"
    command: ["master"]
    volumes:
      - kudu-master-1:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      # TODO: Use `host.docker.internal` instead of KUDU_QUICKSTART_IP when it
      # works on Linux (https://github.com/docker/for-linux/issues/264)
      - >
        MASTER_ARGS=--fs_wal_dir=/var/lib/kudu/master
        --rpc_bind_addresses=0.0.0.0:7051
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP:?Please set KUDU_QUICKSTART_IP environment variable}:7051
        --webserver_port=8051
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8051
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true

  kudu-master-2:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    ports:
      - "7151:7151"
      - "8151:8151"
    command: ["master"]
    volumes:
      - kudu-master-2:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        MASTER_ARGS=--fs_wal_dir=/var/lib/kudu/master
        --rpc_bind_addresses=0.0.0.0:7151
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7151
        --webserver_port=8151
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8151
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-master-3:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    ports:
      - "7251:7251"
      - "8251:8251"
    command: ["master"]
    volumes:
      - kudu-master-3:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        MASTER_ARGS=--fs_wal_dir=/var/lib/kudu/master
        --rpc_bind_addresses=0.0.0.0:7251
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7251
        --webserver_port=8251
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8251
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-tserver-1:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    depends_on:
      - kudu-master-1
      - kudu-master-2
      - kudu-master-3
    ports:
      - "7050:7050"
      - "8050:8050"
    command: ["tserver"]
    volumes:
      - kudu-tserver-1:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        TSERVER_ARGS=--fs_wal_dir=/var/lib/kudu/tserver
        --rpc_bind_addresses=0.0.0.0:7050
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7050
        --webserver_port=8050
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8050
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-tserver-2:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    depends_on:
      - kudu-master-1
      - kudu-master-2
      - kudu-master-3
    ports:
      - "7150:7150"
      - "8150:8150"
    command: ["tserver"]
    volumes:
      - kudu-tserver-2:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        TSERVER_ARGS=--fs_wal_dir=/var/lib/kudu/tserver
        --rpc_bind_addresses=0.0.0.0:7150
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7150
        --webserver_port=8150
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8150
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-tserver-3:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    depends_on:
      - kudu-master-1
      - kudu-master-2
      - kudu-master-3
    ports:
      - "7250:7250"
      - "8250:8250"
    command: ["tserver"]
    volumes:
      - kudu-tserver-3:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        TSERVER_ARGS=--fs_wal_dir=/var/lib/kudu/tserver
        --rpc_bind_addresses=0.0.0.0:7250
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7250
        --webserver_port=8250
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8250
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-tserver-4:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    depends_on:
      - kudu-master-1
      - kudu-master-2
      - kudu-master-3
    ports:
      - "7350:7350"
      - "8350:8350"
    command: ["tserver"]
    volumes:
      - kudu-tserver-4:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        TSERVER_ARGS=--fs_wal_dir=/var/lib/kudu/tserver
        --rpc_bind_addresses=0.0.0.0:7350
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7350
        --webserver_port=8350
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8350
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
  kudu-tserver-5:
    image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
    depends_on:
      - kudu-master-1
      - kudu-master-2
      - kudu-master-3
    ports:
      - "7450:7450"
      - "8450:8450"
    command: ["tserver"]
    volumes:
      - kudu-tserver-5:/var/lib/kudu
    environment:
      - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
      - >
        TSERVER_ARGS=--fs_wal_dir=/var/lib/kudu/tserver
        --rpc_bind_addresses=0.0.0.0:7450
        --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7450
        --webserver_port=8450
        --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8450
        --webserver_doc_root=/opt/kudu/www
        --stderrthreshold=0
        --use_hybrid_clock=false
        --unlock_unsafe_flags=true
volumes:
  kudu-master-1:
  kudu-master-2:
  kudu-master-3:
  kudu-tserver-1:
  kudu-tserver-2:
  kudu-tserver-3:
  kudu-tserver-4:
  kudu-tserver-5:

内容解释

version: "3"：指定使用 Docker Compose 文件的版本。版本 3 是当前常用的版本，支持多种高级功能，适用于 Docker 1.13.0 及更高版本。
services：定义了多个服务，每个服务对应一个 Docker 容器。在本配置中，定义了多个 Kudu Master 和 Kudu Tablet Server（简称 TServer）。
volumes：定义了持久化存储卷，用于存储各个服务的数据，确保即使容器重新创建或删除，数据仍然保留。

3 个 Kudu Master 节点：

kudu-master-1
kudu-master-2
kudu-master-3

5 个 Kudu Tablet Server 节点：

kudu-tserver-1
kudu-tserver-2
kudu-tserver-3
kudu-tserver-4
kudu-tserver-5

Master

每个 Master 节点的配置基本相同，只是端口和持久化卷不同。以下以 kudu-master-1 为例进行解释：

yml 复制代码

kudu-master-1:
  image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}
  ports:
    - "7051:7051"
    - "8051:8051"
  command: ["master"]
  volumes:
    - kudu-master-1:/var/lib/kudu
  environment:
    - KUDU_MASTERS=kudu-master-1:7051,kudu-master-2:7151,kudu-master-3:7251
    - >
      MASTER_ARGS=--fs_wal_dir=/var/lib/kudu/master
      --rpc_bind_addresses=0.0.0.0:7051
      --rpc_advertised_addresses=${KUDU_QUICKSTART_IP:?Please set KUDU_QUICKSTART_IP environment variable}:7051
      --webserver_port=8051
      --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8051
      --webserver_doc_root=/opt/kudu/www
      --stderrthreshold=0
      --use_hybrid_clock=false
      --unlock_unsafe_flags=true

udu-master-1：服务名称，标识第一个 Kudu Master 节点。
image: apache/kudu:${KUDU_QUICKSTART_VERSION:-latest}：指定使用的 Docker 镜像。apache/kudu 是官方 Kudu 镜像。
${KUDU_QUICKSTART_VERSION:-latest}：使用环境变量 KUDU_QUICKSTART_VERSION 指定版本，如果未设置，则默认使用 latest。
ports："7051:7051"：将主机的 7051 端口映射到容器的 7051 端口，用于 Kudu Master 的 RPC 通信。"8051:8051"：将主机的 8051 端口映射到容器的 8051 端口，用于 Kudu Master 的 Web 界面。
command: ["master"]：指定容器启动时运行的命令，这里启动 Kudu Master 进程。
volumes：kudu-master-1:/var/lib/kudu：将 Docker 卷 kudu-master-1 挂载到容器内的 /var/lib/kudu 目录，用于持久化存储数据。
KUDU_MASTERS：定义了集群中所有 Master 节点的地址，格式为 hostname:port，以逗号分隔。

启动服务

shell 复制代码

export KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}' | tail -1)
docker-compose up -d

访问服务

shell 复制代码

http://localhost:8051/

这样我们的服务就顺利的启动了：

大数据-146 Apache Kudu 安装运行 Dockerfile 模拟集群 启动测试

点一下关注吧！！！非常感谢！！持续更新！！！

目前已经更新到了：

章节内容

安装准备

硬件准备

操作系统

存储要求

环境说明

Dockerfile

内容解释

Master

启动服务

访问服务

大数据-146 Apache Kudu 安装运行 Dockerfile 模拟集群启动测试