clickhouse集群的安装与部署

clickhouse集群的安装与部署

1、准备工作

硬件要求

三台服务器(节点1、节点2和节点3)

每台服务器建议至少4核CPU、8GB内存、100GB SSD存储

确保每台服务器之间网络互通(建议千兆内网)

系统要求

Linux操作系统(推荐Ubuntu 20.04/22.04或CentOS 7/8)

确保时间同步(安装并配置NTP)

2、安装ClickHouse

在每台节点上执行以下命令:

bash

#Ubuntu/Debian

sudo apt-get install -y apt-transport-https ca-certificates dirmngr

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E0C56BD4

echo "deb https://repo.clickhouse.com/deb/stable/ main/" | sudo tee /etc/apt/sources.list.d/clickhouse.list

sudo apt-get update

sudo apt-get install -y clickhouse-server clickhouse-client

#CentOS/RHEL

sudo yum install -y yum-utils

sudo rpm --import https://repo.clickhouse.com/CLICKHOUSE-KEY.GPG

sudo yum-config-manager --add-repo https://repo.clickhouse.com/rpm/stable/x86_64

sudo yum install -y clickhouse-server clickhouse-client

3、配置集群

在每台节点上创建配置文件 /etc/clickhouse-server/config.xml,注意没个节点上的配置文件ip和主机名不一样

复制代码
<!--
  注意:用户级别和查询级别的设置在"users.xml"文件中配置。
  如果错误地在此文件配置了用户级设置,服务器将无法启动。
  解决方法:
  1. 将设置移动到"users.xml"文件中的正确位置
  2. 或者在此添加 <skip_check_for_incorrect_settings>1</skip_check_for_incorrect_settings>
-->
<clickhouse>
    
    <!-- 日志配置 -->
    <logger>
        <!-- 可选日志级别:
          - none (关闭日志)
          - fatal
          - critical
          - error
          - warning
          - notice
          - information
          - debug
          - trace
          - test (不用于生产环境)
        -->
        <level>trace</level>
        <log>/data/clickhouse/logs/clickhouse-server/clickhouse-server.log</log>
        <errorlog>/data/clickhouse/logs/clickhouse-server/clickhouse-server.err.log</errorlog>
        
        <!-- 日志轮转策略 -->
        <size>1000M</size>  <!-- 单个日志文件最大大小 -->
        <count>10</count>    <!-- 保留的日志文件数量 -->
    </logger>

    <!-- URL方案映射 -->
    <url_scheme_mappers>
        <s3>
            <to>https://{bucket}.s3.amazonaws.com</to>
        </s3>
        <gs>
            <to>https://{bucket}.storage.googleapis.com</to>
        </gs>
        <oss>
            <to>https://{bucket}.oss.aliyuncs.com</to>
        </oss>
    </url_scheme_mappers>

    <!-- HTTP OPTIONS请求的响应头配置 (用于CORS预检请求) -->
    <http_options_response>
        <header>
            <name>Access-Control-Allow-Origin</name>
            <value>*</value>
        </header>
        <header>
            <name>Access-Control-Allow-Headers</name>
            <value>origin, x-requested-with, x-clickhouse-format, x-clickhouse-user, x-clickhouse-key, Authorization</value>
        </header>
        <header>
            <name>Access-Control-Allow-Methods</name>
            <value>POST, GET, OPTIONS</value>
        </header>
        <header>
            <name>Access-Control-Max-Age</name>
            <value>86400</value>
        </header>
    </http_options_response>

    <!-- HTTP API端口 (也用于ODBC/JDBC驱动和Web界面) -->
    <http_port>8123</http_port>
    
    <!-- 原生协议端口 (用于clickhouse-client和分布式查询) -->
    <tcp_port>9000</tcp_port>
    
    <!-- MySQL协议兼容端口 -->
    <mysql_port>9004</mysql_port>
    
    <!-- PostgreSQL协议兼容端口 -->
    <postgresql_port>9005</postgresql_port>
    
    <!-- 副本间通信端口 (用于数据交换) -->
    <interserver_http_port>9009</interserver_http_port>
    <interserver_http_host>10.233.0.151</interserver_http_host>

    <!-- 监听地址 (0.0.0.0表示监听所有网络接口) -->
    <listen_host>0.0.0.0</listen_host>
    
    <!-- 最大连接数 -->
    <max_connections>4096</max_connections>
    
    <!-- HTTP 1.1 keep-alive超时时间(秒) -->
    <keep_alive_timeout>10</keep_alive_timeout>

    <!-- gRPC协议配置 -->
    <grpc>
        <enable_ssl>false</enable_ssl>
		<ssl_cert_file>/path/to/ssl_cert_file</ssl_cert_file>
        <ssl_key_file>/path/to/ssl_key_file</ssl_key_file>
		<ssl_require_client_auth>false</ssl_require_client_auth>
		<ssl_ca_cert_file>/path/to/ssl_ca_cert_file</ssl_ca_cert_file>
        <transport_compression_type>none</transport_compression_type>
        <transport_compression_level>0</transport_compression_level>
        <max_send_message_size>-1</max_send_message_size>
        <max_receive_message_size>-1</max_receive_message_size>
        <verbose_logs>false</verbose_logs>
    </grpc>

    <!-- SSL/TLS配置 -->
    <openSSL>
        <server>
            <verificationMode>none</verificationMode>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
			<invalidCertificateHandler>
                <name>RejectCertificateHandler</name>
            </invalidCertificateHandler>
        </server>
        <client>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
			<invalidCertificateHandler>
                <name>RejectCertificateHandler</name>
            </invalidCertificateHandler>
        </client>
    </openSSL>
	
    <concurrent_threads_soft_limit_num>0</concurrent_threads_soft_limit_num>
    <concurrent_threads_soft_limit_ratio_to_cores>2</concurrent_threads_soft_limit_ratio_to_cores>
    <!-- 并发查询限制 -->
    <max_concurrent_queries>1000</max_concurrent_queries>
    
    <!-- 服务器内存使用限制 (0表示使用默认值) -->
    <max_server_memory_usage>0</max_server_memory_usage>
	
	<max_thread_pool_size>10000</max_thread_pool_size>
    
    <!-- 服务器内存使用与物理内存的比例 -->
    <max_server_memory_usage_to_ram_ratio>0.9</max_server_memory_usage_to_ram_ratio>

	<total_memory_profiler_step>4194304</total_memory_profiler_step>
	
	<total_memory_tracker_sample_probability>0</total_memory_tracker_sample_probability>

    <!-- 未压缩数据缓存大小 (字节) -->
    <uncompressed_cache_size>8589934592</uncompressed_cache_size>
    
    <!-- 标记缓存大小 (字节) -->
    <mark_cache_size>5368709120</mark_cache_size>
    
    <!-- 二级索引标记缓存大小 (字节) -->
    <index_mark_cache_size>5368709120</index_mark_cache_size>
	
	<mmap_cache_size>1000</mmap_cache_size>
	
	<compiled_expression_cache_size>134217728</compiled_expression_cache_size>
	
	<compiled_expression_cache_elements_size>10000</compiled_expression_cache_elements_size>
	
	<custom_cached_disks_base_directory>/data/clickhouse/data/caches/</custom_cached_disks_base_directory>

    <validate_tcp_client_information>false</validate_tcp_client_information>

    <!-- 数据目录路径 (以斜杠结尾) -->
    <path>/data/clickhouse/data/</path>
    
    <!-- 临时文件目录 -->
    <tmp_path>/data/clickhouse/data/tmp/</tmp_path>
    
    <!-- 用户文件目录 (用于file表函数) -->
    <user_files_path>/data/clickhouse/data/user_files/</user_files_path>

    <!-- 密码认证配置 -->
    <allow_plaintext_password>1</allow_plaintext_password>
    <allow_no_password>1</allow_no_password>
    <allow_implicit_no_password>1</allow_implicit_no_password>
    <default_password_type>sha256_password</default_password_type>
    <bcrypt_workfactor>12</bcrypt_workfactor>
	
	<ldap_servers>
	</ldap_servers>

    <!-- 用户目录配置 -->
    <user_directories>
        <users_xml>
            <path>users.xml</path>
        </users_xml>
        <local_directory>
            <path>/data/clickhouse/data/access/</path>
        </local_directory>
    </user_directories>
	
	<access_control_improvements>
        <users_without_row_policies_can_read_rows>true</users_without_row_policies_can_read_rows>
     
        <on_cluster_queries_require_cluster_grant>true</on_cluster_queries_require_cluster_grant>

        <select_from_system_db_requires_grant>true</select_from_system_db_requires_grant>

        <select_from_information_schema_requires_grant>true</select_from_information_schema_requires_grant>

        <settings_constraints_replace_previous>true</settings_constraints_replace_previous>

        <role_cache_expiration_time_seconds>600</role_cache_expiration_time_seconds>
    </access_control_improvements>

    <!-- 默认设置配置文件 -->
    <default_profile>default</default_profile>
    
    <!-- 自定义设置前缀 (用于兼容MySQL) -->
    <custom_settings_prefixes>SQL_</custom_settings_prefixes>
    
    <!-- 默认数据库 -->
    <default_database>default</default_database>
    
    <!-- 服务器时区 -->
    <timezone>Asia/Shanghai</timezone>

    <!-- 启动后锁定内存以减少延迟 -->
    <mlock_executable>true</mlock_executable>
    
    <!-- 不重新映射可执行内存 -->
    <remap_executable>false</remap_executable>

    <!-- 集群配置 (用于分布式表) -->
    <remote_servers>
        <!-- 默认测试集群配置 -->
        <default>
            <shard>
                <replica>
                    <host>localhost</host>
                    <port>9000</port>
                </replica>
            </shard>
        </default>
        
        <!-- 生产集群配置 -->
        <lwlk_cluster>
            <shard>  <!-- 仅1个分片 -->
                <internal_replication>true</internal_replication>
                <replica>
                    <host>app1</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>**********</password>
                </replica>
                <replica>
                    <host>app2</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>**********</password>
                </replica>
                <replica>
                    <host>app3</host>
                    <port>9000</port>
                    <user>default</user>
                    <password>**********</password>
                </replica>
            </shard>
        </lwlk_cluster>
    </remote_servers>

    <!-- ClickHouse Keeper配置 (替代ZooKeeper) -->
    <keeper_server>
        <feature_flags> 
            <check_not_exists>true</check_not_exists>
            <create_if_not_exists>true</create_if_not_exists>
        </feature_flags>
        <tcp_port>9181</tcp_port>  <!-- Keeper通信端口 -->
        <server_id>1</server_id>   <!-- 节点ID -->
        <log_storage_path>/data/clickhouse/coordination/log</log_storage_path>
        <snapshot_storage_path>/data/clickhouse/coordination/snapshots</snapshot_storage_path>
        <coordination_settings>
            <operation_timeout_ms>10000</operation_timeout_ms>
            <session_timeout_ms>60000</session_timeout_ms>
            <raft_logs_level>warning</raft_logs_level>
        </coordination_settings>
        <raft_configuration>
            <server>
                <id>1</id>
                <hostname>app1</hostname>
                <port>9234</port>
            </server>
            <server>
                <id>2</id>
                <hostname>app2</hostname>
                <port>9234</port>
            </server>
            <server>
                <id>3</id>
                <hostname>app3</hostname>
                <port>9234</port>
            </server>
        </raft_configuration>
    </keeper_server>

    <!-- ZooKeeper配置 (实际使用ClickHouse Keeper替代) -->
    <zookeeper>
        <node>
            <host>app1</host>
            <port>9181</port>
        </node>
        <node>
            <host>app2</host>
            <port>9181</port>
        </node>
        <node>
            <host>app3</host>
            <port>9181</port>
        </node>
    </zookeeper>

    <!-- 宏定义 (用于复制表) -->
    <macros>
		<cluster>lwlk_cluster</cluster>  <!-- 集群名称 -->
        <shard>01</shard>
        <replica>app1</replica>
    </macros>
	
	<builtin_dictionaries_reload_interval>3600</builtin_dictionaries_reload_interval>

    <max_session_timeout>3600</max_session_timeout>

    <default_session_timeout>60</default_session_timeout>


    <!-- Prometheus监控端点 -->
    <prometheus>
        <endpoint>/metrics</endpoint>
        <port>9363</port>
        <metrics>true</metrics>
        <events>true</events>
        <asynchronous_metrics>true</asynchronous_metrics>
    </prometheus>

    <!-- 查询日志配置 -->
    <query_log>
        <database>system</database>
        <table>query_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
		<buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </query_log>

    <!-- 跟踪日志配置 -->
    <trace_log>
        <database>system</database>
        <table>trace_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
		<max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </trace_log>

    <!-- 查询线程日志配置 -->
    <query_thread_log>
        <database>system</database>
        <table>query_thread_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
		<max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </query_thread_log>
	
	<query_views_log>
        <database>system</database>
        <table>query_views_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_views_log>

    <!-- 部件日志配置 (记录MergeTree表部件的操作) -->
    <part_log>
        <database>system</database>
        <table>part_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
		<max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </part_log>
	
	<metric_log>
        <database>system</database>
        <table>metric_log</table>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <collect_interval_milliseconds>1000</collect_interval_milliseconds>
        <flush_on_crash>false</flush_on_crash>
    </metric_log>
	
	<asynchronous_metric_log>
        <database>system</database>
        <table>asynchronous_metric_log</table>
        <flush_interval_milliseconds>7000</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </asynchronous_metric_log>
	
	<opentelemetry_span_log>
        <engine>
            engine MergeTree
            partition by toYYYYMM(finish_date)
            order by (finish_date, finish_time_us, trace_id)
        </engine>
        <database>system</database>
        <table>opentelemetry_span_log</table>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </opentelemetry_span_log>
	
	<crash_log>
        <database>system</database>
        <table>crash_log</table>
        <partition_by />
        <flush_interval_milliseconds>1000</flush_interval_milliseconds>
        <max_size_rows>1024</max_size_rows>
        <reserved_size_rows>1024</reserved_size_rows>
        <buffer_size_rows_flush_threshold>512</buffer_size_rows_flush_threshold>
        <flush_on_crash>true</flush_on_crash>
    </crash_log>
	
	<processors_profile_log>
        <database>system</database>
        <table>processors_profile_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
    </processors_profile_log>

    <asynchronous_insert_log>
        <database>system</database>
        <table>asynchronous_insert_log</table>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <max_size_rows>1048576</max_size_rows>
        <reserved_size_rows>8192</reserved_size_rows>
        <buffer_size_rows_flush_threshold>524288</buffer_size_rows_flush_threshold>
        <flush_on_crash>false</flush_on_crash>
        <partition_by>event_date</partition_by>
        <ttl>event_date + INTERVAL 3 DAY</ttl>
    </asynchronous_insert_log>
	
	<backup_log>
        <database>system</database>
        <table>backup_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </backup_log>

    <s3queue_log>
        <database>system</database>
        <table>s3queue_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </s3queue_log>

    <blob_storage_log>
        <database>system</database>
        <table>blob_storage_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
        <ttl>event_date + INTERVAL 30 DAY</ttl>
    </blob_storage_log>
	
	<top_level_domains_lists></top_level_domains_lists>

    <dictionaries_config>*_dictionary.*ml</dictionaries_config>

    <dictionaries_lazy_load>true</dictionaries_lazy_load>

    <wait_dictionaries_load_at_startup>true</wait_dictionaries_load_at_startup>

    <user_defined_executable_functions_config>*_function.*ml</user_defined_executable_functions_config>

	<encryption_codecs></encryption_codecs>
	
	 <!-- 分布式DDL查询配置 -->
    <distributed_ddl>
        <path>/clickhouse/task_queue/ddl</path>
    </distributed_ddl>
	
	<graphite_rollup_example>
        <pattern>
            <regexp>click_cost</regexp>
            <function>any</function>
            <retention>
                <age>0</age>
                <precision>3600</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>60</precision>
            </retention>
        </pattern>
        <default>
            <function>max</function>
            <retention>
                <age>0</age>
                <precision>60</precision>
            </retention>
            <retention>
                <age>3600</age>
                <precision>300</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>3600</precision>
            </retention>
        </default>
    </graphite_rollup_example>


    <format_schema_path>/data/clickhouse/data/format_schemas/</format_schema_path>

    <google_protos_path>/usr/share/clickhouse/protos/</google_protos_path>
	
	<send_crash_reports>
        <enabled>false</enabled>
        <anonymize>false</anonymize>
        <endpoint>https://6f33034cfe684dd7a3ab9875e57b1c8d@o388870.ingest.sentry.io/5226277</endpoint>
    </send_crash_reports>

    <!-- 查询缓存配置 -->
    <query_cache>
        <max_size_in_bytes>1073741824</max_size_in_bytes>
        <max_entries>1024</max_entries>
        <max_entry_size_in_bytes>1048576</max_entry_size_in_bytes>
        <max_entry_size_in_rows>30000000</max_entry_size_in_rows>
    </query_cache>

    <!-- 备份配置 -->
    <backups>
        <allowed_path>backups</allowed_path>
        <remove_backup_files_after_failure>true</remove_backup_files_after_failure>
    </backups>

    <!-- 默认副本路径和名称模板 -->
    <default_replica_path>/clickhouse/tables/{shard}/{database}/{table}</default_replica_path>
    <default_replica_name>{replica}</default_replica_name>

</clickhouse>

在每台节点上创建配置文件 /etc/clickhouse-server/users.xml:

复制代码
<clickhouse>
    <!-- 另见users.d目录中的文件,可在其中覆盖这些设置 -->

    <!-- 设置配置文件 -->
    <profiles>
        <!-- 默认设置 -->
        <default>
            <max_query_size>1048576</max_query_size><!-- 设置为1024KB -->
            <max_result_bytes>100000000</max_result_bytes> <!-- 可选:禁止覆盖设置 -->
            <max_memory_usage>5905580032</max_memory_usage> <!-- 5GB -->
        </default>

        <!-- 仅允许读查询的配置文件 -->
        <readonly>
            <readonly>1</readonly>
        </readonly>
    </profiles>

    <!-- 用户和访问控制列表(ACL) -->
    <users>
        <!-- 如果未指定用户名,则使用'default'用户 -->
        <default>
            <!-- 另见users.d目录中的文件,可在其中覆盖密码设置

                 密码可以以明文或SHA256(十六进制格式)指定。

                 如果要以明文指定密码(不推荐),请将其放在'password'元素中。
                 示例:<password>qwerty</password>
                 密码可以为空。

                 如果要指定SHA256,请将其放在'password_sha256_hex'元素中。
                 示例:<password_sha256_hex>65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5</password_sha256_hex>
                 SHA256的限制:无法使用MySQL JS客户端连接ClickHouse(截至2019年7月)。

                 如果要指定双重SHA1,请将其放在'password_double_sha1_hex'元素中。
                 示例:<password_double_sha1_hex>e395796d6546b1b65db9d665cd43f0e858dd4303</password_double_sha1_hex>

                 如果要使用预定义的LDAP服务器(参见主配置中的'ldap_servers')进行认证,
                 请在'ldap'元素内的'server'元素中指定其名称。
                 示例:<ldap><server>my_ldap_server</server></ldap>

                 如果要通过Kerberos认证用户(假设已启用Kerberos,参见主配置中的'kerberos'),
                 请使用'kerberos'元素代替'password'(及类似)元素。
                 发起者的规范主体名称的名称部分必须与用户名匹配才能认证成功。
                 还可以在'kerberos'元素内添加'realm'元素,进一步限制只允许领域匹配的请求。
                 示例:<kerberos />
                 示例:<kerberos><realm>EXAMPLE.COM</realm></kerberos>

                 如何生成合适的密码:
                 执行:PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'
                 第一行是密码,第二行是对应的SHA256。

                 如何生成双重SHA1:
                 执行:PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'
                 第一行是密码,第二行是对应的双重SHA1。
            -->
            <password>*******</password>

            <!-- 允许访问的网络列表

                 要允许从任何地方访问,指定:
                    <ip>::/0</ip>

                 仅允许从本地主机访问,指定:
                    <ip>::1</ip>
                    <ip>127.0.0.1</ip>

                 列表中的每个元素可以是以下形式之一:
                 <ip> IP地址或网络掩码。例如:213.180.204.3 或 10.0.0.1/8 或 10.0.0.1/255.255.255.0
                     2a02:6b8::3 或 2a02:6b8::3/64 或 2a02:6b8::3/ffff:ffff:ffff:ffff::。
                 <host> 主机名。例如:server01.clickhouse.com。
                     为了检查访问权限,会执行DNS查询,并将所有接收到的地址与对等地址进行比较。
                 <host_regexp> 主机名的正则表达式。例如:^server\d\d-\d\d-\d\.clickhouse\.com$
                     为了检查访问权限,会对对等地址执行DNS PTR查询,然后应用正则表达式。
                     然后,对于PTR查询的结果,执行另一个DNS查询,并将所有接收到的地址与对等地址进行比较。
                     强烈建议正则表达式以$结尾
                 所有DNS请求的结果都会被缓存,直到服务器重启。
            -->
            <networks>
                <ip>::/0</ip>
            </networks>

            <!-- 用户的设置配置文件 -->
            <profile>default</profile>

            <!-- 用户的配额 -->
            <quota>default</quota>

            <!-- 用户可以创建其他用户并授予权限 -->
            <access_management>1</access_management>

            <!-- 用户可以操作命名集合 -->
            <named_collection_control>1</named_collection_control>

            <!-- 可以在此处授予用户权限 -->
            <!--
            <grants>
                <query>GRANT ALL ON *.*</query>
            </grants>
            -->
        </default>
    </users>

    <!-- 配额设置 -->
    <quotas>
        <!-- 配额名称 -->
        <default>
            <!-- 时间间隔的限制。可以指定多个具有不同限制的时间间隔 -->
            <interval>
                <!-- 间隔长度 -->
                <duration>3600</duration>

                <!-- 无限制。仅计算时间间隔内的资源使用情况 -->
                <queries>0</queries>
                <errors>0</errors>
                <result_rows>0</result_rows>
                <read_rows>0</read_rows>
                <execution_time>0</execution_time>
            </interval>
        </default>
    </quotas>
</clickhouse>

4、配置网络和防火墙

确保每台节点可以互相访问:

TCP端口9000(客户端连接和节点间通信)

TCP端口8123(HTTP接口)

TCP端口9009(复制数据)

5、启动服务

在每台节点上启动ClickHouse服务:

复制代码
systemctl start clickhouse-server

6、验证集群

在任一节点上执行以下命令验证集群配置:

sql

SELECT * FROM system.clusters WHERE cluster='node_cluster';

应该能看到每个节点的信息。

7、创建分布式表

sql

-- 在每个节点上创建本地表

CREATE TABLE local_table ON CLUSTER two_node_cluster (

id UInt32,

date Date,

value Float64

) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/local_table', '{replica}')

PARTITION BY toYYYYMM(date)

ORDER BY (date, id);

-- 创建分布式表

CREATE TABLE distributed_table ON CLUSTER two_node_cluster AS local_table

ENGINE = Distributed(two_node_cluster, default, local_table, rand());

8、测试集群

插入数据测试集群功能:

sql

-- 向分布式表插入数据

INSERT INTO distributed_table VALUES

(1, '2023-01-01', 10.5),

(2, '2023-01-02', 11.3);

-- 查询数据

SELECT * FROM distributed_table;

9、注意事项

确保两台服务器的时间同步(使用NTP)

生产环境建议使用DNS名称而非IP地址

对于高可用性,建议配置3节点ZooKeeper集群

10、操作示例

删除本地表

复制代码
DROP TABLE nidb_biz_ck.biz_gpsdatas_source_local  ON CLUSTER lwlk_cluster SYNC;

客户端命令迁移表数据

复制代码
clickhouse-client --query "insert into nidb_biz_ck.biz_track_data809_sub_normal_new1 select * from nidb_biz_ck.biz_track_data809_sub_normal where toDate(gps_time) >= '2025-07-01' AND toDate(gps_time) <= '2025-07-07';" &

修改表引擎

  1. DETACH 表,保留数据
    DETACH TABLE nidb_biz_ck.biz_ai_alarm_info_local;
  2. 修改元数据文件(关键步骤)
    ClickHouse 的元数据默认在 /var/lib/clickhouse/metadata/{database}/{table}.sql

修改文件引擎 ReplicatedMergeTree 为 ReplicatedReplacingMergeTree

  1. 删除分区

    SYSTEM DROP REPLICA 'app4' FROM ZKPATH '/clickhouse/tables/nidb_biz_ck/01/biz_ai_alarm_info_local';

  2. ATTACH 回表

    ATTACH TABLE nidb_biz_ck.biz_ai_alarm_info_local;

  3. restore表

    system restore replica nidb_biz_ck.biz_ai_alarm_info_local;

  4. sync表

    system sync replica nidb_biz_ck.biz_ai_alarm_info_local;

相关推荐
风中凌乱2 小时前
ClickHouse-Backup的安装与部署
clickhouse
白眼黑刺猬2 小时前
ClickHouse从入门到企业级实战全解析课程简介
clickhouse
chenglin0163 天前
ClickHouse、Doris、OpenSearch、Splunk、Solr系统化分析
clickhouse·solr·lucene
慕y2743 天前
Java学习第一百一十七部分——ClickHouse
java·学习·clickhouse
zuozewei9 天前
随笔之 ClickHouse 列式分析数据库安装注意事项及基准测试
数据库·clickhouse
牛牛木有坏心眼(大数据进阶)10 天前
linux系统离线环境安装clickhouse客户端
linux·clickhouse
许心月10 天前
Clickhouse#表记录转换为insert语句
clickhouse
许心月10 天前
Clickhouse#记录隐藏字段
clickhouse
weixin_3077791310 天前
ClickHouse Windows迁移方案与测试
linux·c++·数据仓库·windows·clickhouse