clickhouse-20版本安装部署

**1,安装包系在路径:

https://mirrors.aliyun.com/clickhouse/tgz/stable/\*\*

2,环境准备

1.配置主机名

hostnamectl set-hostname ${hostname}

2.配置hosts

ip hostname

3.关闭防火墙

vim /etc/selinux/config 改为 SELINUX=disabled

ps:

1)关闭systemctl stop firewalld

2)查看防火墙端口是否开发

lsof -i:80 or netstat -apnt |grep 80

重启防火墙:systemctl restart firewalld.service

查看所有开放端口:firewall-cmd --list-ports

查看防火墙状态:firewall-cmd --state

开启防火墙:systemctl start firewalld.service

设置防火墙开机自启动:systemctl enable firewalld.service

重启防火墙:systemctl restart firewalld.service

3,clickhouse 前期准备

1.创建用户

useradd clickhouse

passwd clickhouse

2.clickhouse用户授权

visudo

添加 clickhouse ALL=(ALL) ALL

4.jdk 安装

5.zk 安装

三:安装clickhouse

1.安装

sudo rpm -ivh clickhouse-*.rpm
2. 修改config.xml 文件

yaml 复制代码
<?xml version="1.0"?>
<yandex>
    <!--根据实际替换-->    
    <listen_host>::1</listen_host>
   <listen_host>0.0.0.0</listen_host>
    <include_from>/etc/clickhouse-server/metrika.xml</include_from>
    <remote_url_allow_hosts></remote_url_allow_hosts>
    <logger>
        <level>trace</level>
        <log>/var/log/clickhouse-server/clickhouse-server.log</log>
        <errorlog>/var/log/clickhouse-server/clickhouse-server.err.log</errorlog>
        <size>1000M</size>
        <count>10</count>
    </logger>
    <!--根据实际替换-->
    <http_port>8123</http_port>
    <tcp_port>9000</tcp_port>
    <openSSL>
        <server>
            <certificateFile>/etc/clickhouse-server/server.crt</certificateFile>
            <privateKeyFile>/etc/clickhouse-server/server.key</privateKeyFile>
            <dhParamsFile>/etc/clickhouse-server/dhparam.pem</dhParamsFile>
            <verificationMode>none</verificationMode>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
        </server>
        <client>
            <loadDefaultCAFile>true</loadDefaultCAFile>
            <cacheSessions>true</cacheSessions>
            <disableProtocols>sslv2,sslv3</disableProtocols>
            <preferServerCiphers>true</preferServerCiphers>
            <invalidCertificateHandler>
                <name>RejectCertificateHandler</name>
            </invalidCertificateHandler>
        </client>
    </openSSL>
    <interserver_http_port>9009</interserver_http_port>
    <max_connections>4096</max_connections>
    <keep_alive_timeout>600</keep_alive_timeout>
    <max_concurrent_queries>150</max_concurrent_queries>
    <uncompressed_cache_size>8589934592</uncompressed_cache_size>
    <mark_cache_size>5368709120</mark_cache_size>
    <path>/var/lib/clickhouse/</path>
    <tmp_path>/var/lib/clickhouse/tmp/</tmp_path>
    <user_files_path>/var/lib/clickhouse/user_files/</user_files_path>
    <users_config>users.xml</users_config>
    <default_profile>default</default_profile>
    <default_database>default</default_database>
    <default_replica_path>/clickhouse/tables/{shard}/{database}/{table}</default_replica_path>
    <default_replica_name>{replica}</default_replica_name>
    <mlock_executable>false</mlock_executable>
    <remote_servers incl="clickhouse_remote_servers" ></remote_servers>
    <zookeeper incl="zookeeper-servers" optional="true" />
    <macros incl="macros" optional="true" />
    <builtin_dictionaries_reload_interval>3600</builtin_dictionaries_reload_interval>
    <max_session_timeout>3600</max_session_timeout>
    <default_session_timeout>60</default_session_timeout>
    <format_schema_path>/var/lib/clickhouse/format_schemas/</format_schema_path>
    <query_log>
        <database>system</database>
        <table>query_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_log>
    <trace_log>
        <database>system</database>
        <table>trace_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </trace_log>
    <query_thread_log>
        <database>system</database>
        <table>query_thread_log</table>
        <partition_by>toYYYYMM(event_date)</partition_by>
        <flush_interval_milliseconds>7500</flush_interval_milliseconds>
    </query_thread_log>
    <dictionaries_config>*_dictionary.xml</dictionaries_config>
    <compression incl="clickhouse_compression"></compression>
    <distributed_ddl>
        <path>/clickhouse/task_queue/ddl</path>
    </distributed_ddl>
    <graphite_rollup_example>
        <pattern>
            <regexp>click_cost</regexp>
            <function>any</function>
            <retention>
                <age>0</age>
                <precision>3600</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>60</precision>
            </retention>
        </pattern>
        <default>
            <function>max</function>
            <retention>
                <age>0</age>
                <precision>60</precision>
            </retention>
            <retention>
                <age>3600</age>
                <precision>300</precision>
            </retention>
            <retention>
                <age>86400</age>
                <precision>3600</precision>
            </retention>
        </default>
    </graphite_rollup_example>
</yandex>

2.修改users.xml

yaml 复制代码
<?xml version="1.0"?>
<yandex>
    <!-- Profiles of settings. -->
    <profiles>
        <!-- Default settings. -->
        <default>
            <!-- Maximum memory usage for processing single query, in bytes. -->
            <max_memory_usage>10000000000</max_memory_usage>

            <!-- Use cache of uncompressed blocks of data. Meaningful only for processing many of very short queries. -->
            <use_uncompressed_cache>0</use_uncompressed_cache>
             <skip_unavailable_shards>1</skip_unavailable_shards>  <!-- 1=true,0=false -->
            <!-- How to choose between replicas during distributed query processing.
                 random - choose random replica from set of replicas with minimum number of errors
                 nearest_hostname - from set of replicas with minimum number of errors, choose replica
                  with minimum number of different symbols between replica's hostname and local hostname
                  (Hamming distance).
                 in_order - first live replica is chosen in specified order.
                 first_or_random - if first replica one has higher number of errors, pick a random one from replicas with minimum number of errors.
            -->
            <load_balancing>random</load_balancing>
        </default>

        <!-- Profile that allows only read queries. -->
        <readonly>
            <readonly>1</readonly>
        </readonly>
     
    </profiles>

    <!-- Users and ACL. -->
    <users>
        <!-- If user name was not specified, 'default' user is used. -->
        <default>
            <!-- Password could be specified in plaintext or in SHA256 (in hex format).

                 If you want to specify password in plaintext (not recommended), place it in 'password' element.
                 Example: <password>qwerty</password>.
                 Password could be empty.

                 If you want to specify SHA256, place it in 'password_sha256_hex' element.
                 Example: <password_sha256_hex>65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5</password_sha256_hex>
                 Restrictions of SHA256: impossibility to connect to ClickHouse using MySQL JS client (as of July 2019).

                 If you want to specify double SHA1, place it in 'password_double_sha1_hex' element.
                 Example: <password_double_sha1_hex>e395796d6546b1b65db9d665cd43f0e858dd4303</password_double_sha1_hex>

                 If you want to specify a previously defined LDAP server (see 'ldap_servers' in main config) for authentication, place its name in 'server' element inside 'ldap' element.
                 Example: <ldap><server>my_ldap_server</server></ldap>

                 How to generate decent password:
                 Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-'
                 In first line will be password and in second - corresponding SHA256.

                 How to generate double SHA1:
                 Execute: PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-'
                 In first line will be password and in second - corresponding double SHA1.
            -->
            <password></password>

            <!-- List of networks with open access.

                 To open access from everywhere, specify:
                    <ip>::/0</ip>

                 To open access only from localhost, specify:
                    <ip>::1</ip>
                    <ip>127.0.0.1</ip>

                 Each element of list has one of the following forms:
                 <ip> IP-address or network mask. Examples: 213.180.204.3 or 10.0.0.1/8 or 10.0.0.1/255.255.255.0
                     2a02:6b8::3 or 2a02:6b8::3/64 or 2a02:6b8::3/ffff:ffff:ffff:ffff::.
                 <host> Hostname. Example: server01.yandex.ru.
                     To check access, DNS query is performed, and all received addresses compared to peer address.
                 <host_regexp> Regular expression for host names. Example, ^server\d\d-\d\d-\d\.yandex\.ru$
                     To check access, DNS PTR query is performed for peer address and then regexp is applied.
                     Then, for result of PTR query, another DNS query is performed and all received addresses compared to peer address.
                     Strongly recommended that regexp is ends with $
                 All results of DNS requests are cached till server restart.
            -->
            <networks incl="networks" replace="replace">
                <ip>::/0</ip>
            </networks>

            <!-- Settings profile for user. -->
            <profile>default</profile>

            <!-- Quota for user. -->
            <quota>default</quota>

            <!-- User can create other users and grant rights to them. -->
            <!-- <access_management>1</access_management> -->
        </default>
    </users>

    <!-- Quotas. -->
    <quotas>
        <!-- Name of quota. -->
        <default>
            <!-- Limits for time interval. You could specify many intervals with different limits. -->
            <interval>
                <!-- Length of interval. -->
                <duration>3600</duration>

                <!-- No limits. Just calculate resource usage for time interval. -->
                <queries>0</queries>
                <errors>0</errors>
                <result_rows>0</result_rows>
                <read_rows>0</read_rows>
                <execution_time>0</execution_time>
            </interval>
        </default>
    </quotas>
</yandex>

这里重点添加了<skip_unavailable_shards>1</skip_unavailable_shards>
3.配置metrika.xml 文件

yaml 复制代码
<?xml version="1.0"?>
<yandex>
    <!-- 集群配置(正确) -->
    <clickhouse_remote_servers>
        <shards_0replicas>  <!-- 集群名称需与 SQL 中 ON CLUSTER 一致 -->
            <!-- 分片1 -->
            <shard>
                <replica>  <!-- 20.x 必须用 <replica> 包裹节点 -->
                    <host>bigdata57</host>  <!-- 节点 IP -->
                    <port>9000</port>  <!-- TCP 端口(正确) -->
                </replica>
            </shard>
            <!-- 分片2 -->
            <shard>
                <replica>
                    <host>bigdata93</host>
                    <port>9000</port>
                </replica>
            </shard>
        </shards_0replicas>
    </clickhouse_remote_servers>

    <!-- ZooKeeper 配置(修正标签:<host> → <node>) -->
    <zookeeper-servers>
        <node index="1">  <!-- 正确标签:<node> -->
            <host>bigdata57</host>  <!-- 节点主机名(内层 <host> 正确) -->
            <port>2181</port>  <!-- ZooKeeper 端口(正确) -->
        </node>
        <node index="2">
            <host>bigdata93</host>
            <port>2181</port>
        </node>
        <node index="3">
            <host>bigdata70</host>
            <port>2181</port>
        </node>
    </zookeeper-servers>

    <!-- 网络配置(正确) -->
    <networks>
        <ip>::/0</ip>  <!-- 允许所有 IP 访问 -->
    </networks>

    <!-- 压缩配置(正确) -->
    <clickhouse_compression>
        <case>
            <min_part_size>10000000000</min_part_size>
            <min_part_size_ratio>0.01</min_part_size_ratio>
            <method>lz4</method>
        </case>
    </clickhouse_compression>
</yandex>

注意格式一定要按照图中的来,新版本和这个版本区别很大

启动

service clickhouse-server start

验证

相关推荐
先吃饱再说8 小时前
存储的进化:从 MySQL 到浏览器缓存,数据到底住在哪?
数据库
Nturmoils8 小时前
字段太多看不全,ksql 的展开模式和输出控制怎么用
数据库·后端
Databend10 小时前
Agent 轨迹分析与归因的数据工程实践
大数据·数据库·agent
这个DBA有点耶11 小时前
SQL改写进阶:标量子查询的“隐形代价”与消除实战
数据库·mysql·架构
smallyoung12 小时前
数据库乐观锁深度解析:MySQL、PostgreSQL 实战 + Spring Boot 集成指南
数据库·mysql·postgresql
parade岁月12 小时前
MySQL JOIN解析:朴实无华但食之有味
数据库·后端
用户31693538118313 小时前
MySQL服务无法启动问题解决全记录
数据库
vivo互联网技术16 小时前
从 10 分钟到 1 秒:ES 深度分页任意跳页的三轮优化实战
服务器·数据库·redis·elasticsearch·深度分页
数据技术说16 小时前
MySQL 迁移实战——如何实现真正的"零改造"平滑切换
mysql
倔强的石头_1 天前
《Kingbase护城河》——猎捕慢查询:执行计划的微观解析与索引调优实战
数据库