TDEngine-OSS-3.3.7.5开源版搭建手册(包含单节点与三副本高可用方案搭建)

包含单节点与三副本方案

单节点请看第1章节即可,三副本方案请看全部

针对高可用方案:开源版本仅支持三副本方案,各种高可用方案对比如下

# 三副本 双副本(企业版) 双活(企业版)
集群数目 部署一个集群 部署一个集群 部署两个不同集群
最小节点数 三个数据节点 两个数据节点,一个仲裁节点 两个数据节点
选主原理 Raft 协议 管理节点中仲裁选主 无需选主
同步原理 Raft 协议 Raft 协议 通过 taosX 进行数据同步
同步延迟 无延迟 无延迟 依赖于 taosX 的同步速度,秒级延迟
数据安全性 无数据丢失 无数据丢失 依赖于 WAL 的保存时长
数据一致性 RAFT 一致性 RAFT 一致性 最终一致性
高可用性 任一节点宕机不影响服务 任一节点宕机不影响服务,但不能处理连续宕机场景 一个实例存活即可提供服务

企业版区别如下,可酌情选择是否购买


网络端口开放情况和服务器所需资源要求请见官网

1 单节点搭建(也是三副本方案主节点搭建方案)

1-1 下载安装包

powershell 复制代码
# 创建文件夹并下载涛思压缩包、解压
mkdir /root/TDinstall-v3.3.7.5 && cd /root/TDinstall-v3.3.7.5 && wget https://downloads.taosdata.com/tdengine-tsdb-oss/3.3.7.5/tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz && tar -zxvf tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz

1-2 安装涛思基本环境

执行后无特殊配置,一直enter即可

shell 复制代码
/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/install.sh

1-3 执行脚本配置基础信息,路径可自行更改

shell 复制代码
cd /etc/taos
# 日志保存路径、保留时长
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taosadapter.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taosadapter.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taoskeeper.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' explorer.toml
sed -i 's/# keepDays = 30/keepDays = 10/g' explorer.toml
# 用户名、密码更改-自行判断是否需要更改
#sed -i 's/username = "root"/username = "admin"/g' taoskeeper.toml
#sed -i 's/password = "taosdata"/password = "hello_world"/g' taoskeeper.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taoskeeper.toml
# 实例id更改,三副本方案必须更改
sed -i 's/# instanceId = 1/instanceId = 2/g' explorer.toml
sed -i 's/instanceId = 32/instanceId = 102/g' taosadapter.toml
sed -i 's/instanceId = 64/instanceId = 152/g' taoskeeper.toml
grep '^instanceId' *

1-4 涛思配置文件更改

日志和数据目录自行配置,fqdn为当前主机的hostname,hostname不可为localhost,如果为localhost需要先配置为node1(自定义,与下面fqdn相同)命令:hostname set-hostname node1

备份配置文件并将下面的配置放入配置中:mv /etc/taos/taos.cfg /etc/taos/taos.cfg.bak && vim /etc/taos/taos.cfg

shell 复制代码
#firstEp                    node1:6030
#secondEp                   node2:6030
fqdn                       node1
#supportVnodes               96
logDir                     /data/taos/log
dataDir                    /data/taos/data
tempDir                    /data/taos/tmp
keepColumnName              1
maxNumOfDistinctRes         10000000
timezone                    UTC-8
locale                      en_US.UTF-8
charset                     UTF-8
maxShellConns               100000
maxConnections              100000
audit                       0
auditFqdn                   localhost
monitor                     1
monitorFqdn                 localhost
logKeepDays                 10
debugflag                   131
shellActivityTimer          120
numOfRpcSessions            30000
numOfCommitThreads          16
#numOfVnodeQueryThreads      80
queryUseMemoryPool          0
forceReadConfig             1

1-5 创建涛思相关目录并配置相关参数

将下面脚本保存为文件并执行

shell 复制代码
#!/bin/bash
# 创建目录
mkdir -p /data/taos/{data,log,core,tmp,soft,dump}
chmod 777 /data/taos/log
chmod 777 /data/taos/tmp

# 资源限制
echo "fs.nr_open = 2147483584" >>/etc/sysctl.conf
echo "fs.file-max = 2147483584" >>/etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 10000 65534" >>/etc/sysctl.conf

echo "* soft nproc  65536" >>/etc/security/limits.conf
echo "* soft nofile 1048576" >>/etc/security/limits.conf
echo "* soft stack  65536" >>/etc/security/limits.conf
echo "* hard nproc  65536" >>/etc/security/limits.conf
echo "* hard nofile 1048576" >>/etc/security/limits.conf
echo "* hard stack  65536" >>/etc/security/limits.conf

echo "root soft nproc  65536" >>/etc/security/limits.conf
echo "root soft nofile 1048576" >>/etc/security/limits.conf
echo "root soft stack  65536" >>/etc/security/limits.conf
echo "root hard nproc  65536" >>/etc/security/limits.conf
echo "root hard nofile 1048576" >>/etc/security/limits.conf
echo "root hard stack  65536" >>/etc/security/limits.conf
# 设置core目录
echo "ulimit -c unlimited" >>/etc/profile
echo "kernel.core_pattern=/data/taos/core/core-%e-%p" >>/etc/sysctl.conf

# 关闭SELinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

# 关闭防火墙(云服务器请至少开放6060、6030、6041端口)
systemctl stop firewalld
systemctl disable firewalld

1-6 启动涛思程序

执行命令/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/start-all.sh

2 节点2搭建(三副本方案,单节点执行以上操作即可)

2-1 下载安装包

powershell 复制代码
# 创建文件夹并下载涛思压缩包、解压
mkdir /root/TDinstall-v3.3.7.5 && cd /root/TDinstall-v3.3.7.5 && wget https://downloads.taosdata.com/tdengine-tsdb-oss/3.3.7.5/tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz && tar -zxvf tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz

2-2 安装涛思基本环境(与单节点不同)

执行后无特殊配置,弹框配置fqdn要求时,输入node1:6030(主节点的hostname加6030端口)

shell 复制代码
/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/install.sh

2-3 执行脚本配置基础信息,路径可自行更改(注意实例id不能与其他实例相同)

shell 复制代码
cd /etc/taos
# 日志保存路径、保留时长
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taosadapter.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taosadapter.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taoskeeper.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' explorer.toml
sed -i 's/# keepDays = 30/keepDays = 10/g' explorer.toml
# 用户名、密码更改-自行判断是否需要更改
#sed -i 's/username = "root"/username = "admin"/g' taoskeeper.toml
#sed -i 's/password = "taosdata"/password = "hello_world"/g' taoskeeper.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taoskeeper.toml
# 实例id更改,与节点一搭建的实例id不能相同
sed -i 's/# instanceId = 1/instanceId = 3/g' explorer.toml
sed -i 's/instanceId = 32/instanceId = 103/g' taosadapter.toml
sed -i 's/instanceId = 64/instanceId = 153/g' taoskeeper.toml
grep '^instanceId' *

2-4 涛思配置文件更改

日志和数据目录自行配置,fqdn为当前主机的hostname,hostname不可为localhost,如果为localhost需要先配置为node2(自定义,与下面fqdn相同)命令:hostname set-hostname node2

备份配置文件并将下面的配置放入配置中:mv /etc/taos/taos.cfg /etc/taos/taos.cfg.bak && vim /etc/taos/taos.cfg

shell 复制代码
firstEp                    node1:6030
secondEp                   node2:6030
fqdn                       node2
#supportVnodes               96
logDir                     /data/taos/log
dataDir                    /data/taos/data
tempDir                    /data/taos/tmp
keepColumnName              1
maxNumOfDistinctRes         10000000
timezone                    UTC-8
locale                      en_US.UTF-8
charset                     UTF-8
maxShellConns               100000
maxConnections              100000
audit                       0
auditFqdn                   localhost
monitor                     1
monitorFqdn                 localhost
logKeepDays                 10
debugflag                   131
shellActivityTimer          120
numOfRpcSessions            30000
numOfCommitThreads          16
#numOfVnodeQueryThreads      80
queryUseMemoryPool          0
forceReadConfig             1

2-5 创建涛思相关目录并配置相关参数

将下面脚本保存为文件并执行

shell 复制代码
#!/bin/bash
# 创建目录
mkdir -p /data/taos/{data,log,core,tmp,soft,dump}
chmod 777 /data/taos/log
chmod 777 /data/taos/tmp

# 资源限制
echo "fs.nr_open = 2147483584" >>/etc/sysctl.conf
echo "fs.file-max = 2147483584" >>/etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 10000 65534" >>/etc/sysctl.conf

echo "* soft nproc  65536" >>/etc/security/limits.conf
echo "* soft nofile 1048576" >>/etc/security/limits.conf
echo "* soft stack  65536" >>/etc/security/limits.conf
echo "* hard nproc  65536" >>/etc/security/limits.conf
echo "* hard nofile 1048576" >>/etc/security/limits.conf
echo "* hard stack  65536" >>/etc/security/limits.conf

echo "root soft nproc  65536" >>/etc/security/limits.conf
echo "root soft nofile 1048576" >>/etc/security/limits.conf
echo "root soft stack  65536" >>/etc/security/limits.conf
echo "root hard nproc  65536" >>/etc/security/limits.conf
echo "root hard nofile 1048576" >>/etc/security/limits.conf
echo "root hard stack  65536" >>/etc/security/limits.conf
# 设置core目录
echo "ulimit -c unlimited" >>/etc/profile
echo "kernel.core_pattern=/data/taos/core/core-%e-%p" >>/etc/sysctl.conf

# 关闭SELinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

# 关闭防火墙(云服务器请至少开放6060、6030、6041端口)
systemctl stop firewalld
systemctl disable firewalld

2-6 启动涛思程序

执行命令/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/start-all.sh

3 节点3搭建(三副本方案,单节点执行以上操作即可)

3-1 下载安装包

powershell 复制代码
# 创建文件夹并下载涛思压缩包、解压
mkdir /root/TDinstall-v3.3.7.5 && cd /root/TDinstall-v3.3.7.5 && wget https://downloads.taosdata.com/tdengine-tsdb-oss/3.3.7.5/tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz && tar -zxvf tdengine-tsdb-oss-3.3.7.5-linux-x64.tar.gz

3-2 安装涛思基本环境(与单节点不同)

执行后无特殊配置,弹框配置fqdn要求时,输入node1:6030(主节点的hostname加6030端口)

shell 复制代码
/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/install.sh

3-3 执行脚本配置基础信息,路径可自行更改(注意实例id不能与其他实例相同)

shell 复制代码
cd /etc/taos
# 日志保存路径、保留时长
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taosadapter.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taosadapter.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' taoskeeper.toml
sed -i 's/# path = "\/var\/log\/taos"/path = "\/data\/taos\/log"/g' explorer.toml
sed -i 's/# keepDays = 30/keepDays = 10/g' explorer.toml
# 用户名、密码更改-自行判断是否需要更改
#sed -i 's/username = "root"/username = "admin"/g' taoskeeper.toml
#sed -i 's/password = "taosdata"/password = "hello_world"/g' taoskeeper.toml
sed -i 's/keepDays = 30/keepDays = 10/g' taoskeeper.toml
# 实例id更改,与其他搭建的实例id不能相同
sed -i 's/# instanceId = 1/instanceId = 4/g' explorer.toml
sed -i 's/instanceId = 32/instanceId = 104/g' taosadapter.toml
sed -i 's/instanceId = 64/instanceId = 154/g' taoskeeper.toml
grep '^instanceId' *

3-4 涛思配置文件更改

日志和数据目录自行配置,fqdn为当前主机的hostname,hostname不可为localhost,如果为localhost需要先配置为node3(自定义,与下面fqdn相同)命令:hostname set-hostname node3

备份配置文件并将下面的配置放入配置中:mv /etc/taos/taos.cfg /etc/taos/taos.cfg.bak && vim /etc/taos/taos.cfg

shell 复制代码
firstEp                    node1:6030
# 配置备用节点的endpoint
secondEp                   node2:6030
fqdn                       node3
#supportVnodes               96
logDir                     /data/taos/log
dataDir                    /data/taos/data
tempDir                    /data/taos/tmp
keepColumnName              1
maxNumOfDistinctRes         10000000
timezone                    UTC-8
locale                      en_US.UTF-8
charset                     UTF-8
maxShellConns               100000
maxConnections              100000
audit                       0
auditFqdn                   localhost
monitor                     1
monitorFqdn                 localhost
logKeepDays                 10
debugflag                   131
shellActivityTimer          120
numOfRpcSessions            30000
numOfCommitThreads          16
#numOfVnodeQueryThreads      80
queryUseMemoryPool          0
forceReadConfig             1

3-5 创建涛思相关目录并配置相关参数

将下面脚本保存为文件并执行

shell 复制代码
#!/bin/bash
# 创建目录
mkdir -p /data/taos/{data,log,core,tmp,soft,dump}
chmod 777 /data/taos/log
chmod 777 /data/taos/tmp

# 资源限制
echo "fs.nr_open = 2147483584" >>/etc/sysctl.conf
echo "fs.file-max = 2147483584" >>/etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 10000 65534" >>/etc/sysctl.conf

echo "* soft nproc  65536" >>/etc/security/limits.conf
echo "* soft nofile 1048576" >>/etc/security/limits.conf
echo "* soft stack  65536" >>/etc/security/limits.conf
echo "* hard nproc  65536" >>/etc/security/limits.conf
echo "* hard nofile 1048576" >>/etc/security/limits.conf
echo "* hard stack  65536" >>/etc/security/limits.conf

echo "root soft nproc  65536" >>/etc/security/limits.conf
echo "root soft nofile 1048576" >>/etc/security/limits.conf
echo "root soft stack  65536" >>/etc/security/limits.conf
echo "root hard nproc  65536" >>/etc/security/limits.conf
echo "root hard nofile 1048576" >>/etc/security/limits.conf
echo "root hard stack  65536" >>/etc/security/limits.conf
# 设置core目录
echo "ulimit -c unlimited" >>/etc/profile
echo "kernel.core_pattern=/data/taos/core/core-%e-%p" >>/etc/sysctl.conf

# 关闭SELinux
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

# 关闭防火墙(云服务器请至少开放6060、6030、6041端口)
systemctl stop firewalld
systemctl disable firewalld

3-6 启动涛思程序

执行命令/root/TDinstall-v3.3.7.5/tdengine-tsdb-oss-3.3.7.5/start-all.sh

三副本数据节点和主从节点配置

图一

图二

shell 复制代码
taos
create dnode "node2:6030";
create dnode "node3:6030";
show dnodes;
# 下面的2和3分别对应 show dnodes; 查询出来的node2和node3对应的id,如上图一所示
create mnode on dnode 2;
create mnode on dnode 3;
# 执行下面命令查看割接点主从状态,如图二
show mnodes;
create database test_3_replica replica 3 vgroups 1 buffer 1024;

如果执行完成后各数据节点为ready状态,停掉主节点等待一会儿重新执行show mnodes;如下图所示,自动推举其他节点为主节点则说明三副本方案搭建成功

相关推荐
TDengine (老段)3 小时前
TDengine 聚合函数 SPREAD 用户手册
大数据·数据库·sql·物联网·时序数据库·tdengine·涛思数据
TDengine (老段)3 小时前
TDengine 时区配置问题全解
大数据·数据库·时序数据库·tdengine·涛思数据
海豚调度6 小时前
3.1.9 生产“稳”担当:Master 服务启动源码全方位解析
大数据·开源·任务调度·大数据调度·apache dolphinscheduler
代码匠心1 天前
从零开始学Flink:数据转换的艺术
java·大数据·flink
在狂风暴雨中奔跑4 天前
厌倦了复杂的编译?一键集成 AeroFFmpeg,让Android音视频开发更简单!
ffmpeg·开源
武子康4 天前
大数据-102 Spark Streaming 与 Kafka 集成全解析:Receiver 与 Direct 两种方式详解 附代码案例
大数据·后端·spark
杨杨杨大侠4 天前
Atlas Mapper 案例 03:企业级订单实体设计文档
java·开源·github
杨杨杨大侠4 天前
手把手教你写 httpclient 框架(二)- 核心注解系统设计与实现
java·开源·github
FreeCode4 天前
StarRocks表设计之数据分布策略
大数据