一、NewSQL简介
NewSQL是新一代分布式关系型数据库,结合了SQL的便利性和NoSQL的可扩展性:
解决的问题:
- 传统关系型数据库无法水平扩展
- NoSQL不支持完整的SQL和事务
- 需要强一致性但又要高可扩展性
代表产品:
- TiDB(PingCAP)
- CockroachDB(CockroachLabs)
- OceanBase(蚂蚁金服)
- YugabyteDB
二、TiDB实战
1. 架构设计
TiDB Server(SQL层,无状态)连接 TiKV 集群(TiKV 1/2/3,每个TiKV管理多个Region),PD Server负责调度。
核心组件:
- TiDB Server:SQL解析和执行,无状态
- TiKV:分布式KV存储引擎(RocksDB)
- PD:集群调度器
2. Docker部署
yaml
version: '3'
services:
pd:
image: pingcap/pd:latest
container_name: pd
ports:
- "2379:2379"
volumes:
- ./pd-data:/pd-data
command: --name=pd --client-urls=http://0.0.0.0:2379
--advertise-client-urls=http://pd:2379
--initial-cluster=pd=http://pd:2380
tikv1:
image: pingcap/tikv:latest
container_name: tikv1
ports:
- "20160:20160"
volumes:
- ./tikv1-data:/tikv-data
depends_on:
- pd
command: --addr=0.0.0.0:20160 --advertise-addr=tikv1:20160
--pd=pd:2379 --data-dir=/tikv-data
tikv2:
image: pingcap/tikv:latest
container_name: tikv2
ports:
- "20161:20160"
volumes:
- ./tikv2-data:/tikv-data
depends_on:
- pd
command: --addr=0.0.0.0:20160 --advertise-addr=tikv2:20160
--pd=pd:2379 --data-dir=/tikv-data
tidb:
image: pingcap/tidb:latest
container_name: tidb
ports:
- "4000:4000"
depends_on:
- pd
- tikv1
- tikv2
command: --store=tikv --path=pd:2379
3. MySQL兼容
TiDB完全兼容MySQL协议:
sql
CREATE TABLE orders (
id BIGINT AUTO_INCREMENT PRIMARY KEY,
order_no VARCHAR(32) NOT NULL,
user_id BIGINT NOT NULL,
shop_id BIGINT NOT NULL,
order_amount DECIMAL(12,2) NOT NULL,
order_status TINYINT DEFAULT 1,
create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
INDEX idx_user_id (user_id),
INDEX idx_shop_id (shop_id),
INDEX idx_order_no (order_no)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
BEGIN;
INSERT INTO orders (order_no, user_id, shop_id, order_amount)
VALUES ('ORDER001', 1001, 2001, 99.99);
UPDATE inventory SET stock = stock - 1 WHERE product_id = 3001;
COMMIT;
BEGIN PESSIMISTIC;
SELECT * FROM orders WHERE id = 1 FOR UPDATE;
UPDATE orders SET order_status = 2 WHERE id = 1;
COMMIT;
4. Java客户端
java
@Configuration
public class TiDBConfig {
@Bean
public DataSource dataSource() {
HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:mysql://localhost:4000/order_db");
config.setUsername("root");
config.setPassword("");
config.setMaximumPoolSize(50);
config.setMinimumIdle(10);
config.setConnectionTimeout(30000);
return new HikariDataSource(config);
}
}
@Service
public class OrderService {
@Autowired
private DataSource dataSource;
@Transactional
public void createOrder(OrderRequest request) {
try (Connection conn = dataSource.getConnection();
PreparedStatement ps = conn.prepareStatement(
"INSERT INTO orders (order_no, user_id, shop_id, order_amount) VALUES (?, ?, ?, ?)")) {
ps.setString(1, request.getOrderNo());
ps.setLong(2, request.getUserId());
ps.setLong(3, request.getShopId());
ps.setBigDecimal(4, request.getAmount());
ps.executeUpdate();
}
}
}
5. HTAP能力
TiDB同时支持OLTP和OLAP:
sql
ALTER TABLE orders SET TIFLASH REPLICA 2;
SELECT
DATE(create_time) as order_date,
shop_id,
COUNT(*) as order_count,
SUM(order_amount) as total_amount
FROM orders
WHERE create_time >= '2024-01-01'
GROUP BY DATE(create_time), shop_id
ORDER BY total_amount DESC
LIMIT 10;
三、CockroachDB实战
1. 架构设计
Gateway Node(请求路由,无状态)连接多个Node节点,每个Node包含Store,默认3副本强一致(Raft协议)。
核心特性:
- 多副本强一致性(Raft协议)
- 自动分片和负载均衡
- 故障自动恢复
- 地理分布支持
2. Docker部署
yaml
version: '3'
services:
cockroach1:
image: cockroachdb/cockroach:v23.1
container_name: cockroach1
ports:
- "26257:26257"
- "8080:8080"
command: start --insecure --join=cockroach1,cockroach2,cockroach3
volumes:
- ./cockroach1-data:/cockroach/cockroach-data
cockroach2:
image: cockroachdb/cockroach:v23.1
container_name: cockroach2
ports:
- "26258:26257"
command: start --insecure --join=cockroach1,cockroach2,cockroach3
volumes:
- ./cockroach2-data:/cockroach/cockroach-data
cockroach3:
image: cockroachdb/cockroach:v23.1
container_name: cockroach3
ports:
- "26259:26257"
command: start --insecure --join=cockroach1,cockroach2,cockroach3
volumes:
- ./cockroach3-data:/cockroach/cockroach-data
3. SQL语法
sql
CREATE DATABASE IF NOT EXISTS order_db;
USE order_db;
CREATE TABLE orders (
id UUID DEFAULT gen_random_uuid(),
order_no VARCHAR(32) NOT NULL,
user_id INT64 NOT NULL,
shop_id INT64 NOT NULL,
order_amount DECIMAL(12,2) NOT NULL,
order_status INT8 DEFAULT 1,
create_time TIMESTAMP DEFAULT now(),
PRIMARY KEY (id),
INDEX idx_user_id (user_id),
INDEX idx_shop_id (shop_id),
UNIQUE INDEX idx_order_no (order_no)
);
BEGIN;
INSERT INTO orders (order_no, user_id, shop_id, order_amount)
VALUES ('ORDER001', 1001, 2001, 99.99);
UPDATE inventory SET stock = stock - 1 WHERE product_id = 3001;
COMMIT;
CREATE CHANGEFEED FOR TABLE orders
INTO 'kafka://localhost:9092'
WITH updated, key_in_value;
4. Java客户端
java
@Configuration
public class CockroachDBConfig {
@Bean
public DataSource dataSource() {
PGSimpleDataSource ds = new PGSimpleDataSource();
ds.setUrl("jdbc:postgresql://localhost:26257/order_db?sslmode=disable");
ds.setUser("root");
ds.setPassword("");
ds.setConnectionOptions("application_name=order_service");
return ds;
}
}
@Service
public class OrderService {
@Autowired
private DataSource dataSource;
public void createOrderWithRetry(OrderRequest request, int maxRetries) {
int retry = 0;
while (retry < maxRetries) {
try (Connection conn = dataSource.getConnection()) {
conn.setAutoCommit(false);
insertOrder(conn, request);
updateInventory(conn, request);
conn.commit();
return;
} catch (SQLTransactionRetryException e) {
retry++;
try {
Thread.sleep(100 * retry);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
throw new RuntimeException("重试次数耗尽");
}
}
5. 地理分布
sql
CREATE TABLE orders (
id UUID DEFAULT gen_random_uuid(),
order_no VARCHAR(32) NOT NULL,
user_id INT64 NOT NULL,
region STRING NOT NULL,
create_time TIMESTAMP DEFAULT now(),
PRIMARY KEY (id, region)
) PARTITION BY LIST (region) (
PARTITION us_east VALUES IN ('NY', 'MA', 'FL'),
PARTITION us_west VALUES IN ('CA', 'WA', 'OR'),
PARTITION eu VALUES IN ('UK', 'DE', 'FR'),
PARTITION asia VALUES IN ('CN', 'JP', 'KR')
);
ALTER TABLE orders SET locality = REGIONAL BY TABLE AS ROW;
四、TiDB vs CockroachDB对比
| 维度 | TiDB | CockroachDB |
|---|---|---|
| 创始公司 | PingCAP | CockroachLabs |
| 底层存储 | RocksDB (TiKV) | RocksDB |
| 一致性 | 强一致 | 强一致 |
| SQL兼容 | MySQL | PostgreSQL |
| HTAP | 支持(TiFlash) | 不支持 |
| 地理分布 | 一般 | 优秀 |
| 性能 | 优秀 | 较好 |
| 生态 | 丰富 | 一般 |
| 许可证 | Apache 2.0 | BSL 1.1 |
| 社区 | 活跃 | 活跃 |
五、选型建议
选择TiDB的场景:
- MySQL迁移项目
- 需要HTAP能力(同时支持TP和AP)
- 现有MySQL技术栈
- 数据量级TB级
选择CockroachDB的场景:
- PostgreSQL迁移项目
- 需要强地理分布
- 需要全球化部署
- 对许可证敏感
六、迁移方案
从MySQL迁移到TiDB
bash
cat > task.yaml << EOF
name: mysql-to-tidb
task-mode: all
target-database:
host: tidb-host
port: 4000
user: root
mysql-instances:
- source-id: mysql-source
block-allow-list: "instance-1"
routes:
route-1:
schema-pattern: "order_db"
target-schema: "order_db"
EOF
dmctl --master-addr=127.0.0.1:8261 start-task task.yaml
dmctl --master-addr=127.0.0.1:8261 query-status
七、总结
NewSQL是分布式关系型数据库的未来:
- TiDB:MySQL兼容 + HTAP能力强
- CockroachDB:PostgreSQL兼容 + 地理分布强
- 强一致:Raft协议保证
- 水平扩展:自动分片和负载均衡
选型建议:
- MySQL技术栈 → TiDB
- PostgreSQL技术栈 → CockroachDB
- 需要分析能力 → TiDB
- 需要全球化部署 → CockroachDB
个人观点,仅供参考