SQLite3在NFS下会不会导致文件写乱？

一、搭建NFS实验环境：

以centos8系统为例：

Linux 4.19.90-23.8.v2101.ky10.x86_64 #1 SMP Mon May 17 17:08:34 CST 2021 x86_64 x86_64 x86_64 GNU/Linux

nfs服务端：

bash 复制代码

yum install -y nfs-utils rpcbind
mkdir -p /usr/share/nfs/share
echo "/usr/share/nfs/share 10.10.10.123/24(rw,sync,no_root_squash)" >> /etc/exports
exportfs -a
systemctl enable rpcbind nfs-server
systemctl start rpcbind nfs-server
exportfs -v
firewall-cmd --permanent --zone=public --add-service=nfs --add-service=mountd --add-service=rpc-bind --add-service=nscd --add-service=nfs-lock --add-service=nfs-idmap
firewall-cmd --reload

nfs客户端：

bash 复制代码

yum install -y nfs-utils
mkdir -p /mnt/nfs_share
mount -t nfs 10.10.10.123:/usr/share/nfs/share /mnt/nfs_share

二、基于NFS压测SQLite3的WAL模式：

压测脚本 wal_performance_test.py：

python 复制代码

import time
import sqlite3
import subprocess
import re
from datetime import datetime

def wal_performance_test():
    # WAL模式
    dev, ip = get_default_interface_ip()
    start = time.perf_counter()
    for i in range(100):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/wal_test.db')
            # 设置WAL模式必须将journal_mode设为WAL
            conn.execute("PRAGMA journal_mode=WAL;")  
            # 调整同步策略为NORMAL提升性能
            conn.execute("PRAGMA synchronous=NORMAL;") 
            # 设置WAL文件自动清理的检查点阈值
            conn.execute("PRAGMA wal_autocheckpoint=1000;") 
            conn.execute('CREATE TABLE IF NOT EXISTS data (id INT, host VARCHAR(15))')
            conn.execute("BEGIN TRANSACTION")
            for ii in range(100):
                conn.execute("INSERT INTO data VALUES (?,?)", (i*100+ii,ip,))
            conn.commit()
            conn.close()
            time.sleep(1)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(1)
    wal_time = time.perf_counter() - start
    print(f"WAL模式: {wal_time:.6f}s")

def sqlite_performance_test():
    # 常规模式
    dev, ip = get_default_interface_ip()
    start = time.perf_counter()
    for i in range(100):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/wal_test.db')
            conn.execute('CREATE TABLE IF NOT EXISTS data (id INT, host VARCHAR(15))')
            conn.execute("BEGIN TRANSACTION")
            for ii in range(100):
                conn.execute("INSERT INTO data VALUES (?,?)", (i*100+ii,ip,))
            conn.commit()
            conn.close()
            time.sleep(1)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(1)
    normal_time = time.perf_counter() - start
    print(f"常规模式: {normal_time:.6f}s")

def get_default_interface_ip():
    """通过路由表获取默认网卡和IP"""
    try:
        # 获取默认路由接口 - 使用 stdout=subprocess.PIPE
        result = subprocess.run(
            ['ip', 'route', 'show', 'default'],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            universal_newlines=True,
            check=True
        )
        # 解析默认路由行
        match = re.search(r'dev\s+(\w+)', result.stdout)
        if match:
            interface = match.group(1)
            # 获取该接口的IP地址
            ip_result = subprocess.run(
                ['ip', '-4', 'addr', 'show', interface],
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                universal_newlines=True,
                check=True
            )
            # 解析IP地址
            ip_match = re.search(r'inet\s+(\d+\.\d+\.\d+\.\d+)', ip_result.stdout)
            if ip_match:
                return interface, ip_match.group(1)
    except subprocess.CalledProcessError as e:
        print(f"命令执行失败: {e}")
        print(f"错误输出: {e.stderr}")
    except Exception as e:
        print(f"错误: {e}")
    return None, None


wal_performance_test()

分别在客户机124和125执行：

bash 复制代码

python3 wal_performance_test.py

两个客户端交替出现错误 disk I/O error，总计出现6条：

$test@124 wal$ $ python3 wal_performance_test.py

2025-12-09 14:33:21.776389 disk I/O error

2025-12-09 14:33:47.146377 disk I/O error

2025-12-09 14:34:39.921037 disk I/O error

WAL模式: 105.810731s

$test@125 wal$ $ python3 wal_performance_test.py

2025-12-09 14:33:20.745065 disk I/O error

2025-12-09 14:34:13.513216 disk I/O error

2025-12-09 14:34:38.881051 disk I/O error

WAL模式: 101.554718s

三、验证数据：

在123机器执行：

bash 复制代码

yum install -y sqlite
sqlite3 wal_test.db
sqlite> select count(*) from data;
19400
sqlite> select count(*) from data where host='10.10.10.124';
9700
sqlite> select count(*) from data where host='10.10.10.125';
9700

总的执行数据量2*100*100=20000，减去6次失败 20000-600=19400，数据量正好。

也就是不会导致数据写乱，但是加锁失败会报错，需要客户端自己重试，如果不重试数据会丢失。

四、基于NFS压测SQLite的三种事务类型：

压测脚本 nfs_transaction_test.py：

python 复制代码

import time
import sqlite3
import sys
from datetime import datetime

def nfs_deferred_transaction_test():
    start = time.perf_counter()
    for i in range(10000):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db')
            conn.execute("BEGIN TRANSACTION")
            row = conn.execute('SELECT * FROM data LIMIT 1').fetchone()
            counter = row[0] + 1
            conn.execute("UPDATE data SET counter=?", (counter,))
            conn.commit()
            conn.close()
            time.sleep(0.01)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(0.01)
    cost_time = time.perf_counter() - start
    print(f"deferred: {cost_time:.6f}s")

def nfs_immediate_transaction_test():
    start = time.perf_counter()
    for i in range(10000):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db')
            conn.execute("BEGIN IMMEDIATE TRANSACTION")
            row = conn.execute('SELECT * FROM data LIMIT 1').fetchone()
            counter = row[0] + 1
            conn.execute("UPDATE data SET counter=?", (counter,))
            conn.commit()
            conn.close()
            time.sleep(0.01)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(0.01)
    cost_time = time.perf_counter() - start
    print(f"immediate: {cost_time:.6f}s")

def nfs_exclusive_transaction_test():
    start = time.perf_counter()
    for i in range(10000):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db')
            conn.execute("BEGIN EXCLUSIVE TRANSACTION")
            row = conn.execute('SELECT * FROM data LIMIT 1').fetchone()
            counter = row[0] + 1
            conn.execute("UPDATE data SET counter=?", (counter,))
            conn.commit()
            conn.close()
            time.sleep(0.01)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(0.01)
    cost_time = time.perf_counter() - start
    print(f"exclusive: {cost_time:.6f}s")

def main():
    conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db')
    conn.execute('CREATE TABLE IF NOT EXISTS data (counter INT)')
    row = conn.execute('SELECT * FROM data LIMIT 1').fetchone()
    if row is None:
        conn.execute("INSERT INTO data (counter) VALUES (?)", (0,))
    conn.commit()
    conn.close()
    command = sys.argv[1] if len(sys.argv)>1 else ""
    if command == "" or command == "1":
        nfs_deferred_transaction_test()
    elif command == "2":
        nfs_immediate_transaction_test()
    elif command == "3":
        nfs_exclusive_transaction_test()
    else:
        print("错误：参数不足或命令无效")

if __name__ == "__main__":
    main()

1、deferred事务：

两台客户机都会出现大量的报错：

$test@124 wal$ $ python3 nfs_transaction_test.py 1

。。。。。。

2025-12-10 01:06:16.459700 database is locked

2025-12-10 01:06:16.695814 database is locked

2025-12-10 01:06:16.777952 database is locked

deferred: 287.597685s

$test@125 wal$ $ python3 nfs_transaction_test.py 1

。。。。。。

2025-12-10 01:06:16.591695 database is locked

2025-12-10 01:06:16.657538 database is locked

2025-12-10 01:06:16.848156 database is locked

deferred: 385.892006s

数据库中的计数也少了：

sqlite> select * from data;

12948

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

13018

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

13284

sqlite> select * from data;

15139

sqlite> select * from data;

15139

2、immediate事务：

两台客户机仅有一条报错产生：

$test@124 wal$ $ python3 nfs_transaction_test.py 2

immediate: 347.385314s

$test@125 wal$ $ python3 nfs_transaction_test.py 2

2025-12-10 00:42:48.581516 disk I/O error

immediate: 353.346707s

从123使用linux工具查询（deferred事务）会偶尔出现报错，但最终的数据是正确的。

$root@localhost share$ # sqlite3 nfs_test.db

SQLite version 3.32.3 2020-06-18 14:00:33

Enter ".help" for usage hints.

sqlite> select * from data;

865

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

1275

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

9723

sqlite> select * from data;

20000

sqlite>

3、exclusive事务：

两台客户机都没有报错：

$test@124 wal$ $ python3 nfs_transaction_test.py 3

exclusive: 329.862279s

$test@125 wal$ $ python3 nfs_transaction_test.py 3

exclusive: 342.372154s

从123使用linux工具查询（deferred事务）会偶尔出现报错，但最终的数据是正确的。

$root@localhost share$ # sqlite3 nfs_test.db

SQLite version 3.32.3 2020-06-18 14:00:33

Enter ".help" for usage hints.

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

427

sqlite> select * from data;

Error: database is locked

sqlite> select * from data;

685

sqlite> select * from data;

20000

sqlite>

--end--