SQLite3在NFS下会不会导致文件写乱？

一、搭建NFS实验环境：

以centos8系统为例：

Linux 4.19.90-23.8.v2101.ky10.x86_64 #1 SMP Mon May 17 17:08:34 CST 2021 x86_64 x86_64 x86_64 GNU/Linux

nfs服务端：

bash 复制代码

yum install -y nfs-utils rpcbind
mkdir -p /usr/share/nfs/share
echo "/usr/share/nfs/share 10.10.10.123/24(rw,sync,no_root_squash)" >> /etc/exports
exportfs -a
systemctl enable rpcbind nfs-server
systemctl start rpcbind nfs-server
exportfs -v
firewall-cmd --permanent --zone=public --add-service=nfs --add-service=mountd --add-service=rpc-bind --add-service=nscd --add-service=nfs-lock --add-service=nfs-idmap
firewall-cmd --reload

nfs客户端：

bash 复制代码

yum install -y nfs-utils
mkdir -p /mnt/nfs_share
mount -t nfs 10.10.10.123:/usr/share/nfs/share /mnt/nfs_share

二、基于NFS压测SQLite3的WAL模式：

压测脚本 wal_performance_test.py：

python 复制代码

import time
import sqlite3
import subprocess
import re
from datetime import datetime

def wal_performance_test():
    # WAL模式
    dev, ip = get_default_interface_ip()
    start = time.perf_counter()
    for i in range(100):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/wal_test.db')
            # 设置WAL模式必须将journal_mode设为WAL
            conn.execute("PRAGMA journal_mode=WAL;")  
            # 调整同步策略为NORMAL提升性能
            conn.execute("PRAGMA synchronous=NORMAL;") 
            # 设置WAL文件自动清理的检查点阈值
            conn.execute("PRAGMA wal_autocheckpoint=1000;") 
            conn.execute('CREATE TABLE IF NOT EXISTS data (id INT, host VARCHAR(15))')
            conn.execute("BEGIN TRANSACTION")
            for ii in range(100):
                conn.execute("INSERT INTO data VALUES (?,?)", (i*100+ii,ip,))
            conn.commit()
            conn.close()
            time.sleep(1)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(1)
    wal_time = time.perf_counter() - start
    print(f"WAL模式: {wal_time:.6f}s")

def sqlite_performance_test():
    # 常规模式
    dev, ip = get_default_interface_ip()
    start = time.perf_counter()
    for i in range(100):
        try:
            conn = sqlite3.connect('/mnt/nfs_share/wal_test.db')
            conn.execute('CREATE TABLE IF NOT EXISTS data (id INT, host VARCHAR(15))')
            conn.execute("BEGIN TRANSACTION")
            for ii in range(100):
                conn.execute("INSERT INTO data VALUES (?,?)", (i*100+ii,ip,))
            conn.commit()
            conn.close()
            time.sleep(1)
        except Exception as e:
            now = datetime.now()
            print(now, e)
            time.sleep(1)
    normal_time = time.perf_counter() - start
    print(f"常规模式: {normal_time:.6f}s")

def get_default_interface_ip():
    """通过路由表获取默认网卡和IP"""
    try:
        # 获取默认路由接口 - 使用 stdout=subprocess.PIPE
        result = subprocess.run(
            ['ip', 'route', 'show', 'default'],
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            universal_newlines=True,
            check=True
        )
        # 解析默认路由行
        match = re.search(r'dev\s+(\w+)', result.stdout)
        if match:
            interface = match.group(1)
            # 获取该接口的IP地址
            ip_result = subprocess.run(
                ['ip', '-4', 'addr', 'show', interface],
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
                universal_newlines=True,
                check=True
            )
            # 解析IP地址
            ip_match = re.search(r'inet\s+(\d+\.\d+\.\d+\.\d+)', ip_result.stdout)
            if ip_match:
                return interface, ip_match.group(1)
    except subprocess.CalledProcessError as e:
        print(f"命令执行失败: {e}")
        print(f"错误输出: {e.stderr}")
    except Exception as e:
        print(f"错误: {e}")
    return None, None


wal_performance_test()

分别在客户机124和125执行：

bash 复制代码

python3 wal_performance_test.py

两个客户端交替出现错误 disk I/O error，总计出现6条：

test@124 wal\]$ python3 wal_performance_test.py 2025-12-09 14:33:21.776389 disk I/O error 2025-12-09 14:33:47.146377 disk I/O error 2025-12-09 14:34:39.921037 disk I/O error WAL模式: 105.810731s \[test@125 wal\]$ python3 wal_performance_test.py 2025-12-09 14:33:20.745065 disk I/O error 2025-12-09 14:34:13.513216 disk I/O error 2025-12-09 14:34:38.881051 disk I/O error WAL模式: 101.554718s ### 三、验证数据： 在123机器执行： ```bash yum install -y sqlite sqlite3 wal_test.db sqlite> select count(*) from data; 19400 sqlite> select count(*) from data where host='10.10.10.124'; 9700 sqlite> select count(*) from data where host='10.10.10.125'; 9700 ``` 总的执行数据量2\*100\*100=20000，减去6次失败 20000-600=19400，数据量正好。 也就是不会导致数据写乱，但是加锁失败会报错，需要客户端自己重试，如果不重试数据会丢失。 ### 四、基于NFS压测SQLite的三种事务类型： 压测脚本 nfs_transaction_test.py： ```python import time import sqlite3 import sys from datetime import datetime def nfs_deferred_transaction_test(): start = time.perf_counter() for i in range(10000): try: conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db') conn.execute("BEGIN TRANSACTION") row = conn.execute('SELECT * FROM data LIMIT 1').fetchone() counter = row[0] + 1 conn.execute("UPDATE data SET counter=?", (counter,)) conn.commit() conn.close() time.sleep(0.01) except Exception as e: now = datetime.now() print(now, e) time.sleep(0.01) cost_time = time.perf_counter() - start print(f"deferred: {cost_time:.6f}s") def nfs_immediate_transaction_test(): start = time.perf_counter() for i in range(10000): try: conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db') conn.execute("BEGIN IMMEDIATE TRANSACTION") row = conn.execute('SELECT * FROM data LIMIT 1').fetchone() counter = row[0] + 1 conn.execute("UPDATE data SET counter=?", (counter,)) conn.commit() conn.close() time.sleep(0.01) except Exception as e: now = datetime.now() print(now, e) time.sleep(0.01) cost_time = time.perf_counter() - start print(f"immediate: {cost_time:.6f}s") def nfs_exclusive_transaction_test(): start = time.perf_counter() for i in range(10000): try: conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db') conn.execute("BEGIN EXCLUSIVE TRANSACTION") row = conn.execute('SELECT * FROM data LIMIT 1').fetchone() counter = row[0] + 1 conn.execute("UPDATE data SET counter=?", (counter,)) conn.commit() conn.close() time.sleep(0.01) except Exception as e: now = datetime.now() print(now, e) time.sleep(0.01) cost_time = time.perf_counter() - start print(f"exclusive: {cost_time:.6f}s") def main(): conn = sqlite3.connect('/mnt/nfs_share/nfs_test.db') conn.execute('CREATE TABLE IF NOT EXISTS data (counter INT)') row = conn.execute('SELECT * FROM data LIMIT 1').fetchone() if row is None: conn.execute("INSERT INTO data (counter) VALUES (?)", (0,)) conn.commit() conn.close() command = sys.argv[1] if len(sys.argv)>1 else "" if command == "" or command == "1": nfs_deferred_transaction_test() elif command == "2": nfs_immediate_transaction_test() elif command == "3": nfs_exclusive_transaction_test() else: print("错误：参数不足或命令无效") if __name__ == "__main__": main() ``` #### 1、deferred事务： 两台客户机都会出现大量的报错： \[test@124 wal\]$ python3 nfs_transaction_test.py 1 。。。。。。 2025-12-10 01:06:16.459700 database is locked 2025-12-10 01:06:16.695814 database is locked 2025-12-10 01:06:16.777952 database is locked deferred: 287.597685s \[test@125 wal\]$ python3 nfs_transaction_test.py 1 。。。。。。 2025-12-10 01:06:16.591695 database is locked 2025-12-10 01:06:16.657538 database is locked 2025-12-10 01:06:16.848156 database is locked deferred: 385.892006s 数据库中的计数也少了： sqlite\> select \* from data; 12948 sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 13018 sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 13284 sqlite\> select \* from data; 15139 sqlite\> select \* from data; 15139 #### 2、immediate事务： 两台客户机仅有一条报错产生： \[test@124 wal\]$ python3 nfs_transaction_test.py 2 immediate: 347.385314s \[test@125 wal\]$ python3 nfs_transaction_test.py 2 2025-12-10 00:42:48.581516 disk I/O error immediate: 353.346707s 从123使用linux工具查询（deferred事务）会偶尔出现报错，但最终的数据是正确的。 \[root@localhost share\]# sqlite3 nfs_test.db SQLite version 3.32.3 2020-06-18 14:00:33 Enter ".help" for usage hints. sqlite\> select \* from data; 865 sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 1275 sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 9723 sqlite\> select \* from data; 20000 sqlite\> #### 3、exclusive事务： 两台客户机都没有报错： \[test@124 wal\]$ python3 nfs_transaction_test.py 3 exclusive: 329.862279s \[test@125 wal\]$ python3 nfs_transaction_test.py 3 exclusive: 342.372154s 从123使用linux工具查询（deferred事务）会偶尔出现报错，但最终的数据是正确的。 \[root@localhost share\]# sqlite3 nfs_test.db SQLite version 3.32.3 2020-06-18 14:00:33 Enter ".help" for usage hints. sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 427 sqlite\> select \* from data; Error: database is locked sqlite\> select \* from data; 685 sqlite\> select \* from data; 20000 sqlite\> --end--