# C# + SQL Server 实现自动清理功能的完整方案：按数量与按日期双模式

C# + SQL Server 实现自动清理功能的完整方案：按数量与按日期双模式

一、引言

在现代软件开发中，数据管理是一个至关重要的环节。随着时间的推移，数据库中会积累大量过期或冗余的数据，这些数据不仅占用存储空间，还会影响系统性能。传统的手动清理方式效率低下且容易出错，因此，实现一个智能的自动清理系统显得尤为重要。

本文将详细介绍如何使用C#和SQL Server构建一个功能完善的自动清理系统，该系统支持两种核心清理策略：

1. 按数量清理：保留最新的N条记录，删除超出数量的旧记录
2. 按日期清理：删除超过指定天数的历史记录

---

二、系统架构设计

2.1 整体架构

┌─────────────────────────────────────────┐
│            应用层（C#）                  │
│  ┌─────────────┐  ┌─────────────┐      │
│  │ 清理服务     │  │ 后台服务    │      │
│  └─────────────┘  └─────────────┘      │
│           │                    │        │
└───────────┼────────────────────┼────────┘
            │                    │
            ▼                    ▼
┌─────────────────────────────────────────┐
│           数据访问层                    │
│  ┌───────────────────────────┐          │
│  │      CleanupRepository    │          │
│  └───────────────────────────┘          │
└─────────────────────────────────────────┘
            │
            ▼
┌─────────────────────────────────────────┐
│         数据存储层（SQL Server）         │
│  ┌─────────────┐  ┌─────────────┐      │
│  │  存储过程    │  │  配置表     │      │
│  └─────────────┘  └─────────────┘      │
└─────────────────────────────────────────┘

2.2 技术栈选择

• 后端框架：.NET 6+（支持跨平台部署）
• 数据库：SQL Server 2016+
• 调度组件：BackgroundService（内置）或 Quartz.NET（高级调度）
• 部署方式：Windows Service、Web API 或 Console Application

---

三、数据库设计

3.1 核心数据表结构

3.1.1 清理策略配置表（CleanupPolicies）

CREATE TABLE CleanupPolicies (
    Id INT IDENTITY(1,1) PRIMARY KEY,
    PolicyType NVARCHAR(20) CHECK (PolicyType IN ('Count', 'Date')) NOT NULL,
    TargetTable NVARCHAR(100) NOT NULL,        -- 目标表名
    ConditionColumn NVARCHAR(100) NOT NULL,    -- 条件字段
    ThresholdValue NVARCHAR(255) NOT NULL,     -- 阈值
    MaxRecords INT NULL,                       -- 最大记录数（用于按数量清理）
    RetentionDays INT NULL,                    -- 保留天数（用于按日期清理）
    IsEnabled BIT DEFAULT 1,                   -- 是否启用
    LastExecuted DATETIME2 NULL,               -- 最后执行时间
    ExecutionFrequency NVARCHAR(20) DEFAULT 'Daily', -- 执行频率
    CreatedAt DATETIME2 DEFAULT GETDATE(),
    Description NVARCHAR(500)                  -- 策略描述
);

3.1.2 清理历史记录表（CleanupHistory）

CREATE TABLE CleanupHistory (
    Id INT IDENTITY(1,1) PRIMARY KEY,
    PolicyId INT NOT NULL,
    TargetTable NVARCHAR(100) NOT NULL,
    DeletedCount INT NOT NULL,
    ExecutionTime DATETIME2 DEFAULT GETDATE(),
    Status NVARCHAR(20) NOT NULL,              -- Success/Error/NoAction
    ErrorMessage NVARCHAR(MAX) NULL,
    FOREIGN KEY (PolicyId) REFERENCES CleanupPolicies(Id)
);

3.2 存储过程设计

3.2.1 按数量清理存储过程

CREATE PROCEDURE sp_CleanupByCount
    @TableName NVARCHAR(100),
    @ConditionColumn NVARCHAR(100),
    @MaxRecords INT
AS
BEGIN
    SET NOCOUNT ON;
    BEGIN TRANSACTION;
    
    DECLARE @TotalCount INT, @DeleteCount INT;
    
    -- 获取总记录数
    EXEC('SELECT @Count = COUNT(*) FROM ' + @TableName);
    
    -- 计算需要删除的数量
    IF @TotalCount > @MaxRecords
    BEGIN
        SET @DeleteCount = @TotalCount - @MaxRecords;
        
        -- 删除最旧的记录
        EXEC('
            DELETE FROM ' + @TableName + '
            WHERE Id IN (
                SELECT TOP ' + @DeleteCount + ' Id
                FROM ' + @TableName + '
                ORDER BY ' + @ConditionColumn + ' ASC
            )');
    END
    
    COMMIT TRANSACTION;
END

3.2.2 按日期清理存储过程

CREATE PROCEDURE sp_CleanupByDate
    @TableName NVARCHAR(100),
    @ConditionColumn NVARCHAR(100),
    @RetentionDays INT
AS
BEGIN
    SET NOCOUNT ON;
    BEGIN TRANSACTION;
    
    -- 删除超过保留天数的记录
    EXEC('
        DELETE FROM ' + @TableName + '
        WHERE ' + @ConditionColumn + ' < DATEADD(DAY, -' 
            + @RetentionDays + ', GETDATE())');
    
    COMMIT TRANSACTION;
END

---

四、C# 核心实现

4.1 实体类定义

public enum CleanupPolicyType { Count, Date }
public enum ExecutionFrequency { Hourly, Daily, Weekly, Monthly }

public class CleanupPolicy
{
    public int Id { get; set; }
    public CleanupPolicyType PolicyType { get; set; }
    public string TargetTable { get; set; }
    public string ConditionColumn { get; set; }
    public int? MaxRecords { get; set; }
    public int? RetentionDays { get; set; }
    public bool IsEnabled { get; set; }
    public DateTime? LastExecuted { get; set; }
    public ExecutionFrequency ExecutionFrequency { get; set; }
}

public class CleanupResult
{
    public bool Success { get; set; }
    public int DeletedCount { get; set; }
    public string Message { get; set; }
    public DateTime ExecutionTime { get; set; }
}

4.2 数据访问层

public class CleanupRepository : ICleanupRepository
{
    private readonly string _connectionString;
    
    public async Task<List<CleanupPolicy>> GetEnabledPoliciesAsync()
    {
        var policies = new List<CleanupPolicy>();
        
        using (var connection = new SqlConnection(_connectionString))
        {
            var command = new SqlCommand(@"
                SELECT Id, PolicyType, TargetTable, ConditionColumn, 
                       MaxRecords, RetentionDays, IsEnabled,
                       LastExecuted, ExecutionFrequency
                FROM CleanupPolicies
                WHERE IsEnabled = 1", connection);
            
            using (var reader = await command.ExecuteReaderAsync())
            {
                while (await reader.ReadAsync())
                {
                    policies.Add(new CleanupPolicy
                    {
                        Id = reader.GetInt32(0),
                        PolicyType = reader.GetString(1) == "Count" 
                            ? CleanupPolicyType.Count : CleanupPolicyType.Date,
                        TargetTable = reader.GetString(2),
                        ConditionColumn = reader.GetString(3),
                        MaxRecords = reader.IsDBNull(4) ? null : (int?)reader.GetInt32(4),
                        RetentionDays = reader.IsDBNull(5) ? null : (int?)reader.GetInt32(5),
                        IsEnabled = reader.GetBoolean(6),
                        LastExecuted = reader.IsDBNull(7) ? null : (DateTime?)reader.GetDateTime(7),
                        ExecutionFrequency = (ExecutionFrequency)Enum.Parse(
                            typeof(ExecutionFrequency), reader.GetString(8))
                    });
                }
            }
        }
        
        return policies;
    }
}

4.3 清理服务核心

public class CleanupService : ICleanupService
{
    private readonly ICleanupRepository _repository;
    private readonly ILogger<CleanupService> _logger;
    
    public async Task ExecuteAllPoliciesAsync()
    {
        _logger.LogInformation("开始执行自动清理任务");
        
        var policies = await _repository.GetEnabledPoliciesAsync();
        
        foreach (var policy in policies)
        {
            if (await ShouldExecuteAsync(policy))
            {
                await ExecutePolicyAsync(policy);
            }
        }
    }
    
    public async Task<CleanupResult> ExecutePolicyAsync(CleanupPolicy policy)
    {
        try
        {
            CleanupResult result;
            
            if (policy.PolicyType == CleanupPolicyType.Count)
            {
                result = await _repository.ExecuteCleanupByCountAsync(
                    policy.TargetTable,
                    policy.ConditionColumn,
                    policy.MaxRecords ?? 1000);
            }
            else // Date类型
            {
                result = await _repository.ExecuteCleanupByDateAsync(
                    policy.TargetTable,
                    policy.ConditionColumn,
                    policy.RetentionDays ?? 30);
            }
            
            // 记录执行历史
            await _repository.LogCleanupHistoryAsync(
                policy.Id,
                policy.TargetTable,
                result.DeletedCount,
                result.Success ? "Success" : "Error",
                result.Message);
            
            _logger.LogInformation($"表{policy.TargetTable}: 删除了{result.DeletedCount}条记录");
            
            return result;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, $"清理策略执行失败: {policy.TargetTable}");
            throw;
        }
    }
}

4.4 后台服务实现

public class CleanupBackgroundService : BackgroundService
{
    private readonly ICleanupService _cleanupService;
    private readonly ILogger<CleanupBackgroundService> _logger;
    private readonly TimeSpan _interval = TimeSpan.FromHours(1);
    
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        _logger.LogInformation("自动清理后台服务已启动");
        
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                await _cleanupService.ExecuteAllPoliciesAsync();
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "清理任务执行失败");
            }
            
            await Task.Delay(_interval, stoppingToken);
        }
    }
}

---

五、配置和使用

5.1 依赖注入配置

// Program.cs (.NET 6+)
var builder = WebApplication.CreateBuilder(args);

// 注册服务
builder.Services.AddScoped<ICleanupRepository, CleanupRepository>();
builder.Services.AddScoped<ICleanupService, CleanupService>();

// 添加后台服务
builder.Services.AddHostedService<CleanupBackgroundService>();

// 配置数据库连接
builder.Services.AddDbContext<CleanupDbContext>(options =>
    options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

5.2 appsettings.json 配置

{
  "ConnectionStrings": {
    "DefaultConnection": "Server=localhost;Database=CleanupDB;User Id=sa;Password=YourPassword;TrustServerCertificate=true;"
  },
  "CleanupSettings": {
    "DefaultRetentionDays": 30,
    "DefaultMaxRecords": 1000,
    "ExecutionIntervalMinutes": 60,
    "EnableDetailedLogging": true
  }
}

5.3 初始化清理策略

-- 示例：配置系统日志清理策略
INSERT INTO CleanupPolicies 
(PolicyType, TargetTable, ConditionColumn, MaxRecords, 
 RetentionDays, IsEnabled, ExecutionFrequency, Description)
VALUES
-- 系统日志：保留最近1000条
('Count', 'SystemLogs', 'CreatedAt', 1000, NULL, 1, 'Daily', '系统日志保留最近1000条'),

-- 用户操作日志：保留30天
('Date', 'UserActivityLogs', 'CreatedAt', NULL, 30, 1, 'Daily', '用户操作日志保留30天'),

-- 临时文件：立即删除过期记录
('Date', 'TemporaryFiles', 'ExpiresAt', NULL, 0, 1, 'Hourly', '立即删除过期临时文件'),

-- 聊天记录：保留最近5000条
('Count', 'ChatMessages', 'CreatedAt', 5000, NULL, 1, 'Daily', '聊天消息保留最近5000条');

---

六、高级功能扩展

6.1 动态策略配置

// 通过API动态管理清理策略
[ApiController]
[Route("api/[controller]")]
public class CleanupPoliciesController : ControllerBase
{
    private readonly ICleanupService _cleanupService;
    
    [HttpPost]
    public async Task<IActionResult> CreatePolicy([FromBody] CleanupPolicy policy)
    {
        // 创建新的清理策略
        return Ok();
    }
    
    [HttpPost("execute/{id}")]
    public async Task<IActionResult> ExecutePolicy(int id)
    {
        // 立即执行指定策略
        return Ok();
    }
    
    [HttpGet("history")]
    public async Task<IActionResult> GetHistory(
        [FromQuery] DateTime? fromDate, 
        [FromQuery] DateTime? toDate)
    {
        // 获取清理历史记录
        return Ok();
    }
}

6.2 监控和告警

public class CleanupMonitor
{
    public async Task<CleanupStats> GetStatisticsAsync()
    {
        return new CleanupStats
        {
            TotalExecutions = await GetTotalExecutionsAsync(),
            TotalDeleted = await GetTotalDeletedAsync(),
            SuccessRate = await GetSuccessRateAsync(),
            RecentErrors = await GetRecentErrorsAsync()
        };
    }
    
    // 邮件告警
    public async Task SendAlertAsync(string policyName, string error)
    {
        var alert = new EmailAlert
        {
            Subject = $"自动清理任务失败: {policyName}",
            Body = $"清理策略执行失败: {error}\n时间: {DateTime.Now}",
            Recipients = new[] { "admin@example.com" }
        };
        
        await _emailService.SendAsync(alert);
    }
}

6.3 性能优化

-- 创建优化索引
CREATE INDEX IX_CleanupHistory_ExecutionTime 
ON CleanupHistory(ExecutionTime DESC);

CREATE INDEX IX_CleanupPolicies_LastExecuted 
ON CleanupPolicies(LastExecuted);

-- 分区表（适用于大数据量）
CREATE PARTITION FUNCTION pf_CleanupByMonth (DATETIME2)
AS RANGE RIGHT FOR VALUES (
    '2024-01-01', '2024-02-01', '2024-03-01'
);

---

七、部署方案

7.1 作为Windows服务部署

// 安装为Windows服务
public static class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }
    
    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .UseWindowsService()
            .ConfigureServices((context, services) =>
            {
                services.AddHostedService<CleanupBackgroundService>();
            });
}

7.2 使用Docker部署

# Dockerfile
FROM mcr.microsoft.com/dotnet/aspnet:6.0 AS base
WORKDIR /app

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["AutoCleanup.csproj", "./"]
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "AutoCleanup.dll"]

7.3 使用Docker Compose

# docker-compose.yml
version: '3.8'
services:
  sqlserver:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      SA_PASSWORD: "YourStrong!Password"
      ACCEPT_EULA: "Y"
    ports:
      - "1433:1433"
      
  cleanup-service:
    build: .
    environment:
      ConnectionStrings__DefaultConnection: "Server=sqlserver;Database=CleanupDB;User Id=sa;Password=YourStrong!Password;TrustServerCertificate=true;"
    depends_on:
      - sqlserver

---

八、最佳实践和注意事项

8.1 安全性建议

1. 权限控制：为清理服务创建专用数据库用户，仅授予必要的权限
2. 参数验证：对所有输入参数进行验证，防止SQL注入
3. 连接加密：使用SSL/TLS加密数据库连接

8.2 性能优化

1. 批量操作：避免逐条删除，使用批量操作
2. 索引优化：确保条件字段有合适的索引
3. 分时段执行：在系统低峰期执行清理任务
4. 事务控制：合理设置事务大小，避免锁表时间过长

8.3 监控和日志

1. 详细日志：记录每次清理的详细信息
2. 性能指标：监控清理任务的执行时间和影响
3. 错误追踪：建立完善的错误处理和恢复机制
4. 报警机制：设置关键错误的实时报警

8.4 灾难恢复

1. 备份策略：在执行清理前考虑数据备份
2. 回滚机制：实现紧急情况下的操作回滚
3. 数据恢复：制定数据误删的恢复流程

---

九、总结

本文详细介绍了使用C#和SQL Server实现自动清理功能的完整方案。该系统具有以下特点：

1. 双模式支持：同时支持按数量和按日期两种清理策略
2. 配置灵活：所有策略都可配置，无需修改代码
3. 高可靠性：完善的错误处理和事务机制
4. 易扩展：支持多种部署方式和扩展功能
5. 监控完善：提供完整的执行历史和统计信息

通过实现这个自动清理系统，您可以：

• 显著减少数据库存储压力
• 提高系统查询性能
• 降低手动维护成本
• 避免数据过期带来的问题