目录
- 引言
- [1. 抽象包](#1. 抽象包)
- [1.1 定义日志记录接口](#1.1 定义日志记录接口)
- [1.2 定义日志记录抽象类](#1.2 定义日志记录抽象类)
- [1.3 表结构迁移](#1.3 表结构迁移)
- [2. EntityFramework Core 的实现](#2. EntityFramework Core 的实现)
- [2.1 数据库上下文](#2.1 数据库上下文)
- [2.2 实现日志写入](#2.2 实现日志写入)
- [3. MySqlConnector 的实现](#3. MySqlConnector 的实现)
- [3.1 SQL脚本](#3.1 SQL脚本)
- [3.2 实现日志写入](#3.2 实现日志写入)
- [4. 使用示例](#4. 使用示例)
引言
在应用程序中,日志记录是一个至关重要的功能。不仅有助于调试和监控应用程序,还能帮助我们了解应用程序的运行状态。
在这个示例中将展示如何实现一个自定义的日志记录器,先说明一下,这个实现和Microsoft.Extensions.Logging
、Serilog
、NLog
什么的无关,这里只是将自定义的日志数据存入数据库中,或许你也可以理解为实现的是一个存数据的"Repository",只不过用这个Repository来存的是日志🙈。这个实现包含一个抽象包和两个实现包,两个实现分别是用 EntityFramework Core 和 MySqlConnector 。日志记录操作将放在本地队列中异步处理,以确保不影响业务处理。
1. 抽象包
1.1 定义日志记录接口
首先,我们需要定义一个日志记录接口 ICustomLogger
,它包含两个方法:LogReceived 和 LogProcessed。LogReceived 用于记录接收到的日志,LogProcessed 用于更新日志的处理状态。
csharp
namespace Logging.Abstractions;
public interface ICustomLogger
{
/// <summary>
/// 记录一条日志
/// </summary>
void LogReceived(CustomLogEntry logEntry);
/// <summary>
/// 根据Id更新这条日志
/// </summary>
void LogProcessed(string logId, bool isSuccess);
}
定义一个日志结构实体CustomLogEntry
,用于存储日志的详细信息:
csharp
namespace Logging.Abstractions;
public class CustomLogEntry
{
/// <summary>
/// 日志唯一Id,数据库主键
/// </summary>
public string Id { get; set; } = Guid.NewGuid().ToString();
public string Message { get; set; } = default!;
public bool IsSuccess { get; set; }
public DateTime CreateTime { get; set; } = DateTime.UtcNow;
public DateTime? UpdateTime { get; set; } = DateTime.UtcNow;
}
1.2 定义日志记录抽象类
接下来,定义一个抽象类CustomLogger
,它实现了ICustomLogger
接口,并提供了日志记录的基本功能,将日志写入操作(插入or更新)放在本地队列总异步处理。使用ConcurrentQueue
来确保线程安全,并开启一个后台任务异步处理这些日志。这个抽象类只负责将日志写入命令放到队列中,实现类负责消费队列中的消息,确定日志应该怎么写?往哪里写?这个示例中后边会有两个实现,一个是基于EntityFramework Core的实现,另一个是MySqlConnector的实现。
封装一下日志写入命令
csharp
namespace Logging.Abstractions;
public class WriteCommand(WriteCommandType commandType, CustomLogEntry logEntry)
{
public WriteCommandType CommandType { get; } = commandType;
public CustomLogEntry LogEntry { get; } = logEntry;
}
public enum WriteCommandType
{
/// <summary>
/// 插入
/// </summary>
Insert,
/// <summary>
/// 更新
/// </summary>
Update
}
CustomLogger
实现
csharp
using System.Collections.Concurrent;
using Microsoft.Extensions.Logging;
namespace Logging.Abstractions;
public abstract class CustomLogger : ICustomLogger, IDisposable, IAsyncDisposable
{
protected ILogger<CustomLogger> Logger { get; }
protected ConcurrentQueue<WriteCommand> WriteQueue { get; }
protected Task WriteTask { get; }
private readonly CancellationTokenSource _cancellationTokenSource;
private readonly CancellationToken _cancellationToken;
protected CustomLogger(ILogger<CustomLogger> logger)
{
Logger = logger;
WriteQueue = new ConcurrentQueue<WriteCommand>();
_cancellationTokenSource = new CancellationTokenSource();
_cancellationToken = _cancellationTokenSource.Token;
WriteTask = Task.Factory.StartNew(TryWriteAsync, _cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default);
}
public void LogReceived(CustomLogEntry logEntry)
{
WriteQueue.Enqueue(new WriteCommand(WriteCommandType.Insert, logEntry));
}
public void LogProcessed(string messageId, bool isSuccess)
{
var logEntry = GetById(messageId);
if (logEntry == null)
{
return;
}
logEntry.IsSuccess = isSuccess;
logEntry.UpdateTime = DateTime.UtcNow;
WriteQueue.Enqueue(new WriteCommand(WriteCommandType.Update, logEntry));
}
private async Task TryWriteAsync()
{
try
{
while (!_cancellationToken.IsCancellationRequested)
{
if (WriteQueue.IsEmpty)
{
await Task.Delay(1000, _cancellationToken);
continue;
}
if (WriteQueue.TryDequeue(out var writeCommand))
{
await WriteAsync(writeCommand);
}
}
while (WriteQueue.TryDequeue(out var remainingCommand))
{
await WriteAsync(remainingCommand);
}
}
catch (OperationCanceledException)
{
// 任务被取消,正常退出
}
catch (Exception e)
{
Logger.LogError(e, "处理待写入日志队列异常");
}
}
protected abstract CustomLogEntry? GetById(string messageId);
protected abstract Task WriteAsync(WriteCommand writeCommand);
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
public async ValueTask DisposeAsync()
{
await DisposeAsyncCore();
Dispose(false);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (disposing)
{
_cancellationTokenSource.Cancel();
try
{
WriteTask.Wait();
}
catch (AggregateException ex)
{
foreach (var innerException in ex.InnerExceptions)
{
Logger.LogError(innerException, "释放资源异常");
}
}
finally
{
_cancellationTokenSource.Dispose();
}
}
}
protected virtual async Task DisposeAsyncCore()
{
_cancellationTokenSource.Cancel();
try
{
await WriteTask;
}
catch (Exception e)
{
Logger.LogError(e, "释放资源异常");
}
finally
{
_cancellationTokenSource.Dispose();
}
}
}
1.3 表结构迁移
为了方便表结构迁移,我们可以使用FluentMigrator.Runner.MySql
,在项目中引入:
csharp
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="FluentMigrator.Runner.MySql" Version="6.2.0" />
</ItemGroup>
</Project>
新建一个CreateLogEntriesTable
,放在Migrations目录下
csharp
[Migration(20241216)]
public class CreateLogEntriesTable : Migration
{
public override void Up()
{
Create.Table("LogEntries")
.WithColumn("Id").AsString(36).PrimaryKey()
.WithColumn("Message").AsCustom(text)
.WithColumn("IsSuccess").AsBoolean().NotNullable()
.WithColumn("CreateTime").AsDateTime().NotNullable()
.WithColumn("UpdateTime").AsDateTime();
}
public override void Down()
{
Delete.Table("LogEntries");
}
}
添加服务注册
csharp
using FluentMigrator.Runner;
using Logging.Abstractions;
using Logging.Abstractions.Migrations;
namespace Microsoft.Extensions.DependencyInjection;
public static class CustomLoggerExtensions
{
/// <summary>
/// 添加自定义日志服务表结构迁移
/// </summary>
/// <param name="services"></param>
/// <param name="connectionString">数据库连接字符串</param>
/// <returns></returns>
public static IServiceCollection AddCustomLoggerMigration(this IServiceCollection services, string connectionString)
{
services.AddFluentMigratorCore()
.ConfigureRunner(
rb => rb.AddMySql5()
.WithGlobalConnectionString(connectionString)
.ScanIn(typeof(CreateLogEntriesTable).Assembly)
.For.Migrations()
)
.AddLogging(lb =>
{
lb.AddFluentMigratorConsole();
});
using var serviceProvider = services.BuildServiceProvider();
using var scope = serviceProvider.CreateScope();
var runner = scope.ServiceProvider.GetRequiredService<IMigrationRunner>();
runner.MigrateUp();
return services;
}
}
2. EntityFramework Core 的实现
2.1 数据库上下文
新建Logging.EntityFrameworkCore项目,添加对Logging.Abstractions项目的引用,并在项目中安装Pomelo.EntityFrameworkCore.MySql
和Microsoft.Extensions.ObjectPool
。
csharp
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.ObjectPool" Version="8.0.11" />
<PackageReference Include="Pomelo.EntityFrameworkCore.MySql" Version="8.0.2" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Logging.Abstractions\Logging.Abstractions.csproj" />
</ItemGroup>
</Project>
创建CustomLoggerDbContext
类,用于管理日志实体
csharp
using Logging.Abstractions;
using Microsoft.EntityFrameworkCore;
namespace Logging.EntityFrameworkCore;
public class CustomLoggerDbContext(DbContextOptions<CustomLoggerDbContext> options) : DbContext(options)
{
public virtual DbSet<CustomLogEntry> LogEntries { get; set; }
}
使用 ObjectPool 管理 DbContext :提高性能,减少 DbContext 的创建和销毁开销。
创建CustomLoggerDbContextPoolPolicy
csharp
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.ObjectPool;
namespace Logging.EntityFrameworkCore;
/// <summary>
/// DbContext 池策略
/// </summary>
/// <param name="options"></param>
public class CustomLoggerDbContextPoolPolicy(DbContextOptions<CustomLoggerDbContext> options) : IPooledObjectPolicy<CustomLoggerDbContext>
{
/// <summary>
/// 创建 DbContext
/// </summary>
/// <returns></returns>
public CustomLoggerDbContext Create()
{
return new CustomLoggerDbContext(options);
}
/// <summary>
/// 回收 DbContext
/// </summary>
/// <param name="context"></param>
/// <returns></returns>
public bool Return(CustomLoggerDbContext context)
{
// 重置 DbContext 状态
context.ChangeTracker.Clear();
return true;
}
}
2.2 实现日志写入
创建一个EfCoreCustomLogger
,继承自CustomLogger
,实现日志写入的具体逻辑
csharp
using Logging.Abstractions;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.ObjectPool;
namespace Logging.EntityFrameworkCore;
/// <summary>
/// EfCore自定义日志记录器
/// </summary>
public class EfCoreCustomLogger(ObjectPool<CustomLoggerDbContext> contextPool, ILogger<EfCoreCustomLogger> logger) : CustomLogger(logger)
{
/// <summary>
/// 根据Id查询日志
/// </summary>
/// <param name="logId"></param>
/// <returns></returns>
protected override CustomLogEntry? GetById(string logId)
{
var dbContext = contextPool.Get();
try
{
return dbContext.LogEntries.Find(logId);
}
finally
{
contextPool.Return(dbContext);
}
}
/// <summary>
/// 写入日志
/// </summary>
/// <param name="writeCommand"></param>
/// <returns></returns>
/// <exception cref="ArgumentOutOfRangeException"></exception>
protected override async Task WriteAsync(WriteCommand writeCommand)
{
var dbContext = contextPool.Get();
try
{
switch (writeCommand.CommandType)
{
case WriteCommandType.Insert:
if (writeCommand.LogEntry != null)
{
await dbContext.LogEntries.AddAsync(writeCommand.LogEntry);
}
break;
case WriteCommandType.Update:
{
if (writeCommand.LogEntry != null)
{
dbContext.LogEntries.Update(writeCommand.LogEntry);
}
break;
}
default:
throw new ArgumentOutOfRangeException();
}
await dbContext.SaveChangesAsync();
}
finally
{
contextPool.Return(dbContext);
}
}
}
添加服务注册
csharp
using Logging.Abstractions;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.ObjectPool;
namespace Logging.EntityFrameworkCore;
public static class EfCoreCustomLoggerExtensions
{
public static IServiceCollection AddEfCoreCustomLogger(this IServiceCollection services, string connectionString)
{
if (string.IsNullOrEmpty(connectionString))
{
throw new ArgumentNullException(nameof(connectionString));
}
services.AddCustomLoggerMigration(connectionString);
services.AddSingleton<ObjectPoolProvider, DefaultObjectPoolProvider>();
services.AddSingleton(serviceProvider =>
{
var options = new DbContextOptionsBuilder<CustomLoggerDbContext>()
.UseMySql(connectionString, ServerVersion.AutoDetect(connectionString))
.Options;
var poolProvider = serviceProvider.GetRequiredService<ObjectPoolProvider>();
return poolProvider.Create(new CustomLoggerDbContextPoolPolicy(options));
});
services.AddSingleton<ICustomLogger, EfCoreCustomLogger>();
return services;
}
}
3. MySqlConnector 的实现
MySqlConnector 的实现比较简单,利用原生SQL操作数据库完成日志的插入和更新。
新建Logging.Connector项目,添加对Logging.Abstractions项目的引用,并安装MySqlConnector
包
csharp
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="MySqlConnector" Version="2.4.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Logging.Abstractions\Logging.Abstractions.csproj" />
</ItemGroup>
</Project>
3.1 SQL脚本
为了方便维护,我们把需要用到的SQL脚本放在一个Consts
类中
csharp
namespace Logging.MySqlConnector;
public class Consts
{
/// <summary>
/// 插入日志
/// </summary>
public const string InsertSql = """
INSERT INTO `LogEntries` (`Id`, `TranceId`, `BizType`, `Body`, `Component`, `MsgType`, `Status`, `CreateTime`, `UpdateTime`, `Remark`)
VALUES (@Id, @TranceId, @BizType, @Body, @Component, @MsgType, @Status, @CreateTime, @UpdateTime, @Remark);
""";
/// <summary>
/// 更新日志
/// </summary>
public const string UpdateSql = """
UPDATE `LogEntries` SET `Status` = @Status, `UpdateTime` = @UpdateTime
WHERE `Id` = @Id;
""";
/// <summary>
/// 根据Id查询日志
/// </summary>
public const string QueryByIdSql = """
SELECT `Id`, `TranceId`, `BizType`, `Body`, `Component`, `MsgType`, `Status`, `CreateTime`, `UpdateTime`, `Remark`
FROM `LogEntries`
WHERE `Id` = @Id;
""";
}
3.2 实现日志写入
创建MySqlConnectorCustomLogger
类,实现日志写入的具体逻辑
csharp
using Logging.Abstractions;
using Microsoft.Extensions.Logging;
using MySqlConnector;
namespace Logging.MySqlConnector;
/// <summary>
/// 使用 MySqlConnector 实现记录日志
/// </summary>
public class MySqlConnectorCustomLogger : CustomLogger
{
/// <summary>
/// 数据库连接字符串
/// </summary>
private readonly string _connectionString;
/// <summary>
/// 构造函数
/// </summary>
/// <param name="connectionString">MySQL连接字符串</param>
/// <param name="logger"></param>
public MySqlConnectorCustomLogger(
string connectionString,
ILogger<MySqlConnectorCustomLogger> logger)
: base(logger)
{
_connectionString = connectionString;
}
/// <summary>
/// 根据Id查询日志
/// </summary>
/// <param name="messageId"></param>
/// <returns></returns>
protected override CustomLogEntry? GetById(string messageId)
{
using var connection = new MySqlConnection(_connectionString);
connection.Open();
using var command = new MySqlCommand(Consts.QueryByIdSql, connection);
command.Parameters.AddWithValue("@Id", messageId);
using var reader = command.ExecuteReader();
if (!reader.Read())
{
return null;
}
return new CustomLogEntry
{
Id = reader.GetString(0),
Message = reader.GetString(1),
IsSuccess = reader.GetBoolean(2),
CreateTime = reader.GetDateTime(3),
UpdateTime = reader.GetDateTime(4)
};
}
/// <summary>
/// 处理日志
/// </summary>
/// <param name="writeCommand"></param>
/// <returns></returns>
/// <exception cref="ArgumentOutOfRangeException"></exception>
protected override async Task WriteAsync(WriteCommand writeCommand)
{
await using var connection = new MySqlConnection(_connectionString);
await connection.OpenAsync();
switch (writeCommand.CommandType)
{
case WriteCommandType.Insert:
{
if (writeCommand.LogEntry != null)
{
await using var command = new MySqlCommand(Consts.InsertSql, connection);
command.Parameters.AddWithValue("@Id", writeCommand.LogEntry.Id);
command.Parameters.AddWithValue("@Message", writeCommand.LogEntry.Message);
command.Parameters.AddWithValue("@IsSuccess", writeCommand.LogEntry.IsSuccess);
command.Parameters.AddWithValue("@CreateTime", writeCommand.LogEntry.CreateTime);
command.Parameters.AddWithValue("@UpdateTime", writeCommand.LogEntry.UpdateTime);
await command.ExecuteNonQueryAsync();
}
break;
}
case WriteCommandType.Update:
{
if (writeCommand.LogEntry != null)
{
await using var command = new MySqlCommand(Consts.UpdateSql, connection);
command.Parameters.AddWithValue("@Id", writeCommand.LogEntry.Id);
command.Parameters.AddWithValue("@IsSuccess", writeCommand.LogEntry.IsSuccess);
command.Parameters.AddWithValue("@UpdateTime", writeCommand.LogEntry.UpdateTime);
await command.ExecuteNonQueryAsync();
}
break;
}
default:
throw new ArgumentOutOfRangeException();
}
}
}
添加服务注册
csharp
using Logging.Abstractions;
using Logging.MySqlConnector;
using Microsoft.Extensions.Logging;
namespace Microsoft.Extensions.DependencyInjection;
/// <summary>
/// MySqlConnector 日志记录器扩展
/// </summary>
public static class MySqlConnectorCustomLoggerExtensions
{
/// <summary>
/// 添加 MySqlConnector 日志记录器
/// </summary>
/// <param name="services"></param>
/// <param name="connectionString"></param>
/// <returns></returns>
public static IServiceCollection AddMySqlConnectorCustomLogger(this IServiceCollection services, string connectionString)
{
if (string.IsNullOrEmpty(connectionString))
{
throw new ArgumentNullException(nameof(connectionString));
}
services.AddSingleton<ICustomLogger>(s =>
{
var logger = s.GetRequiredService<ILogger<MySqlConnectorCustomLogger>>();
return new MySqlConnectorCustomLogger(connectionString, logger);
});
services.AddCustomLoggerMigration(connectionString);
return services;
}
}
4. 使用示例
下边是一个EntityFramework Core的实现使用示例,MySqlConnector的使用方式相同。
新建WebApi项目,添加Logging.ntityFrameworkCore
csharp
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();
// 添加EntityFrameworkCore日志记录器
var connectionString = builder.Configuration.GetConnectionString("MySql");
builder.Services.AddEfCoreCustomLogger(connectionString!);
var app = builder.Build();
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.UseSwagger();
app.UseSwaggerUI();
}
app.UseAuthorization();
app.MapControllers();
app.Run();
在控制器中使用
csharp
namespace EntityFrameworkCoreTest.Controllers;
[ApiController]
[Route("[controller]")]
public class TestController(ICustomLogger customLogger) : ControllerBase
{
[HttpPost("InsertLog")]
public IActionResult Post(CustomLogEntry model)
{
customLogger.LogReceived(model);
return Ok();
}
[HttpPut("UpdateLog")]
public IActionResult Put(string messageId, MessageStatus status)
{
customLogger.LogProcessed(messageId, status);
return Ok();
}
}