[.NET] Aspire Dashboard: 云原生可观测性

MadLongTom2025-08-21 15:47

Aspire Dashboard 遥测数据采集机制详解

概述

.NET Aspire Dashboard是一个专门为分布式应用程序设计的可观测性平台，它通过OpenTelemetry协议(OTLP)采集和展示应用程序的日志(Logs) 、指标(Metrics) 和**追踪(Traces)**三大类遥测数据。Dashboard同时支持gRPC和HTTP两种传输协议，为应用程序提供了灵活的数据上报方式。

整体架构

Aspire Dashboard的遥测数据采集架构主要包含以下几个核心组件：
graph TB subgraph "应用程序/服务" A1["日志生成"] A2["指标生成"] A3["追踪生成"] A1 --> OTEL A2 --> OTEL A3 --> OTEL OTEL["OpenTelemetry SDK<br/>(OTLP Export)"] end OTEL -->|HTTP/gRPC| DASHBOARD subgraph DASHBOARD ["Aspire Dashboard"] subgraph ENDPOINTS ["OTLP 接收端点"] subgraph HTTP_EP ["HTTP 端点"] H1["/v1/logs"] H2["/v1/traces"] H3["/v1/metrics"] end subgraph GRPC_EP ["gRPC 端点"] G1["OtlpGrpcLogsService"] G2["OtlpGrpcTraceService"] G3["OtlpGrpcMetricsService"] end end ENDPOINTS --> STORAGE subgraph STORAGE ["数据处理与存储层"] subgraph REPO ["TelemetryRepository"] subgraph STORES ["存储组件"] S1["Logs 存储<br/>CircularBuffer&lt;OtlpLogEntry&gt;"] S2["Metrics 存储<br/>Resource Based Storage"] S3["Traces 存储<br/>CircularBuffer&lt;OtlpTrace&gt;"] end end end STORAGE --> UI subgraph UI ["Dashboard UI"] U1["日志查看器"] U2["指标图表"] U3["追踪分析器"] end end style A1 fill:#e1f5fe style A2 fill:#e8f5e8 style A3 fill:#fff3e0 style OTEL fill:#f3e5f5 style S1 fill:#e1f5fe style S2 fill:#e8f5e8 style S3 fill:#fff3e0

核心组件详解

1. OTLP接收端点

Dashboard提供两种接收遥测数据的方式：

HTTP端点 (OtlpHttpEndpointsBuilder)

  • 路径 : /v1/logs, /v1/traces, /v1/metrics
  • 支持格式: Protocol Buffers (application/x-protobuf)
  • CORS支持: 可配置跨域资源共享
  • 认证: 支持API Key认证
csharp 复制代码 
// HTTP端点映射示例
group.MapPost("logs", static (MessageBindable<ExportLogsServiceRequest> request, OtlpLogsService service) =>
{
    if (request.Message == null)
    {
        return Results.Empty;
    }
    return OtlpResult.Response(service.Export(request.Message));
});

gRPC端点

  • 服务 : OtlpGrpcLogsService, OtlpGrpcTraceService, OtlpGrpcMetricsService
  • 协议: OpenTelemetry标准gRPC服务
  • 性能: 更高效的二进制传输
csharp 复制代码 
[Authorize(Policy = OtlpAuthorization.PolicyName)]
public class OtlpGrpcLogsService : LogsService.LogsServiceBase
{
    public override Task<ExportLogsServiceResponse> Export(ExportLogsServiceRequest request, ServerCallContext context)
    {
        return Task.FromResult(_logsService.Export(request));
    }
}

2. 数据处理服务

日志处理 (OtlpLogsService)

csharp 复制代码 
public ExportLogsServiceResponse Export(ExportLogsServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddLogs(addContext, request.ResourceLogs);
    
    return new ExportLogsServiceResponse
    {
        PartialSuccess = new ExportLogsPartialSuccess
        {
            RejectedLogRecords = addContext.FailureCount
        }
    };
}

追踪处理 (OtlpTraceService)

csharp 复制代码 
public ExportTraceServiceResponse Export(ExportTraceServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddTraces(addContext, request.ResourceSpans);
    
    return new ExportTraceServiceResponse
    {
        PartialSuccess = new ExportTracePartialSuccess
        {
            RejectedSpans = addContext.FailureCount
        }
    };
}

指标处理 (OtlpMetricsService)

csharp 复制代码 
public ExportMetricsServiceResponse Export(ExportMetricsServiceRequest request)
{
    var addContext = new AddContext();
    _telemetryRepository.AddMetrics(addContext, request.ResourceMetrics);
    
    return new ExportMetricsServiceResponse
    {
        PartialSuccess = new ExportMetricsPartialSuccess
        {
            RejectedDataPoints = addContext.FailureCount
        }
    };
}

3. 数据存储层 (TelemetryRepository)

TelemetryRepository是Dashboard的核心数据管理组件，负责：

存储结构

  • 日志存储 : CircularBuffer<OtlpLogEntry> - 循环缓冲区，FIFO方式管理
  • 追踪存储 : CircularBuffer<OtlpTrace> - 支持容量管理和自动清理
  • 指标存储: 基于Resource的存储模式
  • 资源管理 : ConcurrentDictionary<ResourceKey, OtlpResource>

数据容量管理

csharp 复制代码 
public sealed class TelemetryLimitOptions
{
    public int MaxLogCount { get; set; } = 10_000;        // 最大日志条数
    public int MaxTraceCount { get; set; } = 10_000;      // 最大追踪条数  
    public int MaxMetricsCount { get; set; } = 50_000;    // 最大指标点数
    public int MaxAttributeCount { get; set; } = 128;     // 最大属性数量
    public int MaxAttributeLength { get; set; } = int.MaxValue; // 最大属性长度
    public int MaxSpanEventCount { get; set; } = int.MaxValue;  // 最大Span事件数
}

数据插入机制

日志插入：支持乱序插入和时间戳排序

csharp 复制代码 
public void AddLogsCore(AddContext context, OtlpResourceView resourceView, RepeatedField<ScopeLogs> scopeLogs)
{
    _logsLock.EnterWriteLock();
    try
    {
        foreach (var record in sl.LogRecords)
        {
            var logEntry = new OtlpLogEntry(record, resourceView, scope, _otlpContext);
            
            // 基于时间戳插入到正确位置
            var added = false;
            for (var i = _logs.Count - 1; i >= 0; i--)
            {
                if (logEntry.TimeStamp > _logs[i].TimeStamp)
                {
                    _logs.Insert(i + 1, logEntry);
                    added = true;
                    break;
                }
            }
            if (!added)
            {
                _logs.Insert(0, logEntry);
            }
        }
    }
    finally
    {
        _logsLock.ExitWriteLock();
    }
}

4. 实时订阅机制

Dashboard使用发布-订阅模式实现实时数据更新：

csharp 复制代码 
// 订阅管理
private readonly List<Subscription> _resourceSubscriptions = new();
private readonly List<Subscription> _logSubscriptions = new();
private readonly List<Subscription> _metricsSubscriptions = new();
private readonly List<Subscription> _tracesSubscriptions = new();

// 触发订阅更新
private void RaiseSubscriptionChanged(List<Subscription> subscriptions)
{
    lock (_lock)
    {
        foreach (var subscription in subscriptions)
        {
            subscription.TryExecute();
        }
    }
}

5. 暂停管理 (PauseManager)

支持暂停数据采集功能，避免在调试期间数据过载：

csharp 复制代码 
public void AddLogs(AddContext context, RepeatedField<ResourceLogs> resourceLogs)
{
    if (_pauseManager.AreStructuredLogsPaused(out _))
    {
        _logger.LogTrace("{Count} incoming structured log(s) ignored because of an active pause.", resourceLogs.Count);
        return;
    }
    // ... 处理日志
}

6. 组件关系图

graph TD subgraph "Dashboard Web Application" DWA[DashboardWebApplication] --> OTLP_HTTP[OtlpHttpEndpointsBuilder] DWA --> OTLP_GRPC[gRPC Services] DWA --> CONFIG[Configuration] subgraph "OTLP Services" OTLP_HTTP --> LS[OtlpLogsService] OTLP_HTTP --> TS[OtlpTraceService] OTLP_HTTP --> MS[OtlpMetricsService] OTLP_GRPC --> GLS[OtlpGrpcLogsService] OTLP_GRPC --> GTS[OtlpGrpcTraceService] OTLP_GRPC --> GMS[OtlpGrpcMetricsService] GLS --> LS GTS --> TS GMS --> MS end subgraph "Data Layer" LS --> TR[TelemetryRepository] TS --> TR MS --> TR TR --> PM[PauseManager] TR --> CB1[CircularBuffer&lt;Logs&gt;] TR --> CB2[CircularBuffer&lt;Traces&gt;] TR --> RM[Resource Manager] TR --> SUB[Subscription System] end subgraph "UI Components" SUB --> LV[Log Viewer] SUB --> MV[Metrics Viewer] SUB --> TV[Trace Viewer] end end subgraph "Host Integration" RP[ResourcePublisher] --> DC[DashboardClient] DC --> TR end style DWA fill:#f9f,stroke:#333,stroke-width:3px style TR fill:#bbf,stroke:#333,stroke-width:2px style SUB fill:#bfb,stroke:#333,stroke-width:2px

配置与端点

端点配置 (OtlpOptions)

csharp 复制代码 
public sealed class OtlpOptions
{
    public string? PrimaryApiKey { get; set; }          // 主API密钥
    public string? SecondaryApiKey { get; set; }        // 备用API密钥  
    public OtlpAuthMode? AuthMode { get; set; }         // 认证模式
    public string? GrpcEndpointUrl { get; set; }        // gRPC端点URL
    public string? HttpEndpointUrl { get; set; }        // HTTP端点URL
    public OtlpCors Cors { get; set; } = new();         // CORS配置
}

CORS配置

csharp 复制代码 
public sealed class OtlpCors
{
    public string? AllowedOrigins { get; set; }   // 允许的来源域名
    public string? AllowedHeaders { get; set; }   // 允许的请求头
    
    [MemberNotNullWhen(true, nameof(AllowedOrigins))]
    public bool IsCorsEnabled => !string.IsNullOrEmpty(AllowedOrigins);
}

应用程序配置

Dashboard在DashboardWebApplication中进行完整的服务配置：

csharp 复制代码 
// 注册OTLP服务
builder.Services.AddSingleton<TelemetryRepository>();
builder.Services.AddTransient<OtlpLogsService>();
builder.Services.AddTransient<OtlpTraceService>();
builder.Services.AddTransient<OtlpMetricsService>();

// 配置gRPC
builder.Services.AddGrpc();

// 映射端点
_app.MapHttpOtlpApi(dashboardOptions.Otlp);      // HTTP端点
_app.MapGrpcService<OtlpGrpcMetricsService>();   // gRPC指标服务
_app.MapGrpcService<OtlpGrpcTraceService>();     // gRPC追踪服务  
_app.MapGrpcService<OtlpGrpcLogsService>();      // gRPC日志服务

与Aspire宿主的集成

资源发布 (ResourcePublisher)

Dashboard通过ResourcePublisher与Aspire宿主通信，获取应用程序资源信息：

csharp 复制代码 
internal sealed class ResourcePublisher
{
    private readonly Dictionary<string, SourceAndResourceSnapshot> _snapshot = [];
    private ImmutableHashSet<Channel<ResourceSnapshotChange>> _outgoingChannels = [];
    
    // 集成资源变更并广播给订阅者
    internal async ValueTask IntegrateAsync(IResource source, ResourceSnapshot snapshot, ResourceSnapshotChangeType changeType)
    {
        lock (_syncLock)
        {
            switch (changeType)
            {
                case ResourceSnapshotChangeType.Upsert:
                    _snapshot[snapshot.Name] = new SourceAndResourceSnapshot(source, snapshot);
                    break;
                case ResourceSnapshotChangeType.Delete:
                    _snapshot.Remove(snapshot.Name);
                    break;
            }
        }
        
        // 通知所有订阅者
        foreach (var channel in channels)
        {
            await channel.Writer.WriteAsync(new(changeType, snapshot), cancellationToken);
        }
    }
}

Dashboard客户端 (DashboardClient)

实现与资源服务的gRPC通信：

csharp 复制代码 
internal sealed class DashboardClient : IDashboardClient
{
    private readonly Dictionary<string, ResourceViewModel> _resourceByName = new();
    private readonly GrpcChannel? _channel;
    private Aspire.DashboardService.Proto.V1.DashboardService.DashboardServiceClient? _client;
    
    // 订阅资源变更
    public async IAsyncEnumerable<IReadOnlyList<ResourceViewModelChange>> SubscribeResourcesAsync()
    {
        // 通过gRPC流式接收资源更新
    }
}

数据流转过程

1. 数据接收流程

sequenceDiagram participant App as 应用程序 participant OTLP as OTLP Exporter participant HTTP as HTTP端点 participant GRPC as gRPC端点 participant Service as 数据处理服务 participant Repo as TelemetryRepository participant Storage as 存储层 App->>OTLP: 生成遥测数据 OTLP->>HTTP: HTTP/JSON 请求 OTLP->>GRPC: gRPC 调用 HTTP->>Service: OtlpLogsService.Export() GRPC->>Service: OtlpGrpcLogsService.Export() Service->>Repo: AddLogs/AddTraces/AddMetrics Repo->>Storage: 写入CircularBuffer Storage-->>Repo: 存储确认 Repo-->>Service: 处理结果 Service-->>HTTP: ExportResponse Service-->>GRPC: ExportResponse

2. 数据查询流程

sequenceDiagram participant UI as Dashboard UI participant Sub as 订阅系统 participant Repo as TelemetryRepository participant Filter as 数据过滤器 participant Storage as 存储层 UI->>Sub: 订阅数据更新 Sub->>Repo: 注册订阅回调 loop 实时数据查询 UI->>Repo: 查询遥测数据 Repo->>Filter: 应用过滤条件 Filter->>Storage: 读取数据 Storage-->>Filter: 返回数据 Filter-->>Repo: 过滤后数据 Repo-->>UI: 返回结果 end Note over Storage: 数据变更时 Storage->>Repo: 触发变更事件 Repo->>Sub: 通知订阅者 Sub->>UI: 实时推送更新

3. 实时更新机制

flowchart LR A[数据变更] --> B[触发订阅] B --> C[推送到UI组件] C --> D[前端实时刷新] subgraph 订阅管理 E[ResourceSubscriptions] F[LogSubscriptions] G[MetricsSubscriptions] H[TracesSubscriptions] end B --> E B --> F B --> G B --> H

性能优化特性

1. 内存管理

  • 循环缓冲区: 自动清理老数据，避免内存泄漏
  • 容量限制: 可配置的数据条数上限
  • 分段锁: 使用ReaderWriterLockSlim减少锁争用

2. 数据压缩

  • HTTP压缩: 支持响应压缩
  • Protocol Buffers: 高效的二进制序列化

3. 异步处理

  • 流式处理: 支持大批量数据的流式处理
  • 异步订阅: 非阻塞的实时数据推送

总结

Aspire Dashboard采用了模块化的架构设计，通过标准的OpenTelemetry协议接收遥测数据，使用高效的存储机制和实时订阅模式，为分布式应用程序提供了完整的可观测性解决方案。其核心优势包括：

  1. 标准化: 完全基于OpenTelemetry标准，确保与各种应用程序的兼容性
  2. 高性能: 使用循环缓冲区和异步处理，支持高吞吐量的数据采集
  3. 实时性: 基于订阅模式的实时数据更新机制
  4. 可配置: 灵活的配置选项，支持不同的部署场景
  5. 多协议: 同时支持HTTP和gRPC两种传输协议

该架构为.NET生态系统中的分布式应用程序监控和调试提供了强大的基础设施支持。

热门推荐
01UV安装并设置国内源02DeepSeek更新！速览DeepSeek V3.1新特性03KGG转MP3工具|非KGM文件|解密音频04Qwen3-Coder 快速上手教程 | Qwen Code + Claude Code05蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接06【2025.08.06最新版】Android Studio下载、安装及配置记录（自动下载sdk）07Claude Code VSCode集成开发指南：AI编程助手完整配置08Spring 调试终于不再痛苦了092025最新国内服务器可用docker源仓库地址大全（2025年8月更新）10【大模型实战篇】部署GPT-OSS-120B踩得坑（vllm / ollama等推理框架）