”观云千里，慧眼深入“：全方位AWS CloudTrail日志收集和洞察攻略

user60173410501382023-11-23 15:34

概述

AWS CloudTrail是AWS服务中不可或缺的一部分，负责记录在AWS账户中发生的各种操作和事件。这包括执行操作的人员、操作发生的时间、执行的操作以及操作的源IP地址。这些宝贵的信息都被存储在指定的Amazon S3存储桶中。

通过深入分析AWS CloudTrail的日志，我们能够获得大量有关AWS账户活动的信息，具体包括：

  1. 审计与合规性： 提供了完整的操作历史记录，能够审计和验证AWS资源的使用情况，满足合规性要求、监视潜在的安全威胁以及确保资源符合政策。
  2. 安全监控：通过分析CloudTrail日志，可以实时监控AWS账户中的活动，检测潜在的安全威胁或异常行为，如未经授权的访问尝试、特定资源的非正常使用或对关键服务的配置更改。
  3. 故障排除：当出现问题或错误时，CloudTrail 日志可以帮助追踪操作发生的时间和上下文，以便更容易诊断问题并解决故障。
  4. 资源管理：通过了解谁对AWS资源执行了什么操作，我们可以更好地管理和优化资源，跟踪资源的创建、修改和删除，并了解团队成员的活动。
  5. 安全策略改进：通过分析CloudTrail日志，有助于改进安全策略，确保只有授权的用户和服务可以执行必要的操作，并及时检测到任何潜在的风险。

总体来说，分析AWS CloudTrail日志能够提供对AWS账户活动的全面洞察，有助于确保安全性、合规性和操作的透明度。而观测云作为一款强大的实时数据监测平台，可以实现高效接入和分析AWS CloudTrail日志。

观测云是一款面向开发、运维、测试及业务团队的实时数据监测平台，能够统一满足云、云原生、应用及业务上的监测需求，快速实现基础设施、中间件、应用层和业务层，云平台的可观测。基础设施监测、日志与指标管理、应用性能监测、用户访问监测、可用性监测、系统级安全巡检、场景和仪表板等均为"观测云"的可观测解决方案，通过统一的数据采集、全面的数据监控、无缝的关联分析、自定义的场景搭建、高度的可编程性及敏捷的成员协作，为用户提供最迅速、最轻松、最全面、最自由的系统可观测平台。

前提条件

  1. 点击注册链接去注册观测云，并创建工作空间。
  2. 在EC2（Linux）上安装datakit。

运行以下安装脚本并运行。EC2需要能访问互联网

3. 开启logtreaming input

c 复制代码 
cd /usr/local/datakit/conf.d/log     //切换到datakit安装目录
cp logstreaming.conf.sample logstreaming.conf     //修改input配置文件名称为logstreaming.conf
datakit service -R     //重启datakit采集器

记录该地址备用：http:// EC2_IP:9529/v1/write/logstreaming（后面配置的lambda函数会将收集到的S3上的日志发送到这个地址）

基本原理

一个完整的AWS CloudTrail日志处理和分析方式包括以下步骤：

  1. 创建CloudTrail事件： 在AWS管理控制台中，创建CloudTrail事件，定义要监控的AWS账户的活动记录，选择一个S3存储桶存储CloudTrail事件日志文件。
  2. 配置 S3 存储桶： 在选择的S3存储桶中配置存储桶策略，确保只有授权的用户可以访问和写入CloudTrail事件日志文件，增强日志数据的安全性和完整性。
  3. S3 事件通知到 Lambda： 配置S3存储桶的事件通知，当有新的CloudTrail事件日志文件上传到存储桶时，触发AWS Lambda函数
  4. Lambda 函数处理事件： 编写Lambda函数处理S3存储桶的事件通知，获取新上传的CloudTrail事件日志文件，并解析其中的事件信息。
  5. 存储到观测云： 一旦Lambda函数解析了CloudTrail事件，将相关信息存储到观测云。

这个架构建立了一个自动化的监控和洞察系统，追踪AWS账户中的活动并进行深度分析。

在AWS上配置CloudTrail

  1. 点击"创建追踪"
  1. 选择追踪属性

输入跟踪名称，现在存储位置为"创建新的S3存储桶"或者"使用现有的S3存储桶"，然后点击"下一步"

  1. 选择日志事件

选择相应的实现类型，比如"管理事件"，"数据事件"等。，然后点击"下一步"

  1. 查看并创建，点击"创建追踪"

配置Lambda

  1. 创建函数

在lambda控制台，点击"创建函数"。

选择"从头开始创建"，输入函数名称，运行时"python3.7"，然后点击"创建函数"

  1. 添加触发器

点击"添加触发器"，选择S3，选择Bucket（该Bucket为创建CloudTail跟踪时所选择的Bucket），选择需要触发的事件类型。

点击"添加"

  1. 复制代码，点击"deploy
python 复制代码 
import json
import urllib3
import boto3
import gzip
import os

# 初始化日志
print('Loading function')

# HTTP端点,EC2_IP请修改成自己的EC2的外网IP，Source可以自定义，改名称会显示在日志查看器的来源里面
http_endpoint = 'http://EC2_IP:9529/v1/write/logstreaming?source=「自定义」'

headers = {
    'Content-Type': 'application/json',
}

s3_client = boto3.client('s3')

def send_record_to_endpoint(record):
    try:
        http = urllib3.PoolManager()
        response = http.request("POST", http_endpoint, body=json.dumps(record).encode(), headers=headers)
        print("HTTP response code:", response.status)
    except Exception as e:
        print("Failed to send record to HTTP endpoint:", str(e))

def lambda_handler(event, context):
    try:
        # 获取S3对象的信息
        records = [x for x in event.get('Records', []) if x.get('eventName') == 'ObjectCreated:Put']
        sorted_events = sorted(records, key=lambda e: e.get('eventTime'))
        latest_event = sorted_events[-1] if sorted_events else {}
        info = latest_event.get('s3', {})
        file_key = info.get('object', {}).get('key')
        bucket = info.get('bucket', {}).get('name')
        print("Processing S3 object: s3://%s/%s" % (bucket, file_key))

        # 下载S3对象到本地临时文件
        local_temp_file = '/tmp/file.gz'
        s3_client.download_file(bucket, file_key, local_temp_file)

        # 解压缩文件
        with gzip.open(local_temp_file, 'rb') as gz_file:
            file_content = gz_file.read().decode('utf-8')

        print("File content read and decompressed")

        # 解析JSON文件
        json_data = json.loads(file_content)

        # 获取所有的records并逐个发送到HTTP端点
        records = json_data.get("Records", [])
        for record in records:
            send_record_to_endpoint(record)

        print("Sent logs to HTTP endpoint")

    except Exception as e:
        print("An error occurred:", str(e))

    return {
        'statusCode': 200,
        'body': json.dumps('Printing access log to datakit!')
    }

授予相应的角色权限

  1. 选择配置--常规配置，然后点击编辑
  1. 选择相应角色

在现有角色中选择相应的角色，角色需要AmazonS3ReadOnlyAccess和AWSLambdaBasicExecutionRole

在观测云上查看CloudTrail日志

7.1 在观测云-日志查看器中可以看到相关日志

后续如果需要对日志进一步处理和分析（比如筛选，基于字段搜索，可视化），需要通过pipeling将相关字段从日志文本中提取出来。

7.2 创建Pipeline

创建视图

  1. 点击"场景"，"新建仪表盘"。
  1. 新建空白仪表盘
  1. 拖动相应视图到下面白板
  1. 修改相应筛选条件

选择日志，选择来源sxt_CloudTrail_log等，点击创建

  1. 也可以导入下面完整json，创建默认视图
json 复制代码 
{
  "dashboardBindSet": [],
  "dashboardExtend": {},
  "dashboardMapping": [],
  "dashboardOwnerType": "node",
  "dashboardType": "CUSTOM",
  "iconSet": {},
  "main": {
    "charts": [
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件总数",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 0,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedGroupByTime": null,
          "fixedTime": "",
          "isRefresh": false,
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "currentChartType": "pie",
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "Get事件类型分布",
        "pos": {
          "h": 13,
          "w": 12,
          "x": 0,
          "y": 37
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [
                {
                  "id": "90c496f0-6c8f-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "eventName",
                  "op": "wildcard",
                  "type": "keyword",
                  "value": "Get*"
                }
              ],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' and  `eventName` = wildcard('Get*')  } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "来源区域数",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 6,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "region",
              "fieldFunc": "count_distinct",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT_DISTINCT(`region`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件来源",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 12,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "eventSource",
              "fieldFunc": "count_distinct",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT_DISTINCT(`eventSource`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "来源服务数量",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 18,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "__content_bytes",
              "fieldFunc": "count",
              "fieldType": "long",
              "fill": "",
              "filters": [
                {
                  "id": "52388b50-6c8d-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "type",
                  "op": "=",
                  "type": "keyword",
                  "value": "AWSService"
                }
              ],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`__content_bytes`)) { `index` = 'default' and  `type` = 'AWSService'  }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "isRefresh": false,
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "currentChartType": "pie",
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件来源分布",
        "pos": {
          "h": 18,
          "w": 12,
          "x": 0,
          "y": 8
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [
                "eventSource"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' } BY `eventSource`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "addColumns": [
              {
                "field": "message",
                "headerName": "事件详情"
              },
              {
                "field": "status",
                "hide": true
              },
              {
                "field": "__docid",
                "hide": true
              }
            ],
            "alias": [],
            "changeWorkspace": false,
            "colors": [],
            "currentChartType": "log",
            "fixedTime": "",
            "globalUnit": [],
            "inColumns": [
              {
                "cellRenderer": "StatusCellRender",
                "field": "time",
                "filterParams": {},
                "headerName": "时间",
                "showTooltip": false
              }
            ],
            "index": "default",
            "isTimeInterval": false,
            "levels": [],
            "showFieldMapping": false,
            "showTitle": true,
            "timeInterval": "",
            "titleDesc": "",
            "unitType": "global",
            "units": [],
            "valColorMappings": [],
            "valMappings": []
          }
        },
        "group": {
          "name": null
        },
        "name": "最近事件内容",
        "pos": {
          "h": 18,
          "w": 12,
          "x": 12,
          "y": 8
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "noTimeRange": false,
            "qtype": "dql",
            "query": {
              "alias": "内容",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "disableMultipleField": false,
              "field": null,
              "fieldFunc": "count",
              "fieldType": "text",
              "fields": [
                {
                  "alias": "事件详情",
                  "field": "message",
                  "isEditAlias": false
                },
                {
                  "field": "status",
                  "hide": true
                },
                {
                  "field": "__docid",
                  "hide": true
                }
              ],
              "fill": null,
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "highlight": true,
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(`message`, `status`, `__docid`) { `index` = 'default' }",
              "queryFuncs": [],
              "type": "simple"
            },
            "type": "log",
            "unit": ""
          }
        ],
        "type": "log"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartType": "areaLine",
            "colors": [],
            "compareColors": {
              "dayCompare": "rgba(11,11,12,0.5)",
              "hourCompare": "#0B0B0C",
              "monthCompare": "rgba(11,11,12,0.12)",
              "weekCompare": "rgba(11,11,12,0.3)"
            },
            "compareColorsDark": {
              "dayCompare": "rgba(213,217,226,0.5)",
              "hourCompare": "#D5D9E2",
              "monthCompare": "rgba(213,217,226,0.12)",
              "weekCompare": "rgba(213,217,226,0.25)"
            },
            "compareColorsLight": {
              "dayCompare": "rgba(11,11,12,0.5)",
              "hourCompare": "#0B0B0C",
              "monthCompare": "rgba(11,11,12,0.12)",
              "weekCompare": "rgba(11,11,12,0.3)"
            },
            "compareType": [],
            "currentChartType": "sequence",
            "density": "medium",
            "fixedTime": "",
            "globalUnit": [],
            "isPercent": false,
            "isTimeInterval": true,
            "legendPostion": "none",
            "legendValues": "",
            "levels": [],
            "mainMeasurementQueryCode": "A",
            "onlyShowGroupName": false,
            "openCompare": false,
            "openStack": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "showFieldMapping": false,
            "showLine": false,
            "showTitle": true,
            "slimit": 20,
            "stackType": "time",
            "timeInterval": "auto",
            "titleDesc": "",
            "unitType": "global",
            "units": [],
            "xAxisShowType": "time",
            "yAxixMaxVal": null,
            "yAxixMinVal": null
          }
        },
        "group": {
          "name": null
        },
        "name": "事件趋势",
        "pos": {
          "h": 11,
          "w": 12,
          "x": 0,
          "y": 26
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "__content_bytes",
              "fieldFunc": "last",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(LAST(`__content_bytes`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "sequence",
            "unit": ""
          }
        ],
        "type": "sequence"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件类型分布",
        "pos": {
          "h": 11,
          "w": 12,
          "x": 12,
          "y": 26
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "Put事件类型分布",
        "pos": {
          "h": 13,
          "w": 12,
          "x": 12,
          "y": 37
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [
                {
                  "id": "ddb47b60-6c8f-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "eventName",
                  "op": "wildcard",
                  "type": "keyword",
                  "value": "Put*"
                }
              ],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' and  `eventName` = wildcard('Put*')  } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      }
    ],
    "groups": [],
    "type": "template",
    "vars": []
  },
  "summary": "",
  "tagInfo": [],
  "thumbnail": "",
  "title": "概览"
}
  1. 创建仪表盘时选择导入自定义模版
  1. 输入仪表盘名称，上传json文件
  1. 仪表盘

截止目前已经将CloudTrail日志接入观测云，并编写了pipeline去提取日志中的相关字段，同时制作了相关可视化的仪表盘对日志数据进行可视化分析。后续也可以通过提取出来的字段创建相关的监控告警器，当某一特定时间出现时触发告警通知。

相关推荐
cocologin1 小时前
RIP 技术深度解析
运维·网络·网络协议
庸子1 小时前
基于Jenkins和Kubernetes构建DevOps自动化运维管理平台
运维·kubernetes·jenkins
Lpy25691 小时前
Docker Desktop 安装到D盘（包括镜像下载等）+ 汉化
运维·docker·容器
眠修2 小时前
Kuberrnetes 服务发布
linux·运维·服务器
好奇的菜鸟2 小时前
Docker 配置项详解与示例
运维·docker·容器
xcs194053 小时前
集运维 麒麟桌面版v10 sp1 2403 aarch64 离线java开发环境自动化安装
运维·自动化
BAOYUCompany3 小时前
暴雨服务器成功中标华中科技大学集成电路学院服务器采购项目
运维·服务器
超龄超能程序猿4 小时前
Bitvisse SSH Client 安装配置文档
运维·ssh·github
奈斯ing4 小时前
【Redis篇】数据库架构演进中Redis缓存的技术必然性—高并发场景下穿透、击穿、雪崩的体系化解决方案
运维·redis·缓存·数据库架构
鳄鱼皮坡4 小时前
仿muduo库One Thread One Loop式主从Reactor模型实现高并发服务器
运维·服务器
热门推荐
01Coze实战第13讲：飞书多维表格读取+豆包生图模型，轻松批量生成短剧封面02手机电脑之间快速传输图片视频文件，不压缩画质、不限制大小的方法！03Coze扣子平台完整体验和实践（附国内和国际版对比）04华为昇腾 910B 部署 DeepSeek-R1 蒸馏系列模型详细指南05免费可用！最强AI数字人对口型神器：让照片开口说话唱歌，支持多人对口型+全身动作，1分钟学会！（附保姆级教程）06扣子（coze）实战|我用扣子搭建了一个自动分析小红薯笔记内容的AI应用|详细步骤拆解072024年 最新 iPhone手机 历代机型、屏幕尺寸、纵横比、分辨率 整理08C#调用WechatOCR.exe实现本地OCR文字识别09DeepSeek各版本说明与优缺点分析10Coze平台 创建AI智能体的详细步骤指南