”观云千里，慧眼深入“：全方位AWS CloudTrail日志收集和洞察攻略

user60173410501382023-11-23 15:34

概述

AWS CloudTrail是AWS服务中不可或缺的一部分，负责记录在AWS账户中发生的各种操作和事件。这包括执行操作的人员、操作发生的时间、执行的操作以及操作的源IP地址。这些宝贵的信息都被存储在指定的Amazon S3存储桶中。

通过深入分析AWS CloudTrail的日志，我们能够获得大量有关AWS账户活动的信息，具体包括：

  1. 审计与合规性： 提供了完整的操作历史记录，能够审计和验证AWS资源的使用情况，满足合规性要求、监视潜在的安全威胁以及确保资源符合政策。
  2. 安全监控：通过分析CloudTrail日志，可以实时监控AWS账户中的活动，检测潜在的安全威胁或异常行为，如未经授权的访问尝试、特定资源的非正常使用或对关键服务的配置更改。
  3. 故障排除：当出现问题或错误时，CloudTrail 日志可以帮助追踪操作发生的时间和上下文，以便更容易诊断问题并解决故障。
  4. 资源管理：通过了解谁对AWS资源执行了什么操作，我们可以更好地管理和优化资源，跟踪资源的创建、修改和删除，并了解团队成员的活动。
  5. 安全策略改进：通过分析CloudTrail日志，有助于改进安全策略，确保只有授权的用户和服务可以执行必要的操作，并及时检测到任何潜在的风险。

总体来说，分析AWS CloudTrail日志能够提供对AWS账户活动的全面洞察，有助于确保安全性、合规性和操作的透明度。而观测云作为一款强大的实时数据监测平台，可以实现高效接入和分析AWS CloudTrail日志。

观测云是一款面向开发、运维、测试及业务团队的实时数据监测平台，能够统一满足云、云原生、应用及业务上的监测需求，快速实现基础设施、中间件、应用层和业务层，云平台的可观测。基础设施监测、日志与指标管理、应用性能监测、用户访问监测、可用性监测、系统级安全巡检、场景和仪表板等均为"观测云"的可观测解决方案，通过统一的数据采集、全面的数据监控、无缝的关联分析、自定义的场景搭建、高度的可编程性及敏捷的成员协作，为用户提供最迅速、最轻松、最全面、最自由的系统可观测平台。

前提条件

  1. 点击注册链接去注册观测云，并创建工作空间。
  2. 在EC2（Linux）上安装datakit。

运行以下安装脚本并运行。EC2需要能访问互联网

3. 开启logtreaming input

c 复制代码 
cd /usr/local/datakit/conf.d/log     //切换到datakit安装目录
cp logstreaming.conf.sample logstreaming.conf     //修改input配置文件名称为logstreaming.conf
datakit service -R     //重启datakit采集器

记录该地址备用：http:// EC2_IP:9529/v1/write/logstreaming（后面配置的lambda函数会将收集到的S3上的日志发送到这个地址）

基本原理

一个完整的AWS CloudTrail日志处理和分析方式包括以下步骤：

  1. 创建CloudTrail事件： 在AWS管理控制台中，创建CloudTrail事件，定义要监控的AWS账户的活动记录，选择一个S3存储桶存储CloudTrail事件日志文件。
  2. 配置 S3 存储桶： 在选择的S3存储桶中配置存储桶策略，确保只有授权的用户可以访问和写入CloudTrail事件日志文件，增强日志数据的安全性和完整性。
  3. S3 事件通知到 Lambda： 配置S3存储桶的事件通知，当有新的CloudTrail事件日志文件上传到存储桶时，触发AWS Lambda函数
  4. Lambda 函数处理事件： 编写Lambda函数处理S3存储桶的事件通知，获取新上传的CloudTrail事件日志文件，并解析其中的事件信息。
  5. 存储到观测云： 一旦Lambda函数解析了CloudTrail事件，将相关信息存储到观测云。

这个架构建立了一个自动化的监控和洞察系统，追踪AWS账户中的活动并进行深度分析。

在AWS上配置CloudTrail

  1. 点击"创建追踪"
  1. 选择追踪属性

输入跟踪名称，现在存储位置为"创建新的S3存储桶"或者"使用现有的S3存储桶"，然后点击"下一步"

  1. 选择日志事件

选择相应的实现类型，比如"管理事件"，"数据事件"等。，然后点击"下一步"

  1. 查看并创建，点击"创建追踪"

配置Lambda

  1. 创建函数

在lambda控制台，点击"创建函数"。

选择"从头开始创建"，输入函数名称，运行时"python3.7"，然后点击"创建函数"

  1. 添加触发器

点击"添加触发器"，选择S3，选择Bucket（该Bucket为创建CloudTail跟踪时所选择的Bucket），选择需要触发的事件类型。

点击"添加"

  1. 复制代码，点击"deploy
python 复制代码 
import json
import urllib3
import boto3
import gzip
import os

# 初始化日志
print('Loading function')

# HTTP端点,EC2_IP请修改成自己的EC2的外网IP，Source可以自定义，改名称会显示在日志查看器的来源里面
http_endpoint = 'http://EC2_IP:9529/v1/write/logstreaming?source=「自定义」'

headers = {
    'Content-Type': 'application/json',
}

s3_client = boto3.client('s3')

def send_record_to_endpoint(record):
    try:
        http = urllib3.PoolManager()
        response = http.request("POST", http_endpoint, body=json.dumps(record).encode(), headers=headers)
        print("HTTP response code:", response.status)
    except Exception as e:
        print("Failed to send record to HTTP endpoint:", str(e))

def lambda_handler(event, context):
    try:
        # 获取S3对象的信息
        records = [x for x in event.get('Records', []) if x.get('eventName') == 'ObjectCreated:Put']
        sorted_events = sorted(records, key=lambda e: e.get('eventTime'))
        latest_event = sorted_events[-1] if sorted_events else {}
        info = latest_event.get('s3', {})
        file_key = info.get('object', {}).get('key')
        bucket = info.get('bucket', {}).get('name')
        print("Processing S3 object: s3://%s/%s" % (bucket, file_key))

        # 下载S3对象到本地临时文件
        local_temp_file = '/tmp/file.gz'
        s3_client.download_file(bucket, file_key, local_temp_file)

        # 解压缩文件
        with gzip.open(local_temp_file, 'rb') as gz_file:
            file_content = gz_file.read().decode('utf-8')

        print("File content read and decompressed")

        # 解析JSON文件
        json_data = json.loads(file_content)

        # 获取所有的records并逐个发送到HTTP端点
        records = json_data.get("Records", [])
        for record in records:
            send_record_to_endpoint(record)

        print("Sent logs to HTTP endpoint")

    except Exception as e:
        print("An error occurred:", str(e))

    return {
        'statusCode': 200,
        'body': json.dumps('Printing access log to datakit!')
    }

授予相应的角色权限

  1. 选择配置--常规配置，然后点击编辑
  1. 选择相应角色

在现有角色中选择相应的角色，角色需要AmazonS3ReadOnlyAccess和AWSLambdaBasicExecutionRole

在观测云上查看CloudTrail日志

7.1 在观测云-日志查看器中可以看到相关日志

后续如果需要对日志进一步处理和分析（比如筛选，基于字段搜索，可视化），需要通过pipeling将相关字段从日志文本中提取出来。

7.2 创建Pipeline

创建视图

  1. 点击"场景"，"新建仪表盘"。
  1. 新建空白仪表盘
  1. 拖动相应视图到下面白板
  1. 修改相应筛选条件

选择日志，选择来源sxt_CloudTrail_log等，点击创建

  1. 也可以导入下面完整json，创建默认视图
json 复制代码 
{
  "dashboardBindSet": [],
  "dashboardExtend": {},
  "dashboardMapping": [],
  "dashboardOwnerType": "node",
  "dashboardType": "CUSTOM",
  "iconSet": {},
  "main": {
    "charts": [
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件总数",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 0,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedGroupByTime": null,
          "fixedTime": "",
          "isRefresh": false,
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "currentChartType": "pie",
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "Get事件类型分布",
        "pos": {
          "h": 13,
          "w": 12,
          "x": 0,
          "y": 37
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [
                {
                  "id": "90c496f0-6c8f-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "eventName",
                  "op": "wildcard",
                  "type": "keyword",
                  "value": "Get*"
                }
              ],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' and  `eventName` = wildcard('Get*')  } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "来源区域数",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 6,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "region",
              "fieldFunc": "count_distinct",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT_DISTINCT(`region`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件来源",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 12,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "eventSource",
              "fieldFunc": "count_distinct",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT_DISTINCT(`eventSource`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "bgColor": "",
            "changeWorkspace": false,
            "colors": [],
            "compareType": "",
            "downsample": "last",
            "fixedTime": "",
            "fontColor": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "levels": [],
            "lineColor": "#3AB8FF",
            "mappings": [],
            "openCompare": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "sequenceChartType": "line",
            "showFieldMapping": false,
            "showLine": false,
            "showLineAxis": false,
            "showTitle": true,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "来源服务数量",
        "pos": {
          "h": 8,
          "w": 6,
          "x": 18,
          "y": 0
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "__content_bytes",
              "fieldFunc": "count",
              "fieldType": "long",
              "fill": "",
              "filters": [
                {
                  "id": "52388b50-6c8d-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "type",
                  "op": "=",
                  "type": "keyword",
                  "value": "AWSService"
                }
              ],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`__content_bytes`)) { `index` = 'default' and  `type` = 'AWSService'  }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "singlestat",
            "unit": ""
          }
        ],
        "type": "singlestat"
      },
      {
        "extend": {
          "fixedTime": "",
          "isRefresh": false,
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "currentChartType": "pie",
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件来源分布",
        "pos": {
          "h": 18,
          "w": 12,
          "x": 0,
          "y": 8
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [
                "eventSource"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' } BY `eventSource`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "addColumns": [
              {
                "field": "message",
                "headerName": "事件详情"
              },
              {
                "field": "status",
                "hide": true
              },
              {
                "field": "__docid",
                "hide": true
              }
            ],
            "alias": [],
            "changeWorkspace": false,
            "colors": [],
            "currentChartType": "log",
            "fixedTime": "",
            "globalUnit": [],
            "inColumns": [
              {
                "cellRenderer": "StatusCellRender",
                "field": "time",
                "filterParams": {},
                "headerName": "时间",
                "showTooltip": false
              }
            ],
            "index": "default",
            "isTimeInterval": false,
            "levels": [],
            "showFieldMapping": false,
            "showTitle": true,
            "timeInterval": "",
            "titleDesc": "",
            "unitType": "global",
            "units": [],
            "valColorMappings": [],
            "valMappings": []
          }
        },
        "group": {
          "name": null
        },
        "name": "最近事件内容",
        "pos": {
          "h": 18,
          "w": 12,
          "x": 12,
          "y": 8
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "noTimeRange": false,
            "qtype": "dql",
            "query": {
              "alias": "内容",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "disableMultipleField": false,
              "field": null,
              "fieldFunc": "count",
              "fieldType": "text",
              "fields": [
                {
                  "alias": "事件详情",
                  "field": "message",
                  "isEditAlias": false
                },
                {
                  "field": "status",
                  "hide": true
                },
                {
                  "field": "__docid",
                  "hide": true
                }
              ],
              "fill": null,
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "highlight": true,
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(`message`, `status`, `__docid`) { `index` = 'default' }",
              "queryFuncs": [],
              "type": "simple"
            },
            "type": "log",
            "unit": ""
          }
        ],
        "type": "log"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartType": "areaLine",
            "colors": [],
            "compareColors": {
              "dayCompare": "rgba(11,11,12,0.5)",
              "hourCompare": "#0B0B0C",
              "monthCompare": "rgba(11,11,12,0.12)",
              "weekCompare": "rgba(11,11,12,0.3)"
            },
            "compareColorsDark": {
              "dayCompare": "rgba(213,217,226,0.5)",
              "hourCompare": "#D5D9E2",
              "monthCompare": "rgba(213,217,226,0.12)",
              "weekCompare": "rgba(213,217,226,0.25)"
            },
            "compareColorsLight": {
              "dayCompare": "rgba(11,11,12,0.5)",
              "hourCompare": "#0B0B0C",
              "monthCompare": "rgba(11,11,12,0.12)",
              "weekCompare": "rgba(11,11,12,0.3)"
            },
            "compareType": [],
            "currentChartType": "sequence",
            "density": "medium",
            "fixedTime": "",
            "globalUnit": [],
            "isPercent": false,
            "isTimeInterval": true,
            "legendPostion": "none",
            "legendValues": "",
            "levels": [],
            "mainMeasurementQueryCode": "A",
            "onlyShowGroupName": false,
            "openCompare": false,
            "openStack": false,
            "openThousandsSeparator": true,
            "precision": "2",
            "showFieldMapping": false,
            "showLine": false,
            "showTitle": true,
            "slimit": 20,
            "stackType": "time",
            "timeInterval": "auto",
            "titleDesc": "",
            "unitType": "global",
            "units": [],
            "xAxisShowType": "time",
            "yAxixMaxVal": null,
            "yAxixMinVal": null
          }
        },
        "group": {
          "name": null
        },
        "name": "事件趋势",
        "pos": {
          "h": 11,
          "w": 12,
          "x": 0,
          "y": 26
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "__content_bytes",
              "fieldFunc": "last",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(LAST(`__content_bytes`)) { `index` = 'default' }",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "sequence",
            "unit": ""
          }
        ],
        "type": "sequence"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "事件类型分布",
        "pos": {
          "h": 11,
          "w": 12,
          "x": 12,
          "y": 26
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      },
      {
        "extend": {
          "fixedTime": "",
          "settings": {
            "alias": [],
            "changeWorkspace": false,
            "chartCombineDefaultColor": "#F56610",
            "chartType": "pie",
            "colors": [],
            "enableCombine": true,
            "fixedTime": "",
            "globalUnit": [],
            "isTimeInterval": false,
            "legendPostion": "bottom",
            "levels": [],
            "mainMeasurementLimit": 20,
            "mainMeasurementQueryCode": "A",
            "mainMeasurementSort": "top",
            "onlyShowGroupName": false,
            "openThousandsSeparator": true,
            "otherColor": "#F56610",
            "precision": "2",
            "showFieldMapping": false,
            "showTitle": true,
            "slimit": 20,
            "timeInterval": "default",
            "titleDesc": "",
            "unitType": "global",
            "units": []
          }
        },
        "group": {
          "name": null
        },
        "name": "Put事件类型分布",
        "pos": {
          "h": 13,
          "w": 12,
          "x": 12,
          "y": 37
        },
        "queries": [
          {
            "color": "",
            "datasource": "dataflux",
            "name": "",
            "qtype": "dql",
            "query": {
              "alias": "",
              "code": "A",
              "dataSource": "Sxt_CloudTrail_Log",
              "field": "*",
              "fieldFunc": "count",
              "fieldType": "keyword",
              "fill": "",
              "filters": [
                {
                  "id": "ddb47b60-6c8f-11ee-bf1a-c51c09d640ed",
                  "logic": "and",
                  "name": "eventName",
                  "op": "wildcard",
                  "type": "keyword",
                  "value": "Put*"
                }
              ],
              "funcList": [],
              "groupBy": [
                "eventName"
              ],
              "groupByTime": "",
              "indexFilter": "default",
              "namespace": "logging",
              "q": "L::`Sxt_CloudTrail_Log`:(COUNT(`*`)) { `index` = 'default' and  `eventName` = wildcard('Put*')  } BY `eventName`",
              "queryFuncs": [],
              "search": "",
              "type": "simple"
            },
            "type": "pie",
            "unit": ""
          }
        ],
        "type": "pie"
      }
    ],
    "groups": [],
    "type": "template",
    "vars": []
  },
  "summary": "",
  "tagInfo": [],
  "thumbnail": "",
  "title": "概览"
}
  1. 创建仪表盘时选择导入自定义模版
  1. 输入仪表盘名称，上传json文件
  1. 仪表盘

截止目前已经将CloudTrail日志接入观测云，并编写了pipeline去提取日志中的相关字段，同时制作了相关可视化的仪表盘对日志数据进行可视化分析。后续也可以通过提取出来的字段创建相关的监控告警器，当某一特定时间出现时触发告警通知。

相关推荐
七夜zippoe2 小时前
CANN Runtime任务描述序列化与持久化源码深度解码
大数据·运维·服务器·cann
Fcy6483 小时前
Linux下 进程(一)（冯诺依曼体系、操作系统、进程基本概念与基本操作）
linux·运维·服务器·进程
袁袁袁袁满3 小时前
Linux怎么查看最新下载的文件
linux·运维·服务器
代码游侠3 小时前
学习笔记——设备树基础
linux·运维·开发语言·单片机·算法
Harvey9034 小时前
通过 Helm 部署 Nginx 应用的完整标准化步骤
linux·运维·nginx·k8s
珠海西格电力科技5 小时前
微电网能量平衡理论的实现条件在不同场景下有哪些差异？
运维·服务器·网络·人工智能·云计算·智慧城市
释怀不想释怀5 小时前
Linux环境变量
linux·运维·服务器
zzzsde5 小时前
【Linux】进程（4）：进程优先级&&调度队列
linux·运维·服务器
聆风吟º7 小时前
CANN开源项目实战指南：使用oam-tools构建自动化故障诊断与运维可观测性体系
运维·开源·自动化·cann
NPE~7 小时前
自动化工具Drissonpage 保姆级教程(含xpath语法)
运维·后端·爬虫·自动化·网络爬虫·xpath·浏览器自动化
热门推荐
01GitHub 镜像站点02Claude Code + GLM4.7 避坑指南：解决 Unable to connect to Anthropic services03openclaw配置教程（linux+局域网ollama）04Linux下V2Ray安装配置指南05OpenClaw Chrome扩展使用教程 - 浏览器中继控制06UV安装并设置国内源07Claude Code Skills 实用使用手册08一文了解国产算子编程语言 TileLang，TileLang 对国产开源生态的影响与启示09Vue-skills的中文文档10让 Trae IDE 智能体 “读懂”文档 Excel+PDF+DOCX ：mcp-documents-reader 工具使用指南