DolphinDB实时告警系统:阈值告警设计

目录

    • 摘要
    • 一、告警系统概述
      • [1.1 告警系统架构](#1.1 告警系统架构)
      • [1.2 告警类型](#1.2 告警类型)
      • [1.3 告警级别](#1.3 告警级别)
    • 二、告警规则定义
      • [2.1 告警规则表](#2.1 告警规则表)
      • [2.2 规则配置接口](#2.2 规则配置接口)
    • 三、阈值检测
      • [3.1 单阈值检测](#3.1 单阈值检测)
      • [3.2 区间阈值检测](#3.2 区间阈值检测)
      • [3.3 动态阈值检测](#3.3 动态阈值检测)
    • 四、告警触发
      • [4.1 告警记录表](#4.1 告警记录表)
      • [4.2 告警触发函数](#4.2 告警触发函数)
      • [4.3 实时告警检测](#4.3 实时告警检测)
    • 五、告警抑制
      • [5.1 时间窗口抑制](#5.1 时间窗口抑制)
      • [5.2 告警聚合](#5.2 告警聚合)
      • [5.3 告警降噪](#5.3 告警降噪)
    • 六、告警通知
      • [6.1 通知渠道](#6.1 通知渠道)
      • [6.2 通知发送](#6.2 通知发送)
      • [6.3 告警升级](#6.3 告警升级)
    • 七、告警闭环
      • [7.1 告警确认](#7.1 告警确认)
      • [7.2 告警处理](#7.2 告警处理)
      • [7.3 告警统计](#7.3 告警统计)
    • 八、实战案例
      • [8.1 完整告警系统](#8.1 完整告警系统)
    • 九、总结
    • 参考资料

摘要

本文深入讲解DolphinDB实时告警系统技术。从告警规则定义到阈值检测,从告警触发到告警通知,从告警抑制到告警闭环,全面介绍实时告警系统的核心方法。通过丰富的代码示例,帮助读者掌握阈值告警设计的核心技能。


一、告警系统概述

1.1 告警系统架构

#mermaid-svg-2UkTWu9WbE4tnw7I{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;fill:#333;}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#mermaid-svg-2UkTWu9WbE4tnw7I .error-icon{fill:#552222;}#mermaid-svg-2UkTWu9WbE4tnw7I .error-text{fill:#552222;stroke:#552222;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-thickness-normal{stroke-width:1px;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-thickness-thick{stroke-width:3.5px;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-pattern-solid{stroke-dasharray:0;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-thickness-invisible{stroke-width:0;fill:none;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-pattern-dashed{stroke-dasharray:3;}#mermaid-svg-2UkTWu9WbE4tnw7I .edge-pattern-dotted{stroke-dasharray:2;}#mermaid-svg-2UkTWu9WbE4tnw7I .marker{fill:#333333;stroke:#333333;}#mermaid-svg-2UkTWu9WbE4tnw7I .marker.cross{stroke:#333333;}#mermaid-svg-2UkTWu9WbE4tnw7I svg{font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:16px;}#mermaid-svg-2UkTWu9WbE4tnw7I p{margin:0;}#mermaid-svg-2UkTWu9WbE4tnw7I .label{font-family:"trebuchet ms",verdana,arial,sans-serif;color:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster-label text{fill:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster-label span{color:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster-label span p{background-color:transparent;}#mermaid-svg-2UkTWu9WbE4tnw7I .label text,#mermaid-svg-2UkTWu9WbE4tnw7I span{fill:#333;color:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I .node rect,#mermaid-svg-2UkTWu9WbE4tnw7I .node circle,#mermaid-svg-2UkTWu9WbE4tnw7I .node ellipse,#mermaid-svg-2UkTWu9WbE4tnw7I .node polygon,#mermaid-svg-2UkTWu9WbE4tnw7I .node path{fill:#ECECFF;stroke:#9370DB;stroke-width:1px;}#mermaid-svg-2UkTWu9WbE4tnw7I .rough-node .label text,#mermaid-svg-2UkTWu9WbE4tnw7I .node .label text,#mermaid-svg-2UkTWu9WbE4tnw7I .image-shape .label,#mermaid-svg-2UkTWu9WbE4tnw7I .icon-shape .label{text-anchor:middle;}#mermaid-svg-2UkTWu9WbE4tnw7I .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#mermaid-svg-2UkTWu9WbE4tnw7I .rough-node .label,#mermaid-svg-2UkTWu9WbE4tnw7I .node .label,#mermaid-svg-2UkTWu9WbE4tnw7I .image-shape .label,#mermaid-svg-2UkTWu9WbE4tnw7I .icon-shape .label{text-align:center;}#mermaid-svg-2UkTWu9WbE4tnw7I .node.clickable{cursor:pointer;}#mermaid-svg-2UkTWu9WbE4tnw7I .root .anchor path{fill:#333333!important;stroke-width:0;stroke:#333333;}#mermaid-svg-2UkTWu9WbE4tnw7I .arrowheadPath{fill:#333333;}#mermaid-svg-2UkTWu9WbE4tnw7I .edgePath .path{stroke:#333333;stroke-width:2.0px;}#mermaid-svg-2UkTWu9WbE4tnw7I .flowchart-link{stroke:#333333;fill:none;}#mermaid-svg-2UkTWu9WbE4tnw7I .edgeLabel{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2UkTWu9WbE4tnw7I .edgeLabel p{background-color:rgba(232,232,232, 0.8);}#mermaid-svg-2UkTWu9WbE4tnw7I .edgeLabel rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2UkTWu9WbE4tnw7I .labelBkg{background-color:rgba(232, 232, 232, 0.5);}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster rect{fill:#ffffde;stroke:#aaaa33;stroke-width:1px;}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster text{fill:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I .cluster span{color:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:"trebuchet ms",verdana,arial,sans-serif;font-size:12px;background:hsl(80, 100%, 96.2745098039%);border:1px solid #aaaa33;border-radius:2px;pointer-events:none;z-index:100;}#mermaid-svg-2UkTWu9WbE4tnw7I .flowchartTitleText{text-anchor:middle;font-size:18px;fill:#333;}#mermaid-svg-2UkTWu9WbE4tnw7I rect.text{fill:none;stroke-width:0;}#mermaid-svg-2UkTWu9WbE4tnw7I .icon-shape,#mermaid-svg-2UkTWu9WbE4tnw7I .image-shape{background-color:rgba(232,232,232, 0.8);text-align:center;}#mermaid-svg-2UkTWu9WbE4tnw7I .icon-shape p,#mermaid-svg-2UkTWu9WbE4tnw7I .image-shape p{background-color:rgba(232,232,232, 0.8);padding:2px;}#mermaid-svg-2UkTWu9WbE4tnw7I .icon-shape .label rect,#mermaid-svg-2UkTWu9WbE4tnw7I .image-shape .label rect{opacity:0.5;background-color:rgba(232,232,232, 0.8);fill:rgba(232,232,232, 0.8);}#mermaid-svg-2UkTWu9WbE4tnw7I .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#mermaid-svg-2UkTWu9WbE4tnw7I .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#mermaid-svg-2UkTWu9WbE4tnw7I :root{--mermaid-font-family:"trebuchet ms",verdana,arial,sans-serif;} 告警系统架构
数据流
规则引擎
告警触发
告警处理
告警通知

1.2 告警类型

类型 说明
阈值告警 超过设定阈值
趋势告警 数据趋势异常
状态告警 状态变更告警
复合告警 多条件组合

1.3 告警级别

级别 说明 响应时间
紧急 严重故障 立即
严重 重要异常 15分钟
警告 一般异常 1小时
提示 轻微异常 4小时

二、告警规则定义

2.1 告警规则表

python 复制代码
// 告警规则表
share table(1:0, 
    `rule_id`rule_name`device_id`metric`operator`threshold`level`enabled,
    [STRING, STRING, SYMBOL, STRING, STRING, DOUBLE, INT, BOOL]) as alert_rules

// 初始化规则
insert into alert_rules values (
    "temp_high", "温度过高", `all, "temperature", ">", 80, 1, true
)
insert into alert_rules values (
    "temp_low", "温度过低", `all, "temperature", "<", 10, 2, true
)
insert into alert_rules values (
    "humidity_high", "湿度过高", `all, "humidity", ">", 90, 2, true
)
insert into alert_rules values (
    "pressure_high", "压力过高", `all, "pressure", ">", 1100, 1, true
)

2.2 规则配置接口

python 复制代码
// 添加告警规则
def addAlertRule(ruleId, ruleName, deviceId, metric, operator, threshold, level) {
    insert into alert_rules values (
        ruleId, ruleName, deviceId, metric, operator, threshold, level, true
    )
}

// 更新告警规则
def updateAlertRule(ruleId, threshold, level) {
    update alert_rules 
    set threshold = threshold, level = level
    where rule_id = ruleId
}

// 禁用告警规则
def disableAlertRule(ruleId) {
    update alert_rules set enabled = false where rule_id = ruleId
}

// 启用告警规则
def enableAlertRule(ruleId) {
    update alert_rules set enabled = true where rule_id = ruleId
}

三、阈值检测

3.1 单阈值检测

python 复制代码
// 单阈值检测
def checkThreshold(value, operator, threshold) {
    if (operator == ">") {
        return value > threshold
    } else if (operator == ">=") {
        return value >= threshold
    } else if (operator == "<") {
        return value < threshold
    } else if (operator == "<=") {
        return value <= threshold
    } else if (operator == "==") {
        return value == threshold
    } else if (operator == "!=") {
        return value != threshold
    }
    return false
}

3.2 区间阈值检测

python 复制代码
// 区间阈值检测
def checkRange(value, lower, upper) {
    return value < lower or value > upper
}

// 带警告区间的检测
def checkRangeWithWarning(value, normalLower, normalUpper, warnLower, warnUpper) {
    if (value < warnLower or value > warnUpper) {
        return "critical"
    } else if (value < normalLower or value > normalUpper) {
        return "warning"
    }
    return "normal"
}

3.3 动态阈值检测

python 复制代码
// 动态阈值(基于历史数据)
def calculateDynamicThreshold(deviceId, metric, multiplier = 3) {
    // 获取历史数据
    data = select eval(metric) as value
           from sensor_data
           where device_id = deviceId
           and timestamp > now() - 86400000 * 7  // 最近7天
    
    // 计算统计量
    mean = avg(data.value)
    std = std(data.value)
    
    return dict(STRING, ANY, [
        ["upper", mean + multiplier * std],
        ["lower", mean - multiplier * std],
        ["mean", mean],
        ["std", std]
    ])
}

四、告警触发

4.1 告警记录表

python 复制代码
// 告警记录表
share table(1:0, 
    `alert_id`rule_id`device_id`alert_time`metric`value`threshold`level`status,
    [STRING, STRING, SYMBOL, TIMESTAMP, STRING, DOUBLE, DOUBLE, INT, STRING]) as alert_log

// 告警计数器
alertCounter = 0

def generateAlertId() {
    alertCounter += 1
    return "ALT" + format(now(), "yyyyMMddHHmmss") + string(alertCounter)
}

4.2 告警触发函数

python 复制代码
// 触发告警
def triggerAlert(ruleId, deviceId, metric, value, threshold, level) {
    alertId = generateAlertId()
    
    insert into alert_log values (
        alertId, ruleId, deviceId, now(), metric, value, threshold, level, "new"
    )
    
    // 发送通知
    sendAlertNotification(alertId, deviceId, metric, value, level)
    
    return alertId
}

4.3 实时告警检测

python 复制代码
// 实时告警检测
def detectAlerts(data) {
    for (row in data) {
        // 获取适用规则
        rules = select * from alert_rules
                where (device_id = row.device_id or device_id = `all)
                and enabled = true
        
        for (rule in rules) {
            value = row[rule.metric]
            
            if (checkThreshold(value, rule.operator, rule.threshold)) {
                triggerAlert(rule.rule_id, row.device_id, rule.metric, 
                            value, rule.threshold, rule.level)
            }
        }
    }
}

// 订阅检测
subscribeTable(, "sensor_stream", "alert_detect", -1,
    def(msg) { detectAlerts(msg) }, true)

五、告警抑制

5.1 时间窗口抑制

python 复制代码
// 告警抑制配置
share table(1:0, 
    `rule_id`device_id`suppress_seconds,
    [STRING, SYMBOL, INT]) as alert_suppress

// 检查是否抑制
def shouldSuppress(ruleId, deviceId) {
    // 查找最近告警
    recent = select * from alert_log
             where rule_id = ruleId
             and device_id = deviceId
             and alert_time > now() - 300000  // 5分钟内
             order by alert_time desc
             limit 1
    
    if (recent.rows() > 0) {
        // 获取抑制时间
        suppress = select suppress_seconds from alert_suppress
                   where rule_id = ruleId and device_id = deviceId
        
        if (suppress.rows() > 0) {
            suppressSeconds = suppress.suppress_seconds[0]
            lastTime = recent.alert_time[0]
            
            if (now() - lastTime < suppressSeconds * 1000) {
                return true
            }
        }
    }
    
    return false
}

5.2 告警聚合

python 复制代码
// 告警聚合
def aggregateAlerts(timeWindow = 300000) {
    return select rule_id, device_id, 
                  count(*) as alert_count,
                  min(alert_time) as first_alert,
                  max(alert_time) as last_alert,
                  avg(value) as avg_value
           from alert_log
           where alert_time > now() - timeWindow
           and status = "new"
           group by rule_id, device_id
}

5.3 告警降噪

python 复制代码
// 告警降噪规则
def filterNoise(alerts) {
    // 过滤短时间内重复告警
    filtered = select * from alerts
               where not shouldSuppress(rule_id, device_id)
    
    return filtered
}

六、告警通知

6.1 通知渠道

python 复制代码
// 通知渠道配置
share table(1:0, 
    `channel_id`channel_type`config`enabled,
    [STRING, STRING, STRING, BOOL]) as notification_channel

// 初始化渠道
insert into notification_channel values (
    "email", "email", "smtp://smtp.company.com", true
)
insert into notification_channel values (
    "sms", "sms", "api://sms.company.com", true
)
insert into notification_channel values (
    "wechat", "wechat", "api://wechat.company.com", true
)

6.2 通知发送

python 复制代码
// 发送告警通知
def sendAlertNotification(alertId, deviceId, metric, value, level) {
    // 构建消息
    message = "告警: 设备" + deviceId + " " + metric + " = " + string(value) + 
              " (级别: " + string(level) + ")"
    
    // 根据级别选择通知渠道
    if (level == 1) {
        // 紧急:所有渠道
        sendEmail(message)
        sendSms(message)
        sendWechat(message)
    } else if (level == 2) {
        // 严重:邮件+微信
        sendEmail(message)
        sendWechat(message)
    } else {
        // 其他:仅邮件
        sendEmail(message)
    }
}

def sendEmail(message) {
    print("发送邮件: " + message)
}

def sendSms(message) {
    print("发送短信: " + message)
}

def sendWechat(message) {
    print("发送微信: " + message)
}

6.3 告警升级

python 复制代码
// 告警升级规则
def escalateAlert(alertId, escalateMinutes = 30) {
    alert = select * from alert_log where alert_id = alertId
    
    if (alert.rows() > 0 and alert.status[0] == "new") {
        alertTime = alert.alert_time[0]
        
        if (now() - alertTime > escalateMinutes * 60000) {
            // 升级告警
            update alert_log set level = level - 1 where alert_id = alertId
            
            // 重新发送通知
            sendAlertNotification(alertId, alert.device_id[0], 
                                 alert.metric[0], alert.value[0], alert.level[0] - 1)
        }
    }
}

七、告警闭环

7.1 告警确认

python 复制代码
// 确认告警
def acknowledgeAlert(alertId, operator, note) {
    update alert_log 
    set status = "acknowledged"
    where alert_id = alertId
    
    // 记录操作
    insert into alert_operation values (
        alertId, "acknowledge", operator, now(), note
    )
}

7.2 告警处理

python 复制代码
// 处理告警
def resolveAlert(alertId, operator, solution) {
    update alert_log 
    set status = "resolved"
    where alert_id = alertId
    
    // 记录处理
    insert into alert_operation values (
        alertId, "resolve", operator, now(), solution
    )
}

7.3 告警统计

python 复制代码
// 告警统计
def getAlertStats(startTime, endTime) {
    return select level, status, count(*) as count
           from alert_log
           where alert_time between startTime and endTime
           group by level, status
}

八、实战案例

8.1 完整告警系统

python 复制代码
// ========== 实时告警系统 ==========

// 1. 创建数据表
share table(1:0, 
    `rule_id`rule_name`device_id`metric`operator`threshold`level`enabled,
    [STRING, STRING, SYMBOL, STRING, STRING, DOUBLE, INT, BOOL]) as alert_rules

share table(1:0, 
    `alert_id`rule_id`device_id`alert_time`metric`value`threshold`level`status,
    [STRING, STRING, SYMBOL, TIMESTAMP, STRING, DOUBLE, DOUBLE, INT, STRING]) as alert_log

// 2. 初始化规则
insert into alert_rules values ("temp_high", "温度过高", `all, "temperature", ">", 80, 1, true)
insert into alert_rules values ("temp_low", "温度过低", `all, "temperature", "<", 10, 2, true)
insert into alert_rules values ("humidity_high", "湿度过高", `all, "humidity", ">", 90, 2, true)

// 3. 创建传感器数据流
share streamTable(100000:0, 
    `device_id`timestamp`temperature`humidity`pressure,
    [SYMBOL, TIMESTAMP, DOUBLE, DOUBLE, DOUBLE]) as sensor_stream

// 4. 实时告警检测
def detectAlerts(data) {
    for (row in data) {
        rules = select * from alert_rules where enabled = true
        
        for (rule in rules) {
            value = row[rule.metric]
            
            if (checkThreshold(value, rule.operator, rule.threshold)) {
                alertId = "ALT" + format(now(), "yyyyMMddHHmmss")
                insert into alert_log values (
                    alertId, rule.rule_id, row.device_id, now(), 
                    rule.metric, value, rule.threshold, rule.level, "new"
                )
                print("告警: " + rule.rule_name + " 设备:" + row.device_id + " 值:" + string(value))
            }
        }
    }
}

subscribeTable(, "sensor_stream", "alert_detect", -1,
    def(msg) { detectAlerts(msg) }, true)

// 5. 模拟数据
def generateMockSensor() {
    while (true) {
        data = table(
            take(1..10, 10) as device_id,
            take(now(), 10) as timestamp,
            rand(20.0..100.0, 10) as temperature,
            rand(40.0..100.0, 10) as humidity,
            rand(1000.0..1100.0, 10) as pressure
        )
        sensor_stream.append!(data)
        sleep(5000)
    }
}

submitJob("mock_sensor", "模拟传感器数据", generateMockSensor)

print("实时告警系统启动完成")

九、总结

本文详细介绍了DolphinDB实时告警系统:

  1. 告警规则:规则定义、规则配置
  2. 阈值检测:单阈值、区间阈值、动态阈值
  3. 告警触发:告警记录、实时检测
  4. 告警抑制:时间窗口、告警聚合、告警降噪
  5. 告警通知:通知渠道、通知发送、告警升级
  6. 告警闭环:告警确认、告警处理、告警统计

思考题

  1. 如何设计高效的告警规则引擎?
  2. 如何平衡告警的敏感度和误报率?
  3. 如何实现告警的智能降噪?

参考资料


相关推荐
七夜zippoe1 天前
DolphinDB质量实时监控:SPC统计过程控制
实时监控·质量·spc·dolphindb·过程控制
七夜zippoe4 天前
DolphinDB生产指标监控:KPI实时计算
监控·生产指标·kpi·dolphindb·实时 计算
七夜zippoe23 天前
DolphinDB WebSocket接入:实时数据流
网络·websocket·网络协议·dolphindb·实时数据流
杨云龙UP23 天前
Spotlight 接入 Oracle 数据库监控操作指南 2026-06-16
数据库·oracle·性能监控·预警·阈值·spotlight·瓶颈分析
七夜zippoe25 天前
DolphinDB HTTP API接入:RESTful数据推送
网络协议·http·api·restful·dolphindb
七夜zippoe1 个月前
DolphinDB Modbus协议接入:PLC数据采集
网络·数据采集·plc·modbus·dolphindb
七夜zippoe1 个月前
DolphinDB OPC-UA协议接入:工业标准通信
网络·通信·dolphindb·工业标准·opc-ua
七夜zippoe1 个月前
DolphinDB MQTT协议接入:工业设备数据采集
运维·mqtt·dolphindb·工业设备·协议接入
七夜zippoe1 个月前
DolphinDB机器学习函数:内置ML能力
人工智能·机器学习·ml·dolphindb·内置