Day 16：大屏呈现——管理层视角 Dashboard 设计

---

🚀 Day 16：大屏呈现------管理层视角 Dashboard 设计

今日目标：

1. 彻底完成代码封版与日志降噪 ：提供全量、无断点的 Python 代码基线。将繁杂的查询细节降级为 DEBUG，只在宏观节点保留 INFO，打造极度纯净的系统后台。
2. 管理层指标提取：抛弃繁琐的中间步骤日志，利用 session_id 强关联，将分散的"计划、证据、报告"聚合为一条包含"风险分、耗时、Token成本"的高价值记录。
3. 高管级Dashboard 展示：编写最终的 Splunk Dashboard XML。以概览界面展示最新风险得分、今日平均风险、Token 月度/今日消耗，以及系统运行成功率。

---

💻 第一部分：后端引擎终极形态

请打开 Add-on Builder 的 Define & Test 编辑器，用这套无任何省略号的最终版代码覆盖原有代码。

(⚠️ 架构师注：所有的日志层级已严格规范化，execute_ai_spl 的底层语句被隐藏在 DEBUG 中，确保日常运行时的 INFO 日志清爽且具备极高的审计价值。)

import os
import sys
import time
import datetime
import json
import uuid
import requests
import splunklib.client as client
import splunklib.results as results

# ==========================================
# HELPER 1: Execute AI Generated SPL
# ==========================================
def execute_ai_spl(helper, service, spl_query):
    """
    Execute SPL generated by AI and return the raw result data.
    """
    spl_query = spl_query.strip()
    if not spl_query.startswith("search") and not spl_query.startswith("|"):
        spl_query = "search " + spl_query
        
    kwargs_oneshot = {"output_mode": "json"}
    
    # Demoted to DEBUG to keep production logs clean
    helper.log_debug(f"[Agentic Engine] Executing SPL: {spl_query}")
    
    try:
        search_results = service.jobs.oneshot(spl_query, **kwargs_oneshot)
        reader = results.JSONResultsReader(search_results)
        result_data = [res for res in reader if isinstance(res, dict)]
        helper.log_debug(f"[Agentic Engine] SUCCESS: Found {len(result_data)} events.")
        return result_data
    except Exception as e:
        helper.log_error(f"[Agentic Engine] FAILED execution: {str(e)}")
        return []

# ==========================================
# HELPER 2: Fetch Real Logs (M-ATH Concept)
# ==========================================
def fetch_rare_logs(helper, service, target_index):
    """
    Fetch the most recent rare/anomalous logs from the target index.
    """
    helper.log_debug("Fetching real rare logs for analysis...")
    spl = f"search index={target_index} | head 5 | table _raw"
    
    try:
        results_data = execute_ai_spl(helper, service, spl)
        if not results_data:
            helper.log_debug("No anomalous logs found in target index.")
            return None
        
        raw_logs = [item.get("_raw", "") for item in results_data if "_raw" in item]
        payload = "\n".join(raw_logs)

        # Context Distillation (Payload Truncation)
        MAX_CHARS = 6000 
        if len(payload) > MAX_CHARS:
            helper.log_info(f"Payload truncated to {MAX_CHARS} chars for token efficiency.")
            payload = payload[:MAX_CHARS] + "\n\n...[TRUNCATED DUE TO CONTEXT LIMITS. ANALYZE AVAILABLE DATA ONLY.]..."
            
        return payload
    except Exception as e:
        helper.log_error(f"Failed to fetch rare logs: {str(e)}")
        return None

# =========================================================================
# HELPER 3: Universal Token Extractor (FinOps Cost Tracking)
# =========================================================================
def extract_token_usage(helper, response_json, response_headers):
    """
    Extract token usage across different LLM providers for FinOps audit.
    """
    try:
        if "usage" in response_json:
            usage = response_json["usage"]
            if "total_tokens" in usage:
                return int(usage["total_tokens"])
            elif "prompt_tokens" in usage and "completion_tokens" in usage:
                return int(usage["prompt_tokens"]) + int(usage["completion_tokens"])
            elif "input_tokens" in usage and "output_tokens" in usage:
                return int(usage["input_tokens"]) + int(usage["output_tokens"])
        
        header_keys = [k.lower() for k in response_headers.keys()]
        for key in header_keys:
            if "token-usage" in key or "x-ratelimit-usage" in key:
                return int(response_headers.get(key, 0))
                
    except Exception as e:
        helper.log_error(f"[FinOps Warning] Token extraction error: {str(e)}")
    
    return 0

# ==========================================
# HELPER 4: The LLM API Connector
# ==========================================
def call_llm_api(helper, api_key, base_url, model, system_prompt, user_prompt, max_tokens):
    """
    Establish HTTP connection to the LLM API with hardware-level token limits.
    """
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    
    payload = {
        "model": model,
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt}
        ],
        "response_format": {"type": "json_object"},
        "max_tokens": max_tokens
    }
    
    endpoint = base_url if base_url.endswith("/chat/completions") else f"{base_url.rstrip('/')}/chat/completions"
    
    try:
        helper.log_debug(f"Initiating network request to LLM API: {endpoint}")
        response = requests.post(endpoint, headers=headers, json=payload, timeout=120)
        response.raise_for_status() 
        
        response_json = response.json()
        llm_content = response_json["choices"][0]["message"]["content"]
        
        total_tokens = extract_token_usage(helper, response_json, response.headers)
        helper.log_debug(f"API Call Success. FinOps Tracked: {total_tokens} tokens consumed.")
        
        return llm_content, total_tokens
        
    except requests.exceptions.RequestException as e:
        helper.log_error(f"Network error during API call: {str(e)}")
        raise

# ==========================================
# MAIN WORKFLOW: The Autonomous Agent
# ==========================================
def collect_events(helper, ew):
    """
    The Ultimate Live Workflow (Production Release).
    Features: Dynamic AI queries, Anti-Hallucination, Truncation, FinOps, Chaos Resilience.
    """
    # Top-level INFO marker for cycle tracking
    helper.log_info("PEAK AI Hunter: CYCLE START.")
    cycle_start_time = time.time()
    
    hunt_session_id = str(uuid.uuid4())
    helper.log_debug(f"Generated Session ID: {hunt_session_id}")

    try:
        session_key = getattr(helper, 'session_key', None) or getattr(helper._input_definition, 'metadata', {}).get('session_key')
        if not session_key:
            raise ValueError("Failed to acquire session_key from Splunk core.")
        service = client.Service(token=session_key)
        
        api_key = helper.get_global_setting("api_key")
        base_url = helper.get_global_setting("base_url")
        model_name = helper.get_global_setting("model_name")
        target_index = helper.get_output_index() or "main"

        if not api_key or not base_url:
             raise ValueError("API Key or Base URL is missing in Global Settings.")

        # ==========================================
        # PHASE 1: PREPARE
        # ==========================================
        rare_logs_payload = fetch_rare_logs(helper, service, target_index)
        if not rare_logs_payload:
            helper.log_info("System Clean: No anomalous logs found. Exiting cycle.")
            return

        sys_prompt_prepare = "You are a Senior Threat Hunter. You MUST reply in JSON format. Be extremely concise. No pleasantries. Schema: 'analysis' (string), 'hypotheses' (array). Each hypothesis MUST have 'ABLE' (must be a nested JSON object with keys: Actor, Behavior, Location, Evidence), 'spl_round_1_validation', and 'spl_round_2_drilldown'."
        usr_prompt_prepare = f"Analyze these logs:\n{rare_logs_payload}\n\nGenerate exactly 2 hypotheses. CRITICAL: For SPL, strictly start with 'search index={{target_index}}'. Output ONLY JSON."

        helper.log_debug("Triggering LLM for Prepare Phase...")
        blueprint_text, prep_tokens = call_llm_api(helper, api_key, base_url, model_name, sys_prompt_prepare, usr_prompt_prepare, max_tokens=1500)
        
        ai_hunting_plan = json.loads(blueprint_text.strip())
        hypotheses = ai_hunting_plan.get("hypotheses", [])

        ew.write_event(helper.new_event(
            source=helper.get_input_type(), index=target_index, sourcetype="_json",
            time=time.time(), 
            data=json.dumps({"session_id": hunt_session_id, "event_type": "PEAK_Plan", "timestamp": round(time.time(), 3), "content": ai_hunting_plan}, ensure_ascii=False)
        ))

        # ==========================================
        # PHASE 2: EXECUTE
        # ==========================================
        all_hunt_evidence = []
        for i, hyp in enumerate(hypotheses):
            hyp_start = time.time()
            spl_r1 = hyp.get("spl_round_1_validation", "").replace("{target_index}", target_index)
            spl_r2 = hyp.get("spl_round_2_drilldown", "").replace("{target_index}", target_index)
            
            r1_hits = len(execute_ai_spl(helper, service, spl_r1))
            r2_hits = len(execute_ai_spl(helper, service, spl_r2))
            
            # Defensive Programming: Safeguard against LLM Schema Hallucinations
            able_data = hyp.get('ABLE', {})
            if isinstance(able_data, dict):
                behavior_text = able_data.get('Behavior', 'Unknown')
            else:
                behavior_text = str(able_data) 

            all_hunt_evidence.append({
                "hypothesis_id": hyp.get("hypothesis_id", i+1),
                "threat_behavior": behavior_text,
                "round_1_hit_count": r1_hits,
                "round_2_hit_count": r2_hits,
                "execution_duration_sec": round(time.time() - hyp_start, 2)
            })

        ew.write_event(helper.new_event(
            source=helper.get_input_type(), index=target_index, sourcetype="_json",
            time=time.time(), 
            data=json.dumps({"session_id": hunt_session_id, "event_type": "PEAK_Evidence", "timestamp": round(time.time(), 3), "content": all_hunt_evidence}, ensure_ascii=False)
        ))

        # ==========================================
        # PHASE 3: ACT
        # ==========================================
        sys_prompt_act = "You are a Security Director. Output ONLY valid JSON. Keep summaries under 30 words. Keys: 'executive_summary', 'threat_qualification', 'risk_score', 'recommended_alert_spl'."
        usr_prompt_act = f"Here is the execution evidence:\n{json.dumps(all_hunt_evidence)}\n\nBased on these hits, qualify the threat, assign a score, and write alert SPL. Reply in JSON."

        helper.log_debug("Triggering LLM for Act Phase...")
        report_text, act_tokens = call_llm_api(helper, api_key, base_url, model_name, sys_prompt_act, usr_prompt_act, max_tokens=800)
        
        try:
            final_report = json.loads(report_text.strip())
        except json.JSONDecodeError as e:
            helper.log_error("JSON Truncation in Act Phase. Engaging fallback.")
            final_report = {"executive_summary": "LLM output truncated.", "risk_score": -1, "raw": report_text}

        # The total_tokens_used is ONLY recorded in the Final Report to prevent dashboard sum inflation
        ew.write_event(helper.new_event(
            source=helper.get_input_type(), index=target_index, sourcetype="_json",
            time=time.time(), 
            data=json.dumps({"session_id": hunt_session_id, "event_type": "PEAK_Final_Report", "timestamp": round(time.time(), 3), "total_tokens_used": prep_tokens + act_tokens, "content": final_report}, ensure_ascii=False)
        ))

        # Clear, concise INFO log for the end of the cycle
        duration = round(time.time() - cycle_start_time, 2)
        helper.log_info(f"PEAK AI Hunter: CYCLE COMPLETE. Session: {hunt_session_id}. Tokens: {prep_tokens + act_tokens}. Took {duration}s.")

    except Exception as e:
        # Enterprise-Grade Graceful Degradation & Error Alerting
        error_msg = str(e)
        helper.log_error(f"FATAL Pipeline Crash: {error_msg}")
        
        try:
            fallback_index = helper.get_output_index() or "main"
            ew.write_event(helper.new_event(
                source=helper.get_input_type(), index=fallback_index, sourcetype="_json",
                time=time.time(),
                data=json.dumps({
                    "session_id": hunt_session_id, 
                    "event_type": "PEAK_Error", 
                    "timestamp": round(time.time(), 3), 
                    "error_message": error_msg,
                    "agent_status": "CRITICAL_FAILURE"
                }, ensure_ascii=False)
            ))
            helper.log_info("Sent Error_State alert to main index successfully.")
        except Exception as write_err:
            helper.log_error(f"Secondary Crash: Could not write PEAK_Error event. {str(write_err)}")

---

💻 第二部分：管理层视角大屏的核心 SPL 逻辑

管理层大屏不需要看到 execute_ai_spl。它需要的是风险以及Token 消耗等的聚合。

这个查询的核心是使用 transaction 或 stats 将三个离散阶段的数据缝合在一起。

index=main sourcetype="_json" (event_type="PEAK_Plan" OR event_type="PEAK_Evidence" OR event_type="PEAK_Final_Report" OR event_type="PEAK_Error")
| stats 
    min(timestamp) as Start_Time_Epoch,
    max(timestamp) as End_Time_Epoch,
    latest(content.threat_qualification) as "Verdict",
    latest(content.risk_score) as "Risk_Score",
    latest(content.executive_summary) as "AI_Summary",
    sum(total_tokens_used) as "Tokens",
    latest(error_message) as "Error"
  by session_id
| eval Duration = round(End_Time_Epoch - Start_Time_Epoch, 2)
| eval Cost_USD = "$" . tostring(round((Tokens / 1000) * 0.002, 6))
| eval Start_Time = strftime(Start_Time_Epoch, "%Y-%m-%d %H:%M:%S")
| sort - Start_Time_Epoch
| table Start_Time, Verdict, Risk_Score, Cost_USD, Duration, AI_Summary

💻 第三部分：Dashboard XML 顶级设计模板

Splunk 仪表板底层是 XML。为了达到"炫酷"效果，我们将使用单值面板（Single Value）展示核心 KPI。

前往 Splunk 的 Dashboards 页面，点击 Create New Dashboard。选择 Classic Dashboards（经典仪表板），进入编辑模式后点击 Source (源码)，将以下 XML 完整贴入。

这就这套系统向管理层汇报时的"终极门面"。它会展示每次PEAK AI Hunter 运行的最新风险得分、今日平均风险、Token 月度/今日消耗，以及系统运行成功率，还能根据风险分值动态变色。

<dashboard>
  <label>PEAK AI Hunter - Executive &amp; FinOps Dashboard</label>
  <description>Automated Threat Hunting Analytics &amp; Token Consumption Tracking</description>
  <row>
    <panel>
      <single>
        <title>Latest Risk Score</title>
        <search>
          <query>index=main event_type="PEAK_Final_Report" | stats latest(content.risk_score)</query>
          <earliest>-24h@h</earliest>
          <latest>now</latest>
        </search>
        <option name="rangeValues">[0,30,70,100]</option>
        <option name="rangeColors">["0x53a051","0xf8be34","0xf1813f","0xdc4e41"]</option>
        <option name="useColors">1</option>
      </single>
    </panel>
    <panel>
      <single>
        <title>Average Risk Score (Today)</title>
        <search>
          <query>index=main event_type="PEAK_Final_Report" | where _time &gt;= relative_time(now(), "@d") | stats avg(content.risk_score) as avg_score | eval avg_score=round(avg_score, 2)</query>
          <earliest>@d</earliest>
          <latest>now</latest>
        </search>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <single>
        <title>Total Token Usage (Month)</title>
        <search>
          <query>index=main event_type="PEAK_Final_Report" | stats sum(total_tokens_used)</query>
          <earliest>@mon</earliest>
          <latest>now</latest>
        </search>
        <option name="underLabel">Tokens Consumed This Month</option>
      </single>
    </panel>
    <panel>
      <single>
        <title>Token Usage (Today)</title>
        <search>
          <query>index=main event_type="PEAK_Final_Report" | where _time &gt;= relative_time(now(), "@d") | stats sum(total_tokens_used)</query>
          <earliest>@d</earliest>
          <latest>now</latest>
        </search>
        <option name="underLabel">Tokens Consumed Today</option>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <single>
        <title>Successful Hunts (Today)</title>
        <search>
          <query>index=main (event_type="PEAK_Plan" OR event_type="PEAK_Evidence" OR event_type="PEAK_Final_Report") | stats count by session_id | search count=3 | stats count</query>
          <earliest>@d</earliest>
          <latest>now</latest>
        </search>
        <option name="rangeValues">[0]</option>
        <option name="rangeColors">["0xdc4e41","0x53a051"]</option>
        <option name="useColors">1</option>
        <option name="underLabel">Completed Closed-Loop Sessions</option>
      </single>
    </panel>
    <panel>
      <single>
        <title>Anomalous/Failed Hunts (Today)</title>
        <search>
          <query>index=main (event_type="PEAK_Plan" OR event_type="PEAK_Evidence" OR event_type="PEAK_Final_Report" OR event_type="PEAK_Error") | stats count, latest(event_type) as last_evt by session_id | where count &lt; 3 OR last_evt="PEAK_Error" | stats count</query>
          <earliest>@d</earliest>
          <latest>now</latest>
        </search>
        <option name="rangeValues">[0]</option>
        <option name="rangeColors">["0x53a051","0xdc4e41"]</option>
        <option name="useColors">1</option>
        <option name="underLabel">Crashed or Incomplete Sessions</option>
      </single>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Recent AI Hunting Reports Summary (Transaction View)</title>
        <search>
          <query>
index=main sourcetype="_json" (event_type="PEAK_Plan" OR event_type="PEAK_Evidence" OR event_type="PEAK_Final_Report" OR event_type="PEAK_Error")
| stats 
    min(timestamp) as Start_Time_Epoch,
    max(timestamp) as End_Time_Epoch,
    latest(content.threat_qualification) as Verdict,
    latest(content.risk_score) as Risk_Score,
    latest(content.executive_summary) as AI_Summary,
    sum(total_tokens_used) as Tokens,
    latest(error_message) as Error
  by session_id
| eval Duration = round(End_Time_Epoch - Start_Time_Epoch, 2)
| eval Start_Time = strftime(Start_Time_Epoch, "%Y-%m-%d %H:%M:%S")
| sort - Start_Time_Epoch
| table Start_Time, Verdict, Risk_Score, Tokens, Duration, AI_Summary, Error
          </query>
          <earliest>-7d@h</earliest>
          <latest>now</latest>
        </search>
        <option name="drilldown">none</option>
      </table>
    </panel>
  </row>
</dashboard>

---

现在，保存你的仪表板。你会看到一个拥有绿黄红警报色、实时折算金钱成本，且能一目了然看清最新狩猎结果的高大上报表！

🔍 验证与收获：管理层视角的直观感受

当你完成以上配置后，面对这张大屏，你不再只是一个"写脚本的安全员"，你正在管理一套全自动进行PEAK AI 狩猎的系统。

🎉 Day 16 总结： 恭喜你！今天你完成了从"技术专家"到"安全架构师"的思维转型。

• 代码封版：规范了日志输出，让运维审计更友好。
• 数据聚合 ：通过 session_id 降维打击，将碎片化数据整合为业务价值。
• 审美达标：利用 XML 构建了满足高管审美的大屏。

明天，我们将突破 AOB 框架霸权，把Dashboard做为插件的显示界面，完成界面重构与大屏呈现！