Elastic Says Added Context Raised Its SOC AI Triage Accuracy from 60% to 92%
Summary
Elastic says its SOC’s AI verdict correctness rose from 60% to 92% after alert triage was enriched with rule guides, Workday user data and 30 days of closed cases. The post details a three-agent workflow and analyst-feedback loop.
Key points
- Elastic reports its AI verdict correctness rose from 60% to 92%, measured by comparing AI verdicts with analyst closure reasons; the figures are from its internal SOC.
- The workflow adds each detection rule’s investigation guide, relevant Workday user details and closed cases from the previous 30 days for the same rule.
- Agent Brainstorm, built with Elastic Workflows and Agent Builder, coordinates 12 child workflows and three agents.
- Pattern Finder compares historical cases, L1 Investigation gathers evidence and reaches a verdict, and Summarizer formats the result for Slack and Kibana cases.
- Analyst closure notes and an optional “AI Incorrect?” field feed into later triage and can inform prompt tuning and added context.
- Elastic says the improved summaries let analysts review and close some cases in one step; the post describes a Slack button that triggers case closure actions.
Article Details
- Defense Focus
- Improve SOC alert-triage verdict accuracy by enriching AI investigations with rule-specific guidance, user context, and historical analyst outcomes, then applying analyst feedback to future triage.
- Detection Methods
- Compare current alerts with closed cases from the prior 30 days that share the same kibana.alert.rule.uuid; review analyst verdicts, closure notes, comments, and prior AI verdicts.
- Use historical cases to identify recurring activity patterns and false-positive or benign rates for a detection rule.
- Have the L1 investigation agent follow the rule-specific investigation guide and assess current alert evidence alongside supporting query results.
- Pivot to correlated alerts within ±5 minutes of the event.
- Compare AI verdicts with analyst-selected closure reasons and AI Incorrect? feedback to track errors and identify recurring misclassifications.
- Data Sources
- Current alert and event data, including rule UUID, user and host details, source IP, event action, process name, and command line.
- Investigation guides attached to detection rules and retrieved through the Kibana API.
- Workday user details, including hire or departure timing, cost center, job family, and job title.
- Closed cases from the prior 30 days for the same rule, including analyst close reasons, summaries, comments, AI verdicts and confidence, and AI Incorrect? feedback.
- ES|QL query results and correlated-alert data.
- Rule Types
- Detection rules with attached investigation guides, identified by kibana.alert.rule.uuid.
- ES|QL queries for predictable lookups and investigation data.
- Kibana API GET requests for investigation-guide retrieval and POST requests for case comments.
- POST requests to Slack for case updates and actions.
- Platforms
- Production SOC environment spanning laptops, servers, and cloud workloads.
- Defensive Actions
- Enrich each alert with its rule-specific investigation guide, relevant user details, and the prior 30 days of closed cases for the same rule before reaching a verdict.
- Use ES|QL queries and API calls for predictable lookups; reserve agents for tasks requiring analysis of case history and other contextual information.
- Track AI verdicts against analyst closure reasons, retain analyst correction notes, and use the results to tune prompts or add relevant context.
- Keep pattern-analysis and summarization agents limited to supplied data, and separate their roles from the investigation agent that performs queries.
- Keep analysts in the loop for case closure and use their feedback to improve triage.
Vendors
Products
Agent BuilderThe post explains how the Agent Brainstorm workflow in Elastic Workflows and Agent Builder uses Pattern Finder, L1 Investigation, and Summarizer agents to generate faster, more trustworthy triage results.Elastic SecurityElastic’s internal SOC serves as “Customer Zero” and runs the newest Elastic Security and Agent Builder versions in production.Elastic WorkflowsThe post explains how the Agent Brainstorm workflow in Elastic Workflows and Agent Builder uses Pattern Finder, L1 Investigation, and Summarizer agents to generate faster, more trustworthy triage results.Kibana#ElasticSlackAfter the changes, analysts could trust the AI summary enough to close cases in one step, with Slack and Kibana case updates integrated into the workflow.WorkdayElastic improved AI verdict correctness in its SOC from 60% to 92% by enriching agent context with investigation guides, Workday user risk data, and 30 days of past case outcomes for the same detection rule.