Kaspersky Explains Automated Filtering of False-Positive Security Alerts
Summary
The article compares algorithmic, single-LLM, and multi-agent approaches to generating filters for false-positive security alerts. It reports that a single-LLM approach reduced false positives by 99% and improved analyst efficiency.
Key points
- Filters should suppress irrelevant alerts without disabling the detection rule or excluding useful events; overly broad or narrow filters can be ineffective.
- The algorithmic approach selects relevant event fields, normalizes strings, and uses FP-Growth to reduce the combinations considered.
- The single-LLM approach generates filter options and explanations, but initially lacked output validation.
- A multi-agent design adds field selection, generation, validation, and commentary, but requires more model calls and may be unsuitable for isolated environments without model access.
- The article reports that the single-LLM and agent-based approaches performed similarly, so the authors chose the simpler, faster single-LLM option.
- The authors report a 99% reduction in false positives, 28% more filters generated per day, and 17% greater efficiency in analysts’ alert handling.
Article Details
- Defense Focus
- Automate generation of filters for false-positive security alerts while avoiding filters that suppress useful detections.
- Detection Methods
- Determine the context of the detection-rule fragment triggered by an alert.
- Select relevant event fields, normalize string values, and generate and rank candidate field combinations using FP-Growth.
- Use an LLM to generate filter options from an alert event, selected fields, and the triggering detection rule.
- Use separate LLM agents to select fields, generate filters, validate and correct them, and explain their choices.
- Evaluate generated filters by analyst approval rate and false-positive reduction over time.
- Data Sources
- Security alert logs and associated event fields
- The detection rule that triggered the alert
- Detection-rule context and recurring event patterns
- File paths, command lines, file hashes, and autorun-entry values
- Rule Types
- Field-value filters using exact or partial string matches
- Boolean-combination filters using AND
- Detection-rule conditions and query fragments
- Defensive Actions
- Have analysts identify alerts assessed as false positives and review suggested filters before deployment.
- Limit filters to the relevant detection context; avoid filters that are too broad or cover only an isolated event.
- Avoid reproducing all fields used by the detection rule in a filter, which could effectively disable that rule.
- Normalize variable file paths, including drive letters and user-folder components, before comparing values.
- Validate filter syntax and relevance, and send invalid filters back for regeneration where the workflow supports it.
- Monitor false-positive counts before and after deploying filters, and track analyst approval of generated options.