How Wiz Built an LLM-Powered Search Engine for Its Cloud Security Graph
Summary
Wiz describes building a natural-language query tool for its cloud security graph using retrieval, curated examples, clustering, and reranking to help users construct queries.
Key points
- The tool uses Anthropic Sonnet 3.5 through Amazon Bedrock to turn natural-language requests into Wiz Security Graph queries.
- Its retrieval process supplies the model with similar query examples, relevant graph entities and properties, allowed operators, values, and relationships.
- An initial benchmark of 200 query examples achieved 80% accuracy by exact match or manual review; real-world requests exposed gaps in the examples and prompts.
- Wiz clustered anonymized customer queries with embeddings, UMAP, and DBSCAN, identifying approximately 150 topics to help find gaps in its example database.
- The team expanded its examples with expert-written queries, LLM-generated descriptions reviewed by experts, and automated examples based on graph schema changes.
- Maximal Marginal Relevance (MMR) was added to retrieve examples that balance relevance and diversity, reducing redundancy in the model's context.
Article Details
- Topic
- Natural-language query generation for Wiz's Security Graph using LLMs, retrieval, clustering, and re-ranking
People
Vendors
AmazonChoosing the Right LLM: Sonnet 3.5 via Amazon Bedrock Anthropicour approach was selecting the appropriate LLM that balances performance, cost, and latency. We chose Anthropic’s Sonnet 3.5 accessed through Amazon Bedrock as our LLM. It gave us the perfect balance of speed,Wizblog post, we take you through our journey of developing a user-friendly text-to-query search engine for Wiz's Security Graph. We’ll discuss the challenges we faced and the innovative solutions we implemented to make
Products
Amazon BedrockChoosing the Right LLM: Sonnet 3.5 via Amazon Bedrock Sonnet 3.5Choosing the Right LLM: Sonnet 3.5 via Amazon Bedrock Wiz Security GraphEnter the Wiz Security Graph— a powerful cybersecurity tool that provides a comprehensive view of an organization's cloud infrastructure and potential security risks. It leverages advanced data analysis and machine