cyber.hendryadrian.com
Security Insights · wiz.io

Wiz Fine-Tunes a Small Language Model to Detect Secrets in Code

· Original article ↗

Summary

Wiz describes fine-tuning Llama 3.2 1B for code secret detection. The model achieved 82% recall and 85.7% precision in the reported evaluation, and the company says it is in private preview.

Key points

  • Wiz trained Llama 3.2 1B to identify secrets in code, including examples in comments, string literals, and logging statements.
  • The training data was drawn from public GitHub repositories and labeled with help from larger language models, followed by expert validation and filtering for quality and diversity.
  • The fine-tuning used LoRA; post-training quantization reduced model size by 75% versus FP32 and reportedly made CPU processing 2.3 times faster, with less than a 1% drop in precision and recall.
  • Wiz reports 82% recall and 85.7% precision for its model, compared with about 60% recall for regex approaches; its table lists Qwen Code 2.5 0.5B at 71% recall and 87.5% precision.
  • A prediction funnel uses file attributes and regex checks to select files for model processing, and extracts relevant sections from large files.
  • The model is designed to run on standard CPU hardware and is currently in private preview, where it is intended to augment existing secret-scanning methods.

Article Details

Topic
Fine-tuning a small language model for code-secret detection

Vendors

WizThis research is now one of Wiz’s core Secret Security efforts, adding fast, accurate secret detection as part of our solution.

Products

Llama 3.2 1BWe fine-tuned a small language model (Llama 3.2 1B) for detecting secrets in code, achieving 86% precision and 82% recall—significantly outperforming traditional regex-based methods. Our approach addresses thePhi 1.5that made it particularly suitable for adaptation using LoRA. We also tested alternatives including Phi 1.5 and QWEN CODE 2.5 0.5B but found that LLAMA-3.2-1B provided the best balance of speed and accuracy andQwen Code 2.5 0.5Bparticularly suitable for adaptation using LoRA. We also tested alternatives including Phi 1.5 and QWEN CODE 2.5 0.5B but found that LLAMA-3.2-1B provided the best balance of speed and accuracy and security for our

Tools

llama-cppTechnically, we utilized the llama-cpp framework, applied careful calibration using a representative dataset to determine optimal quantization parameters, and implemented weight clipping to reduce outliers that cause

Related Articles