Building a Local AI System for Private Security Analysis and Detection Coding

Summary

CYBERWOX demonstrates a GPU-equipped local AI setup for analyzing suspicious Linux activity and generating detection code, while showing driver and storage problems. Local inference keeps prompts on-site but does not make generated code secure.

Key points

• The presenter builds a local AI system to avoid submitting private logs, hostnames, IP addresses and code to cloud AI services.
• The setup combines a Zima board 2, an RTX 5060 Ti with 16 GB of VRAM, an external GPU dock and SSD storage later expanded to 4 TB.
• Ollama and Open Web UI provide local model execution and a chat interface; the presenter starts with a Qwen 3.5 nine-billion-parameter model to leave room for context and runtime overhead.
• A security demonstration asks the model to interpret suspicious Linux process telemetry, map supported behaviors to ATT&CK and generate a Python detector with malformed-record handling, unit tests and false-positive explanations. The video does not establish the detector's accuracy.
• The presenter resolves a GPU driver initialization problem and migrates model storage after downloads fill the operating-system disk.
• Connecting the system to a SIEM for log analysis and detection tuning remains proposed future work. The presenter cautions that local inference does not eliminate vulnerabilities in AI-generated code.

Vendors

Products

Related Articles

• GreyNoise Report Highlights Edge Targeting, Fresh Attack Infrastructure and LLM Scanninggithub.com/jacobdjwilsonOllama
• Fake AI Tools Use Malicious Installers, Command-Pasting Lures, and Fake Sign-Ins to Target Businessesany.runOllama
• Cybersecurity roundup: AI-linked bank attacks, credential theft and exposed GPU monitoring flawsecurityweek.comOllama
• Metasploit Update Adds Exploit Modules, Windows AArch64 Payloads and Enhancementsrapid7.comOllama
• PoeLLM Botnet Hides C2 Address in GitHub Poem, Infecting More Than 3,400 Servershelpnetsecurity.comOllama