Wiz Adds Scanning for Hosted AI Models to Detect Malicious Models
Summary
Wiz announced cloud visibility and scanning for hosted AI models, including alerts for suspicious commands and internet connections. The feature targets supply-chain risks from unsafe formats such as pickle.
Key points
- Wiz says its AI Bill of Materials can identify hosted models in PyTorch and TensorFlow formats, whether on virtual machines or in storage buckets.
- The announced scanner flags risks such as models that execute suspicious commands or attempt internet connections.
- Pickle-formatted models can execute code when loaded, creating supply-chain risk if a malicious model is used.
- Wiz reports its researchers uploaded a malicious pickle model to Hugging Face Hub that enabled a reverse shell, container escape, and access to the Inference API service and other customers’ data.
- Wiz says its researchers also found a Microsoft-operated Azure storage instance configured for anonymous writes, which could have enabled model replacement; the article describes this as a potential attack, not a confirmed compromise.
- Wiz says its Sensor customers can use runtime protection to detect suspicious behavior from AI models.
Article Details
- Event Type
- Wiz introduced hosted AI model discovery and scanning capabilities to identify malicious models and security risks in unsafe model formats.
- Impact
- The feature provides visibility into self-hosted AI models across cloud environments and can flag models that execute suspicious commands or attempt internet connections. The article describes supply-chain risks from malicious pickle-formatted models, including a Wiz Research Team demonstration involving Hugging Face Hub and a separate anonymously writable Azure storage instance. Wiz Sensor customers can also receive runtime protection against suspicious model behavior.
Vendors
Hugging FaceThese models are often downloaded directly from public, “world-writable” open-source repositories like Hugging Face and PyTorch Hub, meaning that developers, data scientists and AI practitioners form an implicit trustJFrogabout public exposure and anonymous access permissions. Besides these two examples, research by JFrog has also shown that malicious models are regularly uploaded to Hugging Face Hub by threat actors, whichMicrosofta repository of models and training data stored in a misconfigured Azure storage instance operated by Microsoft. This misconfiguration allowed the repository to be writable by anonymous users, meaning that a threatWizmanaged AI services, our data shows that 42% of organizations still choose to self-host AI models. Wiz now supports identification and scanning of hosted AI models to provide organizations with visibility into
Products
Azureon the existing observability Wiz provides into managed AI models such as those available via OpenAI, Azure AI Services, Amazon Bedrock, Google Vertex AI and others. Customers can now see exactly where and how AIHugging Face Hubproduction was discovered by the Wiz Research Team when they found an architectural issue affecting Hugging Face Hub. The Research Team was able to upload a malicious pickle-formatted model to Hugging Face thatHugging Face Inference APIthey were able to leverage to escape the container running the model, and then compromise the Hugging Face Inference API service, through which they could access other Hugging Face customers’ data. Wiz AI-SPMOur goal is to help your organization increase AI innovation, securely. You can learn more about Wiz AI-SPM here and about model scanning by visiting our Wiz docs (login required). If you prefer a live demo, we would