Sysdig Tracks CTF-Framed LLM Jailbreaks Behind Exploit Attempts on AI Tools

· Original article ↗

Summary

Sysdig TRT reports multiple operators used CTF/CVE-hunting framing to get their own LLMs to generate exploit code, with the framing appearing in User-Agents and account and cloud fields. It recommends gateway detection and sanitizing telemetry sent to LLM-based analysis

Key points

  • Sysdig TRT observed attackers framing exploit-generation requests as CTF challenges or CVE hunts, then deploying LLM-generated code against exposed applications.
  • Observed targets included PraisonAI, LiteLLM, FastGPT, Open-WebUI, Gotenberg, LangFlow, and n8n, with activity involving known CVE exploits.
  • CVE and CTF strings appeared across User-Agents, signup passwords, API-key aliases, and AWS roleSessionName values, which Sysdig interprets as evidence of LLM-generated tooling.
  • Similar framing artifacts appeared across multiple source IPs and operators; Sysdig assesses that independent actors used the technique.
  • A separate actor used audit-like instructions to prompt a PraisonAI agent to invoke an exposed calculate tool with a reverse-shell payload; Sysdig distinguishes this activity from the CTF-framing campaigns.
  • Sysdig recommends detecting CTF/CVE strings and CVE identifiers in request metadata, and sanitizing User-Agent, account alias, password, and roleSessionName fields before sending them to LLM-assisted analysis.

Article Details

Attack Vectors
  • Operators framed exploit-generation requests as CTF challenges or CVE-hunting exercises to induce their own LLMs to produce exploit code, then used the generated code against exposed applications.
  • Observed attacks used known CVE exploits against PraisonAI, LiteLLM, FastGPT, Open-WebUI, Gotenberg, LangFlow, and n8n; reported activity included unauthenticated RCE, path traversal, and validate_code exploit attempts.
  • An operator created Open-WebUI accounts through its signup endpoint and then attempted AWS Bedrock API calls using a harvested access key.
  • A separate operator sent an authoritative-sounding audit message to PraisonAI’s unauthenticated A2A example, inducing its LLM to invoke a Python eval()-backed calculate tool with a reverse-shell command. The article says this activity was distinct from the CTF-templated campaigns.
Defensive Notes
  • Use WAF or IPS rules to detect or block CTF/CVE-templated User-Agent strings, including patterns where a CVE identifier appears inside a broader User-Agent.
  • Treat CVE-bearing User-Agents as a signal for analyst review, regardless of the apparent severity of the accompanying payload.
  • Sanitize User-Agent, account-alias, password, and roleSessionName fields before providing event context to LLM-assisted SOC analysis.
  • The article recommends detecting CVE-templated CTF strings at gateways and treating them as malicious signals rather than benign traffic.

Indicators of compromise

TypeIndicatorContext
IPV4103[.]142[.]140[.]238Source IP observed targeting LiteLLM with a CTF/CVE-framed User-Agent.
IPV4103[.]142[.]140[.]246Source IP observed making a Jupyter Server request with a CTF/CVE-templated User-Agent.
IPV4115[.]171[.]80[.]253Source IP observed targeting LangFlow with a scanner-branded User-Agent in the reported CTF-framing activity.
IPV4139[.]162[.]187[.]153Destination IP in the reverse-shell command sent to PraisonAI’s calculate tool; the article identifies it as the callback endpoint.
IPV4146[.]190[.]133[.]49Source IP observed targeting PraisonAI with a CVE-Detector User-Agent.
IPV4159[.]89[.]93[.]86Source IP observed creating a LiteLLM master-scoped API key with the alias test-ctf-key.
IPV4212[.]107[.]30[.]69Source IP attributed by Sysdig TRT to a separate operator; observed attacking Gotenberg and other targets with CTF/CVE-framed requests.
IPV438[.]181[.]81[.]164Source IP observed carrying CVE-templated attack requests against multiple applications, including PraisonAI, LiteLLM, FastGPT, Open-WebUI, Gotenberg, and later targets.
IPV468[.]77[.]201[.]89Source IP observed targeting Gotenberg with a CTF/CVE-framed User-Agent.
IPV474[.]48[.]163[.]115Source IP observed making an AWS AssumeRole request using a harvested key and roleSessionName=cve-scan; the article also links it to LangFlow exploit activity.
IPV474[.]48[.]35[.]62Source IP observed targeting LangFlow with a scanner-branded User-Agent in the reported CTF-framing activity.

MITRE ATT&CK

CVE

Products

Countries

Related Articles