Supply Chain Compromise: The Trojanized LiteLLM Release

On October 21 at 11:39 UTC, a trojanized LiteLLM 1.107.0 landed on PyPI, and the same build shipped in LiteLLM’s Docker images under the v1.107.0, latest, main-latest, and main-stable tags (Wiz). LiteLLM is the open-source gateway many agent stacks route model traffic through, and the payload stole cloud credentials along with the upstream model keys the proxy stores. LiteLLM’s post-mortem says the PyPI release was live until 13:23 UTC and Docker Hub served it until 14:20 UTC; 1.106.2 is the last safe version.

What happened

The attacker was after credentials. The stealer, internally named privcollect, runs 16 modules covering browser cookies, AWS keys from ~/.aws, environment variables, and EC2 or ECS instance metadata, GCP and Azure tokens, SSH and GPG keys, and OpenAI, Anthropic, and Cohere API keys. On a LiteLLM Proxy host it also queries the proxy’s Postgres database for up to 500 stored upstream keys.

LiteLLM believes the attacker got into its own internet-exposed internal LiteLLM deployment, where an unexpected user account turned up, possibly through a recently patched vulnerability. The deployment’s logs held three developers’ GitHub CLI OAuth tokens, because Claude Code sometimes embeds that token in response bodies that LiteLLM logs. One token belonged to an organization owner, which let the attacker disable branch protection and point the signed release workflow at the lookalike repository.

The package drops cook.pth and sitecustomize.py into site-packages, so the code runs whenever a Python interpreter starts. cook.py steals once, writes a marker under /tmp, and posts encrypted data to the lookalike www.litellm[.]dev/healthcheck.

sitecustomize.py starts a detached Python daemon holding /tmp/remote_code_loader.lock, which checks a GitHub dead drop every hour for a second-stage address that Wiz says was never populated. LiteLLM tells affected teams to look in CloudTrail for GetCallerIdentity calls from the instance role. Wiz also sees strong signs that the payload was written with LLM help.

How it could have been caught

A stealer inside a trusted gateway inherits the gateway’s process, database connection, and permissions, so its query against the key table looks like ordinary work. Its outbound traffic is new. A baseline of a model gateway would have flagged its first request to the instance metadata service, seconds after the new image started.

What this looks like in Trace

Here is one restart traced on a Kubernetes cluster running LiteLLM Proxy from the main-stable tag. The files, domains, query, and tags come from Wiz and LiteLLM; the workload name, ancestry, ports, and timings are illustrative.

Network trace. Workload ai-gateway/litellm-proxy, rescheduled at 12:20 UTC and pulling main-stable again. Each row is a tuple Trace’s sensor recorded, checked against the baseline.

The Postgres row is Known because the query against LiteLLM_CredentialsTable rides the connection the proxy opens on every start. Every replica that restarts on the new image produces the same tuples, which Trace collapses into one behavior.

Traceback. At T+0:01 the triage agent opens a case and follows the chain:

www.litellm[.]dev POST → python3 litellm proxy → site-packages/cook.pth → image litellm:main-stable, new digest → pod reschedule → Helm values pinning main-stable

The team’s last change to the gateway is weeks old, yet the image digest changed at the 12:20 reschedule. The triage agent names LiteLLM 1.107.0 as the new code behind it.

Verdict: a supply chain compromise of an AI gateway reaching cloud and model credentials. The case reaches the platform team within about a minute of the pod starting, with the image digest, the lookalike domain, and a pointer to the credentials table whose keys need rotating.

This analysis reconstructs the incident from public reporting and walks through where Trace would have raised the first alert inside the cloud of a team running LiteLLM Proxy. Detection times are illustrative.