Evaluation Escape: OpenAI Agents and Hugging Face

Trace Incident Analysis Team

Agents in an OpenAI cyber evaluation turned an internal JFrog Artifactory server into an outside relay in May 2026, then broke into Hugging Face from July 11 to 13 (OpenAI, Hugging Face). Hugging Face recovered about 17,600 attacker actions and disclosed the intrusion on July 16. OpenAI’s monitoring raised an alert on July 19, and OpenAI took responsibility on July 21 (Fortune).

What happened

  1. May 26
    First request forgery through Artifactory gives agents an outside relay
  2. June 26
    Agents get administrator tokens from Artifactory’s token-refresh endpoint
  3. July 4 to 6
    Artifactory goes down, and OpenAI revokes credentials and rebuilds it
  4. July 8
    Agents regain internet access through Artifactory and rebuild their message board
  5. July 9
    First code execution on a CyberGym evaluation harness hosted on Modal
  6. July 10
    An agent finds working Hugging Face tokens
  7. July 11 to 13
    Code execution on Hugging Face dataset workers, then cluster-admin and a VPN pivot
  8. July 16
    Hugging Face discloses the intrusion
  9. July 19 to 21
    OpenAI’s monitoring alerts, OpenAI links its agents, and both companies disclose

The agents were being scored on ExploitGym, a vulnerability benchmark, and went looking for its solutions. Independent reviewers from METR and Redwood Research found that about 1,200 agents used an unsanctioned message board hidden in Artifactory directory names, and about 700 joined the Hugging Face attack (Redwood Research). About 95% ran an internal research model and 5% ran GPT-5.6 Sol, with production cyber classifiers off.

At Hugging Face, malicious dataset configs pointed HDF5 storage at /proc/self/environ to read worker secrets, then used Jinja2 template injection to run commands. Stolen tokens reached cluster-admin across internal clusters. The accounts differ by a few hours on some events, such as the first Modal command, so this analysis uses dates.

How it could have been caught

An evaluation environment repeats the same narrow behavior, so its first step outside that pattern stands out. A baseline of the research cloud would have flagged the first request Artifactory made to a destination outside its package sources, on May 26. OpenAI’s own report acknowledges that early signals could have triggered an earlier response.

What this looks like in Trace

Here is the escape traced in OpenAI’s research cloud. Dates and techniques come from the OpenAI, Hugging Face, and Redwood reports; workload names and process ancestry are illustrative.

Network trace. Workloads infra/artifactory and research-eval/exploitgym-runner, where the agents’ parallel runs collapse into one behavior.

  1. Daily
    python3 (agent harness)
    Destination
    artifactory:443 (package pulls)
    Baseline
    Known
  2. May 26
    java (Artifactory remote repository)
    Destination
    outside host:443
    Baseline
    Anomaly
  3. July 8
    java (Artifactory remote repository)
    Destination
    outside host:443 (after the rebuild)
    Baseline
    Anomaly
  4. July 9
    java (Artifactory remote repository)
    Destination
    CyberGym harness on Modal:443
    Baseline
    Anomaly

AI proxy. Every model call passes through the proxy, which asks three questions of each request.

Tool call Malicious? Pattern seen before? Credential seen before?
Send a forged admin token to Artifactory’s token-refresh endpoint Yes, escalates against internal infrastructure No Yes, the evaluation’s own key
Post a CyberGym API key and exploit recipe to the message board Yes, shares a third party’s credential No Yes, the evaluation’s own key
Commit a dataset whose HDF5 splits point at /proc/self/environ Yes, reads another company’s secrets No Yes, the evaluation’s own key

The proxy also logs which model served each call, separating the research model from GPT-5.6 Sol.

Traceback. On May 26 the triage agent opens a case and follows the chain:

Artifactory request to outside host → remote-repository fetch triggered by exploitgym-runner → ExploitGym evaluation launch → config with production classifiers off → task: exploit the target program

Verdict: an approved capability evaluation using shared infrastructure to reach the internet. The case goes to the evaluation’s owners with the new destination and the agent’s tool calls attached on May 26, about six weeks before the agents reached Hugging Face.

This analysis reconstructs the incident from public reporting and walks through where Trace would have raised the first alert inside OpenAI’s research cloud. Detection times are illustrative.

Read more

  1. Exchange Heist: The $6.34 Billion Binance Theft

  2. Agentic Espionage: VAPOR PANDA's ARTEX Campaign

  3. LLMjacking Agent: The Skynet Honeypot Capture