SDSignal Desk

OpenAI says it broke up a campaign to extract model reasoning, tying a core cluster to Moonshot AI

· Updated · Signal Desk newsroom

Analysis of reporting by OpenAI. How we work

Image: OpenAI

OpenAI's security team says it disrupted a coordinated campaign in July and August that manipulated interactions to expose protected model reasoning at scale, which could help train another system. OpenAI attributes a core cluster to individuals associated with Moonshot AI, maker of Kimi; the post does not include a response from Moonshot.

Key takeaways

  • Earliest activity began in the first week of July, according to OpenAI.
  • Operators did not break encryption or steal stored chats; they manipulated interactions to expose reasoning.
  • Volume spiked July 24 and 25 with about 16,000 extraction-pattern requests from over 4,000 users.
  • Related patterns spanned a cluster of more than 15,000 users before disruption by July 28.
  • OpenAI shared findings with the Frontier Model Forum.

What OpenAI describes

In a post on its security blog, OpenAI describes a coordinated effort, beginning in the first week of July, to extract protected reasoning, the internal working record its models produce while solving problems, in ways that could help train or reproduce another AI system. This practice is known as distillation. OpenAI says the operators did not break encryption or steal stored conversations; instead they manipulated interactions so that reasoning became visible at scale, in violation of its terms of service.

According to OpenAI, activity spiked on July 24 and 25, with about 16,000 requests matching extraction patterns from more than 4,000 users. Related patterns appeared across a cluster of more than 15,000 users before OpenAI disrupted the activity by July 28.

Attribution

OpenAI attributes a core cluster of the activity to individuals associated with Moonshot AI, the company behind the Kimi models. That is OpenAI's attribution; the post does not include a response from Moonshot AI, and the evidence behind it has not been independently assessed. OpenAI says it shared findings through the Frontier Model Forum, an industry body, and that mitigations and coordination with partners continue.

OpenAI has previously published reports on misuse, such as disrupting a covert influence campaign from Russia. This report differs in targeting the model itself as the asset: the concern is not what the model is used to say but what can be copied from it.

Why it matters

Reasoning traces are valuable training data. If they can be extracted through ordinary-looking API use, competitors can narrow capability gaps at a fraction of the cost of original training. OpenAI frames adversarial distillation as a security risk, and the report gives policymakers a first-party account of how such campaigns operate. The disclosure also connects to OpenAI's broader safeguards work, outlined in its path to Astra and its model guide for the GPT-6 family.

For enterprise API customers, likely effects include tighter signup checks, more monitoring and possible friction on high-volume traffic that touches reasoning. Responses from Moonshot AI or others named, matching mitigations from other Frontier Model Forum members and research on related attack classes are what to watch. More in our cybersecurity coverage and OpenAI coverage.

Who feels it

Frontier labs
Need to audit paths by which reasoning can be exposed or replayed.
Enterprise API customers
Expect tighter signup and monitoring on high-volume traffic.
Policymakers
A documented first-party case of adversarial distillation.

What to watch

  1. Moonshot AI's response to the attribution.
  2. Whether other Frontier Model Forum members publish mitigations.
  3. Research on related extraction attacks.

FAQ

What is model distillation?

Using another model's outputs, such as reasoning, to train or reproduce a system.

Did attackers break encryption?

No. OpenAI says they manipulated interactions to expose reasoning.

Who does OpenAI blame?

A core cluster associated with Moonshot AI, per OpenAI's attribution.

How large was the campaign?

About 16,000 extraction requests from over 4,000 users at its peak, per OpenAI.

Read the original

Continue at the source.

OpenAI

Companies: OpenAI

More on OpenAI

All OpenAI coverage →