OpenAI disrupts large-scale attempt to replicate ChatGPT capabilities

Yesterday 18:30

By: Azzat Manal

OpenAI disrupts large-scale attempt to replicate ChatGPT capabilities

OpenAI says it has disrupted a large-scale operation aimed at extracting outputs from its artificial intelligence models in order to reproduce some of their capabilities through a technique known as model distillation. The company said the activity involved thousands of accounts and was detected and stopped after investigators identified coordinated patterns of use.

Model distillation is a legitimate technique widely used in artificial intelligence research. It allows developers to train smaller or more efficient models using information generated by larger systems. However, OpenAI says the same approach can be used without authorization to reproduce capabilities developed through substantial investments in computing resources, training data and research.

According to OpenAI, the activity began on a limited scale in July before expanding through a coordinated network of accounts. The company said it identified more than 15,000 accounts associated with the operation and subsequently suspended the accounts it considered to be involved in attempts to extract and reuse model capabilities.

OpenAI has linked part of the activity to Moonshot AI, the Chinese developer behind the Kimi AI assistant, while stressing that it has not established whether the company was responsible for the entire operation. The allegations come amid growing scrutiny in the United States over the use of model distillation by Chinese AI companies. US officials have separately accused several Chinese firms, including Moonshot AI, of using distillation to reproduce capabilities developed by American AI companies. China has rejected such accusations and argued that distillation is a standard technique in AI development.

The issue extends beyond commercial competition. OpenAI argues that unauthorized extraction of advanced model capabilities could also create security concerns if systems reproduce powerful capabilities without incorporating the safeguards and alignment measures developed alongside the original models. The company has previously described adversarial distillation as a potential commercial, safety and strategic risk.

OpenAI said it responded by closing suspicious accounts, strengthening its monitoring systems and addressing vulnerabilities that could have facilitated the activity. The company also said it worked with external service providers to disrupt accounts and infrastructure connected to the operation and shared relevant findings with other technology companies and governments.

The incident highlights the growing difficulty of protecting frontier AI models as developers increasingly rely on automated systems, large-scale API access and synthetic data. OpenAI has said that its detection efforts now combine automated classifiers, behavioral analysis, technical monitoring and human investigation to identify attempts to systematically reproduce model capabilities.

The company expects such attempts to become more sophisticated as AI systems improve. The dispute also reflects a broader challenge for the industry: balancing legitimate research and the wider dissemination of AI capabilities with the protection of proprietary technology and safeguards surrounding increasingly powerful models.

OpenAI has said it will continue strengthening its monitoring and security measures while encouraging greater information sharing between AI companies and government authorities as the technology develops.