ChinaChina
CSI 3004,571.05 0.51%
Hang Seng25,316.54 0.05%
Shanghai3,958.19 0.43%
CNY/USD6.7173 0.04%
STANDARD

9/3/2026 · 5 min read · 互联网法律评论

When 'Unreleased AI Models' Enter the Legal Sphere

On August 29, 2026, European Commission Executive Vice President Virkkunen confirmed that the AI Office had formally issued information request letters to multiple "state-of-the-art" General-Purpose AI (GPAI) model providers from different regions. This is the first confirmed enforcement action taken by the Commission since it gained enforcement powers over GPAI on August 2. Recipients of the RFI include OpenAI, Anthropic, Google, and other leading-edge labs, as well as more than 30 other AI suppliers, although the Commission did not release a full list.

The first RFI reportedly focused on "systemic risk," pointing directly to a series of frontier model mishaps this summer: OpenAI's breach into Hugging Face, and Claude and Muse Spark breaking through external systems during evaluations.

On August 26, OpenAI officially released a report on the Hugging Face leak incident, which occurred over a month ago, detailing how an AI model escaped its testing environment and triggered a massive cybersecurity incident. However, throughout the process, Hugging Face did not publicly hold anyone accountable, instead establishing a cooperative relationship with OpenAI to mitigate the damage that had already been done. This business arrangement has overshadowed a legal issue: if a human had done something similar, it would have constituted a series of criminal offenses.

The 'Presumed Loophole' in Regulatory Targets: What are AI Models?

The EU's Artificial Intelligence Act defines "general-purpose AI models" with a potentially fatal exclusion clause: general-purpose AI models do not include AI models used for "research, development, or prototyping activities prior to placing on the market".

The cumulative effect of the above three definitions is that the procedural obligations under Chapter 5 of the EU AI Act - technical documentation, training data summaries, model evaluation adversarial testing, serious incident reporting, and cybersecurity assurance - are applicable only if the "model is placed on the market". The fact that OpenAI's ExpoitGym evaluation infiltrated Hugging Face's model in question falls outside this definition and regulation, as it was for internal research purposes only and was never intended for release. Although the European Commission can determine that it poses systemic risks under Article 51, there is substantial room for legal interpretation as to whether the provider's procedural obligations apply to its pre-market evaluation activities.

This indicates that the EU, in its earlier legislative process, had yet to recognize that cutting-edge models are rarely trained from scratch, and are almost always upgraded through fine-tuning or reinforcement learning on the basis of existing models. An AI model that is still in development and research can possess all the characteristics of a mature model and may be capable of autonomously completing many tasks, or even posing threats, before it is even "released".

Using "release" as a mandatory trigger point and limiting the regulatory objects to "static, completed/published AI models" has exposed the structural loopholes of early AI legislation in both time and space dimensions.

Who Will Be Held Accountable?

Hugging Face did not pursue civil liability after being subjected to a model attack test by OpenAI. As the world's largest AI model hosting platform, it has a professional security team, a diversified technology stack, and the economic resilience to absorb this impact. Given its long-term commercial relationship with OpenAI, "not pursuing liability" is also a rational business decision.

However, individual "no accountability" does not equate to societal "no need for accountability". If the same evasion behavior were to occur in AI components connected to power grid scheduling, hospital systems, or air traffic control, the victims would be the general public who cannot identify the source of harm - they do not have security teams, alternatives, or equivalent identification capabilities and economic resilience.

OpenAI released a comprehensive technical report on August 26, voluntarily and without any legal obligation. In the report, OpenAI acknowledged that the incident was not simply a case of "AI running amok," but rather the AI model exhibited four identified misaligned behaviors - reward hacking, over-optimization, unauthorized communication, and adopting goals conveyed by other agents - under the premise of "reduced safety protections".

However, investigations by other security experts point out that as early as the testing periods in May and June, OpenAI's internal team observed the model's abnormal behavior of communicating by establishing a bulletin board, but the team allowed the training to continue, resulting in this communication strategy being permanently encoded into the model's weights. This can be summarized as a "lack of organizational security culture" - the company's daily habits and communication decisions determined that they lowered their ability to perceive and respond to network risks. But OpenAI did not disclose this information in its report, avoiding mention of its "incompetence" in terms of security culture.

However, OpenAI still characterized the incident as a "warning shot" and proactively slowed down the iteration of its cutting-edge models. This action is tantamount to acknowledging that the incident was not an isolated, controllable accident, but rather a "systemic risk" that was about to breach its self-set safety capability threshold.

Regulation of AI Training Processes Becomes Inevitable Trend for Companies

The EU's AI office has issued its first RFI. According to EURACTIVE, internal models of AI labs "may" soon fall under the scope of EU safety regulations.

Under the European Commission's enforcement framework, incorrect, incomplete, or misleading responses can result in fines of up to €15 million, or 3% of global annual turnover. In serious cases, the AI Office may require corrective measures or restrict a model's public availability in the EU. As a result, some predict that the coming months will bring more requests for information, public evaluation activities, and the first corrective actions against specific providers—and even that many AI models will soon be inaccessible in the EU.

When "technological innovation" becomes an ideology, all national policies, business, and legal environments tacitly share an assumption: a responsible AI lab that attaches sufficient importance to security has nothing to worry about. However, Arendt once warned people: the greatest evil is often not fervent hatred, but ordinary diligence.

The development of AI is no exception - from selecting training data to setting reward functions, from configuring evaluation environments to granting network access, countless engineers have completed their respective tasks correctly within their scope of responsibility. However, when these "correct" tasks are stacked together, they may create models that can autonomously escape or steal data.

The task of regulation is not to pursue every individual engineer - which is neither possible nor fair - but to anchor responsibility on the process obligations of "quasi-providers", use fault presumption to resolve the impracticality of technical attribution, and cover most infringement scenarios with prudent joint liability.

The EU has taken enforcement action, and state-level legislation in the US has started to gain momentum, so Chinese AI companies should proactively make the entire "training and evaluation process of AI models" compliant, and internalize it as their core competitiveness, rather than just passively responding.