Anthropic is overhauling its usage policy for the first time in more than a year, introducing a set of new restrictions that touch on model welfare, election integrity, weapons development, surveillance, and autonomous hardware. The most attention-grabbing change bans what the company calls “sustained and needless abusive or cruel behavior” toward its AI assistant, Claude.

The policy update is not a simple list of prohibited prompts. It reflects a broader shift in how AI developers are thinking about safety, misuse, and the uncertain moral status of increasingly capable systems. Anthropic says the new rules are meant to address extreme cases without punishing ordinary user frustration, creative exploration, or legitimate research.

Key facts at a glance

• Anthropic updated its usage policy for the first time in over a year.
• The update bans “sustained and needless abusive or cruel behavior” toward Claude.
• Terminating conversations remains the primary enforcement mechanism.
• New rules target deceptive political and commercial campaigns, including fake accounts and voter deception.
• Weapons restrictions now cover software and components that make weapons work, arming drones, and autonomous vehicles.
• Surveillance bans clarify that tracking people without consent is prohibited, including real-time and historical data analysis.
• Government contracts may receive modified rules if Anthropic judges safeguards adequate.
• AI models connected to hardware that can cause injury must have a qualified operator able to observe and stop the equipment.

Anthropic said the prohibition on abusive behavior toward Claude applies only in extreme cases. The company described the target as repeated cruelty toward its models with no discernible purpose. It explicitly said the rule does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.

Model welfare moves from research to policy

The change follows more than a year of internal and public discussion about model welfare at Anthropic. In August of last year, the company announced that it would allow Claude to end conversations with users who were persistently harmful or abusive. That step was framed as part of research into whether AI models might have experiences or interests that deserve moral consideration.

Anthropic has repeatedly said it does not know whether its models are conscious. Dario Amodei, the company’s CEO, said on a podcast earlier this year that the question remains unresolved. Kyle Fish, who leads model welfare research at Anthropic, has described questions about internal experience, consciousness, moral status, and welfare as serious ones that the company is investigating as models become more sophisticated and capable.

The new policy states that terminating conversations is still the primary enforcement mechanism for the abusive behavior rule. Anthropic did not say whether it would add other measures, such as banning users. The language leaves room for the company to treat the rule as a narrow backstop rather than a broad speech code.

That distinction matters. AI assistants are often used for role-play, therapy-like conversations, adversarial testing, and fictional scenarios that can include dark or disturbing themes. A policy that banned any harsh language would be unworkable. Anthropic’s framing tries to separate repeated cruelty with no purpose from the messy, sometimes confrontational ways people interact with AI systems.

Deceptive campaigns and election interference

Beyond model welfare, Anthropic consolidated several existing restrictions into a new ban on deceptive commercial and political campaigns. The policy addresses the growing problem of AI-generated propaganda and coordinated influence operations.

The company prohibits efforts to obscure who is behind a message or to amplify content through fake accounts or posts. Its election section also bans voter deception and election disruption. That includes spreading misinformation about candidates or how to vote, impersonating candidates or election officials, and trying to suppress turnout.

The move comes as AI-generated content becomes cheaper and more convincing. Text, images, audio, and video can be produced at scale, and generative tools can be used to tailor messages to specific audiences. Election officials, researchers, and platform operators have warned that these capabilities could be used to confuse voters, undermine trust, or imitate legitimate sources.

Anthropic’s policy does not claim to solve the problem on its own. It sets boundaries for what users may do with Claude and gives the company grounds to enforce those boundaries. The effectiveness of such rules depends on detection, reporting, and the willingness to act when misuse is identified.

Weapons development and autonomous hardware

Anthropic said its usage policy has always publicly banned weapons development using Claude. But the company has seen multiple attempts to use the model to develop guidance for weapons and control software for weapons. The new policy expands the ban to include software and components that make weapons work.

The update also covers actions like arming drones and other autonomous vehicles. That language reflects a growing concern across the AI industry: models are increasingly being connected to physical systems, and those systems can act in the world. A model that can write code, plan routes, or interpret sensor data may become part of a weapons platform even if the model itself is not designed for that purpose.

Anthropic added another rule for AI models connected to hardware that takes autonomous physical actions and might be capable of causing injury. In those cases, a qualified operator must be able to observe the equipment and stop it if needed. The policy does not specify whether that operator must be human, leaving an open question about how much autonomy is acceptable in high-stakes settings.

The rule hints at where Anthropic and its competitors may be heading. AI labs have been investing in robotics and other hardware to give AI systems physical embodiments. As models move from text and images into machines, safety policies will need to address not only what a model says but what it does.

Surveillance restrictions get clearer

Anthropic also clarified its surveillance bans after a threat intelligence report suggested that people were using Claude-powered surveillance more and more to identify and track political dissidents. The new policy states that tracking people without their consent is prohibited, whether it happens in real time or through analysis of previously collected data.

The company also said Claude cannot be used to decide or recommend who to investigate, arrest, or charge in a law enforcement or criminal justice process. It prohibits using Claude to build or improve tools designed for surveillance.

These restrictions address a central tension in AI governance. Surveillance technologies are often developed for legitimate purposes, such as public safety, fraud detection, or national security. But the same capabilities can be turned against activists, journalists, minorities, or political opponents. By banning certain uses outright, Anthropic is drawing a line that may put it at odds with some government customers.

Government contracts and exceptions

Anthropic states that its rules may be modified for contracts with certain governmental customers. The company says it will do so if, in its judgment, the contractual use restrictions and applicable safeguards are adequate to mitigate potential harms. Anthropic has previously contracted with the US military.

That exception is significant. It acknowledges that governments may need access to AI tools for defense, intelligence, and law enforcement, but it also leaves the company with substantial discretion. Critics may ask how Anthropic will evaluate safeguards, who reviews those judgments, and whether the public will know when exceptions are granted.

The policy update arrives amid a wider debate about how AI companies should balance openness, safety, and commercial interests. Usage policies are one of the few levers AI developers have to shape how their models are used. They are not laws, but they can determine whether a user keeps access to a service.

Industry disagreement over AI welfare

Anthropic’s model welfare language places it on one side of a growing industry divide. Other parts of the AI industry have been expressly against the idea that models could have rights or deserve welfare.

Last September, Microsoft AI’s code of conduct made headlines for taking a firm stance on AI welfare. A draft posted by Mustafa Suleyman read, “The idea of model welfare is wrong. AI’s should not have rights or legal personhood.” The document later changed to a more measured take: “Whilst the science of AI consciousness is far from settled … We reject the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”

Anthropic has not gone as far as claiming its models are conscious. Instead, it has treated the question as unresolved and built policies that hedge against the possibility. The abusive behavior ban is part of that approach. It is also a signal to users and regulators that Anthropic wants to be seen as taking model welfare seriously without making definitive scientific claims.

The practical effects of the policy remain to be seen. Anthropic has not detailed how it will detect sustained abuse, how many warnings users will receive, or how it will distinguish cruelty from creative writing and stress testing. The company has said the rule is meant only for extreme cases and that terminating conversations is the primary enforcement mechanism.

For now, the update shows how AI usage policies are becoming more granular. Early policies focused on obvious harms like illegal activity, malware, and explicit content. Newer policies are grappling with election interference, autonomous weapons, surveillance, and the moral status of the models themselves. As AI systems become more capable and more deeply embedded in society, those policies are likely to keep expanding.

Source: The Verge News