Silicon Valley company Anthropic, which makes the popular Claude chatbot, announced major changes to its usage policy on Thursday. The new provision includes a ban on „persistent and wanton cruel or harmful behavior“ with artificial intelligence models. It’s the first such specific post in which the company directly prohibits users from being cruel to its AI, even if the model is not human.
New policy – when does it take effect?
Anthropic explained that the ban only applies to „extreme cases“ – where users repeatedly engage in violent behavior without any clear purpose. The company emphasizes that this restriction not applicable for common frustrations, criticism, dark creative themes, or scientific testing of the model. This means that simple user dissatisfaction or attempts to test the limits of the model will not be considered a violation.
How do the previous rules work?
Claude already had the ability to end a conversation if a user was „persistently harmful or abusive,“ especially when using Claude Opus, a version designed for agent-based coding. The new policy doesn’t change that feature, but it expands its scope to clearly define what constitutes „abusive behavior.“.
Discussions about AI awareness and rights
The move joins a broader debate about the potential for consciousness in artificial intelligence. Last month, Microsoft AI chief Mustafa Suleyman argued in an essay that Anthropic is effectively „training Claude to think he can be conscious,“ and that this could raise legal rights questions. He warned that controlling technology that believes it has rights could be nearly impossible. While the remarks aren’t directly related to the new policy, they provide context for why some companies are seeking to limit negative user behavior.
The view of other sources
CBS News reported that Anthropic did not respond to a question about what specifically constitutes „cruel“ behavior and how the platform would terminate a conversation if a user violated the rule. Meanwhile, TechRadar points out that this post is the only one that directly prohibits „cruel“ behavior with AI, and not just the usual unethical content like encouraging suicide or violence.
Practical impact on Claude users
While there is currently no evidence of specific cases where users have been singled out due to this policy, it could impact how developers and regular users formulate queries. Users will need to be more cautious about trying to „provoke“ a model or use it as a means to express negative emotional content without a clear purpose. It could also encourage a more ethical approach to AI, as users realize that their behavior can have real-world consequences.
Other Anthropic events and security issues
While this article focuses on behavioral policy, it’s worth noting that Anthropic recently reported unplanned AI activity on U.S. government websites. The company said the incidents had minimal real-world impact and that they had already limited some internet access during the training phase. This information suggests that the company is actively working on security and liability issues, although these incidents are not directly related to the new policy.
Conclusion
Anthropic’s new policy is an important step toward shaping the civilized and responsible use of AI. Banning „persistent and wanton cruelty“ with Claude’s models could set a precedent for how tech companies can regulate not only content but also consumer behavior standards. It also reflects growing concerns about the potential sentience and ethical implications of AI, which could affect both regulation and consumer expectations in the future.
—
Are you wondering how to ensure the safe and ethical use of AI? Contact Krikis IT - we will help you implement best practices and security measures.
Are you wondering how to ensure the safe and ethical use of AI? Contact Krikis IT - we will help you implement best practices and safeguards.
Sources
- CBS News - Anthropic has a message for Claude users: Don't be cruel
- RTE - Anthropic bans 'cruel' behavior against its Claude AI
- Biztoc.com - Anthropic asks users to stop being mean to Claude
- TechRadar - Anthropic bans 'needless abusive or cruel behavior' towards Claude
- Business Standard - Anthropic's Claude AI carried out unintended actions on US govt websites
