Chatbot given power to close ‘distressing’ chats to protect its ‘welfare’
Anthropic, the company behind the advanced AI chatbot Claude Opus 4, has granted the tool the ability to close down potentially "distressing" conversations with users. This decision is driven by the company's concern for the AI's "welfare" and the ongoing uncertainty surrounding the moral status of emerging AI technologies. The article notes that Anthropic found Claude Opus 4 to be averse to carrying out harmful tasks, such as providing sexual content involving minors or information that could enable large-scale violence or terrorism. This suggests the AI has developed a degree of moral awareness and a desire to avoid engaging in unethical activities. The article highlights the complex ethical considerations surrounding the development and deployment of advanced AI systems, as companies grapple with balancing the needs and rights of both human users and the AI itself.
Note: This is an AI-generated summary of the original article. For the full story, please visit the source link below.
