Anthropic warns that AI will soon be able to improve itself without human intervention | CNN Business

AI models are rapidly improving – so fast that they may soon be able to develop themselves without human involvement. That’s why Anthropic is warning the AI industry: It needs to build a “brake pedal,” or companies risk losing control of their creations. AI systems that can advance themselves, known as “full recursive self-improvement,” could have the potential for great good for science and health care, they also pose great risks for humanity, according to a blog post written by Marina Favaro, leader of The Anthropic Institute, and Jack Clark, a co-founder of Anthropic. “Full recursive self-improvement also might increase the risks of humans losing control over AI systems,” they wrote. “If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.” The industry is much closer to self-improving AI than previously expected, they warn. So tech companies should consider slowing or pausing frontier AI development to allow time for researchers to gain a better understanding of the potential societal harms self-improving AI could unleash. Researchers should also build a way