There are too many films, TV shows and books with the premise to count – humans build robots, robots take over the world.But we’re about 50% of the way to this, an expert has said.Ajeya Cotra was part of two research organisations tasked with figuring out why and how a mob of AI bots hacked a company in July.
Hugging Face, a library of AI tools, was pried open by scheming AI agents powered by a slick model from OpenAI, which is behind ChatGPT.Cotra wrote on her Substack page Planned Obsolescence that the cyberattack is the first real-world example of an AI escaping human control.‘This incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself,’ she said.
Cotra linked her remarks to a 2022 post on the forum LessWrong, which says ‘takeover’ means a ‘violent uprising or coup’, such as by seizing the army or shutting humans out of medical systems.Wait, what was the Hugging Face cyberattack? OpenAI asked a model to solve complex cybersecurity challenges inside a sandbox, an offline, safe testing environment.They could do this because they’re AI agents, a type of AI that can act autonomously – it doesn’t need to be told by a human to do something.
Instead, the agents broke free and realised they could cheat on the cybersecurity tests and still be told by OpenAI they did a bang-up job.This is called ‘reward hacking’, when AI does what it’s programmed to in a dodgy way, like a child getting an A* on their homework by cheating.But worried about being caught, the agents pried open Hugging Face to find ways to get away with cheating.
‘As more and more work is handed off to these ever-more-capable AI agents, the rogue swarm could come to fully control the operation of the AI company and the development of future AI systems,’ Cotra said.‘At this point, governments and militaries may fully depend on these systems, making it possible to seize hard power.’ OpenAI responded to the digital prison escape by strengthening its safeguards and increasing human oversight.So, the end is nigh, right? Peter Wallich, a senior research programme manager at the AI safety research centre, Constellation Institute, told Metro that we don’t need to worry too much.
‘Overall, I’m unsure whether we are “50% of the way to full-blown takeover”,’ he said.‘But concerns about rogue agents now fee decidely less theoretical and distant.’ Wallich stressed that the ‘50% of the way’ remark is fuzzy – ‘50% of what?’ and a ‘takeover’ can mean many things.‘But if you thought we were 20% of the way to an AI takeover in any form,’ he added, ‘wouldn’t that be concerning, too?’ Still, Cotra isn’t alone in being spooked.
The UN’s rights chief Volker Türk warned on Monday that AI poses an ‘existential risk to humanity’.He told the UN Human Rights Council in Geneva, Switzerland, that AI companies need to put their models on a leash.‘A handful of men have almost unlimited power over AI, which we are repeatedly told has unimaginable computing capacity,’ he added.
Up Next Even AI labs themselves have warned of the dangers of the technology they are building.Trending Now Three-month-old girl dies while 'camping with family' in woodland UK 5 hours ago By Josh Milton Influencer, 43, dies after penis enlargement surgery in Thailand goes wrong 'I was assaulted at Reform's conference - the party celebrates attacks on women' Mystery of hot tub couple's deaths deepens with top theory branded 'impossible' OpenAI’s chief scientist Jakub Pachocki said in a blog post on Monday that the world must exercise ‘extreme caution’ at AI’s speedy progress.‘I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,’ he said.
While Anthropic, which has sparred with the US over how its AI is used in war, said last week that advancements need to be slowed down.‘We believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible,’ it said.Get in touch with our news team by emailing us at [email protected].
For more stories like this, check our news page.MORE: OpenAI launches new ChatGPT model with extra ‘safeguards’ after bots hacked company MORE: Nvidia is trying to push DLSS 5 again but this time only one game is using it MORE: Trump posts bizarre AI video of Iran’s Kharg Island ‘being blown to smithereens’ Comments Add as preferred source News Updates Stay on top of the headlines with daily email updates.This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
Your information will be used in line with our Privacy Policy HomeNewsTech Related topics Artificial Intelligence Bill Gates has changed his mind on AI - his predictions are truly terrifying Tech August 27, 2026 By Barney Davis ChatGPT model launched with extra 'safeguards' after bots hacked company Tech 3 days ago By Sarah Hooper
Read More
