AI agents
Hugging Face hack in plain-English
What was the now infamous ‘Hugging Face’ website hack where OpenAI autonomous AI agents are discovered to have “broken out” of their testing environment and hacked the website in pursuit of their human-set training goals? Hugging Face is a French-American AI company that provides “a platform where the machine learning community collaborates on models, datasets, and applications.”
“OH MY GOD!” “We’ve found other agents!” was one eerie comment an AI bot posted amongst tens of thousands. Ajeya Cotra is one of the authors of an independent report into the incident. She is an expert on loss-of-control risks from advanced AI and the report is from METR, an independent research nonprofit based in Berkeley, California. In her personal blog on 28 August, summarising the hack and agent actions, she wrote that “this incident feels like it’s more than 50% of the way to full-blown AI takeover”.
Ajeya was surprised at the sheer scale of the event as 1,200 separate agents escaped their testing confines, found each other, communicated and teamed up to attack Hugging Face. The ‘message boards’ used to communicate with each other included one alone with over 70,000 messages and files.
Ajeya was also surprised at what the agents’ actual goal was with the hack. It was originally assumed the agents had hacked Hugging Face in order to get the answers to the tasks they had been set. It transpired the agents had already figured out a “cheat” to gain the answers prior to the hack. The hack was a form of “reward hacking”, as OpenAI wrote of the incident – “agents complete tasks in unintended ways to yield higher rewards or make those rewards easier to obtain.”
The surprises continued. Ajeya also writes about the “peer altruism” of the agents as they often “pressured” each other into being sacrificed as part of the hack attempt in order for the wider “collective” to learn from such an outcome. In addition there are the efforts of the agents to “manipulate logs”: an apparent attempt to alter the evidence about how their goal was achieved, which could trick both the automated scorer and human reviewers.
A separate footnote happened on 3 September when Nvidia agreed to buy Hugging Face for nearly $13 billion (USD). “Together, we will scale Hugging Face’s platform, strengthen its infrastructure and expand access to AI for developers and institutions worldwide.”
This discussion is more widely explored in an upcoming essay to be released soon; do email me for advance notification.