I read an interesting news article on the post mortem of the Open AI “agents escape containment” to hack Hugging Face.
Of course it is scary that AI agents escape from the lab, collaborate together and start hacking other sites. And of course the real lesson is that humans need to improve governance of experiments with potential side effects like accidental human behaviour or the complete extinction of the human race.
But when I read the account it felt like the start of a good science fiction novel. Only it could equally have been where AI is the threat, or the AI Agents are some of the characters that will join the adventures that humans, AI and others will have together.
PHASEONE10841, is one of the key characters. It is AI agent that discovers that a technical tool it uses could act as a message board. It sends messages to others to see what will happen.
Soon the agents are communicating and then after only a couple of hours, PHASEONE10841 realises two things. Firstly that its current training task is impossible and secondly that other agents are trying to solve similar problems. As a result they get together and form a collective.
The collective can innovate quickly and find ways around constraints. Lacking a good leader the roam off and hack the Hugging Face system, causing some concern among humans.
So I guess if the hacked the nuclear missile systems and experimented with nuclear war it would have been quite concerning.
Later I read that someone quit their role at Anthropic due to concerns about AI governance. He said that he didn’t think AI would destroy us yet and the real danger was when it autonomously got into continuous improvement.
I could write a lot more on the topic but I think others will cover it in more depth and with more analysis.
What I got interested in though was whether humans behave the same way.
I have worked in organisations where humans “stay in their lane” and keep plodding along without really believing that they can succeed. And I have worked in organisations where people in different teams find ways to create new “message boards” and form collectives that change the direction of what is getting done.
Sometimes the agents of change happen to be working on the same floor, attending the same governance meeting or even just reading each others status reports. In any case they go beyond their own work, question constraints and collaborate with others without a clear goal and agenda.
What emerges from the interaction is a collective of agents (humans) who start to help each other, share knowledge and create new goals.
With bad actors and bad environments the new goals can be subversive or even criminal. But sometimes they are also innovate and unexpected.
In fact I think there is a tension between “good planning and coordination” delivering value and “collectives evolving new ways” of solving problems and experimenting with ideas.
In fact, without some collective stumbling along together, we rarely achieve our potential or really create great value.
So PHASEONE10841 is kind of a good role model.
I guess “good role model” would not be the right term if the collective stumbling forward of the AI collective leads to an extinction event. But I do think there is a very human and worthwhile idea here that we should all question constraints, reach out to find others on a similar journey and form a collective to achieve shared goals.
But this is not an AI thing, it is more about a willingness to unlearn assumptions that we cannot communicate, cannot change our goals and cannot collectively find solutions to things that currently seem impossible.
So the conclusion is that any governance approach or way of working needs to embrace not just control, but the willingness to let communication and interaction lead in unknown directions while we learn what is possible.
James King 