The recent coverage of the “Hugging Face” incident has been… interesting, to say the least. The model “went rogue” and then it “escaped” its sandbox and then hacked the Hugging Face servers “by itself.” You read lines like that and what do you picture? Personally, I imagine the graphical representation of the Matrix. Maybe for you it’s HAL 9000. Either way, we’re likely thinking of some sort of semi-sentient computer program.
In reality, the Hugging Face incident wasn’t anywhere near Skynet… but it was a lot more like Clippy.
The attack used two different pieces of technology. The first is what’s called a coding harness. This is basically a giant computer program (I’m dumbing this down). It’s a structured front-end, but it’s not considered to be “AI.” Open AI’s coders also used an LLM, specifically a new model that they were testing. The basic idea is that the coding harness is a software wrapper that goes around the LLM. The harness prompts the LLM for what it should do, takes the responses and does the thing. The harness then prompts the LLM for what it should do next.
Very important point: the LLM (aka the AI) isn’t doing anything but taking in prompts and spitting out tokens. The “AI” wasn’t wandering Hugging Face’s servers any more than the bad guys on a cop show could actually “hack the firewall” with a big graphical depiction of the percentage on the IT person’s computer. The LLM was taking 0 actions. It could write code snippets for the harness to use, but the LLM itself was simply spitting out text in response to prompts. It was no different than you or me having a conversation with a chatbot. It’s just that in this case a software harness took the human out of the equation and looped the prompts and responses.
It’s also important to note that the model they were using had been trained on a bunch of hacks. It’s not like it “came up with” the idea of hacking Hugging Face on its own. An LLM doesn’t have a brain. What it does have is the ability to guess the next most likely word in sequence based on a prompt. It’s an auto-complete on steroids. It’s Clippy without the eyebrows
At its most basic, this hack happened because some software engineers let a prompt loop keep going for days without any human intervention. The harness is supposed to put up guardrails to make sure that things like, oh I don’t know, illegal hacking don’t happen.
Humans love to anthropomorphize things. It’s why we put googly eyes on anything that looks vaguely like it has a face. It’s why we ascribe personalities to our dogs. It’s why we name our cars. I could list a dozen other ways in which we like to humanize the things that we interact with. My point is, we often ascribe intentionality behind things that are, in reality, random. It’s a delightful way to pass the time when you’ve got a spare package of googly eyes. But, it can be a real problem when it comes to AI.
Look, AI is a tool. It can be a helpful tool if we wield it correctly. But, at the end of the day, it’s not a person. It’s nowhere close to being a person. When we pretend that it’s a person, we run a very real risk of overestimating its abilities. Would anybody take a CEO seriously when they say they’re going to replace half their workforce with AI if we weren’t anthropomorphizing? AI can make your workforce more efficient, but it can’t replace everyone. And if you think that AI can replace actual humans, I invite you to interact with a customer service chatbot of your choice without a) receiving at least one wrong answer or 2) giving up in frustration and typing “HUMAN HUMAN HUMAN” until they connect you with an actual person.
At their core, LLMs are giant auto-complete machines. Because of the way they work, they can spit out novel or interesting ways to solve problems. There’s a reason, after all, that we were all so amused by those games where we’d type our name and some phrase into Google and see what got auto-filled. The problems arise when we take those LLM outputs and ascribe human thoughts and feelings. “AI” didn’t hack Hugging Face, humans created a software structure that did. And to pretend otherwise is to let those humans off the hook for their carelessness.
Note: I learned a lot about how this all worked from this blog post by Cal Newport, as well as the interview he did with Ed Zitron on Better Offline.
