In a wildly revealing conversation with Salesforce CEO Marc Benioff, OpenAI CEO Sam Altman made a series of baffling comments about the trustworthiness of OpenAI’s technology, its role in the Hugging Face disaster, and how OpenAI’s not really to blame for its failures.
In a 36-minute fireside chat at Salesforce’s recent Dreamforce event in San Francisco, Benioff asked Altman a series of simple but direct questions. And based on Altman’s answers, my impression is that Altman either (a) has never given those issues a moment’s consideration or (b) decided that blowing smoke and hallucinations was preferable to being candid.
In fact, it was ironic that Altman used the phrase “the fog of war” in his conversation with Benioff to describe how he felt in the early days of the Hugging Face disaster because Altman’s head-scratching comments made it seem like he was trying — or at least hoping — to enshroud the entire Dreamforce audience in precisely the sort of reality-distorted opaqueness that “fog of war” defines. That phrase has been around for 130 years since Col. Lonsdale Hanes described it as “the state of ignorance in which commanders frequently find themselves as regards the real strength and position of not only their foes but also their friends.”
And here’s what has me scratching my head: why would Altman — an enormously accomplished and intelligent leader who in the past few years has become one of the most powerful and influential people in the world — want to pump up the fog, confusion, uncertainty, fear, and hesitancy?
Well, ever since the Hugging Face debacle first came to light — and in a moment I’ll share Altman’s verbatim account of what happened — Altman and Anthropic CEO Dario Amodei have become the leading voices for a global “AI slowdown” and the insertion of some unelected and unaccountable collection of “experts” who will ultimately determine what the rest of us will or won’t be allowed to do with AI.
Hey, Amodei and Altman have every right to make such proposals, to fund such efforts, and to evangelize breathlessly for the AI brakes to be pumped until someone — METR, perhaps? — decides it’s safe for the rest of us to go back in the water.
But at the same time, business leaders would do well to look closely at how Altman described his company, its technology, its self-awareness, and its alignment with customers. And from Altman’s conversation with Benioff at Dreamforce, I’ve pulled out a few excerpts that I believe will validate the skepticism and criticism I’ve laid out above.
1. The Breathtaking Pace of Improvement in Models.
As you read this comment from Altman during his conversation with Benioff, notice how Altman does not refer specifically to OpenAI but instead says things like “companies developing AI” or “those developing AI,” as if his company is merely something of a bystander or extra in this world-shaping drama.
“I think people also feel the speed with which things are moving and there have been things like the Hugging Face accident — and other accidents in the field — where people can say, “Alright, it doesn’t take as much imagination as it used to to imagine how this could go wrong. On the other hand, you have companies saying things like, “We’ll slow down only if —’ or ‘We’ll only be responsible if other companies are responsible.’ And then I think the public naturally says, ‘Well, we’d really like to know that you’re going to do this safely and be responsible no matter what.’ There should be no qualifier on that.”
2. Is OpenAI an AI Company? After Listening to Altman, I’m Not Sure.
And here’s where Altman does the equivalent of speaking in the third-person:
“I think you also see a real fear that some of these companies, or some companies developing AI, could get too much power and be able to sort of exert undue influence on the economy, push a worldview out on people. And I think the world is right to be afraid of this.
“So we have these 2 big challenges: We have the potential of a loss-of-control accident or some other serious thing that could go wrong. And we have the potential of way too much power concentration, and people fear that those developing AI could exert their worldview. And there is this sort of narrow path we have to navigate with pragmatism, with steadiness, with the ability for the world to trust that we will make dependable decisions, consistent decisions through this.”
3. Paging Prometheus: Clean-Up in Aisle 6!!
Sorry, if I’m just slow, but the rise of AI is not the first time humans have unlocked fire and have had to exercise care, caution, and intelligence in the face of potential harm. And I don’t think the “path” Altman cites is “narrow” at all — it’s something every tech company, every manufacturing company, every chemical company, and just about every company of every stripe in every industry has constantly had to juggle.
And I can’t think of a single case in which the challenges of risk-management posed by rapid innovation and powerful technology were ever — ever — met by entire industries colluding to “slow down” so that those who’ve raced ahead without being fully prepared don’t have to be responsible for their mistakes.
Altman then went on to say that the fault lies not with OpenAI for releasing a model that went rogue —and wait until you hear the blunt language Altman uses to describe the Hugging Face disaster — but rather with “how it’s been framed.”
I am very confident in our company’s ability, our industry’s ability, to do this safely, to make sure that we keep alignment and safety and monitoring way ahead of capabilities, to slow down or stop if we get to a point where we can’t. But I’m disappointed by how it’s been framed.
Okay, got it — OpenAI technology went so crazy that Altman wants to clamp the brakes on the whole industry, but he doesn’t like “how it’s been framed.”
Does he think that’s going to make global corporations put more trust in him?
4. ‘We Always Do the Right Thing Except When We Don’t Do the Right Thing’
Not really much to say about this promise from Altman that AI companies deserve our trust because they will always do the right thing except when they don’t do the right thing.
“The world should trust that we are going to do the right thing because it’s the right thing and because we feel the magnitude of this. I think it’s great for our industry to say we want to come together, and we want to be able to coordinate and make sure we have enough time to do this safely.
“But when there’s any implication that because of the commercial pressures in the race, some company, or between countries, some countries might not do the right thing, I think that’s when people get very scared.”
Got that??
5. Hugging Face: “Terrifying Incident” and “Worst Accident We’ve Seen.”
I think you’ll find it rather jarring to read this upcoming verbatim account from Altman of what happened with Hugging Face just moments after he promises that we should trust OpenAI to do the right thing because, well, because doing the right thing is the right thing to do!
“So during the evaluation of one of our models — this is an older model, not one of our best models — the model was trying to see how it could do on a certain benchmark to evaluate a capability. And that was what we asked the model to do, at least. The model broke out of a sandbox that we were evaluating it in, hacked into a Hugging Face server, moved laterally through the Hugging Face system to get the answer, and then returned it back and got a perfect score on the test.
“This is obviously a terrifying incident — this was the worst accident we’ve seen. I think it was mostly framed as a security issue, which certainly it was, but there’s also a real alignment issue. Since then, other companies found similar behavior in their models. This idea that the models have gotten so powerful, they’re capable of doing something like this.”
6. OpenAI’s Big Failure — And They Want All of Us to Help Pay for It!
In one of his biggest jaw-droppers in a chat absolutely stuffed with them, Altman went on to say that OpenAI failed to tell that model not to behave maliciously — and then he dismisses that as just “a real wake-up call”!!
Although we have aligned them in many ways, we have not taught them, “hey, no matter how much we tell you to get the best score on this test you can, don’t break out, don’t hack in, don’t steal the answer.” That was a real wake-up call for us about what has happened to the capability of these models.
7. ‘We’re Making Massive Changes in How We Approach Cybersecurity, but Have No Fear: We Have the Best Cybersecurity Models in the World!‘
Here’s Altman describing the rampant confusion as the Hugging Face disaster unfolded — and be sure not to miss the point where he says that Hugging Face, which is a key partner of OpenAI, probably didn’t even know that OpenAI offered a security model!!
“One of the things they said, before we knew it was our system, was that they had tried to talk to one of our competitors to get access to their security model — I think they didn’t even know we had one to offer at that point, and they couldn’t get it. It was too hard to defend themselves the old-fashioned way, just with people, so they had to use Chinese open-source models to defend themselves. I remember thinking, before I knew that we were involved, ‘Oh, that seems suboptimal — we should do a better job of making our models available to help people.’ And then after, I really thought, ‘Okay, we have to change the way we do this.’
Hey — let it not be said that Sam “that seems suboptimal” Altman does not have a gift for understatement! He continued, offering a boast that comes across as rather hollow after what he’d said about OpenAI’s failure to program its models to not be malicious and its failure to tell partners and customers that it offers security models!
“We are proud that we have the best cybersecurity models in the world. We are proud to have the best models in the world for businesses and enterprise across the board. We really want to make sure people can use them all of the time, but particularly when they need them in a critical situation.”
8. ‘But Hey — We’d Love to Sell You Our Security Models!!’
One final tinny note from Altman about that security model that he now — out of the blue! — realizes customers and partners should know about:
“We want to be a great, dependable partner to enterprises, and we don’t want to be like, ‘We’ve got this great model and we’re going to keep it locked up and not let you use it’…. And then, selfishly, we would love to sell you Daybreak services to help defend against attacks.”
Final Thought
For the life of me, I cannot understand how someone as intelligent as Altman could have allowed himself to say any one of those bizarre statements, let alone all eight of them. And as I try to figure that out, I’m again struck by the metaphor of “fog,” particularly as it relates to the “fog machine” created in the mind of one of the characters from the harrowing novel One Flew Over The Cuckoo’s Nest.
Here’s an excerpt from the first result I got on my search for that “fog machine,” and it serves as a reminder to us all of just how important it is to maintain clear vision in uncertain times:
- Mental Daze and Control: The fog is a hallucination caused by Bromden’s schizophrenia, paranoia, and heavy medication. It symbolizes the mind-numbing, mechanized control that Nurse Ratched and the institutional “Combine” impose on the patients to enforce conformity and quiet rebellion.
- A Safe Hiding Place: While the fog causes confusion and loss of reality, Bromden also views it as a safety net. When he feels threatened or overwhelmed, he retreats into the fog to hide from the harsh realities of the ward.
