#009 - Turnabout is AI play
You're reading Complex Machinery, a newsletter about risk, AI, and related topics. (You can also subscribe to get this newsletter in your inbox.)
Dialing it back a bit
Well that was quite the turnaround.
Last week I mentioned Google's "AI Overviews" system. The one that suggested that people glue cheese onto pizza and nibble rocks for good health. The thing clearly wasn't working as advertised, I said, so Google should pull the plug on it. And now they're cutting back.
I won't take credit for this.
But I also won't not take credit for this.
Mostly I'll celebrate that Google did the right thing. Sure, they released a product that was not ready for prime time. And it's clear they either skipped over key pre-release tests, like red-teaming, or they didn't take those test results seriously. And the product vision was flawed as it relied on summarizing the general web, where folly freely mixes with fact. And once people pointed out that the thing didn't work so well, Google played the "it mostly works" card with a hint of "blame the end-users." And then they waited too long to scale it back. But only after various end-users had published ways to get around this product because they found it so bothersome.
Yes, this reads like a step-by-step of how to not design, release, and roll back an AI product. Because it is. But I guess twelve wrongs eventually make a right? The real AI is the rocks we ate along the way? Something like that?
In case you think I'm being harsh, I'll point you to Google's own words. "AI Overviews: About last week" is a blog post written by head of search Liz Reid. When you mix other reports of Google hand-waving over the problems, plus the blog post's section "About those odd results," my take adds up. Consider this excerpt:
One area we identified was our ability to interpret nonsensical queries and satirical content. Let’s take a look at an example: “How many rocks should I eat?” Prior to these screenshots going viral, practically no one asked Google that question. You can see that yourself on Google Trends.
There isn't much web content that seriously contemplates that question, either. This is what is often called a “data void” or “information gap,” where there’s a limited amount of high quality content about a topic. However, in this case, there is satirical content on this topic … that also happened to be republished on a geological software provider’s website. So when someone put that question into Search, an AI Overview appeared that faithfully linked to one of the only websites that tackled the question.
(I won't get into Reid's point that "practically no one" had asked Google about eating rocks. If she's going to gloss over the fact that some nonzero number of people were asking Google about the recommended dietary allowance of rocks, then I will do the same.)
I can only imagine the heated internal discussions, with one side pointing out that AI Overviews wasn't ready for prime time and the other side claiming that good vibes and messaging would carry the product across the finish line. That second group was winning. Until they weren't.
Which gets to why Google finally caved. It helps to discern between the "end-user" (who uses Google search) and the "customer" (the advertisers who pay Google for end-users' attention). The AI Overviews product is a way for Google to keep end-users from clicking away to other websites, which means they stay on Google properties, which tells advertisers that their dollars would be better spent on Google than on the destination sites.
So it wasn't end-user mockery that drove Google to change gears; it was Google's realization that the product's present state wouldn't move the needle on advertiser revenue. Hence why they're going to dial it back and try to improve it for the next go-round.
Now that Google has learned its lesson, what about the rest of the space? This stumble-and-retreat will be a wake-up call to companies hurriedly cramming AI into their offerings, right?
It should be. But it won't. Not yet.
Yes, companies tend to unquestioningly follow FAANGs' tech practices. Even long after the FAANGs have abandoned said practices. (Remember when Google stopped using its famous brainteaser questions for interviews? And how copycat companies kept going?) But … AI buyers are still in denial. And they're eager, because they see AI as the latest Holy Grail for their balance sheet. Well maybe this other vendor's flying car will actually work…
No worries. They'll stop once they can no longer hand-wave over the losses. Just ask Google. But maybe skip over what you see in the AI Overviews.
Rabbits crying wolf
A few weeks back I mentioned the Fake It Till You Make It problem in AI. One example was the Humane AI Pin wearable: reviews noted it was long on the vision but short on the delivery.
Just two days after that newsletter went out, the Rabbit R1 AI box got a similar treatment. Ron Amadeo at Ars Technica determined that the R1 is:
[...] yet another "AI box" that is trying to replace your smartphone with a voice command device that runs zero apps. Just like the Humane AI Pin, this thing recently launched and seems to be dead on arrival as a completely non-viable device that doesn't solve any real problems, has terrible battery life, and is missing big chunks of core functionality.
Ouch.
The problem with Fake It Till You Make It is that your marketing can get so far ahead of the present day that it laps reality, creating a situation where you can't ever deliver. Then it becomes Fake It Till You're Found Out and eventually Fake It Till It's Over. It's the product equivalent of crying wolf. As I hinted at last time (and should have made more explicit) this will drive people to tune out AI's valid use cases along with the trash.
As a coda to this segment, Humane has allegedly put itself up for sale. Barely a week later, they announced that the device's charging case "may pose a fire safety risk." No word on whether they'll pivot to AI-driven cigarette lighters.
There's no shame in focus
From the Ars Technica piece I linked to earlier, this also caught my eye:
Before the [Rabbit R1] fades into obscurity, though, Android Authority's Mishaal Rahman looked at the software and found the "smartphone replacement" device just runs a smartphone OS. It's Android—both an Android OS and Android app, just in a very limited $200 box.
This isn't necessarily a bad thing. To market the device like it's a game-changer, yes, I take issue with that. But Android, Linux, and a handful of other systems have dramatically reduced the cost to inject a computer (and now, AI) into an arbitrary object. So if you see the Humane AI Pin and the Rabbit R1 from that perspective – as task-specific devices, not smartphone replacements – then these are pretty good ideas, indeed.
In other news …
- Letting ChatGPT guide your crypto investments. Seems legit. (Finbold)
- Google (mistakenly?) published internal docs to a public GitHub repository, under a permissive license. I suppose that's one way to move the spotlight off of the whole AI Overviews matter. (And it's still better than that time Citi accidentally repaid $900M in loans due to a typo.) (9to5Google)
- An AI decorating tool conjured up a … rather interesting design idea. (Schmatta's Substack newsletter)
- Companies just love AI chatbots. Their customers? Not so much. (WSJ)
The wrap-up
This was an issue of Complex Machinery.
Reading this online? You can subscribe to get this newsletter in your inbox every time it is published.
Who’s behind Complex Machinery? I'm Q McCallum. I think a lot about AI and risk, which I write about here.
Disclaimer: This newsletter does not constitute professional advice.