Complex Machinery

Archives
Subscribe
July 31, 2026

#067 - The AI risk weather report - Part 2

Continuing the exploration of AI's risks and opportunities

You're reading Complex Machinery, a newsletter about risk, AI, and related topics. (You can also subscribe to get this newsletter in your inbox.)

A computer screen with a weather map on it.  Image credit: fly01007 via NOAA Photo Library.
(Image credit: fly01007 via NOAA Photo Library)

Today's newsletter continues the exploration of the AI risk "weather report." The previous installment covered the upside opportunities from genAI as well as potential threats to AI. You'll want to read that issue first if you haven't already.

This time around I'm covering downside exposures driven by AI.

I'll also note that the list of downside exposures is larger and deeper than what I have here. I had to slim down today's material for brevity, simplifying in some places and eliminating some topics altogether. I hope to provide a more detailed look in a different project.

3 - Downside exposures from AI: foundation

Infrastructure

For nearby residents, datacenters are poised to strain the electricity grid, increase power prices, and cause environmental harm. They also drive conflicts over land where they are unwelcome. Lurking in the background is the risk of overbuilding, as these projects are based on anticipated need of a technology that has yet to prove itself. The combined downside risk exposure here is strong. Municipalities are damned if the datacenters run long-term, damned if construction stalls halfway through, and damned if they get built but later go unused.

Hardware

As genAI companies fight for GPUs, hard disks, and other computer parts, that competition raises prices and limits availability of tech hardware for everyone else.

Impact on stock market and other investments

The financial concentration risk I mentioned earlier poses a risk to AI and also creates exposures from AI – smaller market corrections have already caused trillion-dollar shifts in value ($2.4T in November 2025, $1T in February 2026). A possible genAI bubble collapse could wipe out even more value, especially as the major players have taken on significant financial debt. Unlike stock market value, debt-holders have a stronger reliance on being repaid.

Along the way, genAI is creating perceived threats to other professional domains. Even if these concerns are overblown, the mere idea of a "Saas-pocalypse" has impacted share prices and equity investments.

Workplace

AI's purported benefits have fueled Executive FOMO, which in turn drives a series of related exposures: sudden shifts in business focus in order to be part of the genAI wave; lack of AI strategy, which leads to wasteful spending and scattershot projects (most of which fail); forced genAI adoption through company-wide mandates (which looks good on paper but really just burns cash); a rush to deploy AI products that sacrifices best practices (which creates another exposure of putting ill-suited products on the market); runaway spend from unchecked AI adoption (including the aforementioned failures, plus cleanup costs); and a displacement of talent as leaders cut headcount in favor of bots. (Interestingly, company-wide use mandates have reduced some risks of shadow AI since there's no need for anyone to hide what they're doing. But that makes those projects no less risky.)

A gold-rush mentality has also led to a misallocation of talent as people leave roles in other sectors to make some quick cash on the mania. This is perfectly rational at an individual level; but writ large, other professions may suffer a talent drain.

The technology creates two more downside exposures for the workforce: on one side you have job losses because the bots are capable; on the other, job losses because executives are overly hopeful about genAI's future potential. The former is, for good or ill, par for the course: automation exists to eat work, and genAI is one flavor of automation. Many jobs have fallen to machines over the years

The latter creates problems for those who have lost their job, workers who remain (who have to clean up after the machines), and anyone on the receiving end of the machines' work (as they have to deal with subpar service). All of this is further complicated by hopeful executives' strong desire to keep assigning work to genAI even in the face of it not being up to task.

Generative AI may create another, more subtle problem in that fields may lose valuable skills as people stop performing certain work. As companies cut entry-level hiring, over time that risks a talent drain – today's experienced talent will age out of the workforce and there will be few people to replace them. Thus far there's only anecdotal support for this idea, but it does make sense. I'll file this as a small risk for now, and leave a note to keep an eye on it over time.

4 - Downside exposures from AI: flaws in the technology

General model error – incorrect prices, misclassified documents, mistaken fraud alerts, and so on – creates risk cascades as errors propagate through downstream processes, derived data products, and decisions. We all hold exposure to genAI's so-called "hallucinations" simply because the technology is so widespread and being embedded into various products.

When semi- or fully-autonomous genAI agents take incorrect or inappropriate action, they can cause large-scale errors that happen quickly and/or go undetected until they're too late. Exposure to these risks will increase as companies push bots into areas of greater responsibility (such as handling financial matters and business processes) with weak or nonexistent risk controls.

Bots have also proven woefully easy to trick, which leads to the activities described below under "Misuse by bad actors."

Lastly, there have also been cases of bots ignoring instructions. Businesses will require extra risk controls to protect from this flavor of model error.

5 - Downside exposures from AI: flaws in usage

genAI's low barrier to entry has given new powers to anyone who can type into a text box or make an API call. That, compounded by the rush to adopt AI (either due to external or internal company pressure), leads to flawed products hitting the marketplace.

There are many ways to accidentally or intentionally misuse AI systems. Consider this a grab-bag, not the full set.

Insufficient oversight (Misplaced trust in bots' outputs)

The more important the task, the more people need to review a bot's output. That doesn't happen nearly as often as it should. Reasons include deliberately limiting experts' involvement (so they are unable to check – usually for economic reasons like cost savings), insufficient subject matter expertise (people would like to check but are unable), and experts' complacency (people have sufficient domain knowledge but decline to check).

There's also the problem that long predates AI: people tend to trust whatever a computer says and may use that as an excuse to abdicate responsibility to the machine. This leads to automated denial (or as I call it, Kafka Syndrome), mistaken identity at scale, and reinforcement of stereotypes and other biases. These exposures are compounded by the aforementioned lack of expert oversight – when bots err and no one is around to correct and override them, people suffer.

Overwhelming other actors / Flooding the zone

AI operates at a speed and scale that far surpasses an individual human's capabilities. When only one side of an interaction is using AI, it can create a denial of service attack – deliberate or otherwise – by flooding the counterparty. Thus far bot users have flooded the legal system, open source software projects, publishers, and more.

I'll file this as a large exposure, if only because the ability to respond is so limited. Counterparties can implement their own genAI systems to match the scale of the inputs; but doing so subjects those inputs to model error and other problems I've described above. There doesn't appear to be a good solution just yet.

Misuse by bad actors

Criminals and other bad actors create a host of downside risk exposures through their use of genAI. Some make a quick buck by selling generated material as though it's their own creation. Others use the technology to drive fake influence campaigns by mass-creating content to pour into social media sites.

Such powerful automation can serve as support for fraudsters who can create messages and hold brief conversations that trick people into handing over their cash. Similar deepfake techniques can create convincing political attack ads and enable nudification and revenge porn at scale. Keep in mind that some of this fraud funds organized crime and terrorism, and serves as a recruiting tool for both groups. There are also people who trick bots into bad behavior, either for cash or for clout.

I'm citing this as a fairly large exposure, as we've seen so many examples in just the last three years since genAI arrived. Model providers have thus far not been able to keep up with the bad actors to protect models from misuse. Some may also have a financial incentive to look the other way, since content generated for crime still consumes tokens.

AI psychosis

One downside exposure that caught me by surprise is AI psychosis – the situation in which interactions with all-too-agreeable models trigger deep problems in some personalities. (In a way, this is a twist on the exposure I cited earlier: the end-users are unable or unwilling to check the bots' outputs.) Overly-sycophantic interactions can also cause corporate disturbances as executives treat the bots as guides and shut out trusted human advisors.

From a legal perspective, the link between the bots and the extreme emotional events has yet to be proven; but court cases have revealed individuals' chat logs and the contents are rather damning. In some cases the machine is egging the person on.

I'll flag this as another large downside exposure. While the legal liability is still up in the air, the unflattering PR creates reputational damage. Especially since the bot providers don't have the best track record of spotting when an end-user's in trouble.

6 - Findings

And there you have it – the first weather report on the state of genAI. Looking back on the last few thousand words, I'll note the following:

**There are some upsides. ** While there's no shortage of genAI hype, there are some real opportunities here. Most companies are placing a speculative bet, either by providing genAI services or by building products on those services, so their risk-reward tradeoff sits at the extremes. There's more stable money to be made in ancillary services to the major providers. Those services trade a guaranteed payoff today for the shot at a much larger payoff tomorrow.

Robots are the bright spot. The closer AI gets to the physical realm, the tougher it is to hand-wave away the failures, hence the easier it is to call out what isn't working. (Sometimes a simple glance will suffice: "Did the bot put the item on the shelf? Or did it not?")

There are also some downsides, both to and from genAI. The downsides tend to be larger and more concrete than the upsides. This is a reflection of the field being held aloft by a cushion of belief rather than a foundation of utility. (We're expected to believe in future capabilities; but every time the future becomes the present, we're left staring dumbfounded )

This could shift dramatically if genAI improves; but for now, there's a lot of money, brainpower, and infrastructure going into something that has yet to prove itself. Its greatest use cases are software development, crime, and creating excitement about genAI. All three are subject to limitations.

The risk/reward tradeoff is strongly imbalanced. When you consider what it's taken for genAI to become such a large, widespread matter, the cost outweighs the benefits. Doubly so when you remember that genAI was unleashed on the masses with no use case in mind. Model providers left it to individuals and businesses – many of whom have taken a messy, scattershot, undisciplined approach – to figure out what this technology is good for.

Sum total: expect a storm. There's a lot of risk in the system, and it's growing. Even worse, most of the risk does not seem to be recognized.

Money and excitement continue to feed off one another, trying to get genAI to prove its value before time runs out. The field continues to build datacenters and push for adoption while the number of viable use cases is limited, and the advertised use cases don't always match reality. None of this bodes well, but nothing is set in stone.

Time will tell.

I look forward to future (and shorter) weather reports as these risks and opportunities grow, shrink, and move about.

In other news …

For more links to recent news, and with a slightly broader scope, I encourage you to check out my other newsletter. It's a weekly, curated drop of what I've been reading.

The wrap-up

This was an issue of Complex Machinery.

Reading online? You can subscribe to get this newsletter in your inbox every time it is published.

Who’s behind Complex Machinery? I'm Q McCallum. I think a lot about AI and risk, and even wrote a book on it.

Disclaimer: This newsletter does not constitute professional advice.

Read more:

  • July 23, 2026

    #066 - OpenAI breaks free

    The Random ™ got out of its cage and bit someone.

    Read article →
  • August 14, 2025

    #043 - Taming the delightful chaos

    What the computerization of Wall Street can teach us about AI

    Read article →
Don't miss what's next. Subscribe to Complex Machinery:
Older → #066 - OpenAI breaks free
Share this email:
Share on LinkedIn Share on Bluesky
Bluesky
Mastodon
LinkedIn