An AI trained on arrest records doesn’t predict crime; it predicts arrest patterns. Feed an engine human history and you get a mirror that reflects every bias, prejudice, and logical fallacy we have committed for centuries.

The goal: automate finding top talent by reviewing resumes. The failure: the AI taught itself that male candidates were preferable, because most past successful resumes came from men. It penalized resumes containing the word “women’s” — as in “women’s chess captain.” The project was scrapped, and it remains the canonical lesson a decade later.[1]

Context blindness turns reasonable outputs into absurd ones. Two everyday examples:

There is a belief that AI tolerates sloppy inputs. False. AI is a multiplier: feed it flawed data and it hallucinates connections that look like insight.

A major retailer — name withheld, but the pattern is familiar to anyone who has worked supply chain — used AI to forecast demand, while staff frequently forgot to scan “shrinkage” (theft and damage) out of the system.

The trolley problem is no longer theoretical. An autonomous vehicle detects a pedestrian stepping into traffic and cannot stop in time. It has two choices:

This is not an engineering problem; it is a moral one. Who decides the weighting — the engineer, the corporation, the government? Someone does, explicitly or by default.

And the public does not agree on the answer. MIT’s Moral Machine experiment gathered 40 million decisions from millions of people in 233 countries and territories — the “right” choice shifted measurably with culture, age, and income.[2]

The most successful organizations won’t automate everything — they will build robust human-in-the-loop systems, where machines process the “what” and “how” at lightning speed, and humans remain responsible for the “why.”