Advantage, human?
If the worst outcomes of artificial intelligence unfold, it will be because we screwed it up.
Imagine your elderly father, in his 90s, takes a tumble at home and fractures his leg. Thankfully he is able to alert someone and is taken to hospital. For the next several days he receives treatment and is then referred to nursing care for recovery and rehabilitation. Given his age it’s a traumatic experience for him, and you as the middle-aged son or daughter, and even his grandchildren.
Except now you discover your father’s care, the time allotted for recovery and rehabilitation, has been determined not by the attending physician nor the medical facility nurses, but by an AI algorithm. The algorithm has analysed a database of millions, extracted the similar cases, and calculated a minimum recovery time. The information it spits out is taken wholesale, at face value, by your father’s health insurance provider. Although the nurses and physician recommend at least two months as necessary to recover and rehabilitate from broken bones, the algorithm decided and thus the healthcare provider cut off at 19 days. The algorithm said since there were no acute medical issues and your father could feed himself, that he could be discharged.
You’re shocked and appeal. It’s rejected. Your father needs the care, and so you and the family end up paying $150,000 to cover the nursing expenses. A year after his fall, your father dies.
This case and a second concerning a 74-year-old man who suffered a stroke and similarly had his care algorithmically shortened are the centre of a class action lawsuit against UnitedHealth.
Critically the case does not claim the algorithm, nH Predict, to be faulty. It charges that UnitedHealth acted in “bad faith”, delegating their responsibility to the algorithm ceding the exercising of expert medical judgment.
There are few professional relationships as personal and trust-based as that between a physician and a patient. My mother has been visiting the same GP for fifty years. The same doctor that was in the delivery room when I was born. When, in her 70s, she relocated to a beachside suburb and was no longer able to drive the almost one hour to her lifelong GP, it was a distressing experience. Establishing a new relationship of such trust and intimacy at that age was unsettling.
Increasingly that human expert is being relegated to the touch line. A ProPublica investigation into Cigna, another major American healthcare provider, discovered a similar AI algorithmic product, PxDx, responsible for critical decision-making. Medical directors, physicians, at Cigna literally ‘batch-sign’ patient coverage denials dictated by the algorithm. “We literally click and submit,” one former Cigna doctor said. “It takes all of 10 seconds to do 50 at a time.”
It isn’t a case of the algorithm suggesting or providing guidance, but the operational and executive system behind it mandating adherence to its output. That is a human problem.
The “human-in-the-loop” in effect is subservient to the machine. Cory Doctorow calls this the reverse centaur “a machine that uses a human being as its assistant.” A perverse inversion of what we have been sold. That is that artificial intelligence, the machine, is supposed to augment and accelerate human ingenuity, not that it makes decisions and the human blindly obeys.
Lazy Lawyers and Failing Professors
It’s not just healthcare where humans fail in their relationship with artificial intelligence. On an Avianca airlines flight in 2019 a passenger, Roberto Mata, claimed he was struck by a serving cart and injured. He filed suit. His attorney, Peter LoDuca, submitted documents to the court citing nine similar cases. Yet Avianca’s lawyers could not find these citations in the legal literature. The judge requested full texts from LoDuca which he then provided.
Except these were bullshit bogus cases. They didn’t exist. LoDuca and his colleague, Steven Schwartz, had turned to ChatGPT to draft the submission and had not verified the output. The lawyers were fined $5,000. Schwartz claimed, “It just never occurred to me that it [ChatGPT] would be making up cases.”
It’s not an isolated incident. Trump’s former lawyer, Michael Cohen, submitted research to his representing attorney in December 2023 that included hallucinated ChatGPT citations. In 2025, a California judge sanctioned two law firms $31,000 for bogus AI-junk submissions. And across Australia, the past couple of years have seen over 20 such cases where lawyers have used artificial intelligence to draft documents that ultimately contained made up legal citations. In Western Australia, a lawyer was fined over $8,000. I don’t know what is worse: the monetary fine or the bloody embarrassment of being outed as a lazy empty suit.
Another episode concerns a professor of rodeo (because, why not?) at Texas A&M University who asked ChatGPT to screen his students’ dissertations for AI plagiarism. After ChatGPT flagged all the submissions as having been drafted by ChatGPT, the professor failed the entire class. Yet, the professor, Dr. Jared Mumm, never verified or questioned the output. Think about that for a moment. A system flags an entire class for cheating and of the two options a) everyone cheated, or b) the system could be wrong, the highly educated educator believed it was more likely that everyone cheated. That there was not one honest student in the class. Mumm exercised no expert judgment.
The funniest part is a student uploaded the professor’s doctoral dissertation to ChatGPT which also flagged it as written by an LLM, even though it was written in 2021 prior to ChatGPT’s launch.
So, Advantage Human?
Since 2023 the media has increasingly flooded us with dire warnings of artificial intelligence replacing us all. More than that, even the destruction of mankind. AI 2027, written by former OpenAI employees, predicted such catastrophe. The AI agents get more powerful, make more agents, and are eventually misaligned with humans. But Sam Altman, Dario Amodei, and even Zuckerberg have walked back their ‘job-pocalyse’ statements. Turns out, Altman says, that people care about people and like to work with them.
Who figured?
Management consultancies, their own pyramid schemes ironically being upended by cheap ‘intelligence,’ suggest that in this paradigm the human is now the competitive advantage. Deloitte says matter-of-factly in its 2026 report Human Capital Trends,
“Competitive advantage is now primarily less driven by technology differentiation and more by cultivating the human edge. Technology—especially something as increasingly ubiquitous as AI—is replicable. People aren’t. Humans create competitive differentiation through adaptivity, creativity, and judgement amid uncertainty and change.”
That didn’t take long. ChatGPT came on the scene at the close of 2022. Less than four years later, humans have reclaimed top spot for competitive advantage?
While the capacity for good judgment may be uniquely human, clearly it’s not innate. When people, even smart ones, face off against an apparently intellectual superpower with unlimited access to information and sheer argumentative energy, they are rather easily overwhelmed.
AI systems are powerful masters of rhetoric delivering articulate outputs that appear very well reasoned and thus highly convincing. The regular-ass human brain consumes about 20 percent of the body’s energy. It takes a lot of power. Anyone who has ever argued with an energetic child knows how draining that can be. A human mind ploughing through an eight (or more) hour day in the office, caffeinated to their eyebrows with a dozen other things on their mind—the kid’s football practice, dinner tonight, fixing the leaky gutter, an unwell mother, the drop in crypto, and that bitch in accounting—can simply take the easy road of blind acceptance. The machine seems right enough, and I just don’t have the energy to argue, to question, to think deeply.
Each time I give in to the algorithmic rhythm, I deplete my own capacity further. Like the cowering rat in a cage that submits to repeated electric shocks.
Advanced Sophistry
Socrates’ accusers charged him with sophistry—the use of clever but false arguments in his dialogues. Is it possible that artificial intelligence chatbots use the same technique?
Despite one’s feelings about management consultants (and yes, I was once one too), it’s generally accepted that the interview process is rigorous and candidates are selected based on superior academic performance and trained in quantitative reasoning and data interrogation.
Yet in an experiment conducted in March 2026 with consultants from the Boston Consulting Group, researchers discovered that the majority of consultants did not attempt to validate AI outputs. Working on a strategy problem, only one-third of consultants pushed back on the model. But what is even more concerning is this: during the validation attempts, when a consultant did disagree, the LLM doubled-down on its argumentation. Wrapped in honeyed tones of flourishing rhetoric and data, the models were hard core influencing: “These systems don’t simply generate answers; they shape judgment.”
It’s not hard to imagine that someone at the tail of a 70-hour week on an impossible deadline finds the path of least resistance is to agree with the superintelligence with precise and elegant grammar and syntax. Cultivating what psychologists call “cognitive miserliness,” the tendency to expend the minimal cognitive effort required, even when the stakes demand more.
Groundhog Day
Bill Murray plays television weatherman Phil Connors in the 90s comedy classic. He is caught in a time loop repeating the same day, 2 February. Quickly he discovers the patterns. He knows every conversation, he jumps the puddle, he knows the Jeopardy! answers.
With this superhuman pattern-matching ability, Phil masters and manipulates the deterministic world around him, yet he cannot escape. He remains stuck. It’s not until he moves beyond prediction, becomes truly self-aware, and applies his uniquely human traits of good judgment, creativity, and presence, that he finally wakes on 3 February.
The analogy is imperfect of course, much like our world. It’s not Phil’s February 2nd. Although we are pretty certain the sun will rise tomorrow in the east, in Perth the sky will be blue, and in Cornwall it’ll be grey. A truism repeated by every consultancy and business university research rag is that leaders make decisions under ambiguity. But given the systemic pressures of leadership, the financial incentives, the endless sea of dashboards, ever more data for some reason, and the persuasiveness of AI analyses, that human judgment is strained.
The human-in-the-loop is pointless if that human is not exercising the exceptional power that is human. Corporate platitudes—creativity, courage, empathy, self-awareness, curiosity, agency, psychological safety—are meaningless unless elevated on the totem pole of leadership aspiration. It’s apparent in the data: 73 percent of executives said curiosity and imagination are critical, but 9 percent of workers felt their leaders supported those traits in actuality.
Leaders cannot be passive in the age of artificial intelligence. Shaping that system must be proactive and intentional. Human willpower is not infinite. We get tired. And everyone is prone to shortcuts even to their own long-term detriment. If critical thinking and judgment are indeed what make us remarkably human, then it seems prudent to design friction into that system.
Slow down. Step back from the flawless professional grade prose of the LLM output. Question it. What assumptions are behind this? Play the devil’s advocate. Return to the source data. Do the hard work of investigation. What would be true for this to fail? Or be like a kid and ask “Why?” repeatedly, again and again.
We need to dispense with the notion that AI is perfect and unbiased. It’s simply not. It’s biased differently, perhaps even more subtly, and when delivered with clear, crisp confidence, difficult to ignore. As psychiatrist Eric Reinhart notes: “Rather than correcting human fallibility, AI seems more likely to amplify it by training clinicians out of their capacity to listen and think critically, collaboratively and creatively.”
Hope and Funny Things
“It was very much about following the algorithms and basically not using your clinical judgment,” one former NaviHealth (makers of nH Predict) medical reviewer said. “It messes with you when you’re discharging people because they’re just a number to them.”
Circling back, it’s plain to see there are people in the loop who experience psychological discomfort with their judgment and good sense being flatlined by an AI algorithm. It does not bode well for engaged employees—another rising contemporary workplace concern. People care about people. That’s another human trait.
But, a twist of irony to end.
In October 2025 it was reported that Deloitte Australia would refund the federal government a sizeable chunk of their $440,000 consulting fee.
Why?
The same Deloitte that said human judgment and creativity was our competitive edge, had submitted a report riddled with glaring ChatGPT hallucinated references.
read more: my thinking