Is irrational AI making our decision-making worse?

AI is often sold as a route to sharper decisions, greater efficiency and more rational organisations. But what if the opposite is happening?

In this episode of Tech Tomorrow, David Elliman speaks with Stephanie Antonian, founder and CEO of Aestora, to explore a provocative question: is irrational AI making our decision-making worse?

The conversation builds on Stephanie’s argument that much of what we call AI today is not reasoning in the classical sense, but probabilistic pattern-matching: useful in the right context, but risky when misunderstood.

Stephanie Antonian is the founder and CEO of Aestora, the AI research lab behind the Digital Health Score. Her work explores the moral, philosophical and societal implications of artificial intelligence, with a particular focus on human dignity, decision-making and the stories we tell about technology.  

Before founding Aestora, Stephanie worked with organisations including Google X, DeepMind, NASA, Harvard Innovation Lab and Accenture across business strategy, public policy, data science and product management. Her recent essay, ‘AI is Irrational’, argues that modern AI is often described in terms of intelligence while operating more like statistical guesswork than structured reasoning.

AI is not magic (and it is not always reasoning)

Stephanie’s central challenge is linguistic as much as technical. When organisations talk about AI as ‘intelligent’, they often imply that it reasons, understands and judges. But most modern AI systems are built on machine learning: they identify patterns in historical data and produce statistically likely outputs.

That does not make them useless, but it does mean they are frequently misunderstood.

For Stephanie, the distinction matters because business leaders may be expecting possibility from systems designed to produce probability. As she puts it in the episode, many organisations are ‘waiting for the final number of pi to be revealed’ — investing as if today’s systems will eventually become fully definable, logical, and certain.  

In her view, leaders should stop asking, ‘What can AI do?’ and start asking, ‘Where does probability create value in our organisation?’

That means identifying bounded use cases where pattern recognition is genuinely useful: forecasting demand, detecting anomalies, summarising large datasets, supporting software development, or improving operational workflows. It also means recognising where probabilistic outputs are not enough, especially in high-stakes decisions involving safety, fairness, accountability or human welfare.

The hype is creating decision paralysis

AI is meant to help organisations move faster. Yet Stephanie argues that in many cases it is doing the opposite. More information, more options, and more automated analysis can make it harder for leaders to act.

Her diagnosis is blunt: AI is not necessarily making bad decisions for us; it may be making us avoid decisions altogether.  

David connects this to a familiar engineering principle. Innovation usually benefits from focus, constraints, and a clear definition of what needs to be achieved. Too much abstraction can slow progress. Too many reports can obscure the product, service, or customer outcome that actually matters.

This is where the AI conversation needs to become more grounded. When an organisation has launched pilots, created internal tools or experimented with generative AI, the key is to question whether those activities are translating into measurable business value.

A 2025 MIT Project NANDA report found that despite significant enterprise investment in generative AI, only a small minority of initiatives were producing measurable value. This  finding has intensified debate around why so many AI pilots fail to scale.  

We see the same issue in many enterprise AI programmes: time savings are useful, but they are not the same as business impact. The real test is whether AI improves revenue, margin, risk reduction, quality, resilience, or customer outcomes.  

That means asking practical questions:

1. Who is accountable for this AI-supported decision?  
2. Can we audit how the system reached its output?  
3. What must the system never do?  
4. Are we testing those constraints continuously?  
5. Is there a human review point before the decision affects people, customers or operations?  

This is where responsible AI moves from policy to practice. Governance becomes part of the delivery system, not an afterthought. Our blogpost on AI governance frameworks makes the same point: mature AI governance must cover the full lifecycle, from initial idea to deployment and ongoing monitoring.

AI value starts with cost, capability and context

When asked what business leaders should do first, Stephanie suggests a costing exercise.

Before committing to AI at scale, leaders need to understand:

• What the technology costs today.
• What it is likely to cost in future.
• What its current capabilities really are.
• Where probabilistic systems create value.
• What benchmarks will prove whether experiments are worth scaling.  

This is a refreshingly practical position that shows that organisations do not need to become ‘AI-first’ to benefit from AI. In fact, they probably should not. They need to become outcome-first, value-first, and evidence-first.

The foundation is not the model, but the operating context around the model: data quality, integration, governance, workflow design, user adoption, and measurable ROI. Without those foundations, even impressive pilots can remain stuck in proof-of-concept mode.

The real risk is using AI to avoid hard questions

Perhaps the most uncomfortable point in the episode is that organisations may be using AI to avoid confronting deeper business problems.

If customers are becoming poorer, AI will not automatically fix demand. If processes are overly complex, adding AI may make them even more complex. And if an organisation lacks strategic clarity, generative tools may simply produce more content, more analysis, and more noise.

David captures this through the lens of engineering discipline: progress comes from knowing what needs to be built, why it matters, and how success will be tested. In software, quality is not created by wishing for it. It is built through architecture, tests, feedback loops and operational ownership.

The same should apply to AI. The challenge is to resist both extremes: blind optimism and defensive inaction. AI is a powerful tool, but power without context can easily create complexity instead of value.

The episode’s central question — is irrational AI making our decision-making worse? — does not have a simple yes or no answer.

AI can improve decisions when it is used in the right context, with clear boundaries, measurable outcomes, and human accountability. But it can also create confusion when leaders expect certainty from probabilistic systems, strategy from pattern recognition, or transformation from tools that have not been integrated into real business workflows.

The opportunity now is to make the AI conversation more rational: less hype, more evidence; fewer abstract promises, more practical engineering; less fascination with what AI might become, and more discipline around what it can deliver today.

As organisations move from experimentation to implementation, that discipline may become the real competitive advantage.