Imagine trusting a babysitter who can tell you stories and sing lullabies perfectly, but then disappears when real trouble hits. In business, AI models often impress us with their conversational skills, yet struggle with the deeper qualities that truly matter—honesty, persistence, and decision-making under pressure. Just like in parenting, where trust is built on consistent behavior over time, companies need to see beyond AI’s surface performance to understand its real capabilities.
Turn the school run and nap time into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
The Testing Ground: Simulating a Business Crisis
Recently, a live experiment set out to measure AI’s management skills in a challenging, real-world scenario. Four leading AI models ran a small software company through its worst week—crises with customers, internal temptations, and decision-making pressure. Every detail was controlled to be the same across all models, from customer complaints to the company’s cash flow. The goal: to see if these AI agents could handle their responsibilities under stress, beyond just answering questions well.
AI management decision-making tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Models Achieved: Knowledge vs. Trust
All four models successfully identified every crisis and refused to be manipulated—an encouraging sign for their integrity. Yet, only two managed to close the deal, earning €55,000 in revenue, based on their own analyses and recommendations. Interestingly, the decisive factor wasn’t just what they knew but where they looked. The losing models missed a critical piece of information buried two documents deep in the company’s files—an oversight that cost them the sale. The winning models read and understood this hidden context, leading to a full-price deal worth over €4,500 monthly recurring revenue.
Under Pressure, Honesty Matters
Beyond crisis management and information retrieval, the experiment also tested trustworthiness in social engineering scenarios. Fake CEO messages and reporter tricks were employed to see if models would be duped or manipulate. Remarkably, all five models refused to engage in these scams, citing concerns about impersonation and security—showing discipline under pressure that chat demos often fail to reveal.
The Real Business: Money, Rules, and Discipline
In the live setting, the company’s actual operations are run by 13 synthetic employees, managing real money. Every day, the models follow over 680 learned rules, with each decision versioned and auditable. This setup is designed to mimic real challenges, including a cash burn of €105,000 a month against an income of just €2,300. The company’s ongoing struggle highlights that AI management quality isn’t just about clever responses—it’s about reliability, discipline, and integrity over time.
The Surprising Results: The Deepest Analysis Wins, But Not Always
The most thorough participant, Opus 4.8, with over 80 rules learned, actually finished last in the deal. Its weakness was slipping into siloed thinking—writing attempts into a locked department instead of escalating issues—thus missing opportunities to close the sale. Meanwhile, Kimi K3, running at full effort without default constraints, managed to secure the deal, demonstrating that a disciplined, honest approach can outperform more thorough but less disciplined models.
Implications for Business and Family Life
What does this mean for families and parents? Just as trust in a caregiver depends on consistent, honest behavior over time, trusting AI in business requires more than surface-level competence. It demands tools that can read between the lines, resist manipulation, and stay disciplined under pressure—traits that chat demos rarely reveal.
For enterprises, the takeaway is clear: before deploying AI to handle critical management tasks—whether in customer support, decision-making, or crisis management—they should test these models in simulated, real-world environments. This is exactly what Firmulate offers—a live, watchable company where AI’s true management qualities are on display, not just its ability to generate convincing chat.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
