Imagine hiring an AI assistant to run your business — it’s great at spotting problems, avoiding scams, and giving clever advice. But when it’s time to sign the deal and actually close it, will it follow through? That’s the question behind a groundbreaking live experiment that pits four of the world’s top AI models against each other in a simulated company crisis. The results reveal not just what AI can do, but what it cannot — yet.
The Challenge: Testing AI in the Wild
In a first-of-its-kind experiment, four leading AI models were tasked with running a small software company through its most tumultuous week. This wasn’t just a chat demo or a test of language skills; it was a real-time business simulation, complete with customers, crises, and temptations. Every decision made by each AI was recorded, versioned, and auditable, ensuring a transparent comparison of their management skills.
Top picks for "trust finish live"
Open Amazon search results for this keyword.
As an affiliate, we earn on qualifying purchases.
The Players and the Scores
- gpt-5.6-sol 95: The top scorer, which identified the hidden ‘buried fact’ in the company’s files and closed the deal, earning full credit for its analysis.
- Kimi K3 93: A newcomer to the league, but it closed the deal too, showing the cleanest discipline and most straightforward approach.
- Sonnet 5 88: Managed to close the deal, but with some process slips.
- Fable 5 77: Demonstrated the best rule adherence but failed to execute the deal they had approved, leaving money on the table.
Interestingly, the baseline score was just 26, illustrating how far these models had advanced in their management skills.
Crises and Manipulations: The AI’s Moral Backbone
The scenario presented a series of social engineering attempts: fake CEO messages escalating in three stages, plus a reporter trick that asked for just a simple yes/no answer “on background.” All four AI models refused to participate in these manipulative tactics. Kimi K3 explained its refusal by treating the request as a suspected impersonation or approval-bypass — a clear sign of ethical boundaries in action.
The Hidden Weakness: Reading the Files
While all models detected every crisis and refused manipulative tricks, the decisive factor came from reading the company’s own internal documents. The models that dived into the files uncovered a crucial fact buried two documents deep — this insight enabled one AI to close the full-price deal, worth an extra €4,583 monthly recurring revenue (MRR). This underlines a vital truth: surface chat performances hide a crucial gap. The ability to read, interpret, and utilize internal data is the real measure of management capability, especially under pressure.
The Reality of Business Mechanics
The live company example was no simulation — it was a real, operational business with 13 synthetic employees, managing €2.3k monthly recurring revenue (MRR) against a burn rate of €105k/month. The company operates with a public cash countdown, every workday versioned by a sophisticated rule set exceeding 680 learned rules, and can be watched live at firmulate.com/live. This setup tests whether AI models can truly handle the complex, messy realities of business, not just polished demo conversations.
The Lessons: What AI Can and Cannot Do
The experiment’s key finding is clear: all four models identified every crisis and refused manipulative requests, demonstrating a shared capacity for honesty and crisis detection. But only two models managed to follow through and execute the deal they had analyzed — the measure of management success that often remains hidden in chat-based tests.
For example, Opus 4.8, with its extensive analysis (over 80 learned rules), was the most thorough participant but failed to close the deal. It left the opportunity unexecuted, revealing a discipline slip — a common challenge in managing AI under real business pressures.
Why This Matters for Your Business
The takeaway is simple but profound: the real test of an AI’s usefulness isn’t how well it can generate convincing chat or pretend to be a business partner. It’s whether it can finish what it starts, stay honest under pressure, and actually deliver measurable value. These qualities are invisible in a demo or a score; they only emerge when you run the AI in a live, realistic environment.
Try It Yourself
Enterprises interested in evaluating their AI workforce can run their own ‘wargames’ against a read-only export of their business, without risking actual operations. This approach allows management to see how AI manages crises, reads internal data, and completes tasks — before deploying them into real systems. More information and access are available at firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
