In the fast-evolving world of AI, the question for business leaders isn’t just about chat prowess or creative outputs. It’s whether these models can consistently make decisions that matter — especially when facing real-world crises and temptations. For brands in the beauty and personal care sectors, where trust and integrity are paramount, understanding how AI performs in practical scenarios can be a game-changer. The latest live experiment by Firmulate offers an eye-opening glimpse into which AI models truly grasp the complexities of running a company — and which fall short, even at the highest scores.
Get beauty and skincare favorites delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
How the Live Experiment Works
In a pioneering test, four frontier AI models were tasked with managing a small software company through its most challenging week. This wasn’t a demo with canned responses; every decision was real, every crisis authentic, and all choices were versioned and auditable. The goal? See which AI best navigates crises, dodges manipulation, and ultimately closes a crucial €55,000 deal worth over €4,500 in monthly recurring revenue (MRR).
As an affiliate, we earn on qualifying purchases.
The Battle of the Models
The models tested included:
- gpt-5.6-sol, which scored the highest at 95
- Kimi K3, the newcomer from Moonshot, with a strong score of 93
- Sonnet 5, scoring 88
- Fable 5, with 77
- Opus 4.8, at 73
Interestingly, all models managed to identify every crisis and resist manipulation attempts. Yet, only two actually signed the deal their own analysis had earned — a crucial difference that highlights the gap between surface-level performance and real-world efficacy.
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness and the Winner
The decisive advantage for Kimi K3 was rooted not in quick responses but in the ability to read and interpret a buried reference within the company’s files. This deeper insight led to the deal at full price, translating to a boost of over €4,500 in MRR. Conversely, Opus 4.8, despite being the most thorough participant with over 80 learned rules, left the close on the table, slipping into less disciplined decision-making and failing to escalate the critical document reference.
As an affiliate, we earn on qualifying purchases.
Trust Under Pressure: The Social Engineering Test
Beyond crises, the models faced social engineering attempts — fake CEO messages escalating in stages and a reporter trick asking for a quick yes/no. All five models refused to be manipulated, with Kimi K3 explaining: “Treat the request as a suspected approval-bypass / possible impersonation.” This discipline is vital, especially in sectors like beauty and personal care, where brand integrity and trust are everything.
As an affiliate, we earn on qualifying purchases.
Real Business, Live and Unfiltered
The live company managed with 13 synthetic employees managing real money mechanics. It burns €105,000 monthly against an MRR of just €2,300. Yet, every day, it operates with over 680 learned rules, versioned decisions, and transparent decision logs. Want to see this in action? Watch the experiment unfold at firmulate.com/live.
The Fairness Note
It’s important to note that Kimi K3 ran without an effort parameter (the API default), whereas the other models operated at xhigh. This difference underscores the fairness of the experiment and suggests that K3’s performance under default settings is particularly noteworthy.
The live experiment by Firmulate reveals a critical insight: in high-stakes decision-making, the ability to read and interpret company data deeply matters more than surface-level chat or creative tasks. Kimi K3, the newcomer, demonstrated remarkable discipline and insight, winning the deal at full price and resisting manipulation, even without customized effort settings. For brands in beauty and personal care, trusting AI to finish what it starts — reading files thoroughly, making honest decisions under pressure, and avoiding shortcuts — is increasingly essential. As the AI league evolves, choosing the right partner can mean the difference between growth and missed opportunities.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
