We gave 28 AI models 300 real office tasks. How often did the best one get it right?
Briefing notes, bid evaluations, redactions, translations: the daily paperwork of Canadian government. "Right" means no deal-breaker mistakes and at least 80% of a manager's checklist. Take a guess.
The best AI got the work right 70% of the time.
We gave 28 AI models 300 real tasks from Canadian government desks and marked every answer the way a careful manager would. The typical model managed 54%, and 11 of the 28 got less than half of their work right.
0%: never right100%: always right
Each dot is one model. Tap a dot for its report card.