Atlas · C17 · Executive mind
Judgment & decision under uncertainty
Choosing well with incomplete information, weighing risks and trade-offs.
AI Reliable score
5 / 10
Average adult level · peak 8 · trend ↑ · grade B
Why this score
Forecasting: best LLMs (Brier 0.103) trail superforecasters (0.086), parity projected around 2027. Narrow diagnostic case studies show expert-level peaks. Long-running real decisions (business simulation) remain far below good humans.
On the human scale
2Beginner. Chooses using clear rules
4Average adult. Average adult: sensible everyday decisions ← AI today
6Competent pro. Professional: sound work decisions under uncertainty
8Expert. Expert: well-calibrated high-stakes calls (diagnosis, investment)
10Best humans. Best decision-makers (superforecasters)
Future warning
At its best, AI already reaches 8 on this capability. Peak results are never counted in the index; they show where the water may rise next.
Evidence
ForecastBench (Forecasting Research Institute)A
E21 · 2026-03
- Measures
- Forecasting real future events (Brier score; lower is better)
- AI result
- Best LLMs 0.103 vs superforecasters 0.086; parity projected ~2027.
- Humans
- Superforecasters (top human forecasters)
- On our scale
- Close to, but below, the best human judges.
forecastingresearch.substack.com ↗
Microsoft MAI-DxO diagnostic orchestratorC
E22 · 2025-06/07
- Measures
- Diagnosing complex NEJM case records
- AI result
- 85% correct vs ~20% for physicians working without colleagues or books.
- Humans
- Practising physicians (restricted conditions)
- On our scale
- Peak in a constrained setting; company-reported.
www.techrepublic.com ↗
Vending-Bench 2 (Andon Labs)B
E17 · 2026
- Measures
- Running a simulated vending business for a year
- AI result
- Best: Claude Opus 5 ~$11,182 average final balance from $500.
- Humans
- Good human strategy estimated ~$63k per year (~10x the best model)
- On our scale
- Long-horizon planning and judgment far below a competent human: key limit on Reliable.
epoch.ai ↗
History
First reading: Atlas v0.1, 5 Nov 2026. Every later change will be listed here with its reason.
Disagree? Send a correction
