How do you actually evaluate whether AI models can reason, and not just answer?

Salesforce AI Research faced a critical challenge: verifying long-form, Olympiad-grade math reasoning produced by frontier models like GPT-5, Gemini 2.5 Pro, and Claude Sonnet 4. These problems demand multi-step logic, where a single error can invalidate everything that follows.

To meet this challenge, Salesforce partnered with Turing to execute a high-precision evaluation at scale.

Turing annotated 200+ open-ended math tasks sourced from elite competitions such as IMO and Putnam. Each model response was broken into 8–15 structured reasoning steps, with expert reviewers issuing strict binary judgments for every step. Correct or Incorrect. No partial credit. No hand-wavy logic. And zero tolerance for carry-forward errors.

This rigorous framework delivered:

• 100% reviewer alignment using justification-based verification
• 100% pass-through across Salesforce’s internal 3-tier review pipeline
• Deep visibility into early-stage reasoning flaws that derail complex logic chains

The resulting dataset now helps Salesforce benchmark reasoning fidelity, detect subtle failure modes, and align AI outputs with human-grade reasoning standards for future fine-tuning and scoring research.

Discover how rigorous, expert-led evaluation is shaping the future of trustworthy AI reasoning.