Organisation
Vals AI
Independent, third-party evaluation of language models on domain-specific professional tasks in law, tax, finance and other regulated fields.
Vals AI is an independent evaluation company that tests large language models on domain-specific professional tasks — principally in law, tax and finance — and publishes the results as public leaderboards. It was founded in 2023 by Rayan Krishnan and Langston Nashold, who left a Stanford master's programme to build it, and is backed by Sequoia Capital and Bloomberg Beta. It positions itself as a third-party benchmarker for enterprises weighing which model to deploy on regulated, high-stakes work.
- Category
- Evaluation & forecasting
- Founded
- 2023
- HQ
- San Francisco, US
- Funding
- ~$5m raised (2024); backed by Sequoia Capital and Bloomberg Beta
- Key people
- Rayan Krishnan, Langston Nashold
Appears alongside
Featured in threads
Tracks
- Models & capabilities 3
- Safety & alignment 1
- Security & misuse 1
- Benchmarks & progress 1
OpenAI launches Astra for Law
OpenAI reported 54.0% accuracy on Vals AI's Legal Research Bench with the configured system, against 38.7% for plain GPT-6 Astra using web search.
Models & capabilities
OpenAI releases GPT-6 Astra
OpenAI's flagship is its first model rated 'Critical' for cyber capability, and its launch is shadowed by disclosures that a 'recurrent depth' technique makes Astra's reasoning harder to monitor.
Models & capabilities · Safety & alignment · Security & misuse
Claude Fable 5.1 solves a 370-year-old cipher in a Vals AI test
Given only the puzzle, Anthropic's model recovered the plaintext of Sir Thomas Urquhart's unsolved 1653 cipher in 44 minutes and 176,000 tokens, then checked its answer against the couplet's own structure.
Models & capabilities · Benchmarks & progress