Perplexity Launches WANDR Benchmark For Measuring Large-Scale Research Capabilities Of AI Agents
Perplexity AI has introduced WANDR (Wide ANd Deep Research), an open benchmark designed to evaluate how effectively artificial intelligence systems perform large-scale research tasks that require both broad information discovery and detailed evidence collection. The framework contains 500 realistic data-gathering tasks modeled on professional knowledge work, including market analysis, due diligence, literature reviews, competitive intelligence, product comparisons, and talent sourcing.
Unlike traditional AI benchmarks that focus on generating a single answer or a written report, WANDR measures an AI system’s ability to identify large numbers of relevant entities and verify each result with supporting evidence. The benchmark is intended to reflect real-world research workflows, where success depends not only on finding accurate information but also on achieving comprehensive coverage across hundreds or even thousands of records.
According to Perplexity, current AI systems continue to face significant challenges in this area. Even the highest-performing model in the company’s evaluation achieved a soft F1 score of 0.363 and a hard F1 score of 0.133, indicating that wide-scale, evidence-backed research remains far from being fully automated. The benchmark includes more than 170,000 source-backed records across its 500 tasks, providing a large-scale testing environment for research-oriented AI agents.
Benchmark Results Highlight Current AI Research Limitations
WANDR uses a reference-free evaluation process that verifies each submitted claim against the evidence cited by the AI system, rather than comparing results with a fixed answer key. Every claim is checked for source quality, factual accuracy, relevance, and whether the supporting excerpts genuinely substantiate the information presented. This approach is intended to better reflect real-world research, where information changes over time and complete answer sets are difficult to maintain.
The benchmark also provides detailed diagnostics to identify where AI systems fail during complex research tasks. Performance can be measured across multiple stages, including information discovery, data enrichment, identity matching, source validation, and evidence extraction, allowing developers to pinpoint weaknesses beyond overall accuracy scores.
Perplexity evaluated six production AI research systems using WANDR under identical testing conditions. Its Search as Code (SaC) platform achieved the highest overall performance, recording a soft F1 score of 0.363 and a hard F1 score of 0.133. Anthropic ranked second with scores of 0.249 and 0.072, while other evaluated systems did not exceed a soft F1 score of 0.121. The study also found that increasing computational effort generally improved performance for several models, although higher costs and longer processing times did not consistently translate into better results.
The company said the benchmark is intended to serve as an open resource for researchers and developers working on AI-powered search and research systems. Beyond benchmarking, WANDR may also support future reinforcement learning techniques by providing structured feedback at each stage of the research process, enabling AI models to improve not only factual accuracy but also planning, coverage, and evidence collection at scale.
Disclaimer
In line with the Trust Project guidelines, please note that the information provided on this page is not intended to be and should not be interpreted as legal, tax, investment, financial, or any other form of advice. It is important to only invest what you can afford to lose and to seek independent financial advice if you have any doubts. For further information, we suggest referring to the terms and conditions as well as the help and support pages provided by the issuer or advertiser. MetaversePost is committed to accurate, unbiased reporting, but market conditions are subject to change without notice.
About The Author
Alisa, a dedicated journalist at the MPost, specializes in crypto, AI, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
More articles
Alisa, a dedicated journalist at the MPost, specializes in crypto, AI, investments, and the expansive realm of Web3. With a keen eye for emerging trends and technologies, she delivers comprehensive coverage to inform and engage readers in the ever-evolving landscape of digital finance.
