deep research APIs are a new category. OpenAI, Perplexity, Google, and startups like Parallel are all shipping systems that can browse the web, synthesize sources, and return cited answers in a single API call. these tools are powerful, BUT choosing between them currently is not.

over the last couple weeks, i kept running into the same problem:

1. pricing scattered across docs
2. capabilities buried in changelogs
3. benchmarks inconsistent, missing, or not public
4. “it returns citations” used as a proxy for quality

so instead of guessing, i decided to actually compare them. this post is about what i learned, how i think about evaluation now, and why i built a public index to track this space.

how i approached evaluation

instead of reading docs in isolation, i did three things in parallel:

1. ran real queries: i tested market analysis, technical research, competitive intel, and product strategy prompts across providers — not just trivia or synthetic benchmarks.
2. read provider docs + benchmarks side-by-side: this made inconsistencies obvious very quickly, such as different definitions of “grounding”, benchmarks that aren’t comparable and pricing models that hide real cost drivers
3. tracked tradeoffs: there isn’t a single “best” deep research api. there are tradeoffs, specifically, depth vs latency, or cost vs accuracy or batch workflows vs interactive use. once i framed it that way, the space became much clearer.

why i built the deep research api index

after doing this manually, i realized there was no neutral, centralized place to:

• compare providers side-by-side
• run the same prompt across them
• track how things change over time

so i built one at deep-research-index.vercel.app

it’s an independent index that:

• compares providers across concrete metrics
• lets you run identical prompts across multiple apis
• makes tradeoffs explicit instead of hiding them

this isn’t a startup pitch, it’s more it’s a reference i wanted to exist.