# Introducing PWS-Decisions: A benchmark for web-grounded decisions

Today we’re open-sourcing PWS-Decisions, a benchmark of 300 questions whose answers depend on the live web. On PWS-Decisions, the Parallel Decisions API [Parallel Decisions API ](/blog/decisions-api)outscores the closest competitor by 1.5X.

Every day, businesses answer thousands of small classification questions about the world: _Does this vendor support SSO? Is this the same company we already have in the CRM?_ _Is this lead a fit? _The Parallel Decisions API[Parallel Decisions API](/blog/decisions-api) is the fastest way to act on that information: it checks the live web and returns a typed, cited answer in under a second, for a tenth of a cent.

Run the benchmark on your own model with the code on GitHub[GitHub](https://github.com/parallel-web/pws-decisions-benchmark) and the dataset on Hugging Face[Hugging Face](https://huggingface.co/datasets/parallel-web/pws-decisions-benchmark), or try the Decisions API free in the playground[playground](https://platform.parallel.ai/play/decisions) with 5,000 free requests.

## Real Decisions need Web Grounding

In our post a couple weeks ago[our post a couple weeks ago](/blog/are-all-system-one-models-created-equal), we benchmarked System One models on real, human-labeled tasks from our own production pipelines instead of public leaderboards. Traditional decision benchmarks check what is already in the prompt from a system-one model, not whether an answer is grounded in live web data, so we built PWS-Decisions to test that.

PWS-Decisions is 300 closed-ended questions modeled on the decisions developers use the Decisions API for: qualifying accounts, checking product capabilities, confirming policy rules and tracking fast-moving events. Every question depends on facts that are current, niche, or scattered across the web. Each answer is confirmed by independent web research and a direct source, and every system is scored by exact match under the same rules.

Run PWS-Decisions on any decision model, including your own.

### Testing on live web questions

Benchmarks like JevBench can’t tell you how a model will perform when the answer depends on the live web. Which is why traditional models like OpenAI Decisions and Jev do far worse on PWS-Decisions than on a generic decisions benchmark like JevBench. OpenAI Decisions scores 2.2x lower (41.7% vs. 90.0%), and Jev scores 2.6x lower (33.4% vs. 88.3%).

## Cost per decision should include search

The way you measure cost is critical in determining what the true cost efficiency is. A low-cost model plus a separate search call ends up costing more than one Decisions API call, which includes both. For 1,000 requests that answer closed-ended questions about one entity using current web evidence:

For scale: See some of the examples from our Decisions API launch demos including: rechecking 426 public companies for KYB changes cost $0.85[rechecking 426 public companies for KYB changes cost $0.85](https://perpetualkyb.paralleldemos.com/), and routing 500 inbound leads cost about $0.50[routing 500 inbound leads cost about $0.50](/blog/decisions-api).

## Where the Decisions API fits

Many search requests are really decisions: one value for a spreadsheet cell, a form field or an agent’s next step.

Use the Decisions API when:

• - You need a single, structured answer, such as a yes/no, one option from a list you define, or a score on a scale. This value can be integrated directly into your agent or downstream data without an expensive LLM.
• - The response depends on information that changes often, like a company’s funding, headcount, pricing or policies, so it has to come from the live web rather than a model’s memory.
• - You’re making the same kind of decision across thousands of records in a pipeline, where each request can carry up to 100 questions and costs $1 per 1,000 requests.

There are many different usecases where Decisions API can be useful.

Not sure where the Decisions API fits for your team? Try this demo: [Try this demo: ](https://webjev-use-cases.paralleldemos.com/)enter your company and see which of your workflows could run on web-grounded decisions.

### Which Parallel API should I use?

The Decisions API works best when you need a quick, typed answer (yes/no, a choice, or a score) to a question that depends on what’s happening on the web right now. For everything else, another Parallel API may serve you better.

## Run PWS-Decisions and try the Decisions API free

Run PWS-Decisions on your own model with the code on GitHub[GitHub](https://github.com/parallel-web/pws-decisions-benchmark) and the data on Hugging Face[Hugging Face](https://huggingface.co/datasets/parallel-web/pws-decisions-benchmark). Then try the Decisions API with 5,000 free requests in the playground[playground](https://platform.parallel.ai/play/decisions), or jump into the docs[docs](https://docs.parallel.ai/decisions/decisions-quickstart). Agents can use it free as the `web_decisions` tool in the Parallel MCP server. Running decisions at scale? Talk to us[Talk to us](https://contact.parallel.ai/) about building an eval set from your own traffic.