Search Capability Leaderboard
Ultramarine 1 Flash
Paper InstrumentsOpen weights
#30 in Search Intelligence
Search Intelligence Score
61.1
+37.1 lift from search
050100
- Lift from search
- +37.1
- 24.0 without search
- Cost per 1K tasks
- $87.2
- Below the median score of 64.4, so not ranked on cost
- Time per task
- 165s
- Search plus inference time to complete one task
Accuracy by benchmark
Accuracy on each eval, with search (solid) and without (light).
- DSQA79.3 vs 36.9
- HLE35.0 vs 27.0
- WISER69.0 vs 8.0
Cost per 1K tasks
How the cost splits between model inference, search calls, and page extraction.
- Inference
- $49.9
- 57%
- Search
- $11.6
- 13%
- Extract
- $25.8
- 30%
Compared with
Other models from the same provider alongside models that score closest.
- Ultramarine Flash61.1Cost $87.2 · Lift +37.1 · 165s
- Step 5 Preview61.1Cost $118 · Lift +33.3 · 387s
- MiMo v2.6 Flash60.9Cost $25.2 · Lift +38.5 · 1323s
- GLM 5.3 Flash60.8Cost $16.6 · Lift +40.4 · 99.1s
- GPT-5.6 Luna60.7Cost $36.2 · Lift +33.9 · 98.0s
| Model | Score | Lift | Cost per 1K tasks | Time per task |
|---|---|---|---|---|
| Ultramarine 1 Flash | 61.1 | +37.1 | $87.2 | 165s |
| Step 5 PreviewStep 5 Preview | 61.1 | +33.3 | $118 | 387s |
| MiMo v2.6 FlashMiMo v2.6 Flash | 60.9 | +38.5 | $25.2 | 1323s |
| GLM 5.3 FlashGLM 5.3 Flash | 60.8 | +40.4 | $16.6 | 99.1s |
| GPT-5.6 LunaGPT-5.6 Luna | 60.7 | +33.9 | $36.2 | 98.0s |
Search Intelligence Score averages accuracy on DSQA, HLE, and WISER with equal weight; cost and time are averaged the same way, with cost shown per 1,000 tasks. Latest update October 18, 2026. Full methodology.
# Ultramarine 1 Flash · Search Capability Leaderboard
- Search Intelligence Score: 61.1 (#30 in Search Intelligence)
- Without search: 24.0 · Lift from search +37.1
- Cost per 1K tasks: $87.2 (below the median score of 64.4, so not ranked on cost)
- Time per task: 165s
- Lab: Paper Instruments · Weights: Open weights · Model id: paperinstruments/ultramarine-1-flash
## Accuracy by benchmark
| Suite | With search | Without search |
|---|---|---|
| DSQA | 79.3 | 36.9 |
| HLE | 35.0 | 27.0 |
| WISER | 69.0 | 8.0 |
## Compared with
| Model | Score | Lift | Cost per 1K tasks | Time per task |
|---|---|---|---|---|
| Ultramarine 1 Flash | 61.1 | +37.1 | $87.2 | 165s |
| Step 5 Preview | 61.1 | +33.3 | $118 | 387s |
| MiMo v2.6 Flash | 60.9 | +38.5 | $25.2 | 1323s |
| GLM 5.3 Flash | 60.8 | +40.4 | $16.6 | 99.1s |
| GPT-5.6 Luna | 60.7 | +33.9 | $36.2 | 98.0s |
Latest update October 18, 2026. Methodology: /leaderboard#methodology