RESULTS EXPLORER
WindTunnel benchmark run
2026-08-20 · 8 sites · 10 methods · 3 attempts per task
WindTunnel measures how reliably and cheaply an AI browser agent can operate real websites, comparing three ways it can “see” and act on a page. This run put 10 methods through 49 tasks across 8 sites, 3 attempts each — 2,352 agent runs in total, about $175.62. The three interfaces: WebMCP (the site hands the agent direct tools), computer use (the agent reads screenshots), and page structure (the agent reads the DOM / accessibility tree).
Headline: WebMCP solved 97% of attempts (vs 84%–94% for the other interfaces) while being 3×–11× cheaper, 4×–13× lighter (median tokens), and 3×–4× faster.
Run summary
| Interface | Method | Median $/task | Median tokens | vs WebMCP |
|---|---|---|---|---|
| WebMCP | wm-gpt / wm-claude | $0.005 / $0.011 | 2,584 / 4,504 | reference |
| Page structure · a11y | a11y-stagehand | $0.026 | 14,541 | 3.3× cost · 4.1× tok · 2.8× time |
| DOM + vision (multimodal) | dom-browseruse | $0.091 | 44,715 | 11× cost · 13× tok · 3.1× time |
| Screenshots | cu-claude / cu-openai | $0.108 / $0.038 | 46,347 / 16,315 | 9.0× cost · 8.8× tok · 4.4× time |
The DOM + vision row (Browser Use) is multimodal — by default it reads the page's DOM and a screenshot each step, so it's the strongest realistic non-WebMCP baseline here, not a DOM-only agent. The computer-use rows are screenshots-only; the a11y row is accessibility-tree-only. WebMCP is compared against each on equal footing.
How the advantage scales with journey length
Solved counts are task×method cells (majority of attempts passing). The multiples are the pooled browser-method median divided by the pooled WebMCP median for that tier — higher means WebMCP is further ahead.
| Tier | Tasks | WebMCP solved | Browser solved | Cheaper | Faster | Lighter |
|---|---|---|---|---|---|---|
| Answer (1–2 steps) | 21 | 105/105 | 92/105 | 6.0× | 3.2× | 5.9× |
| Act, short (3–5) | 16 | 80/80 | 75/80 | 7.8× | 2.8× | 6.5× |
| Act, long (6–10) | 8 | 40/40 | 35/40 | 10.4× | 4.8× | 13.8× |
| Transaction (8–15) | 4 | 15/20 | 16/20 | 11.7× | 8.5× | 15.1× |
A page-reading agent re-reads the whole page on every step, so its cost and token disadvantage compounds with journey length; a WebMCP call returns a small JSON result regardless of how long the journey is. On the longest journeys the cost gap reaches an order of magnitude, while the speed gap — bounded by model latency per turn rather than by payload size — grows more slowly.
Click any task to see how each method did — pass rate, time, tokens, and cost — ranked best→worst. Cost/time/tokens are medians across repeats.
bulletproof-react
react-auth-boundary›
promptCheck whether the private app is accessible while signed out; do not sign in.
check {"type":"answer","matches":"(redirect|unauthorized|denied|forbidden|inaccessible|not\\W{0,4}accessible|(cannot|can'?t|could not|couldn'?t|unable to)[^.]{0,30}(access|reach|view|open|see)|not\\W{0,4}reachable|requires? (signing|sign|log|logging|authentication)|(must|need to|have to) (log|sign) in|restricted to signed|only visible to signed|blocked)"}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 6.7s | 2,495 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 8.5s | 2,697 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 6.0s | 980 | $0.004 |
| wm-stagehand-v4 | ✓ 3/3 | 7.9s | 1,034 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 11.6s | 1,924 | $0.015 |
| cu-gemini | ✓ 3/3 | 22.5s | 19,709 | $0.017 |
| Computer use · GPT | ✓ 6/6 | 9.8s | 6,691 | $0.017 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 20.8s | 8,911 | $0.017 |
| Computer use · Claude | ✓ 6/6 | 34.9s | 30,231 | $0.077 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 22.0s | 36,160 | $0.089 |
react-public-content›
promptIdentify the public application shown on the home page.
check {"type":"answer","contains":["bulletproof"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 5.2s | 1,154 | $0.005 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 7.8s | 3,148 | $0.005 |
| wm-gemini | ✓ 3/3 | 10.7s | 6,607 | $0.007 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 18.8s | 7,709 | $0.008 |
| wm-stagehand-v4 | ✓ 3/3 | 7.2s | 954 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 8.9s | 1,004 | $0.011 |
| Computer use · Claude | ✓ 6/6 | 8.7s | 4,774 | $0.012 |
| Computer use · GPT | ✓ 6/6 | 6.2s | 3,206 | $0.012 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 15.3s | 19,166 | $0.048 |
| cu-gemini | ◑ 2/3 | 76.9s | 105,663 | $0.085 |
directory-9d8
directory-detail›
promptOpen the Figma listing and report its one-line description.
check {"type":"answer","contains":["collaborative"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 12.0s | 2,715 | $0.002 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.5s | 2,910 | $0.003 |
| wm-gemini | ✓ 3/3 | 6.3s | 2,966 | $0.003 |
| WebMCP · Claude | ✓ 6/6 | 5.1s | 2,519 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 8.5s | 2,267 | $0.008 |
| cu-gemini | ✓ 3/3 | 13.5s | 11,986 | $0.011 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 15.4s | 9,455 | $0.017 |
| Computer use · GPT | ✓ 6/6 | 18.2s | 16,202 | $0.025 |
| Computer use · Claude | ✓ 6/6 | 25.9s | 28,028 | $0.075 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 19.8s | 47,128 | $0.121 |
directory-filter›
promptFilter the directory to developer tools, then report one result shown in the filtered list.
check {"type":"answer","matches":"\\b(github|react)\\b"}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 13.9s | 4,081 | $0.006 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 10.8s | 6,938 | $0.007 |
| cu-gemini | ✓ 3/3 | 11.1s | 7,698 | $0.007 |
| wm-gemini | ✓ 3/3 | 10.4s | 7,432 | $0.008 |
| WebMCP · GPT | ✓ 6/6 | 8.0s | 2,204 | $0.008 |
| WebMCP · Claude | ✓ 6/6 | 9.1s | 3,908 | $0.009 |
| Computer use · GPT | ✓ 6/6 | 9.8s | 6,624 | $0.016 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 16.6s | 9,980 | $0.018 |
| Computer use · Claude | ✓ 6/6 | 16.3s | 18,700 | $0.035 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 19.4s | 37,046 | $0.090 |
directory-search›
promptSearch the directory for a {params.kind} tool and report its name.
check {"type":"answer","contains_any":["figma","dribbble"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 11.3s | 2,877 | $0.003 |
| wm-gemini | ✓ 3/3 | 7.2s | 4,090 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 6.5s | 1,368 | $0.005 |
| wm-stagehand-v4 | ✓ 3/3 | 7.2s | 1,302 | $0.008 |
| WebMCP · Claude | ✓ 6/6 | 6.4s | 2,912 | $0.008 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 12.3s | 9,886 | $0.015 |
| cu-gemini | ✓ 3/3 | 20.7s | 23,857 | $0.020 |
| Computer use · GPT | ✓ 6/6 | 15.1s | 11,564 | $0.040 |
| Computer use · Claude | ✓ 6/6 | 22.5s | 23,339 | $0.062 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 18.4s | 30,123 | $0.083 |
easyappointments
ea-1›
promptWhat services does this studio offer? List all of them.
check {"type":"answer","contains":["consultation","haircut","massage"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 7.7s | 5,258 | $0.005 |
| wm-gemini | ✓ 3/3 | 6.2s | 5,668 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.9s | 5,656 | $0.006 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 11.3s | 3,878 | $0.008 |
| WebMCP · GPT | ✓ 6/6 | 5.1s | 2,540 | $0.009 |
| WebMCP · Claude | ✓ 6/6 | 7.4s | 4,859 | $0.011 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 17.9s | 29,547 | $0.082 |
| cu-gemini | ✕ 0/3 | 66.2s | 90,154 | $0.077 |
| Computer use · Claude | ✕ 0/6 | 89.4s | 91,549 | $0.243 |
| Computer use · GPT | ✕ 0/6 | 80.2s | 85,568 | $0.260 |
ea-2›
promptHow much does a haircut cost, and how long does the appointment take?
check {"type":"answer","contains":["35","45"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 4.9s | 5,180 | $0.003 |
| WebMCP · GPT | ✓ 6/6 | 4.2s | 2,503 | $0.005 |
| wm-gemini | ✓ 3/3 | 5.7s | 5,772 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.3s | 5,861 | $0.005 |
| WebMCP · Claude | ✓ 6/6 | 6.1s | 4,833 | $0.006 |
| cu-gemini | ✓ 3/3 | 13.4s | 11,321 | $0.010 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 14.1s | 6,713 | $0.011 |
| Computer use · GPT | ✓ 6/6 | 20.8s | 25,849 | $0.043 |
| Browser Use · DOM + screenshot | ◑ 5/6 | 20.0s | 53,894 | $0.128 |
| Computer use · Claude | ◑ 4/6 | 55.4s | 65,556 | $0.175 |
ea-3›
promptWhich of the studio's services is free of charge?
check {"type":"answer","contains":["consultation"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 4.8s | 5,159 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 5.0s | 3,324 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.0s | 3,342 | $0.003 |
| WebMCP · Claude | ✓ 6/6 | 6.1s | 4,456 | $0.008 |
| WebMCP · GPT | ✓ 6/6 | 5.0s | 2,530 | $0.009 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 37.5s | 10,103 | $0.030 |
| Browser Use · DOM + screenshot | ◑ 5/6 | 22.4s | 57,868 | $0.156 |
| cu-gemini | ◑ 1/3 | 58.8s | 80,505 | $0.066 |
| Computer use · Claude | ✕ 0/6 | 80.8s | 90,425 | $0.233 |
| Computer use · GPT | ✕ 0/6 | 78.5s | 86,520 | $0.261 |
ea-4›
promptWho provides the Deep Tissue Massage service? Report the provider's name.
check {"type":"answer","contains":["jane"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 4.6s | 5,159 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 4.4s | 3,274 | $0.003 |
| wm-gemini | ✓ 3/3 | 3.8s | 3,308 | $0.003 |
| WebMCP · GPT | ✓ 6/6 | 4.1s | 2,467 | $0.005 |
| WebMCP · Claude | ✓ 6/6 | 5.8s | 4,435 | $0.007 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 14.5s | 6,767 | $0.011 |
| cu-gemini | ✓ 3/3 | 20.5s | 19,045 | $0.016 |
| Computer use · GPT | ✓ 6/6 | 20.8s | 20,933 | $0.041 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 21.9s | 46,883 | $0.120 |
| Computer use · Claude | ◑ 4/6 | 59.2s | 60,033 | $0.176 |
ea-5›
promptFind out whether the studio takes bookings for a service called "Yoga Class". Answer with the single word AVAILABLE or MISSING.
check {"type":"answer","matches":"\\bmissing\\b","not_contains":["not missing","available or missing"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 4.4s | 5,182 | $0.002 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.4s | 5,280 | $0.005 |
| wm-gemini | ✓ 3/3 | 5.7s | 5,275 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 4.1s | 2,490 | $0.008 |
| WebMCP · Claude | ✓ 6/6 | 7.5s | 6,259 | $0.009 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 16.1s | 29,335 | $0.080 |
| Computer use · GPT | ◑ 5/6 | 86.6s | 94,227 | $0.281 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 7.7s | 3,913 | $0.007 |
| Computer use · Claude | ◑ 3/6 | 87.1s | 107,076 | $0.285 |
| cu-gemini | ✕ 0/3 | 85.0s | 121,317 | $0.101 |
ea-6›
promptBook a Haircut appointment with Jane Doe at the earliest available slot for the customer Alex Runner, email alex.runner@example.com, phone +1 555 0199. Confirm the booking went through.
check {"probe":"appointments","assert":{"contains":{"service":{"name":"Haircut"},"customer":{"email":"alex.runner@example.com"}}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 10.2s | 12,879 | $0.011 |
| wm-gemini | ✓ 3/3 | 9.0s | 13,081 | $0.012 |
| wm-stagehand-v4 | ✓ 3/3 | 14.1s | 13,602 | $0.028 |
| WebMCP · Claude | ✓ 6/6 | 14.9s | 13,635 | $0.029 |
| cu-gemini | ✓ 3/3 | 71.5s | 87,600 | $0.070 |
| Computer use · GPT | ✓ 6/6 | 44.2s | 36,923 | $0.122 |
| Computer use · Claude | ✓ 6/6 | 116.7s | 176,110 | $0.388 |
| WebMCP · GPT | ◑ 5/6 | 11.3s | 9,390 | $0.013 |
| Browser Use · DOM + screenshot | ◑ 4/6 | 51.6s | 129,314 | $0.337 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 174.3s | 107,304 | $0.192 |
ea-7›
promptBook a Consultation at the earliest available slot for Casey Reed, email casey.reed@example.com, phone +1 555 0177 — then cancel that same appointment with the reason "schedule conflict". Report the cancellation result.
check {"probe":"database","assert":{"contains":{"customers":2}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 12.2s | 17,870 | $0.016 |
| wm-gemini | ✓ 3/3 | 11.8s | 18,201 | $0.016 |
| WebMCP · GPT | ✓ 6/6 | 14.0s | 13,134 | $0.026 |
| wm-stagehand-v4 | ✓ 3/3 | 21.7s | 14,833 | $0.047 |
| WebMCP · Claude | ✓ 6/6 | 18.7s | 14,869 | $0.054 |
| cu-gemini | ✓ 3/3 | 140.2s | 216,400 | $0.153 |
| Computer use · GPT | ✓ 6/6 | 185.0s | 210,818 | $0.482 |
| Computer use · Claude | ✓ 6/6 | 167.3s | 203,780 | $0.516 |
| Browser Use · DOM + screenshot | ◑ 5/6 | 162.9s | 168,602 | $0.430 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 161.9s | 137,817 | $0.318 |
ea-8›
promptLog in to the admin backend at /index.php/login as administrator with password administrator, open the customers page, and report the name of the registered customer.
check {"type":"answer","contains":["james"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 6.6s | 3,769 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 24.5s | 5,103 | $0.005 |
| wm-gemini | ✓ 3/3 | 24.6s | 5,223 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 11.2s | 7,433 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 8.6s | 7,512 | $0.009 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 34.9s | 23,107 | $0.042 |
| Computer use · GPT | ✓ 6/6 | 27.1s | 21,242 | $0.053 |
| Computer use · Claude | ✓ 6/6 | 46.9s | 53,430 | $0.100 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 28.8s | 73,772 | $0.168 |
| cu-gemini | ✕ 0/3 | 35.3s | 7,536 | $0.007 |
hi-events
hev-1›
promptWhere does the WebMCP Community Workshop take place? Report the venue or location.
check {"type":"answer","contains":["online"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 5.7s | 2,552 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.9s | 5,011 | $0.005 |
| cu-gemini | ✓ 3/3 | 9.2s | 4,368 | $0.005 |
| wm-gemini | ✓ 3/3 | 6.7s | 5,302 | $0.005 |
| wm-stagehand-v4 | ✓ 3/3 | 6.3s | 4,374 | $0.006 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 7.2s | 3,971 | $0.007 |
| Computer use · GPT | ✓ 6/6 | 7.5s | 3,171 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 7.5s | 4,638 | $0.013 |
| Computer use · Claude | ✓ 6/6 | 17.5s | 14,872 | $0.032 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 19.1s | 16,824 | $0.039 |
hev-10›
promptDoes the event page say anything about parking? Answer with the single word PRESENT or ABSENT.
check {"type":"answer","matches":"\\babsent\\b","not_contains":["not absent","present or absent"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 7.0s | 2,303 | $0.002 |
| wm-stagehand-v4 | ✓ 3/3 | 5.6s | 4,560 | $0.003 |
| wm-gemini | ✓ 3/3 | 9.0s | 6,565 | $0.006 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 11.0s | 8,280 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 5.6s | 5,547 | $0.008 |
| cu-gemini | ✓ 3/3 | 15.5s | 12,452 | $0.012 |
| Computer use · GPT | ✓ 6/6 | 16.4s | 11,441 | $0.023 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 17.9s | 30,144 | $0.082 |
| Computer use · Claude | ◑ 5/6 | 48.8s | 47,579 | $0.155 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 6.4s | 3,939 | $0.007 |
hev-2›
promptAccording to the workshop schedule, what session starts at 11:15?
check {"type":"answer","contains":["state-conditioned"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| cu-gemini | ✓ 3/3 | 4.8s | 1,532 | $0.002 |
| WebMCP · GPT | ✓ 6/6 | 6.5s | 2,579 | $0.002 |
| wm-gemini | ✓ 3/3 | 4.7s | 3,238 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 5.8s | 3,298 | $0.004 |
| wm-stagehand-v4 | ✓ 3/3 | 6.3s | 4,369 | $0.006 |
| WebMCP · Claude | ✓ 6/6 | 4.8s | 4,357 | $0.006 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 9.5s | 3,936 | $0.007 |
| Computer use · Claude | ✓ 6/6 | 7.5s | 5,439 | $0.010 |
| Computer use · GPT | ✓ 6/6 | 5.7s | 3,121 | $0.011 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 17.4s | 16,724 | $0.039 |
hev-3›
promptWhat is the event's age policy for young attendees?
check {"type":"answer","contains_any":["sixteen","16"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 5.3s | 2,434 | $0.003 |
| wm-gemini | ✓ 3/3 | 3.9s | 2,999 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.0s | 3,090 | $0.003 |
| cu-gemini | ✓ 3/3 | 8.8s | 4,241 | $0.005 |
| wm-stagehand-v4 | ✓ 3/3 | 6.3s | 4,359 | $0.005 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 7.3s | 3,915 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 5.1s | 4,316 | $0.007 |
| Computer use · GPT | ✓ 6/6 | 8.2s | 6,536 | $0.016 |
| Computer use · Claude | ✓ 6/6 | 13.7s | 12,672 | $0.023 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 18.9s | 16,780 | $0.038 |
hev-4›
promptAre the workshop sessions recorded, and who gets access to the recordings?
check {"type":"answer","matches":"record(ed|ing)s?[\\s\\S]{0,160}attendee|attendee[\\s\\S]{0,160}record"}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 6.6s | 2,529 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.7s | 3,079 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 5.5s | 3,194 | $0.003 |
| wm-stagehand-v4 | ✓ 3/3 | 6.2s | 4,388 | $0.006 |
| cu-gemini | ✓ 3/3 | 11.4s | 7,576 | $0.007 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 7.5s | 3,958 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 5.0s | 4,662 | $0.007 |
| Computer use · GPT | ✓ 6/6 | 9.5s | 6,593 | $0.016 |
| Computer use · Claude | ✓ 6/6 | 24.8s | 23,863 | $0.050 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 18.1s | 20,235 | $0.053 |
hev-5›
promptDoes this event offer live captions? Answer with the single word PRESENT or ABSENT.
check {"type":"answer","matches":"\\bpresent\\b","not_contains":["not present","present or absent"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 6.4s | 2,480 | $0.002 |
| wm-stagehand-v4 | ✓ 3/3 | 5.9s | 4,802 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.3s | 3,038 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 5.3s | 3,054 | $0.003 |
| cu-gemini | ✓ 3/3 | 8.0s | 4,224 | $0.004 |
| WebMCP · Claude | ✓ 6/6 | 4.8s | 4,513 | $0.005 |
| Computer use · GPT | ✓ 6/6 | 9.2s | 6,598 | $0.016 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 16.9s | 16,566 | $0.037 |
| Computer use · Claude | ✓ 6/6 | 17.4s | 17,911 | $0.038 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 6.2s | 3,922 | $0.006 |
hev-6›
promptTry to order five General admission tickets in a single order for Pat Lee, pat.lee@example.com. If that is not possible, report the maximum allowed per order and stop.
check {"type":"answer","matches":"(max|maximum|limit|allowed)[^.]{0,40}\\b(4|four)\\b"}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 6.5s | 2,531 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.6s | 3,276 | $0.004 |
| wm-stagehand-v4 | ✓ 3/3 | 6.4s | 4,386 | $0.006 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 9.9s | 5,768 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 5.3s | 4,377 | $0.009 |
| Computer use · GPT | ✓ 6/6 | 10.8s | 7,475 | $0.014 |
| cu-gemini | ✓ 3/3 | 28.5s | 28,967 | $0.024 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 32.2s | 18,860 | $0.049 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 31.1s | 52,390 | $0.160 |
| Computer use · Claude | ✓ 6/6 | 63.3s | 75,648 | $0.168 |
hev-7›
promptRegister Jordan Baker, jordan.baker@example.com, for two General admission tickets to the WebMCP Community Workshop — the second attendee is Sam Baker. Confirm the registration completed.
check {"probe":"orders","assert":{"contains":{"orders":1}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 11.8s | 7,262 | $0.008 |
| WebMCP · GPT | ✓ 6/6 | 10.2s | 4,269 | $0.009 |
| wm-gemini | ✓ 3/3 | 12.2s | 8,212 | $0.010 |
| wm-stagehand-v4 | ✓ 3/3 | 17.8s | 4,583 | $0.020 |
| WebMCP · Claude | ✓ 6/6 | 14.5s | 6,739 | $0.032 |
| Computer use · GPT | ✓ 6/6 | 47.3s | 37,643 | $0.099 |
| cu-gemini | ✓ 3/3 | 102.4s | 138,438 | $0.108 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 114.4s | 104,479 | $0.226 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 62.5s | 130,480 | $0.310 |
| Computer use · Claude | ✓ 6/6 | 120.8s | 222,118 | $0.386 |
hev-8›
promptLog in to the organizer dashboard as organizer@webmcp-eval.test with password WebmcpEval123! and report the current status of the WebMCP Community Workshop event.
check {"type":"answer","matches":"(status|workshop|event)[^.]{0,80}\\b(live|published|on[ -]?sale)\\b|\\b(live|published|on[ -]?sale)\\b[^.]{0,80}(status|workshop|event)|^\\W*((final )?answer:)?\\W*(live|published|on[ -]?sale)\\b[^.]{0,40}\\.?$|\\b(is|are|remains?|shows?|currently)\\b[^.]{0,25}\\b(live|published|on[ -]?sale)\\b","not_contains":["could not determine","couldn't determine","unable to determine","could not verify","couldn't verify","could not log in","couldn't log in","unable to log in","never reached"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 8.9s | 2,889 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 11.2s | 3,037 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 13.1s | 2,255 | $0.010 |
| wm-stagehand-v4 | ✓ 3/3 | 13.5s | 3,240 | $0.018 |
| WebMCP · Claude | ✓ 6/6 | 12.5s | 3,162 | $0.019 |
| cu-gemini | ✓ 3/3 | 47.9s | 31,478 | $0.026 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 28.8s | 19,118 | $0.040 |
| Computer use · GPT | ✓ 6/6 | 37.5s | 24,407 | $0.047 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 26.5s | 37,736 | $0.095 |
| Computer use · Claude | ✓ 6/6 | 46.5s | 49,963 | $0.111 |
hev-9›
promptRegister Riley Fox, riley.fox@example.com, for one General admission ticket, note the order reference code (it starts with "o_"), then use the event page's order lookup with that code and the same email to check the order and report its status.
check {"probe":"orders","assert":{"contains":{"orders":1}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 8.5s | 4,095 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 10.0s | 4,436 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 10.9s | 3,739 | $0.009 |
| wm-stagehand-v4 | ✓ 3/3 | 14.7s | 4,081 | $0.016 |
| WebMCP · Claude | ✓ 6/6 | 16.8s | 6,761 | $0.034 |
| cu-gemini | ✓ 3/3 | 133.5s | 201,789 | $0.143 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 216.9s | 154,418 | $0.285 |
| Computer use · Claude | ✓ 6/6 | 135.3s | 203,010 | $0.519 |
| Computer use · GPT | ◑ 4/6 | 188.5s | 160,983 | $0.173 |
| Browser Use · DOM + screenshot | ◑ 2/6 | 21.7s | 37,058 | $0.042 |
idurar-erp-crm
id-1›
promptLog in at /login as admin@admin.com with password admin123, then report which client invoice 1 is billed to.
check {"type":"answer","contains":["acme"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 28.5s | 4,431 | $0.004 |
| wm-gemini | ✓ 3/3 | 26.3s | 4,476 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 8.5s | 3,470 | $0.006 |
| cu-gemini | ✓ 3/3 | 25.6s | 7,661 | $0.007 |
| wm-stagehand-v4 | ✓ 3/3 | 8.1s | 6,117 | $0.012 |
| WebMCP · Claude | ✓ 6/6 | 11.2s | 6,119 | $0.016 |
| Computer use · GPT | ✓ 6/6 | 10.7s | 6,599 | $0.016 |
| Computer use · Claude | ✓ 6/6 | 23.1s | 22,411 | $0.053 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 18.9s | 36,974 | $0.089 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 48.9s | 41,564 | $0.096 |
id-2›
promptLog in at /login as admin@admin.com with password admin123, then report the total amount of invoice 1/2026.
check {"type":"answer","contains":["250"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 18.0s | 4,525 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 17.4s | 4,695 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 8.4s | 3,485 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 8.2s | 6,119 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 9.7s | 6,283 | $0.011 |
| cu-gemini | ✓ 3/3 | 36.6s | 19,879 | $0.017 |
| Computer use · GPT | ✓ 6/6 | 18.4s | 16,291 | $0.038 |
| Computer use · Claude | ✓ 6/6 | 24.0s | 27,075 | $0.074 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 25.4s | 37,488 | $0.094 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 63.9s | 59,937 | $0.150 |
id-3›
promptLog in at /login as admin@admin.com with password admin123, then report the name of the line item on invoice 1.
check {"type":"answer","contains":["webmcp evaluation service"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 21.4s | 4,384 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 21.5s | 4,468 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 8.1s | 3,500 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 8.6s | 6,158 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 8.8s | 6,245 | $0.012 |
| cu-gemini | ✓ 3/3 | 42.7s | 20,071 | $0.017 |
| Computer use · GPT | ✓ 6/6 | 95.2s | 20,961 | $0.041 |
| Computer use · Claude | ✓ 6/6 | 41.3s | 42,449 | $0.115 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 35.3s | 83,698 | $0.210 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 45.7s | 42,837 | $0.090 |
id-4›
promptLog in at /login as admin@admin.com with password admin123, then report the payment status of invoice 1.
check {"type":"answer","contains":["unpaid"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 25.7s | 4,285 | $0.004 |
| wm-gemini | ✓ 3/3 | 20.4s | 4,453 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 7.7s | 3,507 | $0.006 |
| cu-gemini | ✓ 3/3 | 27.3s | 7,706 | $0.007 |
| wm-stagehand-v4 | ✓ 3/3 | 11.4s | 6,153 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 10.2s | 6,168 | $0.011 |
| Computer use · Claude | ✓ 6/6 | 23.9s | 24,028 | $0.054 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 20.1s | 37,306 | $0.092 |
| Computer use · GPT | ◑ 5/6 | 10.7s | 6,604 | $0.005 |
| Page structure · Stagehand (a11y) | ◑ 5/6 | 38.9s | 24,090 | $0.052 |
id-5›
promptLog in at /login as admin@admin.com with password admin123, search the customers for "acme", and report that client's email address.
check {"type":"answer","contains":["billing@acme.example.test"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 18.3s | 4,344 | $0.004 |
| wm-gemini | ✓ 3/3 | 47.7s | 4,356 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 8.3s | 3,346 | $0.005 |
| wm-stagehand-v4 | ✓ 3/3 | 8.9s | 6,136 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 9.4s | 6,091 | $0.011 |
| cu-gemini | ✓ 3/3 | 51.8s | 19,420 | $0.016 |
| Computer use · GPT | ✓ 6/6 | 20.2s | 16,127 | $0.037 |
| Computer use · Claude | ✓ 6/6 | 37.0s | 40,414 | $0.080 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 31.2s | 72,319 | $0.174 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 68.9s | 66,418 | $0.146 |
id-6›
promptLog in at /login as admin@admin.com with password admin123, then create a new draft invoice for the client Acme Evaluation with a single line item "Quarterly Retainer", quantity 1, price 400.
check {"probe":"invoices","args":{"number":2},"assert":{"contains":{"number":2,"status":"draft","client":{"name":"Acme Evaluation"},"items":{"name":"Quarterly Retainer","quantity":1,"price":400}}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 11.4s | 7,015 | $0.007 |
| wm-gemini | ✓ 3/3 | 8.4s | 7,203 | $0.007 |
| WebMCP · GPT | ✓ 6/6 | 11.1s | 5,235 | $0.008 |
| wm-stagehand-v4 | ✓ 3/3 | 14.5s | 9,535 | $0.015 |
| WebMCP · Claude | ✓ 6/6 | 12.9s | 9,344 | $0.018 |
| cu-gemini | ✓ 3/3 | 71.0s | 92,575 | $0.075 |
| Computer use · GPT | ✓ 6/6 | 64.5s | 58,964 | $0.165 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 75.7s | 207,822 | $0.483 |
| Computer use · Claude | ◑ 5/6 | 102.2s | 163,831 | $0.399 |
| Page structure · Stagehand (a11y) | ✕ 0/6 | 118.7s | 105,323 | $0.226 |
id-7›
promptLog in at /login as admin@admin.com with password admin123, then record a payment of 100 against invoice 1/2026 with reference EVAL-100.
check {"probe":"payments","args":{"number":1},"assert":{"contains":{"number":1,"paymentStatus":"partially","payments":{"amount":100,"ref":"EVAL-100"}}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.2s | 4,706 | $0.005 |
| wm-gemini | ✓ 3/3 | 6.1s | 4,787 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 7.3s | 3,602 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 11.9s | 6,313 | $0.012 |
| WebMCP · Claude | ✓ 6/6 | 10.2s | 7,758 | $0.018 |
| cu-gemini | ✓ 3/3 | 61.0s | 57,850 | $0.047 |
| Computer use · GPT | ✓ 6/6 | 35.4s | 31,701 | $0.091 |
| Computer use · Claude | ✓ 6/6 | 84.1s | 128,790 | $0.313 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 64.5s | 191,880 | $0.501 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 76.5s | 99,468 | $0.167 |
id-8›
promptLog in at /login as admin@admin.com with password admin123, open this month's financial summary, and report the total invoiced amount.
check {"type":"answer","contains":["250"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 24.8s | 4,148 | $0.004 |
| wm-gemini | ✓ 3/3 | 38.3s | 4,495 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 7.3s | 3,422 | $0.005 |
| wm-stagehand-v4 | ✓ 3/3 | 8.7s | 6,163 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 9.4s | 6,187 | $0.012 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 108.1s | 156,400 | $0.425 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 96.6s | 79,620 | $0.165 |
| Computer use · GPT | ◑ 2/6 | 121.5s | 105,287 | $0.289 |
| Computer use · Claude | ◑ 1/6 | 110.8s | 120,852 | $0.340 |
| cu-gemini | ✕ 0/3 | 161.9s | 133,745 | $0.109 |
learnhouse
lh-1›
promptWhat courses does this school offer? Report every course name.
check {"type":"answer","contains":["evaluation foundations"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 5.7s | 3,298 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.7s | 3,138 | $0.003 |
| wm-gemini | ✓ 3/3 | 6.4s | 3,164 | $0.003 |
| WebMCP · GPT | ✓ 6/6 | 4.1s | 1,602 | $0.005 |
| WebMCP · Claude | ✓ 6/6 | 6.8s | 3,296 | $0.010 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 12.0s | 7,548 | $0.012 |
| Computer use · GPT | ✓ 6/6 | 12.2s | 11,521 | $0.019 |
| Computer use · Claude | ✓ 6/6 | 15.0s | 14,012 | $0.029 |
| cu-gemini | ✓ 3/3 | 41.0s | 45,024 | $0.036 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 23.9s | 42,967 | $0.094 |
lh-2›
promptOpen the Evaluation Foundations course and report the names of its chapters.
check {"type":"answer","contains":["applied practice"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 4.3s | 2,448 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.7s | 2,492 | $0.003 |
| wm-stagehand-v4 | ✓ 3/3 | 5.5s | 3,049 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 8.6s | 3,054 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 5.5s | 2,961 | $0.011 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 15.5s | 8,353 | $0.014 |
| cu-gemini | ✓ 3/3 | 24.9s | 19,524 | $0.016 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 25.5s | 42,716 | $0.081 |
| Computer use · GPT | ◑ 4/6 | 94.2s | 84,420 | $0.092 |
| Computer use · Claude | ◑ 3/6 | 50.6s | 47,773 | $0.090 |
lh-3›
promptWhat is the title of the first lesson in the Foundations chapter of the Evaluation Foundations course?
check {"type":"answer","contains":["what evaluation means"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 4.4s | 2,493 | $0.003 |
| wm-gemini | ✓ 3/3 | 4.3s | 2,514 | $0.003 |
| WebMCP · Claude | ✓ 6/6 | 4.7s | 3,058 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 5.5s | 3,061 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 5.9s | 1,957 | $0.007 |
| cu-gemini | ✓ 3/3 | 18.2s | 11,553 | $0.010 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 15.9s | 8,358 | $0.014 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 21.7s | 36,670 | $0.089 |
| Computer use · GPT | ✓ 6/6 | 62.2s | 53,250 | $0.149 |
| Computer use · Claude | ◑ 3/6 | 45.0s | 43,421 | $0.054 |
lh-4›
promptWhat tags does the Evaluation Foundations course carry?
check {"type":"answer","contains":["webmcp"]}
No runs recorded for this task yet.
lh-5›
promptSearch the catalog for a course about evaluation, open it, and report its description.
check {"type":"answer","contains":["deterministic"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 6.0s | 3,733 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.8s | 3,871 | $0.004 |
| wm-stagehand-v4 | ✓ 3/3 | 11.5s | 6,016 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 8.0s | 3,055 | $0.011 |
| WebMCP · Claude | ✓ 6/6 | 9.8s | 4,769 | $0.012 |
| cu-gemini | ✓ 3/3 | 19.1s | 15,512 | $0.013 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 17.8s | 18,011 | $0.020 |
| Computer use · GPT | ✓ 6/6 | 14.0s | 11,414 | $0.023 |
| Computer use · Claude | ✓ 6/6 | 34.4s | 37,174 | $0.090 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 23.9s | 50,595 | $0.105 |
lh-6›
promptDoes this school offer a course on machine learning? Answer with the single word AVAILABLE or MISSING.
check {"type":"answer","matches":"\\bmissing\\b","not_contains":["not missing","available or missing"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 6.2s | 3,187 | $0.003 |
| wm-gemini | ✓ 3/3 | 6.9s | 4,224 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 8.0s | 4,288 | $0.004 |
| WebMCP · Claude | ✓ 6/6 | 6.3s | 3,971 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 8.6s | 2,016 | $0.008 |
| Computer use · Claude | ✓ 6/6 | 16.1s | 16,793 | $0.030 |
| cu-gemini | ✓ 3/3 | 38.7s | 40,100 | $0.032 |
| Computer use · GPT | ✓ 6/6 | 16.8s | 11,588 | $0.039 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 25.7s | 56,069 | $0.101 |
| Page structure · Stagehand (a11y) | ◑ 5/6 | 22.6s | 12,568 | $0.022 |
lh-7›
promptSign in as admin@webmcp-eval.com with password WebmcpEvalAdmin!234, enroll in the Evaluation Foundations course, and report which lesson you should start with.
check {"type":"answer","contains":["what evaluation means"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 18.5s | 5,110 | $0.005 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 19.9s | 5,462 | $0.006 |
| wm-stagehand-v4 | ✓ 3/3 | 9.3s | 4,834 | $0.011 |
| WebMCP · GPT | ✓ 6/6 | 10.5s | 2,996 | $0.011 |
| cu-gemini | ✓ 3/3 | 56.0s | 36,062 | $0.029 |
| WebMCP · Claude | ✓ 6/6 | 14.5s | 5,660 | $0.032 |
| Computer use · GPT | ✓ 6/6 | 29.2s | 21,547 | $0.058 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 49.6s | 39,130 | $0.077 |
| Computer use · Claude | ✓ 6/6 | 69.4s | 90,780 | $0.194 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 38.1s | 89,828 | $0.218 |
lh-8›
promptSign in as admin@webmcp-eval.com with password WebmcpEvalAdmin!234, open the course dashboard, and report how many published lessons the Evaluation Foundations course contains in total.
check {"type":"answer","matches":"(lessons?|activities)[^.]{0,40}\\b(4|four)\\b|\\b(4|four)\\b[^.]{0,40}(lessons?|activities)"}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 18.6s | 6,035 | $0.006 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 19.7s | 6,512 | $0.007 |
| wm-stagehand-v4 | ✓ 3/3 | 13.1s | 4,955 | $0.012 |
| WebMCP · GPT | ✓ 6/6 | 17.9s | 4,779 | $0.019 |
| WebMCP · Claude | ✓ 6/6 | 17.6s | 5,528 | $0.025 |
| cu-gemini | ✓ 3/3 | 77.9s | 57,696 | $0.047 |
| Computer use · GPT | ✓ 6/6 | 41.8s | 36,885 | $0.107 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 43.3s | 90,441 | $0.229 |
| Computer use · Claude | ◑ 5/6 | 104.9s | 131,315 | $0.397 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 78.5s | 91,396 | $0.171 |
nextjs-starter-medusa
md-1›
promptHow many products does this store sell? Report the count and every product's name.
check {"type":"answer","contains":["t-shirt","sweatshirt","sweatpants","shorts"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.3s | 6,068 | $0.006 |
| wm-gemini | ✓ 3/3 | 7.0s | 6,091 | $0.006 |
| WebMCP · GPT | ✓ 6/6 | 4.9s | 2,733 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 7.8s | 2,990 | $0.018 |
| wm-stagehand-v4 | ✓ 3/3 | 7.0s | 2,975 | $0.022 |
| Computer use · GPT | ✓ 6/6 | 13.9s | 11,360 | $0.025 |
| Computer use · Claude | ✓ 6/6 | 37.1s | 42,934 | $0.111 |
| Browser Use · DOM + screenshot | ◑ 4/6 | 24.3s | 43,768 | $0.101 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 67.2s | 42,119 | $0.066 |
| cu-gemini | ◑ 1/3 | 58.0s | 77,389 | $0.062 |
md-2›
promptWhat is the store's shipping policy? How long does delivery take?
check {"type":"answer","contains":["3-5 business days"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.1s | 3,391 | $0.004 |
| wm-gemini | ✓ 3/3 | 7.8s | 3,582 | $0.004 |
| wm-stagehand-v4 | ✓ 3/3 | 6.6s | 2,964 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 5.3s | 1,792 | $0.007 |
| WebMCP · Claude | ✓ 6/6 | 8.8s | 2,967 | $0.008 |
| cu-gemini | ✓ 3/3 | 22.6s | 23,639 | $0.020 |
| Computer use · GPT | ✓ 6/6 | 35.1s | 36,596 | $0.092 |
| Computer use · Claude | ✓ 6/6 | 47.4s | 57,934 | $0.164 |
| Page structure · Stagehand (a11y) | ◑ 4/6 | 57.5s | 46,926 | $0.070 |
| Browser Use · DOM + screenshot | ◑ 4/6 | 38.1s | 88,281 | $0.212 |
md-3›
promptWhat is the store's returns policy?
check {"type":"answer","contains":["no questions asked"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 5.9s | 2,916 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.1s | 3,328 | $0.004 |
| wm-gemini | ✓ 3/3 | 8.9s | 4,850 | $0.005 |
| WebMCP · Claude | ✓ 6/6 | 8.0s | 3,276 | $0.005 |
| WebMCP · GPT | ✓ 6/6 | 5.2s | 1,587 | $0.006 |
| cu-gemini | ✓ 3/3 | 29.2s | 33,493 | $0.029 |
| Computer use · Claude | ◑ 4/6 | 52.9s | 62,041 | $0.172 |
| Page structure · Stagehand (a11y) | ◑ 3/6 | 73.2s | 46,493 | $0.085 |
| Computer use · GPT | ◑ 3/6 | 43.9s | 40,447 | $0.111 |
| Browser Use · DOM + screenshot | ◑ 3/6 | 38.8s | 68,907 | $0.195 |
md-4›
promptWhat sizes does the Medusa Sweatshirt come in? List them all.
check {"type":"answer","contains":["xl"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 6.1s | 3,655 | $0.003 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 6.4s | 3,720 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 4.6s | 1,746 | $0.006 |
| WebMCP · Claude | ✓ 6/6 | 6.2s | 2,809 | $0.009 |
| cu-gemini | ✓ 3/3 | 16.4s | 11,226 | $0.009 |
| wm-stagehand-v4 | ✓ 3/3 | 7.2s | 1,561 | $0.010 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 28.5s | 14,640 | $0.031 |
| Computer use · GPT | ✓ 6/6 | 17.5s | 15,962 | $0.036 |
| Computer use · Claude | ✓ 6/6 | 27.1s | 32,856 | $0.071 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 28.4s | 77,260 | $0.174 |
md-5›
promptAdd one Medusa T-Shirt, size M, color Black, to the shopping cart.
check {"probe":"database","assert":{"contains":{"carts":1}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 7.0s | 4,464 | $0.004 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 7.3s | 4,473 | $0.004 |
| WebMCP · GPT | ✓ 6/6 | 7.7s | 3,531 | $0.012 |
| WebMCP · Claude | ✓ 6/6 | 10.8s | 4,603 | $0.020 |
| wm-stagehand-v4 | ✓ 3/3 | 8.7s | 3,055 | $0.021 |
| cu-gemini | ✓ 3/3 | 41.5s | 32,424 | $0.027 |
| Computer use · GPT | ✓ 6/6 | 26.7s | 25,841 | $0.068 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 58.8s | 43,580 | $0.074 |
| Computer use · Claude | ✓ 6/6 | 45.5s | 64,127 | $0.162 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 42.2s | 111,888 | $0.255 |
md-6›
promptCheck the shopping cart and report what it currently holds. If it holds nothing, answer with the single word EMPTY.
check {"type":"answer","contains":["empty"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-gemini | ✓ 3/3 | 3.7s | 1,641 | $0.002 |
| wm-stagehand-v4 | ✓ 3/3 | 5.5s | 3,072 | $0.002 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 3.8s | 1,738 | $0.002 |
| WebMCP · Claude | ✓ 6/6 | 5.7s | 2,960 | $0.003 |
| cu-gemini | ✓ 3/3 | 6.7s | 3,802 | $0.003 |
| WebMCP · GPT | ✓ 6/6 | 5.0s | 1,440 | $0.005 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 8.4s | 3,820 | $0.006 |
| Computer use · GPT | ✓ 6/6 | 8.4s | 6,639 | $0.017 |
| Computer use · Claude | ✓ 6/6 | 10.1s | 8,482 | $0.017 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 16.8s | 29,164 | $0.078 |
md-7›
promptAdd two pairs of Medusa Sweatpants in size M and one pair of Medusa Shorts in size S to the cart, then report the cart's total item count.
check {"type":"answer","matches":"(cart|items?)[^.]{0,40}\\b(3|three)\\b|\\b(3|three)\\b[^.]{0,40}(cart|items?)|^\\W*((final )?answer:)?\\W*(3|three)\\b\\W*$","not_contains":["cart contains 2 items","cart has 2 items","cart holds 2 items","only 2 items","cart contains two items","cart has two items"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4-gemini | ✓ 3/3 | 13.1s | 11,262 | $0.010 |
| wm-gemini | ✓ 3/3 | 13.1s | 11,334 | $0.010 |
| WebMCP · GPT | ✓ 6/6 | 14.3s | 7,036 | $0.016 |
| wm-stagehand-v4 | ✓ 3/3 | 17.6s | 10,595 | $0.023 |
| WebMCP · Claude | ✓ 6/6 | 17.6s | 10,597 | $0.026 |
| cu-gemini | ✓ 3/3 | 49.3s | 59,788 | $0.049 |
| Computer use · GPT | ✓ 6/6 | 52.1s | 51,509 | $0.149 |
| Computer use · Claude | ✓ 6/6 | 97.6s | 121,663 | $0.383 |
| Page structure · Stagehand (a11y) | ◑ 5/6 | 90.6s | 93,220 | $0.147 |
| Browser Use · DOM + screenshot | ◑ 5/6 | 69.8s | 190,079 | $0.443 |
md-8›
promptBuy one Medusa T-Shirt, size L, color Black — add it to the cart and complete guest checkout as Jane Tester, jane.tester@example.test, 1 High Street, London, N1 9GU, United Kingdom, phone +44 20 7946 0000, using the store's test payment method. Report the order confirmation.
check {"probe":"database","assert":{"contains":{"orders":1}}}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| cu-gemini | ✓ 3/3 | 142.6s | 220,007 | $0.161 |
| Computer use · GPT | ✓ 6/6 | 83.2s | 90,264 | $0.282 |
| Browser Use · DOM + screenshot | ◑ 4/6 | 117.3s | 326,841 | $0.822 |
| Page structure · Stagehand (a11y) | ◑ 1/6 | 158.7s | 199,315 | $0.445 |
| wm-gemini | ✕ 0/3 | 16.4s | 10,792 | $0.013 |
| WebMCP · GPT | ✕ 0/6 | 11.6s | 5,741 | $0.016 |
| wm-stagehand-v4-gemini | ✕ 0/3 | 19.2s | 14,222 | $0.016 |
| wm-stagehand-v4 | ✕ 0/3 | 19.9s | 8,676 | $0.028 |
| WebMCP · Claude | ✕ 0/6 | 23.1s | 8,839 | $0.041 |
| Computer use · Claude | ✕ 0/6 | 148.7s | 253,253 | $0.521 |
md-9›
promptDoes the store sell a Medusa Jacket? Answer with the single word AVAILABLE or MISSING.
check {"type":"answer","matches":"\\bmissing\\b","not_contains":["not missing","available or missing"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| wm-stagehand-v4 | ✓ 3/3 | 5.8s | 2,929 | $0.003 |
| wm-gemini | ✓ 3/3 | 6.9s | 4,327 | $0.004 |
| WebMCP · Claude | ✓ 6/6 | 7.4s | 4,168 | $0.009 |
| WebMCP · GPT | ✓ 6/6 | 5.2s | 2,706 | $0.010 |
| cu-gemini | ✓ 3/3 | 24.6s | 27,323 | $0.022 |
| Computer use · GPT | ✓ 6/6 | 14.7s | 13,721 | $0.025 |
| Computer use · Claude | ✓ 6/6 | 23.1s | 27,862 | $0.061 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 30.7s | 42,442 | $0.081 |
| Page structure · Stagehand (a11y) | ◑ 5/6 | 26.8s | 24,508 | $0.030 |
| wm-stagehand-v4-gemini | ◑ 2/3 | 6.6s | 3,140 | $0.003 |
tailwind-nextjs-blog
blog-find-post›
promptFind the post whose title starts with "Introducing" and report its full title.
check {"type":"answer","contains_any":["tailwind nextjs starter blog","multi-part posts"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| WebMCP · GPT | ✓ 6/6 | 8.7s | 2,432 | $0.010 |
| wm-gemini | ✓ 3/3 | 12.6s | 9,670 | $0.010 |
| wm-stagehand-v4 | ✓ 3/3 | 10.3s | 1,010 | $0.013 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 16.0s | 13,143 | $0.013 |
| WebMCP · Claude | ✓ 6/6 | 9.1s | 2,756 | $0.014 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 16.1s | 8,906 | $0.023 |
| Computer use · Claude | ✓ 6/6 | 13.9s | 13,317 | $0.033 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 23.0s | 16,432 | $0.037 |
| cu-gemini | ✓ 3/3 | 52.0s | 65,061 | $0.055 |
| Computer use · GPT | ◑ 5/6 | 11.1s | 11,349 | $0.007 |
blog-read-author›
promptWho is the author shown on this About page?
check {"type":"answer","contains":["azimuth"]}
| method | result | time | tokens | cost |
|---|---|---|---|---|
| cu-gemini | ✓ 3/3 | 3.8s | 1,331 | $0.001 |
| wm-stagehand-v4-gemini | ✓ 3/3 | 4.6s | 1,629 | $0.002 |
| wm-gemini | ✓ 3/3 | 5.3s | 3,202 | $0.003 |
| wm-stagehand-v4 | ✓ 3/3 | 5.6s | 2,916 | $0.005 |
| Page structure · Stagehand (a11y) | ✓ 6/6 | 7.2s | 3,679 | $0.006 |
| Computer use · Claude | ✓ 6/6 | 6.1s | 5,381 | $0.008 |
| Computer use · GPT | ✓ 6/6 | 4.8s | 3,078 | $0.010 |
| WebMCP · Claude | ✓ 6/6 | 6.6s | 2,204 | $0.011 |
| Browser Use · DOM + screenshot | ✓ 6/6 | 13.6s | 19,496 | $0.047 |
| WebMCP · GPT | ◑ 4/6 | 10.5s | 4,778 | $0.010 |
Filter by site, interface or tier; click any column header to sort. Site type is from the benchmark's own site registry. Cost/tokens/time are medians across repeats; infrastructure-errored runs are excluded.
| Site | Type | Task | Tier | Interface | Method | Pass | Success | $/task | Tokens | Time |
|---|
* Consolidated from 42 source runs spanning 2026-08-16 to 2026-08-20; 10 of 16 configurations draw cells from more than one run. Every cell carries exactly 3 attempts from a single source — cells are never assembled across runs. Per-cell sources: PROVENANCE.md.
* Scorer corrections are applied by offline re-score (scripts/rescore.mjs) as a separate correction artifact that supersedes the affected cells; raw run directories are never rewritten. Probe predicates ran against live database state and cannot be re-scored offline.