MCP Evals
A connection test proves your server speaks the protocol. An eval shows whether a model can work out which tool to call and come back with the right answer.
Any reachable MCP server · Read-only tools by default · No server? Use a public mock
Detect your tools
Connect any MCP server you can reach. The tool list is read, and real values are pulled from read-only tools.
Write real tasks
Questions a user would actually ask, like “what is the total for order 1041?”, grounded in your data.
Let an agent drive
A live model gets only your tool names and descriptions, and has to finish each task on its own.
Get a graded report
Every task passes or fails, with the full trace of tool calls behind it. Re-run it after each change.
Right calls, wrong answer
Every call succeeded and every argument validated, and the answer was still wrong. No schema check can see this.
The wrong tool
Two tools sound alike, and the agent reaches for the one your description made look right.
Wasted calls
A description leaves out that a search also matches email, so the agent burns five calls to find one customer.
Step through a real run, screen by screen, exactly as the product shows it.
Run your MCP server against real AI models — a full, end-to-end test, not a mock.
It detects your tools, writes the tasks a real user would ask, then turns a live model loose to complete each one using your server — just like a real client. You watch the whole agentic run: every tool call, where it gets stuck, and whether it lands the right answer. So you learn if AI can actually use your server — and which of 60+ models do it best.
Run this against your own server
Paste a URL. Tools are detected, tasks are written from your real data, and an agent runs them. Writing tools stay out unless you opt in.
A task a model has to complete using only your server’s tools. It checks whether an agent can pick the right tool from your descriptions and reach a correct answer, not whether the endpoint returns 200.
No. Models write, drive and grade the evals, so the same suite can come out differently next time. We only see what your server exposes, never your database or what the right answer should have been. Treat a run as evidence, not proof.
Not by default. Tools that write are left out of the suite unless you opt in.
A client tells you the protocol works. An eval tells you the server is usable. Try the inspector if the protocol is what you want to check.