A passing connection test proves your server speaks the protocol. It does not prove that a model handed your tools can work out which one to call, or that the answer it comes back with is right. An eval is a real task — “what is the total for order 1041?” — given to a model that has nothing but your tool names and descriptions to go on.
The most valuable thing it finds is the failure no schema check can see: every tool call succeeded, the arguments validated, and the answer was still wrong. Below is a finished run on a made-up orders server, replayed exactly as the product renders it.
Run your MCP server against real AI models — a full, end-to-end test, not a mock.
It detects your tools, writes the tasks a real user would ask, then turns a live model loose to complete each one using your server — just like a real client. You watch the whole agentic run: every tool call, where it gets stuck, and whether it lands the right answer. So you learn if AI can actually use your server — and which of 60+ models do it best.
Point it at any MCP server you can reach. It detects the tools, harvests real values from the read-only ones so the tasks are grounded in your actual data, writes the evals, then runs them. Tools that write are left out unless you opt in.
Run evals on your server