# MCP Agent Studio — Run Evals & Test MCP Servers with AI Models > Run any hosted or remote MCP server against 60+ AI models in your browser — Claude, GPT-5, Gemini, DeepSeek, Grok, Qwen and more. Auto-generate evals that prove an agent can answer using your tools, compare models side by side, and save reusable agents. No install. **Source:** https://mcpplaygroundonline.com/mcp-agent-studio --- ## What it does Agent Studio is a browser-based MCP client with a model picker. You paste an MCP server URL, choose a model, and chat. Every tool call is shown inline with its full JSON input and output, so you can see exactly which tool the model picked, what arguments it constructed, and what came back. Nothing is installed. There is no local Node process, no `npx`, and no config file — the connection is made from the browser session. ## Why the model matters The same MCP server behaves differently depending on which model is driving it. Anthropic's API is the only one that speaks MCP natively; every other provider reaches MCP tools through a translation layer. That means tool-selection quality, argument construction, and error recovery all vary by model — and a server that works perfectly in Claude Desktop can fail against GPT or Gemini for reasons that have nothing to do with your code. Agent Studio exists to make that difference visible. Run the same prompt against 60+ models and compare which one picks the right tool. **Model families available:** - Claude (Anthropic) - GPT (OpenAI) - Gemini and Gemma (Google) - DeepSeek - Grok (xAI) - Qwen (Alibaba) - GLM (Z.AI) - Kimi (Moonshot) - MiniMax - Nemotron (NVIDIA) - Mistral ## MCP evals Evals answer a different question than a tester does. A tester tells you the server responds; an eval tells you an *agent can actually accomplish something* with it. Agent Studio reads every tool on your server, writes a test suite from the real schemas, and runs it. The suite has two halves: - **Direct checks — decided deterministically by code.** Output-schema conformance, JSON-RPC error codes, pagination behaviour, result caps, idempotency. No model judges these; they pass or fail on the response. - **Agentic tasks — driven by one model, graded by another.** A realistic question is posed to an agent connected to your server; a second model grades whether the answer is actually supported by the tool output. The result is a pass/fail report per tool, with evidence attached to each result. Re-run the same suite after a change to see what regressed. Full walkthrough: [https://mcpplaygroundonline.com/mcp-evals](https://mcpplaygroundonline.com/mcp-evals) ## What people build with it - Run an eval suite against your own MCP server and find the tool an agent cannot use, before your users do - Chat with a Postgres or Supabase database in plain English, without writing SQL - Triage Sentry errors, find the offending commit, and draft a fix - Run Amazon Ads, Meta Ads or LinkedIn Ads campaigns from chat - Manage Linear, Jira or Asana sprints conversationally - Query HubSpot or Salesforce with natural language - Inspect a Shopify store, Stripe billing or Vercel deployments without leaving the chat - Compare Claude vs GPT-5 vs Gemini on the same MCP task to find the cheapest model that still succeeds ## Getting started 1. Open [https://mcpplaygroundonline.com/mcp-agent-studio](https://mcpplaygroundonline.com/mcp-agent-studio). 2. Paste your MCP server URL, or start from a [template](https://mcpplaygroundonline.com/templates) if you are connecting to a service like Notion, GitHub or Stripe. 3. Pick a model and start chatting. Sign-up grants free credits. No MCP server of your own yet? Use one of the [hosted mock servers](https://mcpplaygroundonline.com/mock-mcp-servers) — they are public URLs that need no sign-up. ## Frequently asked questions ### What is MCP Agent Studio? MCP Agent Studio is a browser-based AI agent that connects to any Model Context Protocol (MCP) server and lets you query your data in plain English using 60+ AI models including Claude, GPT-5, Gemini, DeepSeek, Mistral, Grok, Qwen, NVIDIA Nemotron, and more. It also runs evals against your server — an auto-generated test suite that proves an AI agent can actually answer questions using your tools. Sign up for free — no credit card required. ### What are MCP evals and how do they work? An eval is a test that checks whether an AI agent can complete a real task using your MCP server. MCP Agent Studio connects to your server, reads every tool and its schema, and writes an eval suite for you. Running the suite has four phases: connect (handshake and tool discovery), harvest (pull real values from read-only tools so the tasks use genuine data), run (a driver model works your tools like a real client would), and score (grade the answers). Direct evals are decided deterministically by code — schema conformance, JSON-RPC error codes, pagination termination, result caps, idempotency — so a failure is a confirmed defect, not a model opinion. Agentic evals send a plain-English task through the driver model and a scoring model grades the final answer. ### How do I test if my MCP server actually works with an AI agent? Paste your server URL into the Evals workspace and run a suite. Instead of eyeballing a tool list in an inspector, you get a pass/fail report per tool with the evidence behind each verdict: the arguments that were sent, the raw response, and which check tripped. Typical failures it catches are tool descriptions too vague for the model to pick the right tool, list endpoints that ignore their own limit parameter, missing-record cases that return null instead of an error, and agent loops that burn dozens of calls without answering. ### Do evals call destructive tools on my server? No. Tools are classified before anything runs — read-only, idempotent, unknown, or destructive. Anything that could mutate or delete your data is excluded from the run and reported as "untestable" rather than being quietly called, and you review and approve the tool list before the suite is generated. Mutation checks are proved indirectly with a read-only witness tool instead. ### Can I re-run the same eval suite after I change my server? Yes. Suites are saved against your organization and versioned, so you can replay one after every schema change and compare the new report against the last run. Writing a suite is charged once per version; replaying it only costs the run, which is why regression checks stay cheap. You can also add your own assertions in plain English alongside the generated ones. ### Do I need to write code to use MCP Agent Studio? No. MCP Agent Studio is a no-code interface. You paste your MCP server URL, pick an AI model, and start chatting. The agent discovers the available tools automatically and calls them for you as needed. ### Which AI models can I use with MCP Agent Studio? You can use 60+ models from every major provider, including Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.4, GPT-5.5, Gemini 3.1 Pro, DeepSeek V4, Mistral Small 2603, Grok 4.6, Qwen 3, NVIDIA Nemotron Super 120B, Google Gemma 4, Z.AI GLM 4.5 Air, and more. You can switch models mid-conversation to compare responses on the same task. ### What is an MCP server and do I need one? An MCP server is a lightweight service that exposes tools (functions, database queries, API calls) over the Model Context Protocol so any AI agent can use them. Popular public MCP servers include GitHub, Supabase, Notion, Slack, Stripe, Postgres, Linear, and Playwright. You can also connect your own custom MCP server over HTTP, SSE, or Streamable HTTP. ### How does pricing work? MCP Agent Studio is free to sign up — no credit card required. Running AI models costs credits, which you can purchase in packs from your Settings page. New accounts may receive a small starter credit to try the product. Credit costs vary by model; check the Settings › Credits page for current pricing. ### Can I query my own database with MCP Agent Studio? Yes. Connect the official Postgres or Supabase MCP server (or any MCP server that wraps your database) and ask questions in plain English: "Show me my top 10 customers by revenue this month." The AI generates the SQL, runs it through the MCP server, and returns the results with a natural-language summary. ### How is this different from ChatGPT or Claude Desktop? Claude Desktop only works with Claude. ChatGPT only works with OpenAI models. MCP Agent Studio works with any model from any provider — swap between Claude, GPT, Gemini, DeepSeek, and Grok on the same MCP server without re-configuring anything. It also runs entirely in your browser with no local installation. ### Can I run an MCP server with GPT-5? Yes. MCP Agent Studio connects any MCP server (HTTP, SSE, or Streamable HTTP) to GPT-5.4, GPT-5.5, GPT-5 mini, and the rest of the GPT-5 family. Paste your server URL, pick the model, and every tool call streams live — no OpenAI SDK or glue code required. ### Which AI model is best for MCP tool calling? It depends on your server. Claude Sonnet 4.6 and Opus 4.6 are the strongest default tool-callers; GPT-5.4 leads on structured outputs; Gemini 3.1 Pro handles long tool lists; DeepSeek V4 and GLM 5.1 are the budget picks. Compare Mode in MCP Agent Studio scores all of them on your actual server. ### Is there a hosted MCP server URL I can test with? Yes. Official hosted endpoints include Notion (https://mcp.notion.com/mcp), Stripe (https://mcp.stripe.com/), GitHub (https://api.githubcopilot.com/mcp/), Linear (https://mcp.linear.app/sse), and Sentry (https://mcp.sentry.dev/mcp). Paste any of them into MCP Agent Studio, or deploy your own sandboxed hosted MCP server with a live URL in under a minute. ### How do I connect to a remote or hosted MCP server? Paste the server URL into MCP Agent Studio, add an auth token if the server requires one, and click connect — the agent lists the tools automatically. Remote HTTP, SSE, and Streamable HTTP transports are all supported, entirely in the browser with nothing to install. ### Can I use the GitHub MCP server with Claude? Yes — and with GPT-5, Gemini, Grok, or any of 60+ models. MCP Agent Studio connects the official GitHub MCP server (https://api.githubcopilot.com/mcp/) to whichever model you pick, so you can review PRs or triage issues with Claude and re-run the same prompt on GPT-5 to compare. ### Does MCP work with Gemini? Yes. The Model Context Protocol is model-agnostic, and MCP Agent Studio acts as a Gemini MCP client: connect any server and run it with Gemini 3.1 Pro or Gemini 3 Flash. Gemini Flash is one of the cheapest models that still calls MCP tools reliably. ### Is my data secure? Your MCP server credentials stay in your browser session and are never stored on our servers. Conversations are scoped to your Supabase-authenticated organization. We never train on your data, and you can delete your history at any time from Settings. ### Can I compare multiple AI models on the same task? Yes — this is one of the core reasons people use MCP Agent Studio. Run the same prompt against Claude, GPT-5, and Gemini side by side, inspect every tool call in the live trace panel, and find the cheapest model that still gives great results. Teams routinely cut inference costs 5–10× this way. ### What MCP servers work with Agent Studio? Any MCP server that speaks HTTP, SSE, or Streamable HTTP. That includes official servers for GitHub, Supabase, Postgres, Notion, Slack, Stripe, Linear, Playwright, Filesystem, Memory, Sequential Thinking, and hundreds of community servers. You can also point it at your own custom server deployed on Cloudflare Workers, Vercel, Fly, or anywhere else. ### Do you have pre-built templates I can use? Yes. MCP Agent Studio ships with ready-to-run templates covering Notion, GitHub, Amazon Ads, Jira, Confluence, Sentry, HubSpot, Stripe, Linear, Slack, Supabase, Vercel, Shopify, Cloudflare, Zapier, and more — including multi-server combos like GitHub + Vercel + Sentry and Amazon Ads + Stripe for blended ROAS. Pick a template, add your access token, and start chatting in seconds. Browse all templates at /templates. ### Can I save an agent configuration and reuse it? Yes. Once you set up a model, one or more MCP servers, and a system prompt, you can save the whole configuration as a named agent with one click. Open it again any time from "My Agents" — your servers, tools and model selection are all restored. Agents can be private or shared with your team. ### Can I export my agent as code? Export is coming soon. Save your agent and open Export in the studio toolbar to preview a Next.js App Router layout or a Python starter (MCP wiring, prompts, .env.example, deploy targets). Downloadable ZIP packs and one-click deploy will ship in a follow-up release; until then you can explore the preview and recreate wiring manually if you self-host. --- _Canonical page: https://mcpplaygroundonline.com/mcp-agent-studio_ _MCP Playground (mcpplaygroundonline.com) is a free, browser-based toolkit for the Model Context Protocol: test any remote MCP server, run evals that prove an AI agent can use its tools, build reusable agents across 60+ models, and audit servers for security issues. No install, no local Node, free to start._