# 97% of MCP Tool Descriptions Are Broken — How to Test Yours

> A peer-reviewed study of 856 tools across 103 MCP servers found that 97.1% of tool descriptions carry at least one defect. Here is what the research actually says, why the obvious fix backfires 1 time in 6, and how to find out whether an agent can use your server.

**Source:** https://mcpplaygroundonline.com/blog/mcp-tool-description-quality  
**Author:** Nikhil Tiwari  
**Published:** 2026-08-13  
**Updated:** 2026-08-13  
**Category:** Development  
**Reading time:** 12 min read

---

TL;DR

-   **97.1% of MCP tool descriptions carry at least one defect**, across 856 tools on 103 servers (arXiv 2602.14878).
-   **56% never clearly state what the tool does.** The description is the only thing the model reads before choosing.
-   **Rewriting descriptions helps — by a median of 5.85 percentage points** on task success, not the transformation the headline implies.
-   **It also costs you.** The same study measured a 67.46% increase in execution steps and a regression in 16.67% of cases.
-   **You cannot catch a bad description with a normal MCP test.** Every call returns 200 whether the model chose well or not.
-   **The only way to measure it is to make a model choose** between your tools and score whether it got there.

Table of contents

-   [What the research actually found](#research)
-   [The six ways an MCP tool description fails](#smells)
-   [The uncomfortable part: fixing descriptions has a cost](#cost)
-   [Three real descriptions, rewritten](#rewrites)
-   [Why your test suite cannot see this](#invisible)
-   [The annotations problem nobody talks about](#annotations)
-   [How to actually measure description quality](#measure)
-   [The checklist](#checklist)

Your MCP server works. Every tool returns valid JSON, the schemas validate, the handshake succeeds. And the agent still picks the wrong tool.

When that happens, the instinct is to blame the model. **The model is rarely the problem.** The problem is usually the one artefact you wrote by hand and never tested: the _tool description_.

A model does not see your code. It sees a name, a description, and a JSON Schema. That is the entire basis on which it decides whether to call your tool.

The [MCP specification](https://modelcontextprotocol.io/) is deliberately quiet on description quality — it defines the wire format, not the prose. That leaves the highest-impact field in your server completely unguarded.

In February 2026 a team from Queen's University and Huawei published an empirical study of **MCP tool description quality** across the public ecosystem. The numbers are worse than most people assume, and the fix is more complicated than the headline suggests.

This post covers what the study found, the six defect classes it identified, why naively expanding your descriptions can make things worse, and how to get a real measurement instead of a guess.

## What the research actually found

The paper is [Model Context Protocol (MCP) Tool Descriptions Are Smelly!](https://arxiv.org/abs/2602.14878) by Mohammed Mehedi Hasan, Hao Li, Gopi Krishnan Rajbahadur, Bram Adams and Ahmed E. Hassan. First submitted 16 February 2026, revised 31 May.

**They examined 856 tools spread across 103 MCP servers** and assessed each description against a defect taxonomy borrowed from code-smell research.

The headline result:

Finding

Figure

Descriptions with at least one smell

**97.1%**

Fail to clearly state their purpose

**56%**

Task success gain from augmented descriptions

+5.85pp (median)

Partial goal completion gain

+15.12%

Increase in execution steps

**+67.46%**

Cases where performance regressed

**16.67%**

Read the bottom three rows again. **That is the part almost every summary of this paper leaves out**, and it changes what you should do about it.

## The six ways an MCP tool description fails

Glama built a [Tool Definition Quality Score](https://glama.ai/blog/2026-04-03-tool-definition-quality-score-tdqs) on top of this research, scoring each tool 1–5 across six dimensions. They map cleanly onto the defect classes.

### 1\. Purpose clarity

The description does not say what the tool does. **This is the 56% case** — the single most common defect in the ecosystem.

A description like `Panel runner` is a noun phrase. It names a component in your architecture, not an action a user wants.

### 2\. Usage guidelines

**89% of descriptions never say when you should or should not use the tool.** This is what causes tool confusion on servers with overlapping capabilities.

If you ship `search_docs` and `get_page`, and neither says which to reach for first, the model guesses. Sometimes it guesses wrong.

### 3\. Behavioural transparency

Does the tool mutate anything? Is it idempotent? Does it hit a rate limit? **A model that does not know a call is destructive will treat it like a read.**

### 4\. Parameter semantics

The tool description is fine, but the parameter descriptions are empty. So the model knows to call `get_issue` and has no idea what format `issueId` takes.

This is the defect that produces [JSON-RPC -32602 invalid params errors](/blog/mcp-error-32602) at runtime.

### 5\. Conciseness

**Longer is not better.** Every character of every description sits in the context window on every single turn, for every tool you expose.

A 60-tool server with 400-word descriptions burns real budget before the user has typed anything. I wrote about this in [how MCP tools silently eat your context window](/blog/mcp-token-counter-optimize-context-window).

### 6\. Contextual completeness

The description assumes knowledge the model does not have. Internal entity names, project-specific jargon, an ID scheme documented only in your wiki.

## The uncomfortable part: fixing descriptions has a cost

Here is where most write-ups of this study go wrong. They quote 97%, quote the success gain, and conclude: write longer descriptions.

The paper does not support that conclusion. **Augmenting descriptions raised execution steps by 67.46%** — the agent did more work to get to the answer.

And in **16.67% of cases the augmented description made performance worse.** One time in six, the rewrite was a regression.

The real finding

Description quality genuinely matters, and you cannot fix it by writing more. The study's own ablations found that **compact variants preserved the reliability gain while cutting the token overhead**. Specific beats long.

So the actionable version is not "expand everything". It is: **say the right things, cut the rest, and measure whether it worked.**

That last clause is the one people skip. A 1-in-6 regression rate means you cannot assume your rewrite helped.

## Three real descriptions, rewritten

These patterns come from tools I have pointed evals at. The names are changed; the shapes are exactly what shows up in the wild.

### Scenario 1: the noun phrase

Before

```
{
  "name": "panel_runner",
  "description": "Panel runner."
}
```

**What goes wrong:** the model has no verb to match against the user's request. Asked to "survey our target customers", it never connects that to `panel_runner`.

After

```
{
  "name": "run_research_panel",
  "description": "Run a customer research panel against a target
audience and return per-question response breakdowns. Use for
new survey questions. To re-read a panel that already ran, use
get_panel_results instead."
}
```

**Lead with the verb the user would use.** Then name the sibling tool it gets confused with.

### Scenario 2: the overlapping pair

A docs server ships `search` and `fetch_page`. Both descriptions are accurate. Neither mentions the other.

The observable symptom: the agent calls `search`, gets a list of URLs, calls `fetch_page` on the first one, does not find the answer, and calls `search` again with a slightly different query.

**It gets there eventually.** It burns four calls doing what should take two. On a metered API, that is your bill.

The fix is one clause in each description: _"Returns ranked page URLs and snippets — call fetch\_page on a result to read the full page."_

### Scenario 3: the undocumented ID format

Before

```
{
  "name": "get_issue",
  "description": "Get an issue.",
  "inputSchema": {
    "type": "object",
    "properties": { "issueId": { "type": "string" } },
    "required": ["issueId"]
  }
}
```

The model has to invent an ID format. It tries `"1234"`. Your server wants `"PROJ-1234"`. You get a -32602 or an empty result.

**Add one line to the parameter and the failure disappears:** `"issueId": { "type": "string", "description": "Issue key in PROJ-123 form, as returned by list_issues." }`

That final clause — _as returned by list\_issues_ — tells the model where to get a valid value. It is the highest-leverage sentence you can write.

See how a model reads your tools

Connect any MCP server and watch a real model choose between your tools — no install, no sign-up.

[Test any MCP server free →](https://mcpplaygroundonline.com/mcp-test-server)

## Why your test suite cannot see this

This is the core of the problem. **A description defect is invisible to every form of testing that calls tools directly.**

Run your server through an inspector and every tool returns 200. Run your integration tests and they pass. Validate against the [2026-07-28 spec](/blog/build-mcp-server-2026-spec) and you are conformant.

None of that touches the question that matters: _given these descriptions, does a model pick the right tool?_

The reason is structural. **When you call a tool yourself, you have already made the choice the model has to make.** You skipped the only step that can fail.

The failure mode this produces is the nastiest one in MCP: **every call succeeds and the final answer is still wrong.** Nothing in your logs is red.

## The annotations problem nobody talks about

The [MCP spec](https://modelcontextprotocol.io/) lets a server declare `readOnlyHint`, `idempotentHint` and `destructiveHint` on each tool. These are part of how a model reasons about safety, and the [2026-07-28 spec release](https://blog.modelcontextprotocol.io/posts/2026-07-28/) leaned further into annotations as a risk vocabulary.

**Most servers in the wild declare none of them.** That is a description defect with real consequences, and it is one you can fix in an afternoon.

It matters more than it looks. When I built the eval engine behind [MCP Evals](/mcp-evals), missing annotations forced a hard design decision.

A functional eval calls tools with arguments _designed to succeed_. That is the whole point of it. So pointing one at an unannotated `delete_record` would really delete a record.

**The only safe posture is fail-closed:** an unannotated tool gets treated as destructive and excluded from a default run.

What this costs you

If your server ships no annotations, every automated tool that respects safety has to skip your tools — and so does any cautious agent framework. **Three booleans per tool buys you back that coverage.**

Note that "unknown" and "destructive" are not the same situation, even though both get excluded. An unannotated tool may well be read-only. You just cannot prove it, and guessing is not good enough when the call is designed to work.

## How to actually measure description quality

Given a 1-in-6 regression rate, you need a measurement, not an opinion. There are three practical levels.

### Level 1: read them against the six dimensions

Cheap and genuinely useful. Score each tool 1–5 on purpose clarity, usage guidelines, behavioural transparency, parameter semantics, conciseness and contextual completeness.

**Anything scoring 1 or 2 on purpose clarity is a certain problem.** Fix those first.

### Level 2: make a model choose, once

Connect your server to a real model, ask a question a user would actually ask, and watch which tool it reaches for. This is the fastest way to find a confused pair.

You can do this in a browser with [any of the models on MCP Playground](/blog/testing-mcp-servers-with-real-ai-models) without writing a harness.

### Level 3: run it as an eval suite

The difference between level 2 and level 3 is repeatability. **An eval is a task plus a scored outcome, run the same way every time your descriptions change.**

This is what the academic benchmarks measure at ecosystem scale. [MCP-Atlas](https://arxiv.org/abs/2602.00933) runs 1,000 human-authored tasks across 36 real servers and 220 tools, with a [public leaderboard](https://labs.scale.com/leaderboard/mcp_atlas). [MCP-Bench](https://openreview.net/forum?id=fe8mzHwMxN) exposes 250 tools across 28 servers to test cross-server orchestration.

Those tell you how good a _model_ is. You want the same method pointed at your _server_.

The signal to watch is not just pass or fail. It is _how many calls the model needed_.

A task that passes in two calls and later passes in five did not stay the same. **Call count is your description-quality metric**, and it moves before pass rate does.

That is why the eval engine I built fixes the call budget at generation time. If the budget moved per run, the comparison would be meaningless.

## The checklist

-   **Lead with a verb** the user would use, not a noun from your architecture.
-   **Say when not to use it** and name the sibling tool instead. This is the 89% gap.
-   **Describe every parameter**, and say which tool produces valid values for it.
-   **Declare your annotations.** All three booleans, on every tool.
-   **Cut anything the model does not need.** Compact variants performed as well as long ones.
-   **Re-measure after every rewrite.** One in six makes things worse.
-   **Track call count, not just pass rate.** It degrades first.

## Frequently asked questions

**Does the 97% figure mean my server is probably broken?+**

It means your descriptions almost certainly carry at least one defect from the taxonomy, which is not the same as being broken. Many defects are harmless in servers with few tools and no overlap. The risk rises sharply with tool count and with tools that do similar things.

**Should I just make every description longer?+**

No. The same study that found the 97% figure also measured a 67.46% increase in execution steps and a regression in 16.67% of cases after augmentation. Its ablations found compact variants held the reliability gain at lower token cost. Be specific rather than long.

**Can an MCP inspector detect a bad tool description?+**

Not really. An inspector calls tools directly, which means you have already made the choice the model would have to make. It can tell you a description field is empty, but it cannot tell you whether the wording leads a model to the right tool.

**What is the single highest-leverage fix?+**

Adding a clause to each ID-shaped parameter saying which tool returns valid values, for example "as returned by list\_issues". It removes the guessing that produces most -32602 errors, and it costs one sentence.

## Where this leaves you

The 97% number is real, peer-reviewed and worth taking seriously. **The naive response to it is not.**

Description quality is a measurable property of your server, it degrades quietly, and the only instrument that reads it is a model being made to choose. Call count tells you before pass rate does.

Score your tools against the six dimensions, declare your annotations, add the "as returned by" clause, then check that the rewrite actually helped.

Find out if an agent can use your server

Walk through a finished MCP eval run — tools detected, tasks written, a model made to complete each one.

[See MCP Evals →](https://mcpplaygroundonline.com/mcp-evals) [Test a server free →](https://mcpplaygroundonline.com/mcp-test-server)

New to the protocol? Start with [what the Model Context Protocol actually is](/blog/what-is-model-context-protocol), then [how to test an MCP server step by step](/blog/how-to-test-mcp-servers-step-by-step).

## Frequently asked questions

### Does the 97% figure mean my MCP server is probably broken?

It means your descriptions almost certainly carry at least one defect from the taxonomy, which is not the same as being broken. Many defects are harmless on servers with few tools and no overlap. The risk rises sharply with tool count and with tools that do similar things.

### Should I just make every MCP tool description longer?

No. The same study that produced the 97% figure also measured a 67.46% increase in execution steps and a regression in 16.67% of cases after augmentation. Its own ablations found that compact variants preserved the reliability gain at far lower token cost. Be specific rather than long.

### Can an MCP inspector detect a bad tool description?

Not really. An inspector calls tools directly, which means you have already made the choice the model would have to make. It can tell you a description field is empty, but it cannot tell you whether the wording leads a model to the right tool.

### What is the single highest-leverage tool description fix?

Adding a clause to each ID-shaped parameter saying which tool returns valid values, for example "as returned by list_issues". It removes the guessing that produces most JSON-RPC -32602 invalid params errors, and it costs one sentence.

### Why do missing tool annotations matter for MCP evals?

A functional eval calls tools with arguments designed to succeed, so an unannotated destructive tool would really run. Safe tooling has to treat an unannotated tool as destructive and skip it. Declaring readOnlyHint, idempotentHint and destructiveHint buys back that test coverage.


---

_Canonical page: https://mcpplaygroundonline.com/blog/mcp-tool-description-quality — MCP Playground (mcpplaygroundonline.com), the free browser-based tool for testing MCP servers and building AI agents._