MCP Server Performance Benchmarking: What to Measure, What the Numbers Say, and What Changed in the 2026-07-28 Spec
This review is based on official documentation, published benchmarks, and community reports — we did not run the tool hands-on. The 2026-07-28 spec rewrite made MCP stateless and added caching hints, obsoleting most pre-rewrite performance advice. The numbers that matter come from three independent benchmark sources (TM Dev Lab, Stacklok, mcp-find.org) plus the official spec’s own documentation on caching and client behavior. This guide covers what changed, what the numbers show, and how to build a trustworthy benchmarking setup.
Transport choice is the single largest performance variable in MCP
Transport selection, stdio versus Streamable HTTP, dominates every other performance decision, with order-of-magnitude gaps in published benchmarks. Stacklok’s benchmark found Streamable HTTP with a shared session pool sustaining 293-300 req/s at 200-1000 connections, while stdio at 20 concurrent connections managed only 0.64 req/s with 2 of 50 requests succeeding (Stacklok). The 2026-07-28 spec removed sessions entirely, which changes how you should interpret those numbers, but the transport gap remains the starting point for capacity planning.
Why stdio collapses under concurrency
The stdio cost model is simple: every fresh connection means a process spawn, runtime load, and module initialization. The stdio specification describes the client launching the server as a subprocess with newline-delimited JSON-RPC on stdout and logs on stderr. TrueFoundry’s engineering estimates put a warm local stdio call at roughly 0.3-1 ms and a cold start at 250-400 ms (~600 ms+ at the tail) (TrueFoundry).
The Stacklok numbers make the failure concrete: 0.64 req/s, a 20-second average response time, a 30-second maximum, timeouts, and connection resets (Stacklok). That is an architectural ceiling, not a tuning problem: a subprocess-per-connection model cannot scale without a pooler or a transport swap.
What the 2026-07-28 spec changed about transports
The 2026-07-28 changelog removed protocol-level sessions and the Mcp-Session-Id header from Streamable HTTP. The basic overview states MCP is explicitly stateless: “all the information needed to process a request is contained in the request itself,” and servers MUST NOT rely on prior requests. An open connection or stdio process is not a session.
Stacklok’s unique-session finding (33-36 vs ~293-300 req/s shared) is now moot: sessions no longer exist. State that lived in session handles moves into request-scoped handles or client-supplied context. The stateless model also removes session affinity, a prerequisite for horizontal scaling. The transports overview notes request metadata is mirrored into HTTP headers (MCP-Protocol-Version, etc.) so gateways can route traffic without parsing bodies.
The 2026-07-28 spec made MCP stateless — and that’s a performance win
The 2026-07-28 revision removed the initialize/notifications/initialized handshake, eliminated SSE resumability, and made every request self-contained, simplifying load balancing and removing session-affinity constraints (we covered the rewrite in depth in our stateless spec rewrite breakdown). Every request now carries protocol version and client capabilities in _meta fields, and version mismatches return UnsupportedProtocolVersionError (changelog). For performance, the pre-request handshake round-trips are gone entirely.
What the handshake removal means for latency
Previously a client had to complete an initialize exchange first, two round-trips on a fresh connection. With the handshake removed, your first request is the one you care about. The streamable-http specification now describes every JSON-RPC message as one HTTP POST to a single MCP endpoint, with the body a single request or notification, no batches or client-sent responses.
For short-lived connections the old handshake doubled your latency; now it doesn’t. For long-lived connections the savings are smaller, but the simplified lifecycle is a maintenance win regardless.
SSE resumability removal: don’t build on a broken stream
The changelog is explicit: SSE resumability is gone. No Last-Event-ID, no event IDs. A broken response stream loses the in-flight request, and clients MUST re-issue it with a new request ID (changelog, streamable-http). The GET stream endpoint is also gone; change notifications now live on a single long-lived subscriptions/listen POST-response stream.
If you relied on SSE resumability to survive flaky intermediaries, you need a client-side retry strategy instead. A mid-stream failure costs the full request latency plus the retry, so budget for it. The correctness issue becomes a performance problem under load.
Published benchmarks: TM Dev Lab’s multi-language comparison
TM Dev Lab’s benchmark is the most rigorous multi-language MCP comparison to date, testing four servers over Streamable HTTP with 3.9M total requests, 50 concurrent VUs, and 5-minute sustained runs across 3 rounds on February 10, 2026 (TM Dev Lab, Java SDK blog). It used Docker (1 CPU, 1 GB RAM per server) on Ubuntu 24.04.3 with k6 and a 0% error rate across all implementations.
Table A: TM Dev Lab multi-language benchmark results
| Server (SDK) | Avg latency | Throughput | CPU eff. (RPS/CPU%) | Memory |
|---|---|---|---|---|
| Java (Spring Boot 4.0.0 + Spring AI 2.0.0-M2, Java 21) | 0.835 ms | 1,624 RPS | 57.2 | ~220 MB |
| Go (official SDK v1.2.0) | 0.855 ms | 1,624 RPS | 50.4 | ~18 MB |
| Node.js (@modelcontextprotocol/sdk 1.26.0) | 10.66 ms | 559 RPS | 5.7 | — |
| Python (FastMCP 2.12.0+, single-worker uvicorn) | 26.45 ms | 292 RPS | 3.2 | — |
Reading the table critically
Java averaged 0.835 ms versus 10.66 ms (Node.js) and 26.45 ms (Python), roughly 12.8x and 31.7x faster, with 2.9x and 5.6x throughput advantages respectively (Java SDK blog). The methodology notes matter: Python ran single-worker uvicorn, capping concurrency; Node.js used per-request server instantiation to mitigate CVE-2026-25536, adding overhead per request; no JVM tuning was applied; Docker’s bridge network kept latency at localhost level.
The honest takeaway: Java and Go are in a different performance class here, but the Node.js 10.66 ms figure is explicitly not representative of tuned production Node, per the source’s methodology notes. A v2 benchmark expands to 15 implementations and 39.9M requests (v2 page).
Published benchmarks: Stacklok’s transport comparison
Stacklok’s benchmark isolated the transport variable, running the same server code over stdio, HTTP+SSE, and Streamable HTTP, finding a ~460x throughput gap between worst and best. The echo tool was unrealistically fast and the local cluster plus port forwarding meant near-zero network latency, so absolute numbers are optimistic but relative gaps are the signal.
Table B: Stacklok transport benchmark results
| Transport | Concurrency | Throughput | Avg response time | Notes |
|---|---|---|---|---|
| stdio | 20 concurrent | 0.64 req/s | 20.01 s (max 30.02 s) | 2 of 50 requests succeeded; timeouts + resets |
| HTTP+SSE (deprecated) | 10 RPS offered | 7.23 req/s | 18.56 ms | 100% success of sent requests |
| HTTP+SSE (deprecated) | sustained | 29.87 req/s | 564.57 ms | 100% success |
| Streamable HTTP, shared pool | 20 conns | 48.40 req/s | — | 100% success |
| Streamable HTTP, shared pool | 50 conns | 96.78 req/s | 6.68 ms | 100% success |
| Streamable HTTP, shared pool | 200-1000 conns | ~293-300 req/s | 622 ms → 3.09 s | 100% success |
| Streamable HTTP, unique session/request | — | 33-36 req/s | — | ~10x gap vs shared sessions |
What the Stacklok numbers actually tell you
The stdio result is catastrophic: 0.64 req/s with a 20-second average response time and only 2 of 50 requests succeeding (Stacklok). That’s the subprocess-per-connection ceiling in action. The HTTP+SSE transport (now deprecated) managed 29.87 req/s sustained, better but still an order of magnitude below Streamable HTTP’s ~300 req/s.
The shared-pool versus unique-session gap (~300 vs ~36 req/s, ~10x) was the pre-2026-07-28 story. With the spec now stateless that advice is obsolete, but the lesson survives: connection reuse and pooling remain critical. The streamable-http spec recommends sending X-Accel-Buffering: no to stop proxies buffering SSE events, plus SSE comment-line keep-alives (:\r\n), both cheap wins.
Community benchmark: mcp-find.org on local hardware
mcp-find.org’s benchmark ran on a MacBook Pro M3 with 1,000 calls and 50 warmup calls across desktop hosts, a realistic single-user picture (mcp-find). The headline: stdio P50 latency was 145 ms, HTTP/SSE was 108 ms, and WebSocket (custom, not standard MCP) was 102 ms. Stdio first-call spawn overhead was 80-150 ms per session, the cold-start tax even on fast hardware.
The serialization finding: MessagePack was ~40% faster than JSON only for payloads larger than 10 KB, and negligible (~0.1 ms) below 1 KB. For small JSON arguments and results, switching serialization buys you nothing; for multi-megabyte payloads it’s a real win.
The caching numbers: an 80% hit rate produced a ~5x median improvement (2,100 ms → 431 ms on simulated 50 ms-API tools), and a realistic 50-60% hit rate still yielded 2-3x. Connection pooling saved ~30-100 ms per uncached call by avoiding TLS handshakes.
Caching hints and the token economics of tool discovery
The 2026-07-28 spec added first-class caching hints (SEP-2549) that target MCP’s biggest hidden cost: token consumption from tool discovery. The caching specification defines ttlMs (freshness hint in milliseconds; 0 = immediately stale; absent = default 0) and cacheScope (public/private) on results from tools/list, prompts/list, resources/list, resources/templates/list, resources/read, and server/discover. A list_changed notification invalidates cached results immediately.
The official client best practices doc is blunt about the cost of naive discovery: ~150,000 tokens consumed by tool definitions upfront versus ~2,000 tokens with progressive discovery (client best practices). That’s a 75x difference in token spend before a single tool call executes.
Cache invalidation and prompt cache preservation
Changing the tools array mid-conversation invalidates provider prompt caches. The fix: append tools after the cache breakpoint, or use a stable call_tool({name, args}) meta-tool to preserve cache hits. Cache tools/list results host-side, refresh on list_changed, and treat server disconnection as a conversation boundary (client best practices).
The spec also made deterministic tools/list ordering a SHOULD, explicitly to “improve LLM prompt cache hit rates” (changelog). If your server returns tools in stable order, clients can cache the serialized list and avoid re-sending identical tokens, a zero-cost optimization most servers haven’t implemented.
What to measure: a practical benchmarking checklist
Designing a useful MCP benchmark requires controlling for transport, workload shape, concurrency, and cold versus warm state, the same variables TM Dev Lab isolated in its February 2026 runs (TM Dev Lab). The published benchmarks agree on what matters most, even when their absolute numbers differ.
Workload shape: sequential vs parallel, cold vs warm, payload size
The TM Dev Lab benchmark used four distinct tools: CPU-bound calculate_fibonacci, I/O-bound fetch_external_data, JSON transform, and latency simulation (TM Dev Lab). You need the same mix; a pure echo tool (like Stacklok’s) shows transport overhead but not real handler cost.
Measure cold-start latency separately from steady-state. The mcp-find data shows 80-150 ms spawn overhead per stdio session (mcp-find), and TrueFoundry estimates 250-400 ms for cold stdio starts (TrueFoundry).
Tooling: k6 and the MCP Inspector
k6, the tool TM Dev Lab used, covers HTTP/1.1 and HTTP/2 load testing in its documentation, including SSE stream handling. The MCP Inspector CLI lets you inspect individual requests and responses, invaluable for verifying your server actually sends ttlMs and cacheScope hints. For CI smoke tests, pair it with the patterns in our MCP server testing guide.
The batching trap
MCP does not support JSON-RPC batch requests. Conformance test #378 requires servers to reject batches, and the python-sdk issue #934 confirms this is settled. Parallelism means concurrent requests, not batches: design your load test for concurrency, not batch size.
Engineering estimates: TrueFoundry’s transport cost model
TrueFoundry’s analysis provides a useful latency budget for planning, though its numbers are labeled estimates, not measured telemetry (TrueFoundry). Their numbers: stdio warm local call ~0.3-1 ms (total ~2-3 ms), stdio cold start ~250-400 ms (~600 ms+ at the tail), Streamable HTTP same-DC ~5-10 ms (total ~15-25 ms), and cross-region ~50-120 ms (200 ms+), RTT-dominated.
The operational math: 50 developers × 8 servers = ~400 stdio processes. Their estimate that migrating stdio→HTTP is “a transport swap, not a rewrite” (~5-line patch) fits the spec’s stateless model. Their gateway product claims ~3-4 ms added latency and 350+ RPS on 1 vCPU, a vendor claim rather than an independent benchmark.
The Bottom Line
The 2026-07-28 rewrite is MCP’s biggest performance event: it made the protocol stateless, removed session affinity, eliminated SSE resumability, and added caching hints that cut token consumption by 75x. Transport choice dominates: stdio collapsed to 0.64 req/s in Stacklok’s test while Streamable HTTP sustained hundreds of RPS with shared pools. Runtime choice matters too, but every number is environment-specific. Treat published figures as rough priors, not gospel: benchmark your own workload, measure cold and warm paths separately, and implement the caching hints first.
FAQ
Does the 2026-07-28 spec change how I should pool connections?
Yes. Sessions were removed, so there’s no session affinity to maintain. Any request can go to any server replica. Connection pooling still matters for TLS handshake avoidance (30-100 ms per call per mcp-find), but you no longer need to pin a client to a specific server.
Is stdio ever the right choice for production?
For single-user local development, yes, stdio’s ~0.3-1 ms warm latency beats HTTP’s ~5-10 ms overhead. For anything concurrent or remote, no. Stacklok’s benchmark showed stdio collapsing to 0.64 req/s at 20 concurrent connections. If you need to serve multiple clients, use Streamable HTTP.
What’s the actual token cost of tool discovery?
The official client docs cite ~150,000 tokens for naive full tool-definition loading versus ~2,000 tokens with progressive discovery, a 75x difference. The new ttlMs and cacheScope hints let you cache tools/list results host-side and refresh only when a list_changed notification arrives.
Why is Python so slow in the TM Dev Lab benchmark?
The benchmark ran Python with a single-worker uvicorn, which caps concurrency. The 26.45 ms average latency and 292 RPS reflect that configuration, not Python’s ceiling, and the benchmark’s own methodology notes flag this. Don’t extrapolate the numbers to tuned production Python.
Does MCP support JSON-RPC batching for parallel calls?
No. MCP does not support JSON-RPC batch requests: conformance test #378 requires servers to reject them, and the python-sdk issue #934 confirms the behavior is intentional. Parallelism means concurrent HTTP requests, not batched payloads, so design your client and load tests around concurrency.
What’s the fastest way to improve MCP performance without changing language?
Implement the caching hints and deterministic tools/list ordering first. An 80% cache hit rate gave ~5x median improvement in mcp-find’s benchmark. Then tune transport: ensure X-Accel-Buffering: no is set and SSE keep-alives are sent. These are zero-code or near-zero-code changes.
How This Guide Was Built
This review synthesizes primary sources: the official 2026-07-28 MCP specification and changelog, the official client best practices docs, three independent published benchmarks (TM Dev Lab, Stacklok, mcp-find.org), and TrueFoundry’s engineering estimates. We did not run any benchmarks hands-on; all figures come from the cited sources with their environmental caveats preserved. Deeper reading: the MCP specification changelog, our MCP transport deep dive, the stateless spec rewrite breakdown, prompt cache hit rate engineering, and our production MCP server deployment patterns. For hands-on debugging, the MCP Inspector CLI is the official tool.
← Back to all posts
