Building High-Performance MCP Servers with Streaming Responses and Async Patterns

The Model Context Protocol (MCP) has become the standard for integrating AI agents with external tools and data sources. As agent deployments scale, a critical challenge emerges: how do you efficiently serve large responses—think complete file contents, search results spanning thousands of items, or real-time data streams—without blocking the client or overwhelming memory?

Traditional request-response patterns become a bottleneck when responses exceed typical request timeouts or memory constraints. This is where streaming responses and async-first architecture patterns become essential. Let's explore how to build MCP servers that handle large payloads gracefully and maintain responsiveness under load.

Why Streaming Matters for MCP Servers

An MCP server that loads an entire response into memory before sending it faces several problems:

Memory Pressure: A server returning a 50MB file or 100k search results must allocate that memory upfront. With multiple concurrent clients, this quickly exhausts available resources.

Timeout Risk: Network transmissions take time. If you batch-load everything before sending, the client may timeout waiting for the first byte. Streaming begins transmission immediately, reducing perceived latency.

Client Usability: Clients like Claude can begin processing results immediately instead of waiting for completion. Agents can stream token results to users in real-time, improving interactivity.

Resource Fairness: In multi-tenant deployments, one client's large request shouldn't starve others. Streaming with backpressure ensures fair resource allocation.

Streaming Architectures in MCP

MCP doesn't mandate a specific streaming format—the protocol is transport-agnostic. However, practical implementations typically use Server-Sent Events (SSE) or WebSocket multiplexing for streaming.

For a file-reading tool, instead of:

{
  "type": "resource",
  "content": "[entire 10MB file]"
}

You'd stream chunks:

data: {"type": "resource_chunk", "offset": 0, "data": "[first 64KB]"}
data: {"type": "resource_chunk", "offset": 65536, "data": "[next 64KB]"}
data: {"type": "resource_complete", "total": 10485760}

This approach decouples payload size from response time and lets clients consume data at their own pace.

Async-First Request Handling

Node.js and TypeScript enable async-first patterns through promises and async/await. For MCP servers, this means:

Non-blocking I/O: Never perform blocking operations (file reads, database queries, network calls) on the main thread. Always use async primitives.

// ❌ Blocking
const data = fs.readFileSync('/large-file.txt');

// ✅ Async
const data = await fs.promises.readFile('/large-file.txt');

Concurrency Control: Use worker pools, semaphores, or queues to prevent resource exhaustion. A naive approach that spawns unlimited concurrent operations will crash under high load.

// Limit concurrent database queries to 10
const pool = new PromisePool(10);

async function queryMany(ids: string[]) {
  return Promise.all(ids.map(id => 
    pool.exec(() => db.query('SELECT * FROM items WHERE id = ?', [id]))
  ));
}

Request Prioritization: Distinguish between quick operations (metadata lookups) and expensive ones (full-text search). Route them to different execution paths or thread pools.

Implementing Streaming in Practice

Here's a practical streaming pattern for an MCP server using Node.js streams:

import { Readable } from 'stream';

async function* streamLargeFile(filepath: string) {
  const stream = fs.createReadStream(filepath, { highWaterMark: 65536 });
  
  for await (const chunk of stream) {
    yield {
      type: 'resource_chunk',
      data: chunk.toString('base64'),
      size: chunk.length
    };
  }
}

// In your tool handler:
async function handleFileTool(filename: string) {
  const chunks = [];
  
  for await (const chunk of streamLargeFile(filename)) {
    // Send chunk to client immediately
    mcp.sendResult(chunk);
    
    // Track memory usage
    if (process.memoryUsage().heapUsed > MAX_HEAP) {
      await gc(); // Force garbage collection if needed
    }
  }
  
  return { status: 'complete' };
}

This pattern avoids loading the entire file into memory. Chunks are processed, streamed, and released one at a time.

Backpressure and Flow Control

Real-world systems must handle mismatched producer/consumer speeds. If your server generates data faster than the network can transmit it, buffering will consume memory indefinitely.

Implement backpressure: pause data generation if the output buffer fills.

const Readable = require('stream').Readable;

class ManagedSource extends Readable {
  constructor(private generator: AsyncGenerator) {
    super({ objectMode: true });
  }

  async _read() {
    try {
      const { value, done } = await this.generator.next();
      if (done) {
        this.push(null); // Signal EOF
      } else {
        // push() returns false if buffer is full—pause generation
        if (!this.push(value)) {
          // Wait for drain event before resuming
          await new Promise(resolve => this.once('drain', resolve));
        }
      }
    } catch (err) {
      this.destroy(err);
    }
  }
}

The stream automatically pauses the generator if the client isn't consuming fast enough. This prevents runaway memory growth.

Error Handling in Streaming Contexts

Streaming introduces new error scenarios. If an error occurs midway through a stream, the client may have already received partial data.

Establish a contract: Define whether streams are atomic (all-or-nothing) or partial-delivery-acceptable. Communicate errors in-band:

data: {"type": "resource_chunk", "offset": 1000000, "data": "..."}
data: {"type": "resource_error", "message": "Disk I/O error at offset 2000000", "recoverable": false}

Clients can then decide whether to retry, use partial results, or fail the operation.

Performance Tuning: Chunk Sizes and Concurrency

Optimal settings depend on your workload:

Chunk Size: Larger chunks reduce protocol overhead but increase latency. For streaming files, 64KB–256KB is typical. Smaller (4KB) for real-time streams where latency matters.

Concurrent Requests: Start conservative (e.g., 10–20 concurrent requests). Monitor CPU, memory, and network. Scale up until throughput plateaus or latency degrades.

Thread Pool Size: For CPU-intensive operations, set pool size to CPU core count. For I/O-bound, be more generous—often 2x to 4x cores.

const os = require('os');
const THREAD_POOL_SIZE = Math.max(2, os.cpus().length * 2);

Observability and Debugging

Streaming systems require different observability than request-response models.

Metrics to Track:

• Chunk throughput (chunks/sec)
• Average chunk size
• Stream duration
• Backpressure events (how often the producer paused)
• Error rate mid-stream

Logging: Include stream context (offset, total size, elapsed time) so you can diagnose partial failures.

logger.info('stream_start', {
  tool: 'read_file',
  filename,
  estimatedSize: stat.size
});

logger.info('stream_chunk', {
  filename,
  offset,
  chunkSize,
  elapsedMs: Date.now() - startTime
});

logger.info('stream_complete', {
  filename,
  totalSize,
  durationMs: Date.now() - startTime,
  chunksCount
});

Real-World Considerations

Security: Streaming doesn't bypass authentication—verify permissions before starting the stream. Consider rate-limiting per-user or per-tool to prevent resource-exhaustion attacks.

Retry Logic: If a stream fails partway through, clients need a way to resume from the last successful chunk offset. Implement resumable streaming with explicit offset tracking.

Timeout Management: Long-running streams may outlive default HTTP request timeouts. Use streaming transports (SSE, WebSocket) that don't have built-in timeouts, or implement keep-alive messages.

---

Building efficient, scalable MCP servers requires thinking beyond simple request-response patterns. By embracing streaming, async-first concurrency, and careful backpressure management, you create infrastructure that remains responsive even under extreme load. Start with streaming for known heavy operations (file serving, search), measure your backpressure behavior, and scale your concurrency settings based on actual production workloads.

The agents depending on your MCP servers will be faster, more reliable, and your infrastructure team will thank you.