We shipped Model Context Protocol (MCP) support in Bifrost a few weeks ago, and I wanted to walk through why we built it and how it works under the hood.
<h2>The Problem We Were Solving</h2>
AI models are great at reasoning and generating text, but they hit a wall when you need them to actually do things. Want your agent to search a database? Read files? Query an API? You’d traditionally write custom code for each integration, which gets messy fast.
MCP standardizes this. Instead of building one-off integrations, you connect MCP servers that expose tools - filesystem access, web search, database queries, whatever you need. The AI model discovers these tools at runtime and uses them to complete tasks.
But here’s what nobody talks about: running MCP servers directly in production is a security nightmare. You’re giving AI models access to your infrastructure with minimal oversight. And debugging failures when you can’t see what tools are being called? Good luck.
That’s why we built MCP support directly into Bifrost. It’s not just a passthrough - it’s a complete control plane.
<h2>How It Works</h2>
Bifrost acts as both an MCP client (connecting to tool servers) and optionally as an MCP server (exposing tools to external clients like Claude Desktop).
We support four connection types, each optimized for different scenarios:
InProcess connections run tools directly in Bifrost’s memory. If you’re hosting tools in Go, this gives you sub-millisecond latency (around 0.1ms) with compile-time type safety. Perfect for internal business logic.
STDIO connections launch external processes and communicate via stdin/stdout. Great for local scripts, filesystem operations, or Python/Node.js MCP servers. Latency is typically 1-10ms.
HTTP connections talk to remote MCP servers over HTTP. Use this for microservices, cloud-hosted tools, or third-party services. Latency is network-dependent, usually 10-500ms.
SSE (Server-Sent Events) connections maintain persistent connections for streaming data. Think real-time monitoring, market data feeds, or live event streams.
<h2>The Security Model</h2>
By default, Bifrost does NOT automatically execute tool calls. This is critical. When an AI model wants to use a tool, it returns a suggestion - you have to explicitly approve execution. This prevents your agent from accidentally nuking a database or charging a credit card.
But we also support “agent mode” where specific tools can be auto-executed. You configure which tools are safe to run automatically, and Bifrost handles the rest. This is great for read-only operations or low-risk actions where you trust the model’s judgment.
Here’s what a request-level filter looks like:
<pre><code class="language-bash">curl -X POST http://localhost:8080/v1/chat/completions \
 -H &quot;x-bf-mcp-include-clients: filesystem,websearch&quot; \
 -H &quot;x-bf-mcp-include-tools: filesystem/read_file,websearch/search&quot; \
 -d &#x27;{&quot;model&quot;: &quot;gpt-4o-mini&quot;, &quot;messages&quot;: [...]}&#x27;
</code></pre>
You can filter by client (which MCP servers are available) or by specific tools. This gives you granular control per request. Financial apps might restrict agents to read-only database tools. Customer-facing chatbots might only access approved external APIs.
<h2>Code Mode vs Agent Mode</h2>
We shipped two execution patterns:
Agent mode has the AI model call one tool at a time. It sees the result, thinks about what to do next, calls another tool. This works well for interactive agents where you want human oversight at each step.
Code mode lets the AI write TypeScript that orchestrates multiple tools in a single execution. Instead of multiple round-trips, the model writes code like:
<pre><code class="language-typescript">const files = await listFiles(&quot;/project&quot;);
const results = await Promise.all(
 files.map(file =&gt; analyzeFile(file))
);
return summarize(results);
</code></pre>
The code runs atomically. This is way faster for batch operations or complex workflows where you don’t need per-tool approval.
<h2>Tool Discovery</h2>
Tools are discovered dynamically at runtime. When you connect an MCP server, Bifrost:
<ol>
<li>Establishes the connection</li>
<li>Gets the list of available tools and their schemas</li>
<li>Validates everything</li>
<li>Registers tools in the internal registry</li>
<li>Makes them available to AI models</li>
</ol>
No hardcoded tool definitions. No config files. The AI model just asks “what tools are available?” and gets the full list with schemas.
<h2>Real-World Use Cases</h2>
Filesystem operations: We’re using this ourselves for code analysis. The agent reads files, looks for patterns, and writes reports. STDIO connection to a Node.js MCP server, latency under 5ms.
Database queries: Customer support agents query user data, pull order history, check account status. HTTP connection to a microservice with proper authentication. Read-only access by default.
Web search: Research assistants search the web, extract content, and summarize findings. HTTP connection to an external API, cached responses for repeated queries.
Real-time monitoring: System health checks, log analysis, metric tracking. SSE connection for live updates without polling.
<h2>Observability</h2>
Every tool call gets logged. Full audit trail of what the agent did, which tools it used, what data it accessed, and what happened. Prometheus metrics track latency, success rates, and error patterns per tool. Distributed tracing shows the full request flow from the AI model through Bifrost to the MCP server and back.
We also expose MCP metrics - connection health, tool availability, execution times. If an MCP server goes down, you see it immediately.
<h2>What We Learned</h2>
Security can’t be optional. Default-deny tool execution prevents so many production disasters. Even with agent mode, limiting auto-execution to specific tools keeps things contained.
Latency matters more than you think. When an agent is calling 20 tools to complete a task, every millisecond of overhead adds up. InProcess connections at 0.1ms feel instant. HTTP at 200ms starts to drag.
Type safety is underrated. InProcess tools with Go structs caught so many bugs at compile time. No runtime surprises about parameter types or missing fields.
Filtering is critical. Request-level control over which tools are available prevents privilege escalation. Customer-facing agents don’t need access to admin tools, period.
<h2>Performance Numbers</h2>
We ran benchmarks with a filesystem MCP server (STDIO) and a custom database server (HTTP):
<ul>
<li>STDIO connection overhead: 1.2ms average</li>
<li>HTTP connection overhead: 15ms average (local network)</li>
<li>InProcess tool calls: 0.08ms average</li>
<li>Tool discovery: &lt;100ms for 50 tools</li>
<li>Memory usage: ~2MB per active STDIO connection</li>
</ul>
These numbers are with Go’s runtime. Your mileage will vary with Python or Node.js servers, but the relative differences hold.
<h2>Try It Yourself</h2>
MCP support is available in Bifrost now. Start with STDIO connections if you’re prototyping - they’re the easiest to set up. Point Bifrost at an existing MCP server, configure which tools to enable, and you’re running.
For production, HTTP connections scale better. You can run multiple instances of your MCP server behind a load balancer, handle failures gracefully, and isolate tool execution from your gateway.
The docs cover setup for all four connection types, agent mode configuration, and request-level filtering. We also have examples for common use cases - filesystem, database, web APIs.
If you’re building agents that need to interact with your infrastructure, MCP makes this way cleaner. And Bifrost gives you the control plane to actually run it in production without melting down.
<hr>
Built with Bifrost - <a href="https://docs.getbifrost.ai/features/mcp" target="_blank" rel="noopener nofollow">https://docs.getbifrost.ai/features/mcp</a>

Squad for devs building apps with GenAI and LLMs! Connect, share, exchange ideas, and get feedback. 

Build With GenAI

Bifrost now supports Model Context Protocol (MCP), enabling AI models to use external tools like databases, filesystems, and APIs through a standardized interface. The implementation provides four connection types (InProcess, STDIO, HTTP, SSE) with varying latency profiles, from 0.1ms for in-memory Go tools to 200ms+ for HTTP. Security is enforced through explicit tool execution approval by default, with optional agent mode for trusted operations. Request-level filtering controls which tools are available per request. The system includes full observability with audit trails, metrics, and distributed tracing. Performance benchmarks show STDIO overhead at 1.2ms and HTTP at 15ms for local connections.

Adding MCP to Bifrost: Letting AI Models Use External Tools