LLM Sampling

Use this when a server asks your client to run an LLM completion on its behalf. Sampling is how a server borrows your model. Rather than hold an API key of its own, the server describes the messages it wants completed and asks you to run them — you pick the model, and you pay for the tokens. Your side of that arrangement is one function, a sampling handler, registered when you create the client. The handler receives the conversation the server wants completed, the parameters it asked for, and a request context carrying metadata about the call. Return the generated text as a string and FastMCP wraps it in the protocol’s result for you; return a CreateMessageResult yourself when you want to report the real model name or hand back content that isn’t text. If the handler raises, the client sends the error back in place of a completion and the server’s tool decides what to do about it.

Handler Template

from fastmcp import Client
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
from mcp.types import TextContent


async def sampling_handler(
    messages: list[SamplingMessage],
    params: SamplingParams,
    context: RequestContext,
) -> str:
    """Run the server's messages against your LLM and return the completion."""
    conversation = [
        f"{message.role}: {message.content.text}"
        for message in messages
        if isinstance(message.content, TextContent)
    ]
    system_prompt = params.system_prompt or "You are a helpful assistant."

    # Call your LLM here with `conversation` and `system_prompt`.
    return "Generated response based on the messages"


client = Client("my_mcp_server.py", sampling_handler=sampling_handler)

The client answers with this handler however the server asks for a completion. The default mode="auto" negotiates whichever protocol era the server speaks, and one handler covers both of the routes an era can use — see Request Routes.

Handler Parameters

Everything the server sends arrives in the first two arguments. The messages are the conversation to complete; the parameters are how the server would like it completed. You decide how much of that to honor, since the client owns the model — a preference your provider cannot express is yours to ignore.

SamplingMessage

Literal["user", "assistant"]

The role of the message

TextContent | ImageContent | AudioContent

The content of the message. TextContent has a .text attribute.

SamplingParams

str | None

Optional system prompt the server wants to use

ModelPreferences | None

Server preferences for model selection (hints, cost/speed/intelligence priorities)

float | None

Sampling temperature

int

Maximum tokens to generate

list[str] | None

Stop sequences for sampling

list[Tool] | None

Tools the LLM can use during sampling

ToolChoice | None

Tool usage behavior (auto, required, or none)

Built-in Handlers

Writing the provider call yourself is rarely worth it. FastMCP ships handlers for OpenAI, Anthropic, and Google Gemini that implement the full sampling API, tool use included, and translate the protocol’s parameters into each provider’s own. Give one a default model and pass it where your own handler would go. Write a custom handler when you need routing across providers, caching, or a provider FastMCP does not cover.

OpenAI Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(default_model="gpt-4o"),
)

Point the handler at any OpenAI-compatible API, including a local model server, by passing your own provider client:

from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler
from openai import AsyncOpenAI

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(
        default_model="llama-3.1-70b",
        client=AsyncOpenAI(base_url="http://localhost:8000/v1"),
    ),
)

Install the OpenAI handler with pip install 'fastmcp[openai]'.

Anthropic Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.anthropic import AnthropicSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=AnthropicSamplingHandler(default_model="claude-sonnet-4-5"),
)

Install the Anthropic handler with pip install 'fastmcp[anthropic]'.

Google Gemini Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.google_genai import GoogleGenaiSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=GoogleGenaiSamplingHandler(default_model="gemini-2.0-flash"),
)

Install the Google Gemini handler with pip install 'fastmcp[gemini]'.

The source of these handlers is the best reference for writing your own.

Tool Use

A sampling request can carry tools. When it does, your handler passes them to the model and returns whatever comes back, tool calls included — the server executes the tools itself and sends a follow-up sampling request with the results if it needs another turn. Your handler never runs a tool. Registering any sampling_handler advertises full sampling support, tools included. A handler that only generates text should say so, so servers know not to send tools it will drop:

from fastmcp import Client
from mcp.types import SamplingCapability


async def text_only_handler(messages, params, context) -> str:
    return "Generated response based on the messages"


client = Client(
    "my_mcp_server.py",
    sampling_handler=text_only_handler,
    sampling_capabilities=SamplingCapability(),
)

Request Routes

Servers reach your handler by two routes, and which one applies depends on the protocol era the connection negotiated. A handshake-era server pushes a sampling/createMessage request down the open session while a tool is running and waits for the reply. A modern (2026-07-28) connection has no such channel, so the tool ends its round by returning a request for a completion instead; the client answers from your handler and calls the tool again with the result attached. One registration covers both, so this is rarely something you configure — it matters only when you pin an era, since mode="legacy" is the sole route that carries a pushed request. See protocol negotiation for how the era is chosen, and Sampling under Servers for how a server issues these requests.

Get Started

Servers

Apps

Clients

Integrations

More

Handler Template

Handler Parameters

SamplingMessage

SamplingParams

Built-in Handlers

OpenAI Handler

Anthropic Handler

Google Gemini Handler

Tool Use

Request Routes

​Handler Template

​Handler Parameters

SamplingMessage

SamplingParams

​Built-in Handlers

​OpenAI Handler

​Anthropic Handler

​Google Gemini Handler

​Tool Use

​Request Routes

Handler Template

Handler Parameters

Built-in Handlers

OpenAI Handler

Anthropic Handler

Google Gemini Handler

Tool Use

Request Routes