> ## Documentation Index
> Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Run multi-turn agent evaluations with MCP tools

> Run multi-turn agent evaluations with MCP tools, sending predefined user messages sequentially and recording the full conversation in one evaluation row.

> [!NOTE]
>
> **Applies to:**
>
> * Plan - Any
> * Deployment - Any
> *
> * Test a sequence of user messages and agent responses together as one dataset row, preserving conversation history between turns.

## Summary

A custom SDK evaluation task can send predefined user messages sequentially, preserve each generated agent response, and return the full conversation as one evaluation result. Expose that task as a [remote eval](https://docs.braintrust.dev/evaluate/remote-evals.md#run-a-remote-eval) to launch it from Playground while the code runs in your environment.

## What is happening

When you prepopulate multiple User messages in a Playground prompt, they are sent together before the model responds. Adding separate message fields does not create separate execution steps, so this setup does not generate an assistant response between the user messages.

The custom task controls the conversation loop. It sends the first message, waits for the agent and any tool calls to finish, then appends the next user message to the conversation. Remote evals make that task callable from Playground and stream its results back.

If your task uses MCP tools, configure their connection and authentication in the agent code. The remote process does not automatically inherit the MCP connection attached to the Playground prompt.

## Fix or suggestion

### Option 1: Prefill both messages in Playground (if sequencing is not required)

1. Put shared instructions in the System message. Keep them outside the dataset.
2. Store per-row fields such as `first_message` and `second_message` in your dataset row.
3. In the prompt editor add two separate User messages and reference `{{input.first_message}}` and `{{input.second_message}}`.
4. Run the evaluation. The trace will show one eval row with both user messages sent up front.

This is the simplest approach. Use it only when the second user turn does not depend on the agent response.

### Option 2: Remote eval that emits turns step-by-step (recommended when you need a reply between turns)

1. **Prepare the dataset row.** Make one row represent a full conversation, with `first_message` and `second_message` in its input. Keep shared instructions outside the row.

2. **Implement the conversation task.** Run one conversation per dataset row. Inside the task:

   * Initialize a fresh agent conversation with your MCP tools and authenticate the MCP connection in your code.
   * Send the first predefined user message from `first_message`.
   * Wait for the agent to finish and for any MCP tool calls to complete.
   * Record the agent response, tool history, and conversation state.
   * Send the second predefined user message from `second_message` in the same conversation.
   * Wait for the agent to finish and then return the complete transcript as the task output.

   The following TypeScript pseudocode illustrates this sequence. `Agent` and its methods are placeholders for your agent framework, not Braintrust SDK APIs:

   ```typescript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
   // Replace with your SDK and MCP client imports
   async function runConversation(evalRow) {
     const agent = new Agent({ mcpConfig: { apiKey: process.env.MCP_API_KEY } });

     await agent.startConversation();
     await agent.sendUser(evalRow.first_message);
     const firstReply = await agent.waitForReply();
     // Persist firstReply and any tool calls as spans

     await agent.sendUser(evalRow.second_message);
     const finalReply = await agent.waitForReply();

     return { transcript: agent.getTranscript(), spans: agent.getSpans() };
   }
   ```

   Adjust for your language and SDK. Configure MCP authentication and tool execution explicitly in your agent code. Do not rely on Playground to forward MCP credentials.

3. **Trace the conversation.** Instrument model and MCP calls as child spans under the same eval trace so the entire conversation and tool history appear in one eval row. Returning a `spans` field alone does not log them.

For a concrete example of preserving generated assistant responses between turns, see the [multi-turn task example](https://docs.braintrust.dev/evaluate/llm-as-a-judge.md#score-traces). For setup, follow the [remote eval guide](https://docs.braintrust.dev/evaluate/remote-evals.md#run-a-remote-eval) to register your conversation function as the `Eval(...)` task, serve the eval, and select it in Playground. Your task controls the conversation sequence; remote evals let Playground launch it and display the results.

## Confirm it worked

* One eval row contains the full user and assistant transcript with the agent reply between the two user messages.
* The trace shows child spans for each model call and each MCP tool call under the same eval row.

## Notes

* Use the Playground prefill approach only when the second user message does not need to wait for the agent.
* Use a remote eval when the test must preserve the agent response between turns.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.