> ## Documentation Index
> Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Trace OpenAI SDK calls, evaluate GPT models, and route them through the Braintrust gateway

Braintrust integrates with [OpenAI](https://openai.com) so you can call GPT models from the Braintrust playground, API, and SDKs. Braintrust also traces OpenAI SDK calls from your application, including streaming, structured outputs, and function calling.

## Add OpenAI as an AI provider

To use OpenAI models in the Braintrust playground, API, and gateway, connect OpenAI as a provider in your organization or project AI providers.

1. Go to ** Settings** > [** Models**](https://www.braintrust.dev/app/~/configuration/models/organization).
2. Click  **Organization provider** to make the provider available across every project in the organization. To limit it to the current project, select the **Project** tab and click  **Project provider** instead.
3. Under **Model providers**, click **OpenAI**.
4. Choose your authentication method:
   * **API key**: Visit [OpenAI's API platform](https://platform.openai.com/api-keys), create a new API key, and paste it into the field below the authentication options.

     > [!NOTE]
     >
     > API keys are stored as one-way cryptographic hashes, never in plaintext.

     > [accordion] Use a custom OpenAI endpoint
     >
     > To point this provider at a custom OpenAI endpoint, such as a regional API, expand **More settings** and set these fields.
     >
     > | Field | What to enter | If you leave it blank |
     > | - | - | - |
     > | **API base URL** | The API root, such as `https://us.api.openai.com/v1`. Don't include a route such as `/chat/completions`. | Braintrust uses the default OpenAI endpoint. |
     > | **Endpoint Path** | A single route, including the leading slash, such as `/responses`. Every request goes to that route, whichever API the caller used. | Braintrust appends the route each request targets, so one provider serves both the Chat Completions and Responses APIs. |
     >
     > Leave **Endpoint Path** blank unless you need to pin this provider to one route.
   * **OAuth access token**: Paste an OAuth access token you obtained for your endpoint. Braintrust sends it using Bearer authentication. Replace the token in the provider settings when it expires.
   * **Workload identity federation**: Exchange a Braintrust-signed OIDC token for an OpenAI access token, instead of storing a long-lived OpenAI API key in Braintrust.

     > [!NOTE]
     >
     > Workload identity federation is available only for organization-level providers on Braintrust-hosted organizations with the Braintrust gateway enabled. For project-level providers and self-hosted deployments, use **API key** or **OAuth access token** authentication.
5. If you chose **Workload identity federation**, use the setup values shown in Braintrust to configure OpenAI:

   1. [Create a workload identity provider](https://developers.openai.com/api/docs/guides/workload-identity-federation) in OpenAI. Enter a descriptive **Name**, use the **OIDC issuer URL** and **Audience** shown in Braintrust, and leave uploaded JWKS and attribute transformations disabled.
   2. From the workload identity provider details page, create a mapping. Use `sub` as the **Key** and the subject pattern shown in Braintrust as the **Value**. Add a mapping attribute for each additional claim shown in Braintrust. Choose the OpenAI **Project**, **Service account**, and **Permissions** Braintrust should use.
   3. Paste the OpenAI IDs back into Braintrust:
      * **Identity provider ID**: The workload identity provider ID configured for Braintrust.
      * **Service account ID**: The OpenAI service account ID Braintrust should use.
      * **Subject suffix**: A stable suffix for this OpenAI connection. It must match the final part of the subject pattern used in OpenAI.

   For general OpenAI concepts and dashboard details, see [OpenAI's workload identity federation docs](https://developers.openai.com/api/docs/guides/workload-identity-federation).
6. Click **Create**, or **Update** when editing an existing provider.

> [!NOTE]
>
> For the GPT-5 family, whether the `temperature` parameter is configurable depends on the reasoning effort. GPT-5.1 and later accept `temperature` only when reasoning effort is set to `none`. At any higher reasoning effort, and for older GPT-5 models (`gpt-5`, `gpt-5-mini`, `gpt-5-nano`) and GPT-5 Pro, `temperature` isn't configurable and is disabled in the Braintrust UI.

**TypeScript**

## Tracing

Braintrust traces OpenAI calls automatically with the `braintrust/hook.mjs` import hook, or manually with `wrapOpenAI`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.

> [!TIP]
>
> Using the OpenAI Agents SDK? See the [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) framework docs.

### Setup

Install the Braintrust SDK alongside the OpenAI SDK, then configure your API keys.

**1. Install packages**

```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add braintrust openai
```

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install braintrust openai
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace OpenAI calls without modifying your application code, initialize Braintrust, create a normal OpenAI client, then run your app with Braintrust's import hook to patch the OpenAI SDK at startup.

**1. Initialize Braintrust and call OpenAI**

```javascript title="trace-openai-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";
import { initLogger } from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const result = await client.chat.completions.create({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "What is machine learning?" }],
});
```

**2. Run with the import hook**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
node --import braintrust/hook.mjs trace-openai-auto.js
```

The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects. Compile your TypeScript to JavaScript and run the compiled file with the import hook, or [run TypeScript directly](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) with a TypeScript loader.

> [!NOTE]
>
> If you're using a bundler, see [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for plugin and loader setup.

### Manual instrumentation

To trace OpenAI calls manually, wrap your client with `wrapOpenAI` yourself. Once wrapped, every `chat.completions.create` call (including streaming) emits a span.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";

// Initialize the Braintrust logger
const logger = initLogger({
  projectName: "My Project", // Your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Wrap the OpenAI client with wrapOpenAI
const client = wrapOpenAI(
  new OpenAI({
    apiKey: process.env.OPENAI_API_KEY,
  }),
);

// All API calls are automatically logged
const result = await client.chat.completions.create({
  model: "gpt-5-mini",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is machine learning?" },
  ],
});
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Streaming

`wrapOpenAI` can automatically log metrics like `prompt_tokens`, `completion_tokens`, and `tokens` for streaming LLM calls if the LLM API returns them. Set `include_usage` to `true` in the `stream_options` parameter to receive these metrics from OpenAI.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
model: "gpt-5-mini",
  messages: [{ role: "user", content: "Count to 10" }],
  stream: true,
  stream_options: {
    include_usage: true, // Required for token metrics
  },
});

for await (const chunk of result) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
```

### Structured outputs

OpenAI's structured outputs are supported with the wrapper functions.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { z } from "zod";

// Define a Zod schema for the response
const ResponseSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const completion = await client.beta.chat.completions.parse({
  model: "gpt-5-mini",
  messages: [
    { role: "system", content: "Extract the person's name and age." },
    { role: "user", content: "My name is John and I'm 30 years old." },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "person",
      // The Zod schema for the response
      schema: ResponseSchema,
    },
  },
});
```

### Function calling and tools

Braintrust supports OpenAI function calling for building AI agents with tools.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const tools = [
  {
    type: "function" as const,
    function: {
      name: "get_weather",
      description: "Get current weather for a location",
      parameters: {
        type: "object",
        properties: {
          location: { type: "string" },
        },
        required: ["location"],
      },
    },
  },
];

const response = await client.chat.completions.create({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "What's the weather in San Francisco?" }],
  tools,
});
```

To trace multimodal content, attachments, errors, and masking sensitive data, see the [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md) guide.

### OpenAI Batch API

Braintrust traces asynchronous [OpenAI Batch API](https://platform.openai.com/docs/guides/batch) requests with three helper functions. Each traced input file becomes an `openai.batch` task span with one LLM child span per request in the batch.

Import the helper functions from `braintrust`, then use them in place of the corresponding OpenAI SDK calls.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";
import { createReadStream } from "fs";
import {
  initLogger,
  openaiFilesCreateTraced,
  openaiBatchesRetrieveTraced,
  completeOpenAIBatchTrace,
} from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Upload the batch input file
const inputFile = await openaiFilesCreateTraced(openai.files)({
  file: createReadStream("batch.jsonl"), // Replace with your batch input file
  purpose: "batch",
});

// Create the batch
const batch = await openai.batches.create({
  input_file_id: inputFile.id,
  endpoint: "/v1/chat/completions",
  completion_window: "24h",
});

// Poll until the batch reaches a terminal status
let currentBatch = batch;
const TERMINAL = new Set(["completed", "failed", "expired", "cancelled"]);
while (!TERMINAL.has(currentBatch.status)) {
  await new Promise((resolve) => setTimeout(resolve, 60_000));
  currentBatch = await openaiBatchesRetrieveTraced(openai.batches)(currentBatch.id);
}

// Write per-request outputs to spans
const outputFileContent = currentBatch.output_file_id
  ? await openai.files.content(currentBatch.output_file_id)
  : undefined;
const errorFileContent = currentBatch.error_file_id
  ? await openai.files.content(currentBatch.error_file_id)
  : undefined;
await completeOpenAIBatchTrace({
  inputFileId: inputFile.id,
  inputFileContent: await openai.files.content(inputFile.id),
  outputFileContent,
  errorFileContent,
});
```

### OpenAI Agents API

Braintrust provides manual tracing helpers for the [OpenAI Agents API](https://platform.openai.com/docs/api-reference/agents) (`client.beta.agents.sessions`), available in `openai` v7 or later. Because the Agents API is event-driven and stateful, auto-instrumentation isn't sufficient. Instead, three functions let you forward session events into a Braintrust trace.

Import `startOpenAIAgentsTrace`, `updateOpenAIAgentsTrace`, and `failOpenAIAgentsTrace` from `braintrust`. Call `startOpenAIAgentsTrace` with the same parameters you send to the OpenAI Agents API, pass each streamed event to `updateOpenAIAgentsTrace`, and call `failOpenAIAgentsTrace` if the stream ends in an error.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import OpenAI from "openai";
import {
  initLogger,
  startOpenAIAgentsTrace,
  updateOpenAIAgentsTrace,
  failOpenAIAgentsTrace,
} from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const params = {
  agent: {
    model: "gpt-5-mini",
    instructions: "You are a helpful assistant.",
  },
  input: [{ role: "user", content: "What is the capital of France?" }],
  stream: true,
};

let traceToken = startOpenAIAgentsTrace(params);

try {
  const events = await client.beta.agents.sessions.create(params);
  for await (const event of events) {
    traceToken = await updateOpenAIAgentsTrace(traceToken, event);
    if (
      event.type === "agent.session.turn.completed" ||
      event.type === "agent.session.turn.failed" ||
      event.type === "agent.session.turn.cancelled"
    ) {
      break;
    }
  }
} catch (error) {
  traceToken = await failOpenAIAgentsTrace(traceToken, error);
  throw error;
}
```

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",

  apiKey: process.env.BRAINTRUST_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-5-mini",
  messages: [{ role: "user", content: "What is a proxy?" }],
  seed: 1, // A seed activates the proxy's cache
});
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust instruments the OpenAI client and emits an LLM span per call. Chat and responses spans capture the request messages or input, the response output, and the request parameters as metadata. Audio and image spans, base64 image and file inputs to Chat Completions calls, and base64 image inputs to Responses API and Decisions API calls capture files as attachments when `BRAINTRUST_CAPTURE_ATTACHMENTS` is set to `1` or `true`. Otherwise, the file contents are recorded as `<omitted>`. Embeddings and moderation spans capture their inputs and results.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `chat.completions.create`, `.parse`, and `.stream` (including streaming) |
| `Embedding` | `embeddings.create` (output records the embedding length, not the raw vector) |
| `Moderation` | `moderations.create` |
| `openai.responses.create` | `responses.create` and the `responses.stream` helper (including streaming). Streams that end with `response.incomplete` or `response.failed` record their output and token metrics, and `response.failed` streams also record the error (v3.37.0 or later). |
| `openai.responses.parse` | `responses.parse` (structured outputs) |
| `openai.responses.compact` | `responses.compact` (OpenAI SDK v6.10.0 or later) |
| `openai.batch` | One task span per batch input file, with one LLM child span per request (requires `openaiFilesCreateTraced` and `completeOpenAIBatchTrace`) |
| `openai.agents.sessions.create` | Agents API (`beta.agents.sessions.create` with `stream: true`), traced with `startOpenAIAgentsTrace`, `updateOpenAIAgentsTrace`, and `failOpenAIAgentsTrace`, with tool call child spans |
| `openai.images.generate` | `images.generate`, with generated images captured as attachments when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.images.edit` | `images.edit`, with input image, mask, and edited output captured as attachments when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.images.createVariation` | `images.createVariation`, with source image and variation captured as attachments when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.audio.speech.create` | `audio.speech.create`, with generated audio captured as an attachment when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.audio.transcriptions.create` | `audio.transcriptions.create`, with input audio captured as an attachment when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.audio.translations.create` | `audio.translations.create`, with input audio captured as an attachment when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled |
| `openai.decisions.create` | Decisions API (`decisions.create`, OpenAI SDK v7.30.0 or later), as a `question`-typed span, which the trace view labels as a **Decision** span. Input records the request's `input` messages as `state` and its `questions`, each with an `id`, `type` (`boolean` for predicate questions), `instructions`, and `criteria` for choice and score questions. Output records the `answers`, each with its question `id` and the returned choice, score, confidence, and probabilities. Token usage is recorded as metrics. |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `completion_reasoning_tokens` | Reasoning tokens |
| `time_to_first_token` | First-token latency (chat and responses spans) |
| `cached` | Whether the response was served from the Braintrust cache |

### Tracing resources

* [Braintrust JavaScript SDK](https://github.com/braintrustdata/braintrust-sdk-javascript)
* [OpenAI Node SDK](https://github.com/openai/openai-node)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust `Eval` is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { Eval } from "braintrust";
import { OpenAI } from "openai";

const client = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

Eval("OpenAI Evaluation", {
  // An array of user inputs and expected outputs
  data: () => [
    { input: "What is 2+2?", expected: "4" },
    { input: "What is the capital of France?", expected: "Paris" },
  ],
  task: async (input) => {
    // Your OpenAI LLM call
    const response = await client.chat.completions.create({
      model: "gpt-5-mini",
      messages: [{ role: "user", content: input }],
    });
    return response.choices[0].message.content;
  },
  scores: [
    {
      name: "accuracy",
      // A simple scorer that returns 1 if the output matches the expected output, 0 otherwise
      scorer: (args) => (args.output === args.expected ? 1 : 0),
    },
  ],
});
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

### Use OpenAI as an LLM judge

You can use OpenAI models to score the outputs of other AI systems. This example uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system.

Install the [`autoevals`](https://docs.braintrust.dev/evaluate/autoevals) package to use the `LLMClassifierFromSpec` scorer.

```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add autoevals
```

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install autoevals
```

Create a scorer that uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system. You can then include `relevanceScorer` as a scorer in your `Eval` function (see above).

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { LLMClassifierFromSpec } from "autoevals";

const relevanceScorer = LLMClassifierFromSpec("Relevance", {
  choice_scores: { Relevant: 1, Irrelevant: 0 },
  model: "gpt-5-mini",
  use_cot: true,
});
```

**Python**

## Tracing

Braintrust traces OpenAI calls automatically with `auto_instrument()`, or manually with `wrap_openai`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.

> [!TIP]
>
> Using the OpenAI Agents SDK? See the [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) framework docs.

### Setup

Install the Braintrust SDK alongside the OpenAI SDK, then configure your API keys.

**1. Install packages**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust openai
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace OpenAI calls without modifying your client construction, call `auto_instrument()` before importing the OpenAI SDK. Braintrust patches the SDK so every call emits a span.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

import braintrust

braintrust.auto_instrument()
braintrust.init_logger(project="My Project")

from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

result = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is machine learning?"},
    ],
)
```

### Manual instrumentation

To trace OpenAI calls manually, wrap your client with `wrap_openai` yourself. Once wrapped, every `chat.completions.create` call (including streaming) emits a span.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

from braintrust import init_logger, wrap_openai
from openai import OpenAI

logger = init_logger(project="My Project")
client = wrap_openai(OpenAI(api_key=os.environ["OPENAI_API_KEY"]))

# All API calls are automatically logged
result = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is machine learning?"},
    ],
)
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Streaming

`wrap_openai` can automatically log metrics like `prompt_tokens`, `completion_tokens`, and `tokens` for streaming LLM calls if the LLM API returns them. Set `include_usage` to `true` in the `stream_options` parameter to receive these metrics from OpenAI.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
model="gpt-5-mini",
    messages=[{"role": "user", "content": "Count to 10"}],
    stream=True,
    stream_options={"include_usage": True},  # Required for token metrics
)

for chunk in result:
    print(chunk.choices[0].delta.content or "", end="")
```

### Structured outputs

OpenAI's structured outputs are supported with the wrapper functions.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int

completion = client.beta.chat.completions.parse(
    model="gpt-5-mini",
    messages=[
        {"role": "system", "content": "Extract the person's name and age."},
        {"role": "user", "content": "My name is John and I'm 30 years old."},
    ],
    response_format=Person,
)
```

### Function calling and tools

Braintrust supports OpenAI function calling for building AI agents with tools.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "What's the weather in San Francisco?"}],
    tools=tools,
)
```

### Streaming audio transcriptions

Braintrust traces streaming audio transcription calls for sync and async OpenAI clients. Each span captures the audio file as an attachment and the final transcript as the span output.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
with open("audio.m4a", "rb") as f:
    stream = client.audio.transcriptions.create(
        model="gpt-4o-transcribe",
        file=f,
        stream=True,
    )
    for event in stream:
        if event.type == "transcript.text.delta":
            print(event.delta, end="", flush=True)
```

To trace multimodal content, attachments, errors, and masking sensitive data, see the [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md) guide.

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.braintrust.dev/v1",
    api_key=os.environ["BRAINTRUST_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[{"role": "user", "content": "What is a proxy?"}],
    seed=1,  # A seed activates the proxy's cache
)
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust patches the OpenAI SDK and emits an LLM span per call across the chat, responses, embeddings, moderation, audio, and image APIs, plus a `decision` span per Decisions API call. Chat and responses spans capture the request input, the response output, and the request parameters as metadata. Audio and image spans capture files as attachments. Every span records `provider: "openai"`.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `chat.completions.create` and `.parse` (including streaming) |
| `openai.responses.create`, `openai.responses.parse` | Responses API, with `tool`-typed child spans for tool calls |
| `Embedding` | `embeddings.create` (output records the embedding length, not the raw vector) |
| `Moderation` | `moderations.create` |
| `Transcription`, `Translation`, `Speech` | Audio APIs (audio captured as an attachment) |
| `Image Generation`, `Image Edit`, `Image Variation` | Image APIs (images captured as attachments) |
| `openai.agents.sessions.create` | Agents API (`beta.agents.sessions.create` with `stream=True`), with tool call child spans, including computer use calls (screenshots captured as attachments), browser authentication requests, and computer use approval requests and results |
| `openai.decisions.create` | Decisions API (`decisions.create`, sync and async, OpenAI SDK v3.26.0 or later), as a `decision`-typed span. Input records the `state` and the `questions` (each with an `id`, `type`, `instructions`, and `criteria` for choice and score questions). Output records an `answers` list keyed by question `id`, with probabilities, scores, and confidence. Images in the input are captured as attachments. |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `prompt_cache_creation_tokens` | Tokens written to the prompt cache, when reported |
| `completion_reasoning_tokens` | Reasoning tokens |
| `time_to_first_token` | First-token latency (streaming) |
| `cached` | Whether the response was served from the Braintrust cache |

**Metadata**

| Field | Description |
| - | - |
| `access_programs` | Responses API `access_programs` value from the request and the response, when present |

### Tracing resources

* [Braintrust Python SDK](https://github.com/braintrustdata/braintrust-sdk-python)
* [OpenAI Python SDK](https://github.com/openai/openai-python)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. Braintrust `Eval` is a simple function composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

from braintrust import Eval
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def task(input):
    response = client.chat.completions.create(
        model="gpt-5-mini",
        messages=[{"role": "user", "content": input}],
    )
    return response.choices[0].message.content

def accuracy_scorer(output, expected, **kwargs):
    return 1 if output == expected else 0

Eval(
    "OpenAI Evaluation",
    data=[
        {"input": "What is 2+2?", "expected": "4"},
        {"input": "What is the capital of France?", "expected": "Paris"},
    ],
    task=task,
    scores=[accuracy_scorer],
)
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

### Use OpenAI as an LLM judge

You can use OpenAI models to score the outputs of other AI systems. This example uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system.

Install the [`autoevals`](https://docs.braintrust.dev/evaluate/autoevals) package to use the `LLMClassifierFromSpec` scorer.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install autoevals
```

Create a scorer that uses the `LLMClassifierFromSpec` scorer to score the relevance of the outputs of an AI system. You can then include `relevance_scorer` as a scorer in your `Eval` function (see above).

```python Python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
from autoevals import LLMClassifierFromSpec

relevance_scorer = LLMClassifierFromSpec(
    "Relevance",
    choice_scores={"Relevant": 1, "Irrelevant": 0},
    model="gpt-5-mini",
    use_cot=True,
)
```

**Ruby**

## Tracing

Braintrust traces OpenAI calls automatically when you load `braintrust/setup`, or manually with `Braintrust.instrument!`. Either path produces the same spans. Auto-instrumentation is the recommended path for most users.

### Setup

Install the Braintrust gem alongside the OpenAI gem, then configure your API keys.

**1. Install gems**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
gem install braintrust openai
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>
BRAINTRUST_DEFAULT_PROJECT=<your-project-name>  # Project that spans are logged to

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace OpenAI calls without modifying your client construction, load `braintrust/setup` early in your application. Braintrust intercepts `require "openai"` and patches the gem so every chat completion, response, and moderation call emits a span.

```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust/setup'
require 'openai'

client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))

client.chat.completions.create(
  model: 'gpt-5-mini',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is machine learning?' }
  ]
)
```

> [!TIP]
>
> In a Rails app, add `gem "braintrust", require: "braintrust/setup"` to your Gemfile to enable auto-instrumentation without an explicit `require` line.

### Manual instrumentation

To trace OpenAI calls manually, instrument your client with `Braintrust.instrument!` yourself. Once instrumented, every `chat.completions.create` call (including streaming) emits a span.

```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust'
require 'openai'

# Initialize Braintrust
Braintrust.init(default_project: 'My Project')

# Create OpenAI client
client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))

# Instrument the client with Braintrust tracing
Braintrust.instrument!(:openai, target: client)

# All API calls are automatically logged
client.chat.completions.create(
  model: 'gpt-5-mini',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is machine learning?' }
  ]
)
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'openai'

client = OpenAI::Client.new(
  base_url: 'https://gateway.braintrust.dev/v1',
  api_key: ENV.fetch('BRAINTRUST_API_KEY', nil)
)

client.chat.completions.create(
  model: 'gpt-5-mini',
  messages: [{ role: 'user', content: 'What is a proxy?' }],
  seed: 1 # A seed activates the proxy's cache
)
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust instruments the OpenAI gem's chat, responses, and moderation APIs and emits an LLM span per call. Chat and responses spans capture the request input, the response output, and the request parameters as metadata.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `chat.completions.create` and `stream` (including streaming) |
| `openai.responses.create` | Responses API (`create` and `stream`) |
| `openai.moderations.create` | Moderations API (no token metrics) |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `time_to_first_token` | First-token latency (streaming) |

### Tracing resources

* [Braintrust Ruby SDK](https://github.com/braintrustdata/braintrust-sdk-ruby)
* [OpenAI Ruby SDK](https://github.com/openai/openai-ruby)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. The Braintrust `Eval` API is composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```ruby Ruby theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require 'braintrust'
require 'openai'

Braintrust.init

client = OpenAI::Client.new(api_key: ENV.fetch('OPENAI_API_KEY', nil))

Braintrust::Eval.run(
  project: 'OpenAI Evaluation',
  experiment: 'basic-eval',
  # An array of user inputs and expected outputs
  cases: [
    { input: 'What is 2+2?', expected: '4' },
    { input: 'What is the capital of France?', expected: 'Paris' }
  ],
  # Your OpenAI LLM call
  task: lambda do |input|
    response = client.chat.completions.create(
      model: 'gpt-5-mini',
      messages: [{ role: 'user', content: input }]
    )
    response.choices[0].message.content
  end,
  # A simple scorer that returns 1 if the output matches the expected output, 0 otherwise
  scorers: [
    Braintrust::Eval.scorer('accuracy') do |_input, expected, output|
      output == expected ? 1.0 : 0.0
    end
  ]
)
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

**Go**

## Tracing

The Braintrust Go SDK ships an OpenAI middleware that you can attach manually, or apply automatically at compile time with [Orchestrion](https://github.com/DataDog/orchestrion). Either path produces the same traces. Auto-instrumentation is the recommended path for most users.

### Setup

Install the Braintrust Go SDK alongside the OpenAI Go SDK, then configure your API keys.

**1. Install packages**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go get github.com/braintrustdata/braintrust-sdk-go
go get github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai
go get github.com/openai/openai-go
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace OpenAI calls without modifying your application code, build your app with [Orchestrion](https://github.com/DataDog/orchestrion), a compile-time tool that rewrites every `openai.NewClient` call to attach Braintrust's middleware automatically.

**1. Install Orchestrion**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go install github.com/DataDog/orchestrion@latest
```

**2. Create orchestrion.tool.go in your project root**

```go title="orchestrion.tool.go" #skip-compile #skip-format theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
//go:build tools

package main

import (
	_ "github.com/DataDog/orchestrion"
	_ "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)
```

**3. Write your app**

Orchestrion instruments every `openai.NewClient` call at build time, so no middleware wiring is needed.

```go Go #skip-compile #skip-format theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main

import (
	"context"
	"log"
	"os"

	"github.com/openai/openai-go"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/trace"

	"github.com/braintrustdata/braintrust-sdk-go"
)

func main() {
	ctx := context.Background()

	tp := trace.NewTracerProvider()
	defer tp.Shutdown(ctx)
	otel.SetTracerProvider(tp)

	_, err := braintrust.New(tp,
		braintrust.WithProject("My Project"),
		braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
	)
	if err != nil {
		log.Fatal(err)
	}

	client := openai.NewClient()

	result, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
		Messages: []openai.ChatCompletionMessageParamUnion{
			openai.SystemMessage("You are a helpful assistant."),
			openai.UserMessage("What is machine learning?"),
		},
		Model: "gpt-5-mini",
	})
	if err != nil {
		log.Fatal(err)
	}
	_ = result
}
```

**4. Build and run with Orchestrion**

Build with Orchestrion to enable auto-instrumentation:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
go mod tidy
orchestrion go build -o myapp
./myapp
```

> [accordion] Enable Orchestrion via GOFLAGS
>
> Instead of running `orchestrion go build`, you can set a `GOFLAGS` environment variable to enable Orchestrion for normal `go build` commands:
>
> ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
> export GOFLAGS="-toolexec='orchestrion toolexec'"
> go build ./...
> ```

### Manual instrumentation

To trace OpenAI calls manually, attach Braintrust's tracing middleware yourself by passing `traceopenai.NewMiddleware()` as an option on `openai.NewClient`. Once attached, every `Chat.Completions.New` call (including streaming) emits a span.

```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main

import (
	"context"
	"log"
	"os"

	"github.com/openai/openai-go"
	"github.com/openai/openai-go/option"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/trace"

	"github.com/braintrustdata/braintrust-sdk-go"
	traceopenai "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)

func main() {
	// Set up OpenTelemetry TracerProvider
	tp := trace.NewTracerProvider()
	defer tp.Shutdown(context.Background())
	otel.SetTracerProvider(tp)

	// Initialize Braintrust client
	_, err := braintrust.New(tp,
		braintrust.WithProject("My Project"),
		braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Create OpenAI client with tracing middleware
	client := openai.NewClient(
		option.WithMiddleware(traceopenai.NewMiddleware()),
	)

	// All API calls are automatically logged
	result, err := client.Chat.Completions.New(context.Background(), openai.ChatCompletionNewParams{
		Messages: []openai.ChatCompletionMessageParamUnion{
			openai.SystemMessage("You are a helpful assistant."),
			openai.UserMessage("What is machine learning?"),
		},
		Model: "gpt-5-mini",
	})
	if err != nil {
		log.Fatal(err)
	}
	_ = result
}
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main

import (
	"context"
	"log"
	"os"

	"github.com/openai/openai-go"
	"github.com/openai/openai-go/option"
)

func main() {
	ctx := context.Background()

	client := openai.NewClient(
		option.WithBaseURL("https://gateway.braintrust.dev/v1"),
		option.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
	)

	response, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
		Model: "gpt-5-mini",
		Messages: []openai.ChatCompletionMessageParamUnion{
			openai.UserMessage("What is a proxy?"),
		},
		Seed: openai.Int(1), // A seed activates the proxy's cache
	})
	if err != nil {
		log.Fatal(err)
	}
	_ = response
}
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust's OpenAI middleware emits an LLM span per call and routes by request path, so OpenAI-compatible providers are traced the same way. Chat, completions, and responses spans capture the request input, the response output, and the request parameters as metadata. Streaming web search responses also capture `annotations` (URL citations) in the span output.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `/v1/chat/completions` (including streaming) |
| `Completion` | `/v1/completions` (legacy completions endpoint, including streaming) |
| `openai.responses.create` | `/v1/responses` (including streaming). Streams that send an `error` event, and responses that end with a `failed` status, mark the span as an error and record the error details (v0.17.2 or later). |
| `openai.embeddings.create` | `/v1/embeddings` (output records the embedding length) |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `time_to_first_token` | First-token latency (chat and responses spans) |

### Tracing resources

* [Braintrust Go SDK](https://github.com/braintrustdata/braintrust-sdk-go)
* [OpenAI Go SDK](https://github.com/openai/openai-go)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. The Braintrust `Evaluator` API is composed of a dataset of user inputs, a task, and a set of scorers. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```go Go theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
package main

import (
	"context"
	"log"
	"os"

	"github.com/openai/openai-go"
	"github.com/openai/openai-go/option"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/trace"

	"github.com/braintrustdata/braintrust-sdk-go"
	"github.com/braintrustdata/braintrust-sdk-go/eval"
	traceopenai "github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai"
)

func main() {
	ctx := context.Background()

	// Set up OpenTelemetry TracerProvider
	tp := trace.NewTracerProvider()
	defer tp.Shutdown(ctx)
	otel.SetTracerProvider(tp)

	// Initialize Braintrust
	bt, err := braintrust.New(tp,
		braintrust.WithAPIKey(os.Getenv("BRAINTRUST_API_KEY")),
	)
	if err != nil {
		log.Fatal(err)
	}

	// Create OpenAI client with tracing
	client := openai.NewClient(
		option.WithMiddleware(traceopenai.NewMiddleware()),
	)

	// Create evaluator
	evaluator := braintrust.NewEvaluator[string, string](bt)

	// Run evaluation
	_, err = evaluator.Run(ctx, eval.Opts[string, string]{
		Experiment: "OpenAI Evaluation",
		// Dataset of user inputs and expected outputs
		Dataset: eval.NewDataset([]eval.Case[string, string]{
			{Input: "What is 2+2?", Expected: "4"},
			{Input: "What is the capital of France?", Expected: "Paris"},
		}),
		// Task function with OpenAI LLM call
		Task: eval.T(func(ctx context.Context, input string) (string, error) {
			response, err := client.Chat.Completions.New(ctx, openai.ChatCompletionNewParams{
				Model: "gpt-5-mini",
				Messages: []openai.ChatCompletionMessageParamUnion{
					openai.UserMessage(input),
				},
			})
			if err != nil {
				return "", err
			}
			return response.Choices[0].Message.Content, nil
		}),
		// Simple scorer that returns 1 if output matches expected, 0 otherwise
		Scorers: []eval.Scorer[string, string]{
			eval.NewScorer("accuracy", func(ctx context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) {
				score := 0.0
				if r.Output == r.Expected {
					score = 1.0
				}
				return eval.S(score), nil
			}),
		},
	})
	if err != nil {
		log.Fatal(err)
	}
}
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

**Java**

## Tracing

The Braintrust Java SDK ships an OpenAI interceptor that you can attach manually with `BraintrustOpenAI.wrapOpenAI()`, or have applied automatically by the [Braintrust Java agent](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation). Both paths produce the same spans. Auto-instrumentation is the recommended path for most users.

### Setup

Install the Braintrust Java SDK alongside the OpenAI Java SDK, then configure your API keys.

**1. Install packages**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# add to build.gradle dependencies{} block
implementation 'dev.braintrust:braintrust-sdk-java:<version-goes-here>'
implementation 'com.openai:openai-java:<version-goes-here>'
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace OpenAI calls without modifying your application code, attach the [`braintrust-java-agent`](https://docs.braintrust.dev/instrument/trace-llm-calls#auto-instrumentation) at JVM startup. The agent intercepts every OpenAI client build and applies the Braintrust interceptor automatically.

**1. Add the agent dependency**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# build.gradle
configurations {
    btAgent
}

dependencies {
    btAgent 'dev.braintrust:braintrust-java-agent:<version-goes-here>'
}

tasks.withType(JavaExec).configureEach {
    jvmArgs "-javaagent:${configurations.btAgent.singleFile.absolutePath}"
}
```

**2. Run your app**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
./gradlew run
```

OpenAI client builds in your application code are now intercepted automatically. No call to BraintrustOpenAI.wrapOpenAI() is required.

> [!NOTE]
>
> The agent instruments both synchronous and asynchronous client builds, so `OpenAIOkHttpClient` and `OpenAIOkHttpClientAsync` clients are traced automatically.

### Manual instrumentation

To trace OpenAI calls manually, wrap your client with `BraintrustOpenAI.wrapOpenAI` yourself. Once wrapped, every `chat().completions().create()` call (including streaming) emits a span.

```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import dev.braintrust.Braintrust;
import dev.braintrust.instrumentation.openai.BraintrustOpenAI;

class OpenAITracing {
    public static void main(String[] args) {
        var braintrust = Braintrust.get();
        var openTelemetry = braintrust.openTelemetryCreate();

        // Wrap the OpenAI client with Braintrust instrumentation.
        // To trace an async client, build it with OpenAIOkHttpClientAsync and wrap it the same way,
        // importing BraintrustOpenAI from dev.braintrust.instrumentation.openai.v2_15_0.
        OpenAIClient client = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());

        // All API calls are automatically logged
        var request = ChatCompletionCreateParams.builder()
            .model(ChatModel.GPT_5_MINI)
            .addSystemMessage("You are a helpful assistant.")
            .addUserMessage("What is machine learning?")
            .build();

        var result = client.chat().completions().create(request);
    }
}
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;

class OpenAIProxy {
    public static void main(String[] args) {
        OpenAIClient client = OpenAIOkHttpClient.builder()
            .apiKey(System.getenv("BRAINTRUST_API_KEY"))
            .baseUrl("https://gateway.braintrust.dev/v1")
            .build();

        var response = client.chat().completions().create(
            ChatCompletionCreateParams.builder()
                .model(ChatModel.GPT_5_MINI)
                .addUserMessage("What is a proxy?")
                .seed(1L) // A seed activates the proxy's cache
                .build());
    }
}
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust instruments the OpenAI client at the HTTP layer and emits an LLM span per request, naming the span by endpoint. Chat and responses spans capture the request input, the response output, request metadata (provider, request path and method, and model), and token metrics. Server-side tool calls in Responses API responses appear as child `tool` spans nested under the LLM span.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `chat().completions().create()` (including streaming) |
| `responses` | Responses API |
| `Embeddings` | Embeddings API (token metrics only) |
| Server-side tool spans | One child span per server-side tool call (`web_search_call`, `file_search_call`, `code_interpreter_call`, `mcp_call`) in Responses API responses |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `prompt_cached_tokens` | Tokens read from the prompt cache |
| `completion_reasoning_tokens` | Reasoning tokens (responses spans) |
| `time_to_first_token` | First-token latency (streaming only) |

### Tracing resources

* [Braintrust Java SDK](https://github.com/braintrustdata/braintrust-sdk-java)
* [OpenAI Java SDK](https://github.com/openai/openai-java)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```java Java theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import dev.braintrust.Braintrust;
import dev.braintrust.eval.DatasetCase;
import dev.braintrust.eval.Scorer;
import dev.braintrust.instrumentation.openai.BraintrustOpenAI;
import java.util.function.Function;

class OpenAIEvaluation {
    public static void main(String[] args) {
        var braintrust = Braintrust.get();
        var openTelemetry = braintrust.openTelemetryCreate();
        OpenAIClient client = BraintrustOpenAI.wrapOpenAI(openTelemetry, OpenAIOkHttpClient.fromEnv());

        Function<String, String> taskFunction = (String input) -> {
            var request = ChatCompletionCreateParams.builder()
                .model(ChatModel.GPT_5_MINI)
                .addUserMessage(input)
                .build();
            var response = client.chat().completions().create(request);
            return response.choices().get(0).message().content().orElse("");
        };

        var eval = braintrust.<String, String>evalBuilder()
            .name("OpenAI Evaluation")
            .cases(
                DatasetCase.of("What is 2+2?", "4"),
                DatasetCase.of("What is the capital of France?", "Paris"))
            .taskFunction(taskFunction)
            .scorers(
                Scorer.of("contains_answer", (evalCase, output) ->
                    output.contains("4") || output.contains("Paris") ? 1.0 : 0.0))
            .build();

        var result = eval.run();
        System.out.println(result.createReportString());
    }
}
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

**.NET**

## Tracing

Trace your OpenAI LLM calls for observability and monitoring. Install the SDK, wrap your client, and every call emits a span.

### Setup

Install the Braintrust .NET SDK alongside the OpenAI .NET SDK, then configure your API keys.

**1. Install packages**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# add to .csproj file
dotnet add package Braintrust.Sdk
dotnet add package OpenAI
```

**2. Get an OpenAI API key**

Visit [OpenAI's API platform](https://platform.openai.com/api-keys) and create a new API key, then [add it as a Braintrust AI provider](#add-openai-as-an-ai-provider).

**3. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
OPENAI_API_KEY=<your-openai-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Manual instrumentation

To trace OpenAI calls, wrap your client with `BraintrustOpenAI.WrapOpenAI`. Once wrapped, every `CompleteChat` and `CompleteChatAsync` call emits a span.

```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using Braintrust.Sdk;
using Braintrust.Sdk.OpenAI;
using OpenAI;
using OpenAI.Chat;

class OpenAITracing
{
    static async Task Main(string[] args)
    {
        var braintrust = Braintrust.Sdk.Braintrust.Get();
        var activitySource = braintrust.GetActivitySource();

        var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY");
        if (string.IsNullOrEmpty(apiKey))
        {
            Console.WriteLine("Error: OPENAI_API_KEY environment variable is not set.");
            return;
        }

        // Wrap the OpenAI client with Braintrust instrumentation
        var client = BraintrustOpenAI.WrapOpenAI(
            activitySource,
            apiKey
        );

        // All API calls are automatically logged
        var chatClient = client.GetChatClient("gpt-5-mini");
        var messages = new ChatMessage[]
        {
            new SystemChatMessage("You are a helpful assistant."),
            new UserChatMessage("What is machine learning?")
        };

        var result = await chatClient.CompleteChatAsync(messages);
    }
}
```

> [!TIP]
>
> For more control over tracing, learn how to [customize traces](https://docs.braintrust.dev/instrument/advanced-tracing.md).

### Gateway

To call OpenAI through the [Braintrust gateway](https://docs.braintrust.dev/deploy/gateway.md), point your client at the gateway base URL and use your Braintrust API key for authentication. Use any [supported provider's SDK](https://docs.braintrust.dev/integrations/ai-providers.md) to call OpenAI models.

```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using OpenAI;
using OpenAI.Chat;

class OpenAIProxy
{
    static async Task Main(string[] args)
    {
        var apiKey = Environment.GetEnvironmentVariable("BRAINTRUST_API_KEY");
        if (string.IsNullOrEmpty(apiKey))
        {
            Console.WriteLine("Error: BRAINTRUST_API_KEY environment variable is not set.");
            return;
        }

        var client = new OpenAIClient(
            new System.ClientModel.ApiKeyCredential(apiKey),
            new OpenAIClientOptions
            {
                Endpoint = new Uri("https://gateway.braintrust.dev/v1")
            }
        );

        var chatClient = client.GetChatClient("gpt-5-mini");
        var messages = new ChatMessage[]
        {
            new UserChatMessage("What is a proxy?")
        };

        var response = await chatClient.CompleteChatAsync(messages);
    }
}
```

The gateway also supports the OpenAI-compatible `/embeddings` endpoint for generating embeddings. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for an example.

### What Braintrust traces

Braintrust instruments the OpenAI chat completions API and emits an LLM span per call, capturing the request messages, the response choices, the model, and the request parameters as metadata.

**Spans**

| Span | Coverage |
| - | - |
| `Chat Completion` | `CompleteChat` and `CompleteChatAsync` |

**Metrics**

| Metric | Description |
| - | - |
| `prompt_tokens` | Input tokens |
| `completion_tokens` | Output tokens |
| `tokens` | Total tokens |
| `time_to_first_token` | Call latency |

### Tracing resources

* [Braintrust .NET SDK](https://github.com/braintrustdata/braintrust-sdk-dotnet)
* [OpenAI .NET SDK](https://github.com/openai/openai-dotnet)
* [OpenAI API reference](https://platform.openai.com/docs/api-reference)

## Evals

Evaluations help you distill the non-deterministic outputs of OpenAI models into an effective feedback loop that enables you to ship more reliable, higher quality products. To learn more about evaluations, see the [Experiments](https://docs.braintrust.dev/evaluate/run-evaluations.md) guide.

### Basic eval setup

Evaluate the outputs of OpenAI models with Braintrust.

```csharp C# theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
using System;
using System.Threading.Tasks;
using Braintrust.Sdk;
using Braintrust.Sdk.Eval;
using Braintrust.Sdk.OpenAI;
using OpenAI;
using OpenAI.Chat;

class OpenAIEvaluation
{
    static async Task Main(string[] args)
    {
        var braintrust = Braintrust.Sdk.Braintrust.Get();
        var activitySource = braintrust.GetActivitySource();

        var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY");
        if (string.IsNullOrEmpty(apiKey))
        {
            Console.WriteLine("Error: OPENAI_API_KEY environment variable is not set.");
            return;
        }

        var client = BraintrustOpenAI.WrapOpenAI(
            activitySource,
            apiKey
        );

        // Define the task function that uses OpenAI
        string TaskFunction(string input)
        {
            var chatClient = client.GetChatClient("gpt-5-mini");
            var messages = new ChatMessage[]
            {
                new UserChatMessage(input)
            };
            var response = chatClient.CompleteChat(messages);
            return response.Value.Content[0].Text;
        }

        // Create and run the evaluation
        var eval = await braintrust
            .EvalBuilder<string, string>()
            .Name("OpenAI Evaluation")
            .Cases(
                new DatasetCase<string, string>("What is 2+2?", "4"),
                new DatasetCase<string, string>("What is the capital of France?", "Paris")
            )
            .TaskFunction(TaskFunction)
            .Scorers(
                new FunctionScorer<string, string>("accuracy", (expected, actual) =>
                    actual.Contains(expected) ? 1.0 : 0.0)
            )
            .BuildAsync();

        var result = await eval.RunAsync();
        Console.WriteLine(result.CreateReportString());
    }
}
```

> [!TIP]
>
> Learn more about eval [data](https://docs.braintrust.dev/annotate/datasets.md) and [scorers](https://docs.braintrust.dev/evaluate/write-scorers.md).

## Cookbooks

* [Evaluating audio with the OpenAI Realtime API](https://docs.braintrust.dev/cookbook/recipes/Realtime.md)
* [Using Python functions to extract text from images](https://docs.braintrust.dev/cookbook/recipes/ToolOCR.md)
* [Using functions to build a RAG agent](https://docs.braintrust.dev/cookbook/recipes/ToolRAG.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.