Anthropic

​

Add Anthropic as an AI provider

1. Visit Anthropic’s Console and create a new API key.
2. Go to Settings > Models.
3. Click Organization provider to make the provider available across every project in the organization. To limit it to the current project, select the Project tab and click Project provider instead.
4. Under Model providers, click Anthropic.
5. Choose your authentication method:
   • API key: Paste an Anthropic API key into the field below the authentication options.
     API keys are stored as one-way cryptographic hashes, never in plaintext.
   • Workload identity federation: Exchange a Braintrust-signed OIDC token for an Anthropic access token, instead of storing a long-lived Anthropic API key in Braintrust.
     Workload identity federation is available only for organization-level providers on Braintrust-hosted organizations with the Braintrust gateway enabled. Project-level providers and self-hosted deployments must use API key authentication.
6. If you chose Workload identity federation, use the setup values shown in Braintrust to configure Anthropic:
   1. Create an Anthropic service account. Copy the svac_... service account ID.
   2. Register the Braintrust issuer in Anthropic. Use the Issuer URL shown in Braintrust, set JWKS source to OIDC discovery, leave Discovery base URL blank, turn Single-use tokens on, and set Max token lifetime to 1 hour.
   3. Create a federation rule in Anthropic. Use the Subject pattern, Expected audience, and Required claims shown in Braintrust. Select the service account from the previous step and the Anthropic workspace Braintrust should use.
   4. Paste the Anthropic IDs back into Braintrust:
   • Federation rule ID: The fdrl_... value in Anthropic’s Workload identity > Rules table.
   • Organization ID: The ID shown on Anthropic’s Organization settings > Organization page.
   • Service account ID: The svac_... ID from the Anthropic service account.
   • Workspace ID: Use default for Anthropic’s default workspace, or the workspace ID selected in the federation rule.
   • Subject suffix: A stable suffix for this Anthropic connection. It must match the final part of the subject pattern used in Anthropic.
   For general Anthropic concepts and Console details, see Anthropic’s workload identity federation docs.
7. Click Create, or Update when editing an existing provider.

pnpm add braintrust @anthropic-ai/sdk

npm install braintrust @anthropic-ai/sdk

ANTHROPIC_API_KEY=<your-anthropic-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>

import Anthropic from "@anthropic-ai/sdk";
import { initLogger } from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const result = await client.messages.create({
  model: "claude-sonnet-4-5-20250929",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What is machine learning?" }],
});

node --import braintrust/hook.mjs trace-anthropic-auto.js

import { AnthropicBedrock } from "@anthropic-ai/bedrock-sdk";
import { initLogger } from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Resolves AWS credentials from the standard environment chain
const client = new AnthropicBedrock();

const result = await client.messages.create({
  model: "anthropic.claude-3-5-sonnet-20241022-v2:0",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What is machine learning?" }],
});

node --import braintrust/hook.mjs trace-anthropic-bedrock-auto.js

pnpm add braintrust @anthropic-ai/sdk

npm install braintrust @anthropic-ai/sdk

ANTHROPIC_API_KEY=<your-anthropic-api-key>
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>

import Anthropic from "@anthropic-ai/sdk";
import { wrapAnthropic, initLogger } from "braintrust";

// Initialize the Braintrust logger
const logger = initLogger({
  projectName: "My Project", // Your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Wrap the Anthropic client with the Braintrust logger
const client = wrapAnthropic(
  new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }),
);

// All API calls are automatically logged
const result = await client.messages.create({
  model: "claude-sonnet-4-5-20250929",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What is machine learning?" }],
});

import { AnthropicBedrock } from "@anthropic-ai/bedrock-sdk";
import { wrapAnthropic, initLogger } from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Resolves AWS credentials from the standard environment chain
const client = wrapAnthropic(new AnthropicBedrock());

const result = await client.messages.create({
  model: "anthropic.claude-3-5-sonnet-20241022-v2:0",
  max_tokens: 1024,
  messages: [{ role: "user", content: "What is machine learning?" }],
});

import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const response = await client.chat.completions.create({
  model: "claude-sonnet-4-5-20250929",
  messages: [{ role: "user", content: "What is a gateway?" }],
  seed: 1, // A seed activates the gateway's cache
});

import Anthropic from "@anthropic-ai/sdk";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import { z } from "zod";

const client = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

// Define a Zod schema for the response
const ResponseSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const message = await client.messages.parse({
  model: "claude-sonnet-4-5-20250929",
  max_tokens: 1024,
  messages: [
    { role: "user", content: "My name is John and I'm 30 years old." },
  ],
  output_config: {
    format: zodOutputFormat(ResponseSchema),
  },
});

console.log(message.parsed_output); // { name: "John", age: 30 }

import { OpenAI } from "openai";
import { z } from "zod";

const client = new OpenAI({
  baseURL: "https://gateway.braintrust.dev/v1",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

// Define a Zod schema for the response
const ResponseSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const completion = await client.beta.chat.completions.parse({
  model: "claude-sonnet-4-5-20250929",
  messages: [
    { role: "system", content: "Extract the person's name and age." },
    { role: "user", content: "My name is John and I'm 30 years old." },
  ],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "person",
      // The Zod schema for the response
      schema: ResponseSchema,
    },
  },
});

import Anthropic from "@anthropic-ai/sdk";
import { wrapAnthropic, collectAnthropicSession, initLogger } from "braintrust";

initLogger({
  projectName: "My Project", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = wrapAnthropic(
  new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }),
);

// Open a sessions event stream. See the Anthropic Sessions API for parameters.
const stream = client.beta.sessions.events.stream({
  // Session request parameters
});

// Opt the stream into Braintrust span collection before consuming it
collectAnthropicSession(stream);

// The sessions span is recorded once the stream is fully consumed
for await (const event of stream) {
  // Handle each event
}

import { Eval } from "braintrust";
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.ANTHROPIC_API_KEY,
});

Eval("Anthropic Evaluation", {
  // An array of user inputs and expected outputs
  data: () => [
    { input: "What is 2+2?", expected: "4" },
    { input: "What is the capital of France?", expected: "Paris" },
  ],
  task: async (input) => {
    // Your Anthropic LLM call
    const response = await client.messages.create({
      model: "claude-sonnet-4-5-20250929",
      max_tokens: 1024,
      messages: [{ role: "user", content: input }],
    });
    return response.content[0].text;
  },
  scores: [
    {
      name: "accuracy",
      // A simple scorer that returns 1 if the output matches the expected output, 0 otherwise
      scorer: (args) => (args.output === args.expected ? 1 : 0),
    },
  ],
});

pnpm add autoevals

npm install autoevals

import { LLMClassifierFromSpec } from "autoevals";

const relevanceScorer = LLMClassifierFromSpec("Relevance", {
  choice_scores: { Relevant: 1, Irrelevant: 0 },
  model: "claude-sonnet-4-5-20250929",
  use_cot: true,
});

​

Export OpenTelemetry traces from the Claude API

​

Add Braintrust as a destination

​

What Braintrust traces

• Model request spans become LLM spans with the model and token metrics, so Braintrust can estimate cost.
• Tool spans become tool spans named after the tool.
• With Full content, the system prompt, input messages, and output messages of anthropic.messages.create and anthropic.session.turn spans become the span input and output, and tool arguments become the input of tool spans.
• gen_ai.conversation.id, which holds the Managed Agents session thread ID, is stored with the span. Other attributes, such as anthropic.session.id, are stored as metadata.

SELECT
  SUM(metrics.prompt_tokens) AS prompt_tokens,
  SUM(metrics.completion_tokens) AS completion_tokens,
  SUM(estimated_cost()) AS estimated_cost
FROM project_logs('<PROJECT_ID>') -- Replace with your project ID
WHERE root_span_id = '<ROOT_SPAN_ID>' -- Replace with the trace's root span ID
  AND span_attributes.name IN ('anthropic.model_request', 'anthropic.compaction', 'anthropic.session.outcome_evaluation')

​

Connect Claude API spans to your application’s traces