> ## Documentation Index > Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt > Use this file to discover all available pages before exploring further. # Product updates > New updates and product improvements **October 2026** ### Trace Muse Code sessions Trace [Muse Code](https://docs.braintrust.dev/integrations/developer-tools/muse-code.md) sessions, including turns, model calls, tool calls, and subagents. Run `bt trace enable muse` to set up tracing, which requires `bt` v0.28.0 or later and Muse Code 1.3.0 or later. See [Muse Code](https://docs.braintrust.dev/integrations/developer-tools/muse-code.md) for details. ### MCP server tracing The Python SDK v0.49.1 or later and TypeScript SDK v3.42.0 or later support [MCP tracing](https://docs.braintrust.dev/integrations/sdk-integrations/mcp-sdk.md), capturing tool calls, prompt requests, resource reads, and server notifications. The Python SDK also traces outgoing MCP client requests, including standalone FastMCP, with trace context propagated between clients and servers. ### Async queries SQL queries can now run asynchronously, so large exports and long-running reads don't have to finish within a single synchronous request. Braintrust runs an async query in the background and stores the results, and you fetch them in pages. Run async queries with the [API](https://docs.braintrust.dev/api-reference/query.md#async-queries) or with [`bt sql --async`](https://docs.braintrust.dev/reference/cli/sql#async-queries). > [!NOTE] > > Async queries are only available on the [Enterprise plan](https://docs.braintrust.dev/plans-and-limits.md#plans). Requires data plane v2.3.0+ (Self-hosted customers must upgrade to use this feature.) ### Terraform provider The [Braintrust Terraform provider](https://docs.braintrust.dev/reference/terraform.md) manages your organization's configuration as code, including projects, permission groups, access control, AI provider secrets, environment variables, automations, prompts, and functions. Changes go through code review and stay consistent across environments. Install it from the [Terraform Registry](https://registry.terraform.io/providers/braintrustdata/braintrust/latest/docs). > [!WARNING] > > The Terraform provider is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Thread summaries The [new thread view](https://docs.braintrust.dev/observe/examine-traces.md#generate-thread-summaries) can show Loop-generated headings for each turn and a summary of the whole conversation, so you can scan a long agent trace before reading it. Select a phrase in the summary to jump to the turn it describes. > [!WARNING] > > Thread summaries are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Continuous Loop automations Scheduled [Loop automations](https://docs.braintrust.dev/loop/automations.md) can run in continuous mode. Between scheduled runs, Loop checks for new traces and, when enough arrive, analyzes them in the current run's thread. Busy projects get findings sooner without starting a new investigation each time. Requires data plane v2.20.0+ (Self-hosted customers must upgrade to use this feature.) ### GitHub repositories in Loop Connect [GitHub repositories to Loop](https://docs.braintrust.dev/loop/github.md) to investigate failing traces alongside your source code. Loop can find the relevant implementation, explain likely causes, and suggest targeted fixes and regression tests. Access is read-only by default, but you can enable [write access](https://docs.braintrust.dev/loop/github.md#give-loop-write-access) to let Loop push fixes and open pull requests. > [!WARNING] > > The GitHub integration is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.18.0+ for GitHub read access (Self-hosted customers must upgrade to use this feature.) Requires data plane v2.21.0+ for GitHub write access (Self-hosted customers must upgrade to use this feature.) ### Reorganized settings \*\* Settings\*\* groups organization, project, and personal settings by task in one sidebar and merges related pages. For example, \*\* Models\*\* holds organization and project AI providers and Loop's model settings, and \*\* Permissions\*\* holds permission groups, project permissions, and SCIM as tabs. Search the Settings sidebar or the command bar to jump to a setting. Links to previous settings pages redirect to the reorganized pages. See [Manage organizations](https://docs.braintrust.dev/admin/organizations.md) for details. ### Inference activity on the billing page The [**Inference activity**](https://docs.braintrust.dev/admin/billing/monitor-usage.md#view-usage) section of the billing page shows how your Braintrust Gateway token usage over the last 7 days splits across features, such as Online scoring, Topics, and Loop, and across your top projects, so you can see what drives your inference usage. ### Add rows to human review with MCP The Braintrust MCP server can now send project log traces, experiment traces, or dataset rows to [human review](https://docs.braintrust.dev/annotate/human-review/score-traces.md) with the [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#tools), and optionally assign them to reviewers. Loop automations can queue rows for review on a schedule when you allow the tool in their [write tool permissions](https://docs.braintrust.dev/loop/automations.md#write-tool-permissions). Requires data plane v2.19.0+ (Self-hosted customers must upgrade to use this feature.) ### Customize pattern severity and categories You can customize the [severity and category rubric](https://docs.braintrust.dev/observe/patterns/enable.md#customize-severity-and-categories) that Loop uses across a project's Patterns automations and conversations. Define your own categories and severity criteria to match what matters to your team. > [!WARNING] > > Patterns is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.19.0+ (Self-hosted customers must upgrade to use this feature.) ### Pattern severity and categories Patterns assigns [severity](https://docs.braintrust.dev/observe/patterns/review.md#severity) so you can prioritize findings by impact and filter the list, and [categories](https://docs.braintrust.dev/observe/patterns/review.md#categories) to describe the kind of finding, such as a tool failure or safety and privacy risk. You can correct either with an optional reason, and Loop preserves your corrections in later investigations. > [!WARNING] > > Patterns is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) ### PagerDuty notifications for automations Send notifications from [Loop automations](https://docs.braintrust.dev/loop/automations.md#destinations) and [log, time window, and environment alerts](https://docs.braintrust.dev/observe/alerts.md) directly to PagerDuty. Configure an integration key and incident severity for each destination. Requires data plane v2.17.0+ (Self-hosted customers must upgrade to use this feature.) ### Coding agent insights Coding agent insights shows what your team's coding agents cost, who uses them, and where they could work more effectively. A built-in dashboard at \*\* Dashboards\*\* > [**Coding agent insights**](https://www.braintrust.dev/app/~/agent-insights) shows usage by user, API key, model, tool, and branch, with cost breakdowns for all except tools. A daily Loop automation finds recurring problems and inefficiencies across your team's sessions. To learn more, see: * [Improving coding agents with coding agent insights](https://docs.braintrust.dev/cookbook/recipes/CodingAgentInsights.md) * [Coding agent insights dashboard](https://docs.braintrust.dev/observe/dashboards/coding-agent-insights.md) > [!WARNING] > > Coding agent insights is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.13.0+ (Self-hosted customers must upgrade to use this feature.) ### New thread view The new thread view organizes a trace by conversation turn. Each turn shows the request, the work the agent did to answer it, and the final response, so you can follow a long agent trace without reading every intermediate message. See [Use the new thread view](https://docs.braintrust.dev/observe/examine-traces.md#use-the-new-thread-view) for details. > [!WARNING] > > The new thread view is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### SQL charts Build dashboard charts from a SQL query, and switch between the query text and a visual builder that edits the same query. Braintrust adds the dashboard's timeframe, filters, and grouping when the chart renders, so the same chart works on any dashboard you paste it into. You can also chart a query's results in the [\*\* SQL sandbox\*\*](https://www.braintrust.dev/app/~/sql) and save a chart to a dashboard. See [SQL charts](https://docs.braintrust.dev/observe/dashboards/sql-charts.md) and [Chart a query in the SQL sandbox](https://docs.braintrust.dev/observe/dashboards/sql-charts.md#chart-a-query-in-the-sql-sandbox) for details. > [!WARNING] > > SQL charts are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. > [!NOTE] > > SQL charts are only available on [Pro and Enterprise plans](https://docs.braintrust.dev/plans-and-limits.md#plans). Requires data plane v2.1.1+ (Self-hosted customers must upgrade to use this feature.) ### Loop on dashboards Open [Loop](https://docs.braintrust.dev/loop.md) from an individual dashboard or the chart editor to investigate data and build or refine charts in natural language. The dashboard or chart is added to your conversation's context automatically. See [Build charts with Loop](https://docs.braintrust.dev/observe/dashboards/build-charts.md#build-charts-with-loop) for details. ### Copy logs into datasets with MCP and Loop Build datasets from existing project log rows with the [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#run-evals-and-edit-datasets) or [Loop](https://docs.braintrust.dev/loop/capabilities.md#curate-datasets). Copied rows preserve their input, metadata, tags, and links to the source logs without passing their full contents through the assistant's context. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) ### Trace Cursor sessions Trace [Cursor](https://docs.braintrust.dev/integrations/developer-tools/cursor.md) agent sessions from the Cursor CLI and desktop app, including turns, model calls, tool calls, and subagents. Run `bt trace enable cursor` to set up tracing, which requires `bt` v0.26.0 or later. See [Trace Cursor sessions](https://docs.braintrust.dev/integrations/developer-tools/cursor.md#trace-cursor-sessions) for details. ### Redesigned permission groups You can now rename a permission group and edit its description, permissions, and members in one sheet. The permission groups list shows each group's member count, so you can check membership without opening each group. You grant access to a project group from a permission group instead of from the project group's settings. See [Edit or delete a permission group](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#edit-or-delete-a-permission-group) and [Set project group permissions](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#set-project-group-permissions) for details. > [!NOTE] > > Editing a permission group's name, description, and permissions is only available on the [Enterprise plan](https://docs.braintrust.dev/plans-and-limits.md#plans). ### Reorganized navigation On desktop, the left navigation groups project pages under **Observe** and **Evaluate**, and you can collapse it to an icon-only rail with or the `[` keyboard shortcut. The organization and project switchers move to the breadcrumbs at the top of each page, the project overview is labeled **Home**, and the SQL sandbox is renamed [**Query**](https://docs.braintrust.dev/reference/sql.md#sql-sandbox). ### Duplicate prompts and scorers in bulk and across projects Select up to 100 prompts or scorers in the **Prompts** or **Scorers** table and duplicate them in one step, or duplicate a single prompt or scorer from its header or from a saved version in its **Activity** tab. Copies can go into the current project or any project where you can create and update prompts. Each copy's **Activity** links to the source version it came from and notes whether it included unsaved editor changes. See [Duplicate prompts](https://docs.braintrust.dev/evaluate/prompts/manage.md#duplicate-prompts) for details. ### Go workflow evaluations Go evaluations (SDK v0.18.0+) can now send their tasks and scorers to asynchronous provider APIs, such as batch APIs, which providers often offer at a discount. Start a run, then resume it later by polling or from a webhook. With a Redis store, a different process can resume the run. Braintrust records the results in a single experiment. See [Workflow evaluations](https://docs.braintrust.dev/sdks/go/workflow-evaluations.md) for details. > [!WARNING] > > Go workflow evaluations are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### TypeScript application logs [Application logs](https://docs.braintrust.dev/sdks/typescript/application-logs.md) are now available in the TypeScript SDK (v3.39.0+). Emit messages with a logger's severity methods such as `info()` and `error()`, or forward existing `console` output with `wrapConsole()`, then inspect them alongside your traces. > [!WARNING] > > TypeScript application logs are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Default provider for Loop Set the provider that new [Loop](https://docs.braintrust.dev/loop.md) threads and automations use across your organization: Braintrust built-in models or one of your AI providers. Loop uses that provider's recommended model, and members can still pick another for any thread or automation. See [Set a default provider for Loop](https://docs.braintrust.dev/admin/organizations.md#set-a-default-provider-for-loop). ### Account billing across organizations Organizations that Braintrust links under one contract share a billing account. Owners, and members with the **View account billing** permission, can see the account's billing period, billing email, and linked organizations from the billing page. See [View account billing](https://docs.braintrust.dev/admin/billing.md#view-account-billing) for details. ### bt CLI releases * [v0.28.1](https://github.com/braintrustdata/bt/releases/tag/v0.28.1) * **Codex command errors**: [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md#what-braintrust-traces) tool spans record an error when a shell command exits with a nonzero code, even if the tool call as a whole completes, and record each command's result in `metadata.command_outcomes`. * **Docs fetch**: [`bt docs fetch`](https://docs.braintrust.dev/reference/cli/docs) recognizes documentation page links on `docs.braintrust.dev` in the docs index, in addition to `www.braintrust.dev/docs`. * [v0.28.0](https://github.com/braintrustdata/bt/releases/tag/v0.28.0) * **MCP tools from the CLI**: [`bt mcp`](https://docs.braintrust.dev/reference/cli/mcp) (public preview) lists and runs the tools exposed by the [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md) without an MCP client, with a generated flag for each tool argument or with JSON input. * **[Muse Code](https://docs.braintrust.dev/integrations/developer-tools/muse-code.md) tracing**: [`bt trace enable muse`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-enable) traces Muse Code sessions, turns, model calls, tool calls, and subagents. `bt trace run muse` traces a single invocation, and `bt trace import muse` imports past or active sessions. Requires Muse Code 1.3.0 or later. * **Codex imports**: [`bt trace import codex`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-import) imports sessions whose transcripts Codex has compressed into `.jsonl.zst` files. You can't follow a compressed session with `--attach`. * **Tracing fixes**: [OpenCode](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#what-braintrust-traces) tool spans keep images that tools return, so they render as images in the trace. When a [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) turn starts from a custom message instead of a typed prompt, the turn records that message as its input. * [v0.27.2](https://github.com/braintrustdata/bt/releases/tag/v0.27.2) * **Codex hook activation**: [`bt trace enable codex`](https://docs.braintrust.dev/integrations/developer-tools/codex#set-up-tracing) and `bt trace update codex` trust the Braintrust hooks in Codex, so you no longer approve them through `/hooks`. * **Span plugin timeouts**: [Span plugins](https://docs.braintrust.dev/reference/cli/trace.md#transform-spans-with-javascript) have 1 second instead of 50 milliseconds per call, and `bt` retries span updates that time out twice before holding them. [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) lists held sessions and what `bt` is waiting for before it sends their spans. * **Hook executable**: Set `BRAINTRUST_BT_BIN` to an absolute path to have `bt trace hook` [capture events with that `bt` executable](https://docs.braintrust.dev/reference/cli/trace#troubleshooting) instead of the one on `PATH`. * **Trace delivery**: The background tracing process sends spans in batches of up to 30 seconds and frees delivered span data, so long sessions use less memory. Spans are sent right away when a session ends. * [v0.27.1](https://github.com/braintrustdata/bt/releases/tag/v0.27.1) * **Eval fixes**: [`bt eval`](https://docs.braintrust.dev/reference/cli/eval) no longer requires a Go toolchain to run a [prebuilt Go eval binary](https://docs.braintrust.dev/reference/cli/eval.md#runtime-configuration) passed with `--runner`, and reports the actual cause when Go eval discovery fails. * **TLS trust update**: `bt` distrusts certificates affected by the [eMudhra (emSign) certificate authority compromise](https://emudhra.we-have-a-problem.com/). * [v0.27.0](https://github.com/braintrustdata/bt/releases/tag/v0.27.0) * **Topics settings in observability templates**: [`bt observability template`](https://docs.braintrust.dev/reference/cli/observability) templates bundle each facet's Topics automation settings, such as scope, filter, sampling, and model, so a push recreates them in the destination project. With `--force`, a push replaces a facet's topic map binding only in the selected Topics automation. * **Reliable coding-agent trace delivery**: `bt` saves coding-agent spans to disk and retries failed uploads, including after a restart. When a [span plugin](https://docs.braintrust.dev/reference/cli/trace.md#transform-spans-with-javascript) fails, `bt` holds the affected spans until you fix the plugin instead of dropping them. * **Richer coding-agent traces**: Every span records the plugin and agent versions in `context.span_origin`. [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#what-braintrust-traces) and [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md#what-braintrust-traces) turns record how they ended in `metadata.outcome`, and interruptions are no longer recorded as errors. Claude Code also records the effort level and compactions, [Cursor](https://docs.braintrust.dev/integrations/developer-tools/cursor.md) subagent spans nest under the tool call that started them, and [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) sessions record the working directory and model. * **Eval fixes**: [`bt eval`](https://docs.braintrust.dev/reference/cli/eval) finds Go evals from any folder in a Go module, rejects Go packages that don't import `evalrunner`, shows runner errors when output isn't a terminal, such as in CI, and fixes the automatic `vite-node` retry for TypeScript evals. * **Tracing fixes**: Fixes missing Cursor traces on Windows, shows the arguments a Claude Code tool actually ran with, and ends Codex tool spans that never receive output. * **Claude Code changes**: `AskUserQuestion` spans are task spans instead of tool spans. Update queries that select them as tool spans. Resuming a session that already ended starts a new trace with the same `session_id`. * [v0.26.0](https://github.com/braintrustdata/bt/releases/tag/v0.26.0) * **Preprocessors from the CLI**: [`bt preprocessors`](https://docs.braintrust.dev/reference/cli/preprocessors) bootstraps, previews, and pushes preprocessors written in TypeScript or JavaScript with `customPreprocessor()` from `braintrust/preprocessors`. You can also use it to list, view, invoke, and delete published preprocessors. Preview runs a local preprocessor against a real trace without publishing it. * **Scorer preprocessors**: [`bt scorers create --preprocessor`](https://docs.braintrust.dev/reference/cli/scorers#preprocessors) selects the preprocessor that builds a prompt-based scorer's trace template variables. It accepts a pushed preprocessor's slug, a built-in preprocessor, a local file, or `none`. * **Environment variable names**: Command-specific environment variables use the `BRAINTRUST_` prefix, matching the Braintrust SDKs. For example, `BT_EVAL_WATCH` is now `BRAINTRUST_EVAL_WATCH`, and coding-agent tracing's `BT_DAEMON_CONFIG` is now `BRAINTRUST_DAEMON_CONFIG`. The `BT_` names still work as deprecated aliases. See the [v0.26.0 migration instructions](https://docs.braintrust.dev/reference/cli/migrate.md#v0-26-0). * **[Cursor](https://docs.braintrust.dev/integrations/developer-tools/cursor.md) tracing**: [`bt trace enable cursor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-enable) adds capture hooks to Cursor's `hooks.json`, so sessions are traced even when Cursor doesn't load the local plugin. * **Coding-agent tracing**: [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md), [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md), and [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) session spans record the final response of the last completed turn, and pi turn spans record each turn's final response. Codex traces keep images in prompts and tool results, so they render as images in the trace. * [v0.25.0](https://github.com/braintrustdata/bt/releases/tag/v0.25.0) * **Turn attribution**: Coding-agent traces record `metadata.turn_span_id` on each turn span and every span under it, including subagent spans, so you can filter or aggregate a turn's spans without walking the trace tree. * **Span origin versions**: Coding-agent session and turn spans record the plugin, `bt`, and coding agent versions in `context.span_origin`, captured when each span is created. Unknown versions are `null`. See the integration page for your agent, such as [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#what-braintrust-traces). * [v0.24.0](https://github.com/braintrustdata/bt/releases/tag/v0.24.0) * **Plugin updates**: [`bt update`](https://docs.braintrust.dev/reference/cli/update) now updates your installed tracing plugins too, even if `bt` itself is already up to date. Restart your coding agents to pick up the new plugin versions. * **Windows hooks**: Fixes hooks that could hang when starting the background tracing process. * [v0.23.0](https://github.com/braintrustdata/bt/releases/tag/v0.23.0) * **[pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) tracing**: Nests pi 1.0 codemode tool calls under the calling tool, and records virtual model selections, request settings, and tool definitions on model call spans. **Breaking change**: On pi model call spans, `reasoning_tokens` is now `completion_reasoning_tokens`, `cost` is now `estimated_cost`, `metadata.provider_request` is replaced by top-level fields such as `temperature` and `tools`, and skill loads use `tool_name: "read"` with `tool_kind: "skill"`. Update queries that use the earlier fields. * **Long session IDs**: Fixes missing traces for sessions with very long IDs. * **Span metadata**: **Breaking change**: Spans no longer include `metadata.bt_daemon_version`. * [v0.22.2](https://github.com/braintrustdata/bt/releases/tag/v0.22.2) * **Windows credential storage**: On Windows, `bt` stores profiles and credentials in one DPAPI-encrypted file, `%USERPROFILE%\.braintrust\auth\credentials.dpapi`, so commands run from a terminal and from packaged (MSIX) desktop apps share the same logins. The first command that reads saved logins after upgrading migrates them automatically. After downgrading to an earlier `bt` version, log in again. See [Authentication and profiles](https://docs.braintrust.dev/reference/cli/overview.md#authentication-and-profiles) for details. * **Project groups in `bt functions push`**: When `bt functions push` creates a missing project, it places it in the [project group](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#manage-project-groups) set on the project with `projectGroupName` (TypeScript SDK v3.36.0+) or `project_group_name` (Python SDK v0.44.0+). Existing projects are not moved. See [`bt functions push`](https://docs.braintrust.dev/reference/cli/functions#bt-functions-push) for details. * **[Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md) tracing**: Nests subagent results under the turn that started the subagent instead of recording them as a separate turn span. * **Hook errors**: `bt trace hook`, which agent plugins run for each event, exits nonzero when it can't capture the event, such as when tracing settings are invalid or the background tracing process is unreachable. See [Troubleshooting](https://docs.braintrust.dev/reference/cli/trace.md#troubleshooting) for details. * **Private tracing files on Windows**: Tracing settings files, the tracing data directory, and the background tracing process's named pipe are accessible only to the current user and SYSTEM, matching the owner-only permissions on macOS and Linux. On every platform, `bt trace import` keeps its data directory private. * **No console windows on Windows**: Background tracing no longer flashes a console window when it runs helper commands such as `git`. ### Python SDK releases * [v0.49.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.49.1) - [MCP content capture configuration](https://docs.braintrust.dev/integrations/sdk-integrations/mcp-sdk.md#capture-content-python) moves from `auto_instrument()` to `MCPIntegration.setup(capture_content=False)` or the per-object wrappers. Pass `client=True` to `MCPIntegration.setup()` to configure clients as well as servers. * [v0.49.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.49.0) - Added tracing for [MCP clients and servers](https://docs.braintrust.dev/integrations/sdk-integrations/mcp-sdk.md#auto-instrumentation-python), including standalone FastMCP, with automatic trace context propagation between clients and servers. Log uploads now batch nearby rows with a default 1-second delay in long-lived processes and no delay in detected serverless environments, configurable with [`BRAINTRUST_FLUSH_DELAY_MS`](https://docs.braintrust.dev/sdks/python/api-reference#environment-variables). * [v0.48.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.48.1) - Added tracing for [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#what-traced-python) beta tool runners and message threads. [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md#what-traced-python) rerank spans record `relevant_spans`, [Harbor](https://docs.braintrust.dev/integrations/sdk-integrations/harbor.md#what-received-python) omits scores for rewards that Harbor marks invalid, and [workflow evaluations](https://docs.braintrust.dev/sdks/python/api-reference.md#workflowtask-and-workflowscorer) retry a failed `submit` on the next `poll()`. Fixed Anthropic `messages.create(stream=True)` streams failing outside a `with` block and their spans not ending when closed early, [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) Decisions API generator inputs, and [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md#what-traced-python) reasoning token metrics from normalized usage. Anthropic streaming `time_to_first_token` now measures time to the first output content. * [v0.48.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.48.0) - [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md#what-traced-python) traces System One calls (`litellm.systemone` and `litellm.asystemone`) as `decision` spans and logs media passed in `images`, `assets`, `audio`, or `videos` as attachments. [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md#what-traced-python) spans record `safeguard_results` and `alignment`, and [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#what-traced-python) spans record the requested `speed`. [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md#what-traced-python) hook spans keep newer input fields such as `error` and `duration_ms`, and sub-agent runs on older bundled Claude Code CLIs create task spans. Fixed missing token metrics on [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) Responses API streams that end as incomplete or failed, and Claude Agent SDK orchestrator turns being nested under a running sub-agent. The SDK requires `certifi` v2026.10.19 or later, which distrusts certificates affected by the [eMudhra (emSign) certificate authority compromise](https://emudhra.we-have-a-problem.com/), and the wheel no longer ships the SDK's test modules. **Breaking change**: [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#what-traced-python) decision spans record images as `` unless [`BRAINTRUST_CAPTURE_ATTACHMENTS`](https://docs.braintrust.dev/sdks/python/api-reference#environment-variables) is set to `1` or `true`. * [v0.47.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.47.0) - [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#what-traced-python) decision spans now capture box answers and images. Images in the state, instructions, criteria, and `assets` are logged as attachments, and when the request contains exactly one image, box answers record its location as `image_path`. Fixed [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md#what-traced-python) tool calls that are deferred for approval or external execution being logged as errors. * [v0.46.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.46.1) - Base64 data URL images in [DSPy](https://docs.braintrust.dev/integrations/sdk-integrations/dspy.md#what-traced-python) spans and [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md#what-traced-python) completion messages are now logged as attachments, and [Instructor](https://docs.braintrust.dev/integrations/sdk-integrations/instructor.md#what-traced-python) `create_partial` spans log only the final partial model as `output`. Fixed `error_score_handler` not running for every failed row in [`Eval()`](https://docs.braintrust.dev/sdks/python/api-reference#eval) and being ignored by `EvalAsync()`, tags added through `hooks.tags` leaking into other trials, rows, or the dataset, and attachments and other nested values in logged dataclasses not being serialized. Also fixed [Gemini](https://docs.braintrust.dev/integrations/ai-providers/gemini.md#what-traced-python) Interactions API calls failing when `response_format` is a Pydantic model class, streamed Gemini code execution parts being logged without their content, [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) spans dropping message fields, and [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) tracing modifying the response's `metadata`. Reduced repeated project lookups and span serialization overhead. * [v0.46.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.46.0) - [Span customizers](https://docs.braintrust.dev/sdks/python/redact-sensitive-data.md#opentelemetry-spans) now also run on OpenTelemetry spans, and a customizer that raises no longer drops the record. Added tracing for the [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) and [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md#what-traced-python) Decisions APIs as `decision` spans, the [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md#what-traced-python) GA Responses API, and [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) workflow nodes and tool calls. [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#what-traced-python) spans now use the `decision` type. [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md#what-traced-python), [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md), and [Gemini](https://docs.braintrust.dev/integrations/ai-providers/gemini.md#what-traced-python) spans capture more request and response details. Fixed Google ADK model calls counting token usage twice, and OpenAI prompt cache write tokens being left out of cost and token totals. Added Python 3.15 support. * [v0.45.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.45.0) - Added `cancel_event` (`threading.Event`) to [`Eval()`](https://docs.braintrust.dev/sdks/python/api-reference#eval) and `EvalAsync()` to stop a running evaluation. Tasks can read it from `hooks.cancel_event`, and a `timeout` also sets it. Added [`WorkflowBatchTask` and `WorkflowBatchScorer`](https://docs.braintrust.dev/sdks/python/api-reference#workflowbatchtask-and-workflowbatchscorer) to submit many cases in each provider batch in [workflow evaluations](https://docs.braintrust.dev/sdks/python/workflow-evaluations.md). Workflow evaluation types can be imported from the top-level `braintrust` package, and [`process_submission_result()`](https://docs.braintrust.dev/sdks/python/api-reference#workfloweval) no longer requires a `run_id` when you pass an `external_id`. [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#what-traced-python) spans now record the `diagnostics` request parameter, plus `cache_miss_reason` and `cache_missed_input_tokens` in metadata when Anthropic reports a prompt cache miss. [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) Responses API spans now record `access_programs` in metadata, and Agents API sessions now trace computer use calls (with screenshots as attachments), browser authentication requests, and computer use approval items. Fixed duplicate spans when calling `wrap_openai()` after `auto_instrument()`, and [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) tracing swallowing task cancellation, including `asyncio.wait_for()` timeouts, in `query()` and `receive_response()`. * [v0.44.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.44.1) - Reduced the SDK's CPU overhead when queuing and sending logs, and log batches now use the full request size limit instead of half of it, so the SDK sends fewer requests. `span` in `with start_span(...) as span` now type-checks as `Span` under pyright and mypy, and `add_braintrust_span_processor()` parameters are fully typed. [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#what-traced-python) MCP connector tool calls now appear as child tool spans named after the MCP tool, with the tool input (including in streamed responses), instead of generic `mcp` spans. Fixed streamed [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md#workflows-python) workflow output including step progress events, and [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) Agents API sessions not being traced when you collect results with `get_final_result()` or `with_result_collection()`. * [v0.44.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.44.0) - [Span customizers](https://docs.braintrust.dev/sdks/python/redact-sensitive-data.md#span-customizers) now run on all native SDK spans, including spans you create with `traced()`, `start_span()`, and `logger.log()`, not only spans from integrations and wrappers. Dataset rows and feedback are still excluded. With `BRAINTRUST_OTEL_COMPAT` enabled, the SDK logs an error instead of installing customizers. Added `project_group_name` to [`projects.create()`](https://docs.braintrust.dev/admin/projects#create-a-project) to create a new project inside an existing project group. [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md) agent and team run spans record `cancellation_stage` in metadata (Agno v3.0.11 or later), [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) run spans record `verbatim_prompts` when verbatim prompt mode is enabled (Claude Agent SDK v0.2.158 or later), and [AgentScope](https://docs.braintrust.dev/integrations/agent-frameworks/agentscope.md) `TeamPipeline.reply_stream()` calls are now traced (AgentScope v2.0.9 or later). Fixed token metrics from [DSPy](https://docs.braintrust.dev/integrations/sdk-integrations/dspy.md) LM history, STT usage metrics for [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md), audio duration usage for [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md), classifier usage metrics for [Pipecat](https://docs.braintrust.dev/integrations/agent-frameworks/pipecat.md), and iterable `tools` metadata for [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md). Integer environment variables with empty or non-integer values now fall back to their defaults instead of raising an error. ### TypeScript SDK releases * [v3.42.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.42.0) - Added [MCP server tracing](https://docs.braintrust.dev/integrations/sdk-integrations/mcp-sdk.md) for servers built with `@modelcontextprotocol/sdk` v1 or `@modelcontextprotocol/server` v2, through auto-instrumentation or `wrapMCPServer()`. Tool and prompt arguments, results, and notification parameters are recorded by default. To keep the spans but omit this content, set [`mcp.captureContent`](https://docs.braintrust.dev/integrations/sdk-integrations/mcp-sdk#capture-content-typescript) to `false`. Notifications that the server sends, such as progress updates and log messages, are traced, and spans record the client and server names and versions from the `initialize` exchange. On HTTP transports without a session ID, spans after `initialize`, such as `notifications/initialized`, don't record `mcp.protocol.version`. Read the version from the `initialize` span instead. [Pi Coding Agent](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md#what-traced-typescript) tool spans record `metadata.tool_approval`, and spans created while a tool runs nest under the tool span. * [v3.41.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.41.0) - [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md#audio-attachments-typescript-auto) records the caller's and the agent's audio separately with `BRAINTRUST_CAPTURE_USER_AUDIO_ATTACHMENTS` and `BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS`, and `session.close()` resolves after the recording finishes uploading. Install [`@braintrust/audio`](https://docs.braintrust.dev/sdks/typescript/related/audio) to reduce the size of uploaded recordings. Fixed tool tracing for [Eve](https://docs.braintrust.dev/integrations/agent-frameworks/eve.md) 0.72 and later. Fixed missing finish reasons on streamed [Gemini](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) responses that end without content. **Breaking change**: `BRAINTRUST_CAPTURE_ATTACHMENTS` no longer records LiveKit Agents audio. Set the two LiveKit variables instead. * [@braintrust/audio v0.1.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/%40braintrust/audio%400.1.0) - Initial release. [`@braintrust/audio`](https://docs.braintrust.dev/sdks/typescript/related/audio) reduces the size of uploaded [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md#audio-attachments-typescript-auto) call recordings. Requires `braintrust` v3.41.0 or later and Node.js 20 or later. The package is in early development (alpha) until v1.0, so expect backwards-incompatible changes between releases. * [v3.40.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.40.0) - Added Cloudflare [Workers AI](https://docs.braintrust.dev/integrations/sdk-integrations/cloudflare.md#cloudflare-workers-ai-typescript) tracing for Clef and Clairic calls with `wrapWorkersAI()`. Long-lived Node.js processes now batch log rows for up to 1 second, configurable with [`BRAINTRUST_FLUSH_DELAY_MS`](https://docs.braintrust.dev/sdks/typescript/api-reference#environment-variables), so call [`flush()`](https://docs.braintrust.dev/sdks/typescript/api-reference#flush) before `process.exit()` or in signal handlers. AI SDK tool spans log [`toModelOutput`](https://docs.braintrust.dev/integrations/sdk-integrations/vercel#what-traced-typescript) results. [Pi Coding Agent](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md#what-traced-typescript) LLM spans capture system prompts and tools, and auto-instrumentation supports Pi 1.x. Fixed double-counted Pi Coding Agent token totals by recording token metrics only on LLM spans, not on `AgentSession.prompt` task spans. [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md#what-traced-typescript-auto) traces include `livekit.` lifecycle spans. Fixed auto-instrumentation with [TypeScript loaders](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) such as `@poppinss/ts-exec`. * [v3.39.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.39.0) - Added [application logs](#typescript-application-logs). Added tracing for [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md#auto-instrumentation-typescript) v1.9.x pipeline sessions, AI SDK [batch requests](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#trace-batch-requests-typescript) and [code mode](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#trace-code-mode-typescript), and [Cloudflare Think](https://docs.braintrust.dev/integrations/agent-frameworks/cloudflare-think.md) up to v0.20. Fixed output and token usage for streamed AWS Bedrock calls to OpenAI models, merged streamed tool calls from OpenAI, OpenRouter, and Hugging Face, and cache tokens missing from [Pi Coding Agent](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md) and [Flue](https://docs.braintrust.dev/integrations/agent-frameworks/flue.md) `prompt_tokens`. Internal SDK errors now print only when [`BRAINTRUST_DEBUG_LOG_LEVEL`](https://docs.braintrust.dev/sdks/typescript/api-reference#environment-variables) is set, which replaces `DEBUG=@braintrust*`. * [@braintrust/otel v1.1.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/%40braintrust/otel%401.1.0) - [Span customizers](https://docs.braintrust.dev/sdks/typescript/redact-sensitive-data.md#opentelemetry-spans) now also run on OpenTelemetry spans that `BraintrustSpanProcessor` and `BraintrustExporter` export, and in OpenTelemetry compatibility mode. Requires `braintrust` v3.38.0 or later. * [v3.38.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.38.0) - Added tracing for [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-typescript) Decisions API calls and AI SDK [`experimental_decide()`](https://docs.braintrust.dev/integrations/ai-providers/typesafe#manual-instrumentation-ai-sdk-typescript) calls, including AI SDK decision telemetry callbacks. [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#what-traced-typescript) spans capture image inputs, and decision spans capture inline media as attachments when `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled. A failed [span customizer](https://docs.braintrust.dev/sdks/typescript/redact-sensitive-data.md#customizer-failures) now uploads only the record's protected fields and timestamps instead of dropping the record, and customizers run on OpenTelemetry spans with `@braintrust/otel` v1.1.0 or later. Empty `BRAINTRUST_API_URL` and `BRAINTRUST_PROXY_URL` values are ignored. Fixed tree shaking for browser and edge builds, and queued logins replacing active credentials. Rebuild bundles that use an [auto-instrumentation bundler plugin](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) after upgrading. TypeSafe spans use the `decision` span type instead of `question`. Base64 images in OpenAI Responses API inputs are recorded as `` unless `BRAINTRUST_CAPTURE_ATTACHMENTS` is enabled. * [v3.37.2](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.37.2) - Fixed `ReactNode` type conflicts in React 19 projects that use `braintrust/custom-views`. `@types/react` is now an optional peer dependency and is no longer installed for every consumer. * [v3.37.1](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.37.1) - Fixed the `braintrust/vite` [auto-instrumentation plugin](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) crashing during Vite 8 dependency optimization. * [v3.37.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.37.0) - Added `signal` to [`Eval()`](https://docs.braintrust.dev/sdks/typescript/api-reference#eval) task hooks for cooperative cancellation. When an evaluation times out or its `signal` aborts, it aborts `hooks.signal`, stops starting new cases, and waits for running tasks and scorers to finish before it rejects, instead of rejecting right away and leaving tasks running in the background. A `timeout` of `0` now times out immediately. Added [`WorkflowBatchTask` and `WorkflowBatchScorer`](https://docs.braintrust.dev/sdks/typescript/workflow-evaluations) to submit many cases in each provider batch in workflow evaluations. Workflow evaluations also retry a failed `submit` on the next `poll()`, and `processSubmissionResult()` accepts an `externalId` without a `runId`. Added support for [Eve](https://docs.braintrust.dev/integrations/agent-frameworks/eve.md) 0.62 and later, and Eve turns now nest under the caller's span when the caller propagates trace context. [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) agent run spans now include sub-agent token usage across all models, `estimated_cost`, and `total_cost_usd` metadata. Fixed missing output and token metrics on [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) Responses API streams that end with `response.incomplete` or `response.failed`, and the error is now logged for `response.failed`. * [v3.36.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.36.0) - [Span customizers](https://docs.braintrust.dev/sdks/typescript/redact-sensitive-data.md#span-customizers) now run on all native SDK spans, including spans you create with `traced()`, `startSpan()`, and `logger.log()`, not only spans from instrumentation and wrappers. Dataset rows and feedback are still excluded. Span customizers aren't supported in OpenTelemetry compatibility mode. Eval and function parameters now accept Zod v4 schemas as well as Zod v3. Added `projectGroupName` to [`projects.create()`](https://docs.braintrust.dev/admin/projects#create-a-project) to create a new project inside an existing project group. Fixed empty `BRAINTRUST_NUM_RETRIES` and `BRAINTRUST_DEFAULT_BATCH_SIZE` environment variables being read as `0`, missing reasoning text from streamed [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md)-compatible responses that return `reasoning_content`, and missing model names on streamed Anthropic calls traced through [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md). ### Go SDK releases * [v0.19.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.19.0) - Added the [`openapi` package](https://docs.braintrust.dev/sdks/go/api-reference#api-client), a REST API client generated from the Braintrust OpenAPI specification that covers projects, experiments, datasets, prompts, functions, project automations, function invocation, and more. Added `prompt.LoadWithClient` and `eval.NewEvaluatorWithClient`, which take an `*openapi.Client`. The handwritten `api` package, `Client.API()`, `prompt.Load`, and `eval.NewEvaluator` are deprecated but keep working. Use `openapi` for new code. `Client.LoadPrompt` and `braintrust.NewEvaluator` aren't deprecated. * [v0.18.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.18.0) - Added [workflow evaluations](#go-workflow-evaluations). Added `InsertEvents` to the [experiments API client](https://docs.braintrust.dev/sdks/go/api-reference.md#api-client) to insert or merge experiment rows. Fixed [`bt eval --max-concurrency`](https://docs.braintrust.dev/reference/cli/eval#evaluator-concurrency) having no effect on Go evals, which ran evaluators one at a time. * [v0.17.2](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.17.2) - [`bt eval --no-send-logs`](https://docs.braintrust.dev/evaluate/advanced-evaluations#run-without-uploading-results) now runs Go evals locally without logging in, creating an experiment, or uploading spans, and the new [`NoSendLogs`](https://docs.braintrust.dev/sdks/go/api-reference#evaluations) option on `eval.Opts` and `eval.RunOpts` does the same in code. [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md#what-traced-go) model provider spans, such as Google GenAI `generate_content` spans, now nest under their `call_llm` span, including streaming calls. Fixed [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-go) Responses API spans not being marked as errors when a stream sends an `error` event or a response ends with a `failed` status. OpenAI, Anthropic, and Google GenAI spans are now marked as errors when the response can't be read or parsed, such as when a stream is cut off partway through. * [v0.17.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.17.1) - When a [span customizer](https://docs.braintrust.dev/sdks/go/redact-sensitive-data.md#when-a-customizer-fails) fails, the SDK now exports that span with its content stripped and `context.customizer_error` set to `true`, instead of dropping the whole export batch. Customizer failures no longer make `ForceFlush` return an error. [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md#what-traced-go) sub-agent spans now nest under the agent that called them, and [LangChainGo](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md#what-traced-go) agent action and finish spans now nest under the agent executor's chain span. * [v0.17.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.17.0) - [Span customizers](https://docs.braintrust.dev/sdks/go/redact-sensitive-data.md#how-customizers-run) now run before attachment upload, so they can remove inline base64 data to keep files out of Braintrust. Customizer errors are no longer included in logs, since they can contain span data. Fixed partially converted attachments when the upload queue is full, and the missing `background` parameter in [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) Responses API metadata. ### Java SDK releases * [v0.3.26](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.26) - When a [span customizer](https://docs.braintrust.dev/sdks/java/redact-sensitive-data.md#how-customizers-run) fails, the SDK now exports only that span's identity and timing instead of dropping the whole export batch. Non-inference calls, such as batches, files, and token counting, now produce plain `anthropic.http`, `openai.http`, or `google.http` spans instead of LLM spans. Fixed a `ClassCastException` that failed instrumented requests during JVM shutdown. ### Ruby SDK releases * [v0.6.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.6.0) - Added [span customizers](https://docs.braintrust.dev/sdks/ruby/redact-sensitive-data.md), which you register with the `span_customizers:` argument of `Braintrust.init` to redact sensitive values or add tags and metadata before the SDK exports each span. Added tracing for [Roast](https://docs.braintrust.dev/integrations/agent-frameworks/roast.md) workflows (`roast-ai` v1.0.0 or later), with a span for each workflow run and each cog, and the LLM calls of `chat` cogs nested under their cog. Fixed `Braintrust::Trace.permalink` URLs for organization and project names that contain spaces or slashes. ### C# SDK releases * [v0.2.14](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.14) - Fixed Microsoft Agent Framework function tracing adding a second tool-call loop, which broke chat history persistence for agents using `RequirePerServiceCallChatHistoryPersistence = true`. Streaming agent runs now keep their LLM and tool spans nested under the agent span. Fixed evals failing for service accounts with only project-level access, a regression in v0.2.12. **Breaking change**: `ChatClientBuilder.UseBraintrustTracing()` has been removed and `UseBraintrustFunctionTracing()` has moved from `ChatClientBuilder` to `AIAgentBuilder`. Replace `UseBraintrustTracing()` with `UseBraintrustLLMTracing()` on the `ChatClientBuilder` and `UseBraintrustFunctionTracing()` on the agent builder. See [Microsoft Agent Framework](https://docs.braintrust.dev/integrations/agent-frameworks/microsoft-agent-framework.md) for details. ### trace-claude-code releases * [v3.0.5](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v3.0.5-claude) - Internal maintenance update. Run `bt trace update claude` to update. * [v3.0.4](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v3.0.4-claude) - Fixes Cursor blocking prompts when it imports your installed Claude Code plugins. `trace-claude-code` no longer runs its tracing hooks in Cursor, so [Cursor tracing](https://docs.braintrust.dev/integrations/developer-tools/cursor.md#troubleshooting-tracing) records Cursor sessions on its own. Run `bt trace update claude` to update. * [v3.0.3](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v3.0.3) - Internal maintenance update. Run `bt trace update claude` to update. * [v3.0.2](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v3.0.2) - Claude Code runs `bt` as an executable with arguments instead of a shell command string, so hooks no longer flash a console window on Windows. With `bt` v0.23.0 or later, [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) reports earlier plugin versions as out of date. Run `bt trace update claude` to update. ### trace-codex releases * [v2.2.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.2.2-codex) - Internal maintenance update. Run `bt trace update codex` to update. * [v2.2.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.2.1-codex) - Internal maintenance update. Run `bt trace update codex` to update. * [v2.1.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.1.1-codex) - The plugin sends Codex's `Interrupt` and `SessionEnd` hook events to `bt`, which uploads a session's pending spans as soon as the session ends. Run `bt trace update codex` to update. With `bt` v0.27.2 or later, the command also trusts the updated hooks. With earlier versions, restart Codex and trust the updated Braintrust hook through `/hooks`. This release includes the changes from v2.1.0, which wasn't published. * [v2.0.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.2-codex) - Internal maintenance update. With `bt` v0.23.0 or later, [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) reports earlier plugin versions as out of date. Run `bt trace update codex` to update. ### trace-grok releases * [v0.0.5](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.0.5-grok) - Fixes tracing, which didn't work in v0.0.4. Run `bt trace update grok`, then `/reload-plugins`. ### Google Antigravity tracing releases * [v0.1.5](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.1.5-antigravity) - Tool spans start when the tool starts instead of being reconstructed afterward, and tracing failures can't block tool calls. ### @braintrust/pi-extension releases * [v2.2.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.2.2-pi) - The extension sends events to `bt` in the background instead of waiting for each one, so tracing no longer slows down pi. With `bt` v0.28.0 or later, session spans record the running pi package name in `metadata.host_name`, and trace links appear as soon as the first turn's spans are delivered. In Oh My Pi, traces record the Oh My Pi version instead of the version of the pi package installed alongside the extension. This doesn't apply to Oh My Pi's standalone binary. Run `bt trace update pi` to update. * [v2.2.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.2.1-pi) - The extension reads the background tracing process's socket path from `BRAINTRUST_DAEMON_SOCKET`, falling back to `BT_DAEMON_SOCKET`, matching the [`BRAINTRUST_` environment variable names](https://docs.braintrust.dev/reference/cli/migrate#v0-26-0) that `bt` uses starting in v0.26.0. * [v2.2.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.2.0-pi) - Records the pi version on session spans for pi 0.99 and 1.0. With `bt` v0.22.0 or later, sessions traced by an extension earlier than v1.0.0 continue in their existing trace. ### @braintrust/trace-opencode releases * [v2.1.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.1.1-opencode) - Adds new turns from a [resumed OpenCode session](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#common-workflows) to its existing trace instead of starting a new trace each time OpenCode restarts. [Built-in Braintrust tools](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#built-in-braintrust-tools) read the path to `bt` from `BRAINTRUST_BT_BIN`, falling back to the deprecated `BT_EXECUTABLE`, and no longer open a console window on Windows. Run `bt trace update opencode` to update. ### Cursor tracing releases * [v0.5.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.5.1-cursor) - Internal maintenance update. * [v0.5.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.5.0-cursor) - Subagent spans open when a subagent begins instead of when it finishes, so they record the subagent's task and duration, and they nest under the `Task` tool call that spawned the subagent. The new hooks always let the tool call or subagent proceed. Requires `bt` v0.27.0 or later. Run `bt trace update cursor` to update. See [Cursor](https://docs.braintrust.dev/integrations/developer-tools/cursor.md#what-braintrust-traces) for details. * [v0.4.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.4.2-cursor) - On Windows, the hook launcher reads the path to `bt` from `BRAINTRUST_BT_BIN`, falling back to `BT_BIN`. * [v0.4.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.4.1-cursor) - `bt trace enable cursor` adds capture hooks to Cursor's `hooks.json`, so sessions are traced even when Cursor doesn't load the local plugin. Requires `bt` v0.26.0 or later. See [Cursor](https://docs.braintrust.dev/integrations/developer-tools/cursor.md) for details. ### Improvements * Sort, filter, group, and save views of the [Patterns list](https://docs.braintrust.dev/observe/patterns/review.md#find-a-pattern) (public preview). On each pattern, set status, severity, quality, and categories with an optional reason, and follow changes and comments in [**Activity**](https://docs.braintrust.dev/observe/patterns/review.md#activity). * Each [Patterns automation](https://docs.braintrust.dev/observe/patterns/enable.md#customize-severity-and-categories) (public preview) can have its own severity and category rubric. Automations without one use the project's rubric. Requires data plane v2.21.0+ (Self-hosted customers must upgrade to use this feature.) * Braintrust computes each measurable pattern's [**Affected traces** chart](https://docs.braintrust.dev/observe/patterns/review.md#affected-traces) and **Last seen** time (public preview) in the background, so pattern pages load faster and **Last seen** stays current even when Loop adds no new evidence. Requires data plane v2.21.0+ (Self-hosted customers must upgrade to use this feature.) * A [**Models**](https://docs.braintrust.dev/observe/patterns/review.md#models) breakdown on each measurable pattern (public preview) compares each model's share of the pattern's matching traces with its share of all traffic, so you can spot models where the behavior is overrepresented. * [Loop automations](https://docs.braintrust.dev/loop/automations.md#create-a-loop-automation) can follow your organization's [Loop default provider](https://docs.braintrust.dev/admin/organizations.md#set-a-default-provider-for-loop). With **Auto** on, each run uses the current default and its recommended model and reasoning effort. * You can now [copy or export a Loop thread](https://docs.braintrust.dev/loop/manage.md#copy-or-export-a-thread) as Markdown, including tool calls and their output, to share an investigation or hand it to a coding agent. * [Loop automations](https://docs.braintrust.dev/loop/automations.md#review-past-runs) now surface failed Slack and webhook deliveries in their automation status. * GPT-6.1 Sol is available in [Loop](https://docs.braintrust.dev/loop.md#choose-a-provider-and-model) and scheduled automations, supports Max reasoning effort, and is the default for new threads. * On desktop, the thread now opens in a panel beside the trace, so you can read the conversation and inspect spans side by side. See [Inspect a trace](https://docs.braintrust.dev/observe/examine-traces.md#inspect-a-trace) for details. * [Custom preprocessors](https://docs.braintrust.dev/observe/topics/enable.md#write-a-custom-preprocessor) run faster on traces with many spans. If a custom preprocessor throws an error on any span, the run fails instead of silently skipping that span. * You can now edit simple point-and-click filters in the Basic filter editor with their values filled in, instead of only in SQL. See [Filter and search logs](https://docs.braintrust.dev/observe/filter.md#apply-a-filter) for details. * [Dashboard chart export](https://docs.braintrust.dev/observe/dashboards/build-charts.md#export-chart-data) now covers SQL charts and table charts, and the click menu on chart data points now includes an **Export data** option that downloads JSON or CSV. * [Online scoring rules](https://docs.braintrust.dev/evaluate/score-online.md) and [human review score configurations](https://docs.braintrust.dev/annotate/human-review.md) now appear in the [audit log](https://docs.braintrust.dev/admin/audit-logs.md) as `project_score` events (created, updated, and deleted), including pause and resume status changes. Reordering human review scores isn't recorded. * With [row-level audit logging](https://docs.braintrust.dev/admin/audit-logs.md#async-queries-and-applied-scorers) enabled, the audit log also records submitting and canceling [async queries](https://docs.braintrust.dev/api-reference/query.md#async-queries), fetching their results, and applying scorers to an experiment with **Apply scorers**. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) * The [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) chart now opens the chart action menu when you click a bar, and you can split it into series by any field from a trace's details. In the **Traces** row type, the chart counts traces instead of spans. See [Explore the logs chart](https://docs.braintrust.dev/observe/view-logs.md#explore-the-logs-chart) for details. * On the [Patterns page](https://docs.braintrust.dev/observe/patterns/review.md) (public preview), you can now investigate a finding with Loop inline, with the selected pattern attached as context. * [Copy for agent](https://docs.braintrust.dev/observe/patterns/review.md#hand-it-to-a-coding-agent) no longer waits for trace details before copying a brief. The brief includes up to three prioritized trace IDs and CLI instructions for retrieving additional evidence. * You can now inspect trace evidence beside a pattern on the [Patterns page](https://docs.braintrust.dev/observe/patterns/review.md#examples) (public preview). * The [custom view](https://docs.braintrust.dev/annotate/custom-views.md) editor preserves your Loop conversation when you close and reopen the same view in the same browser. * [Loop](https://docs.braintrust.dev/loop.md) queues follow-up messages while it works, so you can continue the conversation without waiting for the current turn to finish. * Loop can now help [troubleshoot scoring failures](https://docs.braintrust.dev/evaluate/score-online.md#scoring-failures) using the scorer and error context from project logs. * The new [Thread view](https://docs.braintrust.dev/observe/examine-traces.md) preserves unrecognized input and output payloads alongside parsed messages. * You can now [translate](https://docs.braintrust.dev/observe/examine-traces.md#translate-message-content) and copy LLM responses and user messages in the [new thread view](https://docs.braintrust.dev/observe/examine-traces.md#use-the-new-thread-view), as in the original thread view. * [Patterns Slack notifications](https://docs.braintrust.dev/observe/patterns/enable.md) now include individual alerts for new or reactivated patterns, alongside a digest of new and existing active patterns with frequency estimates and links to additional findings. * Loop can now create [human review scores](https://docs.braintrust.dev/annotate/human-review.md#configure-review-scores) from a description, including categorical choices, sliders, and free-form inputs. * Table charts that return raw rows of spans from project logs now offer **Open trace**, which opens the exact trace and span behind the clicked row instead of a sample of matching traces. See [Read a table](https://docs.braintrust.dev/observe/dashboards/sql-charts.md#read-a-table) for details. * Service tokens with an expiration can no longer create service tokens that outlive them. These requests return a 403 error that states the maximum allowed lifetime. See [Create service tokens](https://docs.braintrust.dev/admin/organizations.md#create-service-tokens) for details. * [Patterns](https://docs.braintrust.dev/observe/patterns.md) now checks each finding's cause and suggested fix against relevant source code in a [connected repository](https://docs.braintrust.dev/loop/github.md) and requirements or contracts from an [external MCP server](https://docs.braintrust.dev/loop/mcp.md) before saving it, and distinguishes confirmed causes from hypotheses when a relevant source isn't available. * On the [Patterns page](https://docs.braintrust.dev/observe/patterns/review.md) (public preview), the pattern header shows the name of the automation or person who discovered it. When a project has more than one Patterns automation, the list shows that automation's name and you can filter the list to one of them. A pattern a person created shows their name in the list as well. * Organization owners can now list every member's API keys with `GET /v1/organization/api_keys`, for example to audit keys programmatically. See [Manage API keys](https://docs.braintrust.dev/admin/organizations.md#manage-api-keys) for details. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) * You can now push code functions and scorers from Python 3.14 on Braintrust-hosted and self-hosted AWS deployments. See the [supported Lambda runtimes](https://docs.braintrust.dev/evaluate/remote-evals.md#2-register-your-sandbox). Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) * You can now paste images from your clipboard directly into the [Loop](https://docs.braintrust.dev/loop.md) composer. * [Loop](https://docs.braintrust.dev/loop/manage.md#models-and-providers) can use project-level AI providers when running in a sandbox and recognizes supported models with gateway prefixes. * The [Patterns Frequency chart](https://docs.braintrust.dev/observe/patterns/review.md#affected-traces) (public preview) now defaults to the share of all traces in your project. The list also uses overall frequency for its saved percentages and frequency ranking. * [Patterns](https://docs.braintrust.dev/observe/patterns/review.md#duplicate-patterns) (public preview) consolidates duplicate findings into one canonical pattern, preserving useful evidence and linking the closed duplicates to it. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) * [Continue in Loop](https://docs.braintrust.dev/observe/patterns/review.md#keep-investigating) (public preview) starts a new chat when a conversation is already active, keeping the pattern investigation separate from that conversation. * The trace view labels TypeSafe [Jev](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#inspect-jev-decisions) spans as **Decision** spans instead of **Question** spans, and **Filter by span type** lists them under **Decision**. * The [SQL sandbox](https://docs.braintrust.dev/reference/sql.md#sql-sandbox) now supports split view, so you can run and compare two queries side by side. * [SQL chart top lists](https://docs.braintrust.dev/observe/dashboards/sql-charts.md#donut-charts) now support a **Donut** display mode that shows each group's share of the overall value. Requires data plane v2.18.0+ (Self-hosted customers must upgrade to use this feature.) * Dashboard filters now default to span filtering, so only matching spans contribute to each chart and charts load faster. You can switch to cross-trace filtering for fields on other spans in the trace, and existing saved dashboards keep cross-trace filtering. See [Filter and group data](https://docs.braintrust.dev/observe/dashboards.md#filter-and-group-data) for details. * The [new thread view](https://docs.braintrust.dev/observe/examine-traces.md#use-the-new-thread-view) (public preview) is now the default in the thread panel. You can switch back to the original thread view from the thread settings menu. * [Dashboard charts](https://docs.braintrust.dev/observe/dashboards.md) that fail to load now label recognized failures, such as an invalid query, a timeout, or missing access, and show a suggested next step and the error message when available. * The [Patterns list](https://docs.braintrust.dev/observe/patterns/review.md#find-a-pattern) (public preview) can filter by more than one automation, and a **Backfill** action asks Loop to assign a severity to patterns that don't have one. * A **Traces linked to patterns** [preset chart](https://docs.braintrust.dev/observe/dashboards/build-charts.md) ranks patterns on a dashboard by the number of traces attached to each as evidence. * You can now duplicate dataset [custom views](https://docs.braintrust.dev/annotate/custom-views.md#duplicate-views) to other datasets in your organization, so you can reuse a dataset view without rebuilding it. * Members, service accounts, and permission groups with every permission on every row of a project group's permissions, including through organization-wide **All projects** grants, now count as Owners of that project group and can create service accounts and service tokens for it. See [Project group owners](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#project-group-owners) for details. * The trace viewer no longer includes the public-preview Debugger. Saved Debugger links open the trace view. Use [Loop](https://docs.braintrust.dev/loop/capabilities.md#analyze-traces) to investigate individual traces. * The [Coding agent insights dashboard](https://docs.braintrust.dev/observe/dashboards/coding-agent-insights.md#group-the-table) (public preview) can now break down usage and cost by permission group or service account, so you can compare spending across teams and identify activity authenticated by service accounts. * Dashboard charts now support [markers](https://docs.braintrust.dev/observe/dashboards/build-charts.md#add-markers), which highlight threshold values and ranges by severity, such as a latency target or an error rate limit. * You can now resize the columns of [table charts](https://docs.braintrust.dev/observe/dashboards/sql-charts.md#read-a-table) on dashboards. Braintrust remembers your column widths for each chart in your browser. * Patterns (public preview) improvements: * [Slack notifications](https://docs.braintrust.dev/observe/patterns/enable.md#slack-notifications) send one alert per new or reactivated pattern, including severity and categories, instead of an end-of-run digest. * [Pattern wording](https://docs.braintrust.dev/observe/patterns.md#goals) refreshes when the writing guidance or finding changes, preserving the finding's meaning and evidence when only the guidance changes. * The pattern header shows when the behavior was last seen in a trace. Hover over the label for the exact timestamp. Requires data plane v2.19.0+ (Self-hosted customers must upgrade to use this feature.) * When you ask Loop to change a pattern's [corrected severity or categories](https://docs.braintrust.dev/observe/patterns/review.md#severity) (public preview), Loop now makes the change. Patterns automations still keep your corrections. Requires data plane v2.21.0+ (Self-hosted customers must upgrade to use this feature.) * The [`new_pattern` MCP tool](https://docs.braintrust.dev/integrations/developer-tools/mcp#tools) requires a `source_review` that records whether source code was reviewed before the pattern was saved. Pattern writes that report matching traces must include `last_seen`, and `first_seen` when the pattern doesn't have one. Requires data plane v2.21.0+ (Self-hosted customers must upgrade to use this feature.) * SQL queries now support [`is_uuid()`](https://docs.braintrust.dev/reference/sql/functions#string-functions), which checks whether a string is a valid UUID, so you can exclude malformed IDs in `WHERE` clauses and aggregates. Requires data plane v2.19.0+ (Self-hosted customers must upgrade to use this feature.) * [Loop](https://docs.braintrust.dev/loop/manage.md#models-and-providers) can now use GPT models served through a Databricks AI provider, such as `databricks-gpt-6-1-sol`. Requires data plane v2.19.0+ (Self-hosted customers must upgrade to use this feature.) * Coding agents can now save dataset [custom views](https://docs.braintrust.dev/annotate/custom-views.md) and reuse existing ones as templates with the [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#run-evals-and-edit-datasets). Requires data plane v2.21.0+ (Self-hosted customers must upgrade to use this feature.) * The [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#explore-your-data) can now resolve [attachment](https://docs.braintrust.dev/instrument/attachments.md) references from query results into short-lived download URLs with `resolve_attachments`, so coding agents can retrieve the images and files attached to your logs, experiments, and datasets. Requires data plane v2.22.0+ (not yet released for self-hosting) (Self-hosted customers must upgrade to use this feature.) * On the [Patterns page](https://docs.braintrust.dev/observe/patterns/review.md#examples), the first example trace opens beside the pattern automatically, and you can step through the other examples without leaving the page. In the [new thread view](https://docs.braintrust.dev/observe/examine-traces.md#use-the-new-thread-view), evidence appears as cards below the highlighted text, with controls to jump between examples. * On desktop, you can now choose whether a trace shows its conversation and custom views, its spans, or both side by side. See [Choose a trace layout](https://docs.braintrust.dev/observe/examine-traces.md#choose-a-trace-layout) for details. * [**Usage analytics**](https://docs.braintrust.dev/admin/billing/monitor-usage.md#view-usage) now charts built-in model and Topics tokens together in a built-in inference chart, broken down by model or token kind. On every plan except Starter without on-demand usage, you can switch the charts to cost. On Enterprise plans, cost mode shows list-rate estimates for inference and hides the logs and scores charts. * [Decision spans](https://docs.braintrust.dev/instrument/trace-application-logic.md#log-bounding-boxes) now support `box` answers, which the trace view draws as bounding boxes over the input image. * The span panel's **Details** tab now shows a span's input and output with its other fields, and the trace header shows the root span's scores. See [View as a hierarchy](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-hierarchy) for details. * You can now pause, resume, or delete several [online scoring rules](https://docs.braintrust.dev/evaluate/score-online.md#pause-an-automation) at once instead of editing each rule. * The [Account details page for Enterprise account billing](https://docs.braintrust.dev/admin/billing.md#view-account-billing) lists the account's invoices, each linking to its hosted invoice. * The [`set_topics_automation` MCP tool](https://docs.braintrust.dev/integrations/developer-tools/mcp#tools) no longer replaces existing topic maps when you change facets. `facets` only adds facets, and the tool removes facets or topic maps only when you name them in `remove_facets` or `remove_topic_map_ids`. **Breaking change**: `facet_mode: "replace"` returns an error. Use `remove_facets` and `remove_topic_map_ids` instead. A new `unpin_topic_maps` tool removes topic map version pins without changing other settings. Requires data plane v2.22.0+ (not yet released for self-hosting) (Self-hosted customers must upgrade to use this feature.) **September 2026** ### Apply scorers to experiments You can now add scorers to an experiment after it runs. Braintrust scores the experiment's existing traces in the background without re-running the task, and continues scoring traces added to the experiment later. See [Apply scorers to all traces](https://docs.braintrust.dev/evaluate/interpret-results.md#apply-scorers-to-all-traces) for details. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) ### Dashboard notes Add Markdown notes to the dashboards you create to explain what a dashboard tracks, link to runbooks, or annotate groups of charts. You can copy notes to other dashboards, the same as charts. See [Add notes](https://docs.braintrust.dev/observe/dashboards/build-charts.md#add-notes) for details. > [!NOTE] > > Dashboard notes are only available on [Pro and Enterprise plans](https://docs.braintrust.dev/plans-and-limits.md#plans). Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) ### TypeScript workflow evaluations [Workflow evaluations](https://docs.braintrust.dev/sdks/typescript/workflow-evaluations.md), which run eval tasks and scorers through provider batch APIs, are now available in the TypeScript SDK (v3.33.0+), following their [Python release](#python-workflow-evaluations). > [!WARNING] > > TypeScript workflow evaluations are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Span customizers Span customizers let you modify spans inside your application before the SDK uploads them to Braintrust, for example to redact sensitive values, remove fields, or add tags and metadata. They're available in the TypeScript (v3.35.0+), Python (v0.43.0+), Go (v0.16.0+), and Java (v0.3.25+) SDKs. See [Redact before upload](https://docs.braintrust.dev/admin/data-management/protect-sensitive-data.md#redact-before-upload) for details. ### External MCP servers in Loop Connect [external MCP servers to Loop](https://docs.braintrust.dev/loop/mcp.md) so it can use their tools and data alongside your Braintrust traces. Save an API key or OAuth connection for the project to share access with Loop and scheduled automations. > [!WARNING] > > External MCP connections in Loop are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) ### Chart data point actions Clicking a data point on a chart now opens a menu of actions for that point. Open matching traces in a side panel without leaving the page, explore the rows behind the point in the SQL sandbox, or filter the dashboard to the clicked series. See [Interact with chart data](https://docs.braintrust.dev/observe/dashboards.md#interact-with-chart-data) for details. ### Python workflow evaluations Python evaluations (SDK v0.39.0+) can now run their tasks and scorers through provider batch APIs, which providers often offer at a discount. Start a run, let the provider work, and pick the run back up later by polling or by responding to a webhook. The process that resumes the run doesn't have to be the one that started it. When every case finishes, Braintrust records the results in a single experiment. See [Workflow evaluations](https://docs.braintrust.dev/sdks/python/workflow-evaluations.md) for details. > [!WARNING] > > Python workflow evaluations are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Application logs Send application logs to Braintrust and inspect them alongside your traces, with or without an LLM call. Emit messages or forward standard-library `logging` records with the Python SDK (v0.42.0+), or export OpenTelemetry logs from any language. Browse and filter logs by severity with the new **Logs** row type, view each message in the trace that emitted it, or query logs with [SQL](https://docs.braintrust.dev/reference/sql.md). See [Send application logs](https://docs.braintrust.dev/instrument/application-logs.md) for details. > [!WARNING] > > Application logs are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.14.0+ (Self-hosted customers must upgrade to use this feature.) ### Claude Code OpenTelemetry mapping Braintrust now maps Claude Code's native OpenTelemetry traces and events to LLM and tool spans with token metrics, cost, and conversation messages. See [Trace Claude Code with OpenTelemetry](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#trace-with-opentelemetry) for details. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) ### OpenInference tracing Braintrust now maps [OpenInference traces](https://docs.braintrust.dev/integrations/sdk-integrations/opentelemetry/attributes.md#openinference-attributes) to structured messages, tool calls, span types, token metrics, tags, and scores. Use OpenInference instrumentors for frameworks such as LangChain, [LlamaIndex](https://docs.braintrust.dev/integrations/sdk-integrations/llamaindex.md#opentelemetry-python), and DSPy with your OpenTelemetry pipeline. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) ### Transform coding-agent traces locally Redact secrets, remove sensitive fields, or add tags and metadata to [coding-agent traces](https://docs.braintrust.dev/reference/cli/trace.md) before they leave your machine. The `bt` CLI (v0.21.0+) runs JavaScript span plugins that you register with `bt trace enable`, `bt trace run`, or `bt trace import`. If a plugin fails, `bt` withholds the affected spans instead of sending untransformed data. See [Transform spans with JavaScript](https://docs.braintrust.dev/reference/cli/trace.md#transform-spans-with-javascript) for details. ### Pattern frequency [Patterns](https://docs.braintrust.dev/observe/patterns/review.md#read-a-pattern) shows how often a finding occurs among comparable traces or across your project, with a chart for the selected time range. Scheduled runs refresh measurements for existing active patterns. > [!WARNING] > > Patterns is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.15.0+ (Self-hosted customers must upgrade to use this feature.) ### AI provider setup and model controls AI provider updates include a redesigned setup experience, new model access controls, and a fix to hosted request routing: * **Simpler setup and testing:** A clearer setup form organizes credentials and advanced settings. The new **Models** tab lets you [browse models](https://docs.braintrust.dev/admin/ai-providers.md#browse-and-test-models) and copy a Gateway command to test one. * **Model restrictions:** New [model restrictions](https://docs.braintrust.dev/admin/ai-providers.md#restrict-available-models) work with [AI provider governance](https://docs.braintrust.dev/admin/ai-providers.md#permissions-and-access) to control which models each team can access. Organization providers support model selection on both Braintrust-hosted and self-hosted deployments. For project providers, the new **Selected models** option is available on Braintrust-hosted deployments. * **Routing fix (behavior change):** On Braintrust-hosted deployments, project providers [now override](https://docs.braintrust.dev/admin/ai-providers.md#request-routing) organization providers only for models they serve. Eligible organization providers can serve other models using their own credentials and billing accounts. If you rely on project restrictions to limit model access or spending, review organization provider model selections and permissions. ### Usage analytics by project The [**Usage analytics**](https://docs.braintrust.dev/admin/billing/monitor-usage.md#view-usage) section of the billing page now breaks usage down by project, so you can see which projects drive your logs, scores, and Topics token spend and export the breakdown as a CSV. Project data is available from September 1, 2026. Requires data plane v2.13.0+ (Self-hosted customers must upgrade to use this feature.) ### DeepSeek V4.1 Flash DeepSeek V4.1 Flash is available as a [built-in model](https://docs.braintrust.dev/admin/ai-providers.md#available-models) in playgrounds, prompts, and scorers, with multimodal reasoning support and no AI provider setup required. You can also request `deepseek-v4.1-flash` through the Braintrust Gateway. Usage draws from your monthly [model credits](https://docs.braintrust.dev/plans-and-limits.md#model-credits), shared with Topics. ### TypeSafe Jev TypeSafe provides [Jev](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md), a model that returns a specific choice or numeric result instead of generating text. You define the question and its allowed answers before the call, so nothing downstream has to find a decision inside prose or malformed JSON. You can now trace Jev calls from your application to inspect the context, question criteria, and selected answers together, and to identify ambiguous decisions. Requires [Python v0.41.0+](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.41.0) or [TypeScript v3.34.0+](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.34.0). You can also use Jev as the judge in a [scorer or classifier](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md#llm-as-a-judge), either with a built-in model or your own TypeSafe key, to score outputs against predefined criteria or label logs with a fixed set of categories. It fits frequent, focused checks with a clear answer, and Braintrust records the confidence and per-choice probabilities behind each decision so you can inspect uncertain judgments. Requires data plane v2.15.0+ (Self-hosted customers must upgrade to use this feature.) ### Loop thread sharing Share [Loop threads](https://docs.braintrust.dev/loop/manage.md#share-a-thread) with project members to review an investigation or continue it together, with view-only or chat access. Loop responds to each message using the sender's permissions, and only that sender can approve or cancel the work their message starts. Requires data plane v2.15.0+ (Self-hosted customers must upgrade to use this feature.) ### Trace Grok and Google Antigravity sessions Understand how [Grok](https://docs.braintrust.dev/integrations/developer-tools/grok.md) and [Google Antigravity](https://docs.braintrust.dev/integrations/developer-tools/antigravity.md) coding sessions progress across prompts, model calls, and tool activity. The `bt` CLI (v0.19.3+) captures each session as a Braintrust trace without interrupting the coding agent if tracing fails. ### Observability templates Reuse facets and Loop automations across projects instead of rebuilding your observability setup each time. The [`bt observability template`](https://docs.braintrust.dev/reference/cli/observability) commands in `bt` v0.19.0+ export a project's configuration as a portable JSON template and apply it to another project. > [!WARNING] > > Observability templates are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Metric columnstore Braintrust can now store trace-level metrics in a columnar format, so the queries behind the **Logs** page read far less data on high-volume projects. Enable **Metric columnstore** in project settings alongside log search optimization, and Braintrust writes columnar artifacts during compaction, speeding up loading and sorting traces. Aggregation queries are unaffected and continue to follow [SQL best practices](https://docs.braintrust.dev/reference/sql/best-practices.md). See [Speed up log filtering](https://docs.braintrust.dev/admin/projects.md#speed-up-log-filtering) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### Faster log search Brainstore now runs log searches on an asynchronous execution engine that streams results as segments finish scanning, instead of waiting for every segment to complete. This covers full-text and phrase search, along with trace searches that combine several [`ANY_SPAN()`](https://docs.braintrust.dev/reference/sql/query-structure#single-span-filters) clauses, including negated ones. Searches over large, cold log volumes benefit most, and unsorted searches gain the most. Query syntax is unchanged and no configuration is required. See [Filter and search logs](https://docs.braintrust.dev/observe/filter.md) for search syntax. Requires data plane v2.13.0+ (Self-hosted customers must upgrade to use this feature.) ### SQL `SELECT DISTINCT` for unique rows `SELECT DISTINCT` on scalar columns now deduplicates SQL query results, returning one row per unique combination of values. See [SELECT DISTINCT](https://docs.braintrust.dev/reference/sql/query-structure.md#select-distinct) for supported usage and restrictions. Requires data plane v2.15.0+ (Self-hosted customers must upgrade to use this feature.) ### Patterns Braintrust now investigates your traces for you. [\*\* Patterns\*\*](https://www.braintrust.dev/app/~/patterns) runs Loop against your project on a schedule, searching for recurring problems and trends that no scorer is already measuring, such as a failure mode affecting one cohort of users or an LLM call whose cost has climbed. Each finding is saved with the traces that support it, and with a suggested fix when Loop has one to offer, so you start from evidence rather than a hunch. From a pattern you can create a scorer or classifier to track the behavior continuously, continue the investigation in Loop, or copy the finding as a prompt for your coding agent. See [Enable Patterns](https://docs.braintrust.dev/observe/patterns/enable.md) to turn it on. Patterns support is limited to two model families: GPT-6 Astra and GPT-5.6 (GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna). Braintrust is working to add more models. Use Braintrust's native inference when possible, so scheduled discovery runs on supported models without provider setup. > [!WARNING] > > Patterns is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.13+ (Self-hosted customers must upgrade to use this feature.) ### Debugger A long agent trace can span dozens of LLM and tool calls, which makes reading it end to end impractical. The **Debugger** reports the failure modes it considers likely for a single trace, citing the spans, tool calls, and model outputs behind each one. **Analyze trace** answers the complementary question, grouping a run into **Work sections** that explain what it did rather than what went wrong. > [!WARNING] > > The Debugger is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. Requires data plane v2.12.0+ (Self-hosted customers must upgrade to use this feature.) ### Loop works across your whole project [\*\* Loop\*\*](https://www.braintrust.dev/app/~/loop) now runs in a Braintrust-managed runtime rather than your browser, so it takes on longer investigations and your threads persist for you to resume. It works across your logs, experiments, and datasets rather than only the page you opened it on, and it can create and edit the objects you'd otherwise build by hand, including prompts, scorers, datasets, facets, custom views, dashboards, and automations. Every write pauses for your approval unless you turn on auto-accept, and results open in a workspace beside the chat so you can review them without leaving the thread. See [What Loop can do](https://docs.braintrust.dev/loop/capabilities.md). Loop support is limited to two built-in model families: GPT-6 Astra and GPT-5.6 (GPT-5.6 Sol, GPT-5.6 Terra, and GPT-5.6 Luna). Braintrust is working to add more models. Use Braintrust's native inference when possible, so teammates can start threads on supported models without configuring provider credentials. Requires data plane v2.9.0+ (Self-hosted customers must upgrade to use this feature.) ### Loop automations Loop can now do recurring work without someone starting a chat. A [Loop automation](https://docs.braintrust.dev/loop/automations.md) is a scheduled run with its own instruction, model, and write permissions, useful for reviewing cost or latency on a cadence, checking whether quality is degrading, or summarizing recent errors. Each run leaves a read-only thread you can **Continue** to ask follow-up questions, and results can be sent to a Slack channel or webhook. Requires data plane v2.12.0+ (Self-hosted customers must upgrade to use this feature.) ### Blind reviews By default, human review now hides peer scores, comments, and aggregates for a row until a reviewer fills in their own scores, and reviewers open unscored rows with a blank form. Enable the **Blind human reviews** project setting to keep those results hidden for the whole review, so independence holds even after a reviewer finishes. Reviewers with the project `Update` permission are exempt and always see all reviews. See [Enable blind reviews](https://docs.braintrust.dev/annotate/human-review.md#enable-blind-reviews) for setup steps. ### Required scores for human review Turn on **Require all scores** in \*\* Settings\*\* > [\*\* Human review\*\*](https://www.braintrust.dev/app/~/configuration/review) to require assigned reviewers to complete every score visible to them. The review completes automatically after all assigned reviewers finish scoring, and manual completion is not available while the setting is on. See [Require all scores](https://docs.braintrust.dev/annotate/human-review.md#require-all-scores) for details. ### GLM-5.3 Flash GLM-5.3 Flash joins GLM-5.2, Kimi K3, and DeepSeek V4 Flash 0731 as a built-in open-source model served by Braintrust, with no AI provider setup required. Select it under the **Braintrust** provider in playgrounds, prompts, and scorers, or request `glm-5.3-flash` through the Braintrust Gateway. GLM-5.3 Flash is a multimodal reasoning model. Usage draws from your monthly [model credits](https://docs.braintrust.dev/plans-and-limits.md#model-credits), shared with Topics. See [Available models](https://docs.braintrust.dev/admin/ai-providers.md#available-models) for details. ### Dashboard sections Group a dashboard's charts into named **sections** you can reorder, duplicate, and collapse to keep a long dashboard readable. Braintrust remembers each section's collapsed state in your browser, and duplicating a section keeps it. See [Organize charts into sections](https://docs.braintrust.dev/observe/dashboards/build-charts.md#organize-charts-into-sections) for details. > [!NOTE] > > Dashboard sections are only available on [Pro and Enterprise plans](https://docs.braintrust.dev/plans-and-limits.md#plans). Requires data plane v2.13.0+ (Self-hosted customers must upgrade to use this feature.) ### Project groups Group projects into a named collection, then grant a permission group access to that collection once instead of repeating the same grant on every project. Every project in the group inherits the access, and a project you add later inherits it immediately. Create and manage them at \*\* Settings\*\* > [\*\* Project groups\*\*](https://www.braintrust.dev/app/~/configuration/org/project-groups). See [Manage project groups](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#manage-project-groups) for details. > [!NOTE] > > Project groups are only available on the [Enterprise plan](https://docs.braintrust.dev/plans-and-limits.md#plans). ### bt CLI releases * [v0.22.1](https://github.com/braintrustdata/bt/releases/tag/v0.22.1) * Fixed separately started coding-agent sessions, such as two agents in the same terminal, merging into one trace. * Fixed trace events getting stuck while waiting to link to a parent session. * Fixed [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) reporting authentication as ready when the background tracing process could not authenticate. * [v0.22.0](https://github.com/braintrustdata/bt/releases/tag/v0.22.0) * **Custom views CLI**: New [`bt custom-views`](https://docs.braintrust.dev/reference/cli/custom-views) commands let you bootstrap starter files, preview views locally against real trace and dataset data with hot reload, and push finished views to Braintrust. Requires `braintrust` SDK v3.33.0 or later. * **npm distribution**: `bt` is now published as [`@braintrust/bt`](https://www.npmjs.com/package/@braintrust/bt) on npm, independent of the `braintrust` JavaScript SDK. Install with `npm install --save-dev @braintrust/bt` or globally with `npm install -g @braintrust/bt`. * **Python dataset pipeline fix**: The `scope: "trace"` setting in `DatasetPipeline` source definitions now works correctly in the Python runner, matching existing TypeScript behavior. Previously the scope was ignored and all Python transforms received span-scoped arguments. * **Tool errors**: Failed tool spans from [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md), [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md) MCP tools, [Grok](https://docs.braintrust.dev/integrations/developer-tools/grok.md), and [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md) record the agent's own error message, such as `Command exited with code 1`, instead of a generic one such as `Tool execution failed`. * [v0.21.0](https://github.com/braintrustdata/bt/releases/tag/v0.21.0) * **Span plugins**: Adds `--plugin` to [`bt trace enable`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-enable), [`bt trace run`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-run), and [`bt trace import`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-import) to [transform spans with JavaScript](https://docs.braintrust.dev/reference/cli/trace.md#transform-spans-with-javascript) before they are sent. [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) reports plugin failures. * **[Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md) tracing**: Records the permission mode on each turn span, and the client system prompt on the session span when Claude Code provides it. * **[Google Antigravity](https://docs.braintrust.dev/integrations/developer-tools/antigravity.md) tracing**: `bt trace enable antigravity` and `bt trace update antigravity` support Windows. * **[pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) tracing**: Fixes `bt trace update pi`. `bt trace enable pi` installs the extension without a version range. * **Trace structure**: Prevents traces with more than one root span when an update arrives for a span whose first write was not saved. Codex, Google Antigravity, OpenCode, and pi no longer send empty traces for sessions that end without agent activity. Codex sessions imported below a parent span stay in the parent's trace. * **Background process upgrades**: When different `bt` versions send tracing events, an older `bt` reuses a newer background tracing process instead of restarting it. * [v0.20.0](https://github.com/braintrustdata/bt/releases/tag/v0.20.0) * **[Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md) tracing**: Stops marking successful tool calls as errors, fixes recovered tool outcomes, and no longer turns stale child transcripts into synthetic subagent spans. * **[Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md) tracing**: Preserves turn inputs and native timestamps, ignores empty error fields, and labels compaction and resume events correctly instead of as startup. * **[OpenCode](https://docs.braintrust.dev/integrations/developer-tools/opencode.md) and [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) tracing**: Flush delivery in the background so a slow backend flush no longer blocks idle handling, shutdown, or the next input. * **[Antigravity](https://docs.braintrust.dev/integrations/developer-tools/antigravity.md) and [Grok](https://docs.braintrust.dev/integrations/developer-tools/grok.md) tracing**: Use each harness's native hook timeouts. * **Breaking change**: Removes the `--relabel-overlap` flag from [`bt topics config enable`](https://docs.braintrust.dev/reference/cli/topics#bt-topics-config-enable) and [`bt topics config set`](https://docs.braintrust.dev/reference/cli/topics#bt-topics-config-set). The flag was deprecated and ignored since data plane v2.15.0. See the [v0.20.0 migration instructions](https://docs.braintrust.dev/reference/cli/migrate.md#v0-20-0). * [v0.19.3](https://github.com/braintrustdata/bt/releases/tag/v0.19.3) * **Browser links**: Adds [`--app-public-url`](https://docs.braintrust.dev/reference/cli/overview#global-flags) and [`BRAINTRUST_APP_PUBLIC_URL`](https://docs.braintrust.dev/reference/cli/overview#environment-variables) for deployments whose browser-facing app URL differs from the authentication endpoint. * **Claude Code tracing**: Prevents empty traces from session startup and idle notifications. * **Custom certificates**: Tracing and data uploads that use the Rust SDK can trust private or corporate certificate authorities through [`BRAINTRUST_CUSTOM_CA_BUNDLE`](https://docs.braintrust.dev/reference/cli/overview#environment-variables). Supply PEM certificate contents, not a file path. * **Eval discovery**: Automatic [`bt eval`](https://docs.braintrust.dev/reference/cli/eval#file-selection) discovery skips `vendor` directories. * **Go evals**: [`bt eval`](https://docs.braintrust.dev/reference/cli/eval#runtime-configuration) runs Go packages that use the Go SDK's `evalrunner` package. It accepts a package directory, discovers the package with `go list`, or runs a prebuilt binary passed with `--runner`. Go evals are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md). * **Grok tracing**: Adds CLI support for [tracing Grok sessions](https://docs.braintrust.dev/integrations/developer-tools/grok.md). * **pi tracing**: [`bt trace run pi`](https://docs.braintrust.dev/integrations/developer-tools/pi#common-workflows) now uses v2 of the tracing extension, matching persistent setup. * **Project commands**: `bt projects create`, `view`, and `delete` support [JSON output](https://docs.braintrust.dev/reference/cli/projects.md) for automation. `view --json` returns project details without opening a browser. * **Trace metadata**: Google Antigravity, Grok, OpenCode, and pi session spans include the local username when available. Grok traces also include Git repository metadata. * **Trace tags**: Adds [session tags](https://docs.braintrust.dev/reference/cli/trace.md#tag-traces) for Claude Code, Codex, Google Antigravity, OpenCode, and pi through `--tag` or `BRAINTRUST_TAGS`. * **Tracing removal**: [`bt trace disable`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-disable) attempts to remove the tracing configuration even if uninstalling the plugin fails. * **Tracing updates**: [`bt trace update`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-update) updates installed coding-agent integrations without rewriting their tracing settings. [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) warns about outdated plugins and mismatched package version ranges. * **Breaking changes**: `bt trace enable opencode` and `bt trace enable pi` now install v2 of their respective tracing integrations. See the [v0.19.3 migration instructions](https://docs.braintrust.dev/reference/cli/migrate.md#v0-19-3). * [v0.19.2](https://github.com/braintrustdata/bt/releases/tag/v0.19.2) * **Tracing performance**: Reduces the time tracing adds to coding-agent commands. * **Smaller traces**: Reduces repeated conversation history in model-call spans after the agent summarizes its context. * **Behavior change**: [`bt sync pull`](https://docs.braintrust.dev/reference/cli/sync) includes all matching experiment and dataset history unless you pass `--window` or set `BT_SYNC_WINDOW`. Project logs retain the default three-day window. See the [v0.19.2 migration instructions](https://docs.braintrust.dev/reference/cli/migrate.md#v0-19-2). * [v0.19.1](https://github.com/braintrustdata/bt/releases/tag/v0.19.1) * **Google Antigravity tracing**: [`bt trace enable antigravity`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-enable) activates the plugin after installing it. [`bt trace import antigravity`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-import) imports existing conversations. * **Self-hosted tracing**: Fixes coding-agent tracing for self-hosted and hybrid deployments by resolving each organization's data-plane API URL. * **Trace recovery**: Prevents duplicate spans across supported coding agents when tracing resumes after an interruption. * [v0.19.0](https://github.com/braintrustdata/bt/releases/tag/v0.19.0) * **Observability templates**: Added public preview [`bt observability template`](https://docs.braintrust.dev/reference/cli/observability) commands to export and apply portable JSON configuration for facets and Loop automations. * **SQL**: [`bt sql`](https://docs.braintrust.dev/reference/cli/sql) now supports `FROM logs` for querying the selected project. * **Tracing setup**: [`bt trace enable`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-enable) is now the recommended setup command (`bt trace setup` remains an alias). [`bt trace disable`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-disable) removes an integration. [`bt trace doctor`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-doctor) diagnoses tracing configuration and authentication. Rerunning setup removes obsolete top-level destination and authentication fields from global OpenCode and pi configurations. Claude Code setup warns about obsolete `BRAINTRUST_CC_PROJECT` and `BRAINTRUST_CC_DEBUG` settings without removing `BRAINTRUST_API_KEY`. * **Session tracing**: Added initial Google Antigravity support. The `enable`, `run`, and `import` commands now accept custom root-span metadata through `--additional-metadata` or `BRAINTRUST_ADDITIONAL_METADATA`. Fixed tracing for non-interactive `bt trace run claude` invocations. * **Authentication and context**: Expanded [`bt status --all`](https://docs.braintrust.dev/reference/cli/status) output. Added [`bt profiles doctor`](https://docs.braintrust.dev/reference/cli/profiles) and [`bt profiles repair`](https://docs.braintrust.dev/reference/cli/profiles) for credential diagnostics and repair. Stable profile IDs preserve tracing references after renames. [`bt logout --all`](https://docs.braintrust.dev/reference/cli/logout) signs out of all saved profiles. Trace viewing and experiment comparisons using URLs now select the URL's organization when an explicit profile is passed, unless `--org` overrides it. * **Breaking changes**: Normal Claude Code and Codex sessions no longer read `TRACE_TO_BRAINTRUST` or `BRAINTRUST_ADDITIONAL_METADATA` from the environment. `bt trace run codex` rejects `--dangerously-bypass-hook-trust` to prevent duplicate Braintrust hooks. [`bt login`](https://docs.braintrust.dev/reference/cli/login) requires confirmation before saving `BRAINTRUST_API_KEY`. See the [v0.19.0 migration instructions](https://docs.braintrust.dev/reference/cli/migrate.md#v0-19-0). ### Python SDK releases * [v0.43.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.43.0) - Added [span customizers](https://docs.braintrust.dev/sdks/python/redact-sensitive-data.md#span-customizers) (`SpanCustomizer`, `set_span_customizers()`, and the `span_customizers` argument to `auto_instrument()`) to modify or redact records from instrumented spans before upload. [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) tracing captures refusal `stop_details`, `compaction` and `context_management` request parameters, and the `speed` and `fallback_credit` usage fields. Integration modules load lazily, which reduces import time. Fixed `OtelExporter` sending spans without authentication when the API key was resolved after construction with `opentelemetry-exporter-otlp-proto-http` 1.45, and fixed usage capture for the [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) when partial messages are enabled. * [v0.42.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.42.0) - Fixed remote scorers to correctly serialize attachment inputs, so scorers that receive attachments no longer fail. Fixed org context propagation for scorers added from the playground during remote evals. Preserved [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md) session IDs across resumed runs. * [v0.41.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.41.1) - Improved span creation performance under heavy tracing. * [v0.41.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.41.0) - Added [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md) instrumentation for synchronous and asynchronous SDK calls. Added [Google Discovery Engine](https://docs.braintrust.dev/integrations/sdk-integrations/google-discovery-engine.md) instrumentation for answer generation, conversational search, grounding checks, and ranking. * [v0.40.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.40.0) - Adds OpenAI [Agents API](https://docs.braintrust.dev/integrations/ai-providers/openai.md#what-traced-python) instrumentation: `beta.agents.sessions.create(stream=True)` calls are now traced as `openai.agents.sessions.create` task spans, capturing agent and session metadata, tool call child spans, and token usage. Fixes remote eval reruns to preserve upsert IDs across reruns. * [v0.39.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.39.0) - Added [workflow evaluations](https://docs.braintrust.dev/sdks/python/workflow-evaluations.md) in public preview, which submit tasks and scorers to asynchronous provider APIs and collect results through polling or webhooks. Added `has_error`, `metadata`, and `duration` fields to `SpanFilters` for `trace.get_spans()`, allowing trace-level scorers to filter spans by error state, metadata values, and execution time bounds. See [custom code scorers](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces) for details. * [v0.38.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.38.0) - Scorer functions can now return `{"score": 0.5}` without a `name` field. The scorer function's name is used as the score key. Fixed [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) extended-thinking spans to capture `completion_reasoning_tokens`. Fixed [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) agent spans to include `instructions` and `system_prompt` in span input, tools in model-call metadata, and reasoning token counts for Anthropic and Google models. * [v0.37.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.37.0) - Added invocation parameters (`temperature`, `max_tokens`, and similar model settings) to [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) LLM span metadata when model settings are provided. * [v0.36.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.36.0) - Added eval instrumentation for [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md): `AccuracyEval`, `AgentAsJudgeEval`, `ReliabilityEval`, `PerformanceEval`, and eval suites are now traced automatically via `auto_instrument()`. Eval suite runs open a Braintrust experiment automatically. Added standard verifier output upload to the [Harbor](https://docs.braintrust.dev/integrations/sdk-integrations/harbor.md) plugin (`test-stdout.txt`, `test-stderr.txt`, and `ctrf.json`) with `attachments` and `redact_patterns` configuration. Fixed API key propagation in `load_prompt`. ### TypeScript SDK releases * [v3.35.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.35.0) - Added [span customizers](https://docs.braintrust.dev/sdks/typescript/redact-sensitive-data.md#span-customizers), set with `spanCustomizers` in [`configureInstrumentation()`](https://docs.braintrust.dev/sdks/typescript/api-reference#configureinstrumentation), to modify or redact records from instrumented spans before they leave your process. Added instrumentation for [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) audio and image APIs, [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) image generation, image editing, and video generation, and [Groq](https://docs.braintrust.dev/integrations/ai-providers/groq.md) audio APIs (speech, transcription, and translation). Set `BRAINTRUST_CAPTURE_ATTACHMENTS` to `1` or `true` to upload files from OpenAI image and audio calls as attachments. Fixed token metrics for AI SDK middleware, nesting of child spans under [Pi Coding Agent](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md) tool spans, capture of provider-managed tool calls and the provider name when using the [Vercel AI Gateway](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md), and dataset snapshots silently including deleted rows. **Breaking change**: Base64 image and file inputs to OpenAI Chat Completions calls are recorded as `` unless `BRAINTRUST_CAPTURE_ATTACHMENTS` is set to `1` or `true`. Earlier versions uploaded them as attachments. * [v3.34.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.34.0) - Added [TypeSafe](https://docs.braintrust.dev/integrations/ai-providers/typesafe.md) instrumentation and tracing for [Vercel AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) `experimental_evaluate()` calls. Improved `flush()` so events queued after a flush starts do not delay it. Fixed OpenAI Batch API instrumentation for `Uint8Array` and `Buffer` JSONL sources and the `/v1/embeddings` endpoint. * [v3.33.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.33.0) - Added [workflow evaluations](https://docs.braintrust.dev/sdks/typescript/workflow-evaluations.md) (`defineWorkflowEval()`) in public preview, which submit tasks and scorers to asynchronous provider APIs and collect results through polling or webhooks. Added `SpanFilters` to `Trace.getSpans()`, so [trace-level scorers](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces) can filter spans by type, name, error state, metadata, and execution time. Added auto-instrumentation for the [LangGraph Platform SDK](https://docs.braintrust.dev/integrations/agent-frameworks/langgraph.md) (`@langchain/langgraph-sdk`), tracing `RunsClient.wait()` and `RunsClient.stream()` calls. Added `startOpenAIAgentsTrace`, `updateOpenAIAgentsTrace`, and `failOpenAIAgentsTrace` for tracing the [OpenAI Agents API](https://docs.braintrust.dev/integrations/ai-providers/openai.md#agents-api-typescript) (`openai.beta.agents.sessions`) from `openai` v7+. Added multimodal embedding support to the [Gemini](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) integration for `gemini-embedding-2` models, capturing image inputs as Braintrust attachments. **Breaking change**: Workflow evaluations replace the experimental `defineDurableEval()` API from v3.28.0. `defineDurableEval()` and its `processBatchResult()` method, `BatchTask`, `BatchScorer`, `DurableEvalStore`, `DurableEvalMemoryStore`, and `DurableEvalRedisStore` are removed. * [v3.32.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.32.0) - Added [ElevenLabs](https://docs.braintrust.dev/integrations/ai-providers/elevenlabs.md) instrumentation (`wrapElevenLabs`) for tracing text-to-speech and speech-to-text calls, including streaming and timestamped audio. Added [`@google/generative-ai`](https://docs.braintrust.dev/integrations/ai-providers/gemini#google-generative-ai-typescript) instrumentation (`wrapGoogleGenerativeAI`) for the legacy Google Generative AI SDK (v0.24.x). Added `input_audio` content type support in Mustache and Nunjucks prompt template rendering. * [v3.31.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.31.0) - Test runner scorer functions can now return a score object without a `name` field, with `"score"` used as the key when `name` is omitted. [Vercel AI SDK v6](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) tool call spans now include `toolCallId` in span metadata. Fixed Anthropic token metrics in LangChain spans and instrumentation hook injection for Next.js versions earlier than 15. * [v3.30.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.30.0) - Added OpenAI Batch API instrumentation with `openaiFilesCreateTraced`, `openaiBatchesRetrieveTraced`, and `completeOpenAIBatchTrace` for tracing asynchronous OpenAI Batch requests. See [OpenAI Batch API](https://docs.braintrust.dev/integrations/ai-providers/openai.md#batch-api-typescript) for details. ### Go SDK releases * [v0.16.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.16.0) - Adds [span customizers](https://docs.braintrust.dev/sdks/go/redact-sensitive-data.md): register ordered customizers with `braintrust.WithSpanCustomizers` to modify or redact every span before export. If a customizer fails, the SDK drops the whole export batch instead of sending unmodified spans. The OpenAI Responses API integration also captures `context_management` and `conversation` in request and response span metadata. * [v0.15.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.15.0) - Adds tracing for the [A2A (Agent2Agent) protocol](https://docs.braintrust.dev/integrations/agent-frameworks/a2a.md) (`trace/contrib/a2a`), covering client `SendMessage()` and `SendStreamingMessage()` calls and server request handling, with trace context propagated so client and server spans share one trace. Supports Orchestrion auto-instrumentation or manual setup with `InstrumentClient()` and `InstrumentServer()`. Captures `completion_reasoning_tokens` for Anthropic extended thinking, and preserves `server_tool_use` and `mcp_tool_use` blocks in streamed Anthropic output. Captures `previous_response_id` and `include` in OpenAI Responses API span metadata, so chained responses can be linked. Fixes streaming OpenAI chat completion spans to record response metadata, `logprobs`, and refusal text, and to keep each choice separate when `n` is greater than 1. * [v0.14.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.14.0) - Adds tracing for the legacy OpenAI `/v1/completions` endpoint as a new `Completion` span. Captures Anthropic `context_management` config and response `applied_edits` together in span metadata. Adds `prompt_cache_key`, `safety_identifier`, `web_search_options`, `prediction`, and streaming citation `annotations` to OpenAI chat completion and responses spans. HTTP 4xx/5xx responses mark spans as errors. * [v0.13.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.13.0) - Adds prompt support: load a Braintrust prompt in Go, render its variables locally, and call any LLM client with the result. Prompts can also drive eval parameters, and rendered calls link back to their prompt in Braintrust traces. See [Go API reference](https://docs.braintrust.dev/sdks/go/api-reference.md#prompts) for details. ### Java SDK releases * [v0.3.25](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.25) - Adds [span customizers](https://docs.braintrust.dev/sdks/java/redact-sensitive-data.md): implement `SpanCustomizer` and register it with `addSpanCustomizer()` on `BraintrustConfig.Builder` to modify or redact every span before export, including attachment data before upload. If a customizer fails, the SDK drops the whole export batch. **Breaking change**: `context.span_origin.instrumentation.name` now holds the name of the tracer that created the span, such as `openai`, `anthropic`, `aws-bedrock`, `genai`, `langchain4j`, or `springai` for integration spans, or your tracer's name for spans you create. Earlier versions always used `braintrust-java`, which still appears on eval and dev server spans. Update queries that filter on `braintrust-java`. * [v0.3.24](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.24) - The S3 attachment uploader now recovers automatically after transient errors instead of shutting down. New `BraintrustConfig` options tune its behavior: `BRAINTRUST_ATTACHMENT_UPLOADER_QUEUE_SIZE` (default: `1024`), `BRAINTRUST_ATTACHMENT_UPLOADER_REQUEST_TIMEOUT_MILLIS` (default: `60000`), `BRAINTRUST_ATTACHMENT_UPLOADER_MAX_RETRIES` (default: `8`), and `BRAINTRUST_ATTACHMENT_UPLOADER_INITIAL_RETRY_DELAY_MILLIS` (default: `500`). See [environment variables](https://docs.braintrust.dev/sdks/java/api-reference.md#environment-variables) for details. * [v0.3.23](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.23) - Eval cases now run concurrently (default: 3 at a time). `Eval.start()` starts an eval in the background and returns an `EvalResult` as soon as the experiment exists, so you can surface the link before all cases finish. `Eval.run()` awaits completion as before. `EvalResult` gains lifecycle APIs: `getStatus()`, `isDone()`, `awaitCompletion()`, `getStartedAt()`, `getDuration()`, `getCasesExecuted()`, and `getAbortCause()`. Set concurrency with `.maxConcurrency(int)` on `Eval.Builder` or `Devserver.Builder`, or supply your own `Executor` with `.executor(Executor)`. Global defaults for concurrency and OTel batch tuning are available via `BRAINTRUST_DEFAULT_MAX_CONCURRENCY`, `BRAINTRUST_OTEL_MAX_QUEUE_SIZE`, `BRAINTRUST_OTEL_MAX_EXPORT_BATCH_SIZE`, and `BRAINTRUST_OTEL_EXPORT_INTERVAL_MILLIS`. **Breaking change**: `Task`, `Scorer`, and `Classifier` implementations must now be thread-safe. See [API reference](https://docs.braintrust.dev/sdks/java/api-reference.md#evaluations) for details. ### Ruby SDK releases * [v0.5.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.5.0) - Added support for [RubyLLM](https://docs.braintrust.dev/integrations/sdk-integrations/ruby-llm.md) 2.0. Improving eval flushing performance. Fixed `rails g braintrust:server` failing because the initializer template was missing from the published gem. * [v0.4.2](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.4.2) - Fixed a bug where the Playground dropped `origin`, `tags`, and `metadata` on inline dataset rows sent to a remote eval endpoint, forwarding only `input` and `expected`. The Playground uses `origin` to match streamed progress events back to its grid rows, so results appeared to never arrive. ### GitHub eval action releases * [v2.1.0](https://github.com/braintrustdata/eval-action/releases/tag/v2.1.0) - Added `report_scores` and `report_metrics` inputs to filter which scores and metrics appear in the PR comment. Each accepts a comma- or newline-separated list of names. Omitting either input includes all results in that category. The PR comment table now labels scores and metrics in separate sections within a single table. See [Run in CI/CD](https://docs.braintrust.dev/evaluate/run-in-ci.md) for details. ### trace-claude-code releases * [v3.0.1](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v3.0.1) - Claude Code runs `bt` directly for each hook instead of through a Bash script, and stops sending hook events that tracing does not use, such as idle notifications. The plugin no longer installs `bt` when it is missing, so `bt` must be on the `PATH` of the environment that launches Claude Code. See [Claude Code tracing troubleshooting](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#troubleshooting-tracing). * [v3.0.0](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v3.0.0) - The `trace-claude-code` tracing integration is unchanged. **Breaking change**: The Braintrust Claude marketplace no longer distributes the MCP-specific `braintrust` plugin. Configure Braintrust MCP directly in [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#connect-to-braintrust-mcp), or use a remote connector in [Cowork](https://docs.braintrust.dev/integrations/developer-tools/claude-cowork.md#connect-to-braintrust-mcp). ### trace-codex releases * [v2.0.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.1-codex) - Codex runs `bt` directly for each hook on every platform, so Windows no longer requires Bash. The plugin no longer installs `bt` when it is missing. After updating, trust the changed Braintrust hook through `/hooks`. See the [Codex integration guide](https://docs.braintrust.dev/integrations/developer-tools/codex.md#troubleshooting-tracing). * [v2.0.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.0-codex) - The `trace-codex` tracing integration is unchanged. **Breaking change**: The Braintrust Codex marketplace no longer distributes the MCP-specific `braintrust` plugin and its bundled routing skill. Configure Braintrust MCP directly in [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md#connect-to-braintrust-mcp). ### trace-grok releases * [v0.0.4](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.0.4-grok) - Runs `bt` directly instead of through a Bash script. * [v0.0.3](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.0.3-grok) - Adds per-turn cache token counts, reasoning tokens, model-call count, API duration, and raw cost metrics when Grok reports them. Recovers the system prompt and first user message on the first model-call span when available. * [v0.0.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.0.2-grok) - Updates the plugin README with authentication instructions and the per-session hook reload requirement. Tracing behavior is unchanged. * [v0.0.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.0.1-grok) - Adds [Grok session tracing](https://docs.braintrust.dev/integrations/developer-tools/grok.md), including model-call details, tool activity, and token usage for each turn. Requires `bt` v0.19.3 or later. ### Google Antigravity tracing releases * [v0.1.3](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.1.3-antigravity) - Antigravity runs `bt` directly for each hook, so tracing no longer requires a Unix-compatible `sh`. With `bt` v0.21.0 or later, you can [set up Antigravity tracing](https://docs.braintrust.dev/integrations/developer-tools/antigravity.md) on Windows. * [v0.1.2](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.1.2-antigravity) - Uses Antigravity's native hook timeouts for trace capture. * [v0.1.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.1.1-antigravity) - Internal maintenance update. ### @braintrust/trace-opencode releases * [v2.1.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.1.0-opencode) - Refreshes session summaries while the agent is idle. Fixes a delivery flushing issue where a slow flush could block idle handling and shutdown. * [v2.0.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.1-opencode) - Internal maintenance update to OpenCode tracing. * [v2.0.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.0-opencode) - Prevents duplicate traces when you use `bt trace run` with the tracing plugin already installed, and fixes the built-in tool for logging data to Braintrust. **Breaking changes**: The plugin no longer reads environment-variable overrides for tracing enablement, profile, organization, project, or metadata. Save these settings with `bt trace enable opencode`. Nested profile, organization, and project settings now take precedence over their legacy top-level equivalents. See [OpenCode migration instructions](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#migrate-from-an-older-plugin-version). ### @braintrust/pi-extension releases * [v2.1.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.1.1-pi) - Reduces tracing overhead during streamed responses by sending only the first text or thinking update for each model call, which preserves time to first token. A slow tracing process no longer delays pi startup or event capture. * [v2.1.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.1.0-pi) - Fixes a delivery flushing issue where a slow flush could block idle handling and shutdown. * [v2.0.1](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.1-pi) - Fixes extension startup failures on compatible pi versions. See the [pi integration guide](https://docs.braintrust.dev/integrations/developer-tools/pi.md). * [v2.0.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v2.0.0-pi) - **Breaking change**: The extension no longer reads environment-variable overrides for tracing enablement, profile, organization, project, or metadata. Save these settings with `bt trace enable pi`. Nested profile, organization, and project settings now take precedence over their legacy top-level equivalents. See [pi migration instructions](https://docs.braintrust.dev/integrations/developer-tools/pi.md#migrate-from-an-older-extension-version). ### Improvements * **Analyze trace** now shows guide sections as collapsible timeline groups, preserving span order, bars, and hierarchy. * Loop and the [MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#tools) now resolve Braintrust organization members, API keys, and service tokens by name, so you can ask which teammate or key produced a set of traces instead of reading raw IDs. See [Explore your data](https://docs.braintrust.dev/loop/capabilities.md#explore-your-data) for details. * The [MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#tools) now resolves and generates links to patterns, prompts, and scorers within their containing projects. Both tools accept `org_name` to disambiguate name lookups across organizations. * Custom column names that conflict with built-in table fields are now rejected at creation time. See [Create custom columns](https://docs.braintrust.dev/observe/view-logs.md#create-custom-columns) for details. * GPT-6 Astra, a multimodal reasoning model, is now available in [Loop's model picker](https://docs.braintrust.dev/loop.md#choose-a-provider-and-model), including scheduled automations. * Long [Loop threads](https://docs.braintrust.dev/loop/manage.md#view-thread-history) now load faster by loading recent history first, with earlier messages available on demand. * Loop automations that cannot run because model credits are depleted, the selected built-in model is unavailable, or built-in models are disabled for the organization now show as **Paused** in \*\* Settings\*\* > [\*\* Automations\*\*](https://www.braintrust.dev/app/~/configuration/automations) with a warning indicator. Opening the automation shows a banner explaining the reason and links to the credit or AI provider settings. See [Pause an automation](https://docs.braintrust.dev/loop/automations.md#pause-an-automation). * You can now select GPT-6 Astra in [Loop](https://docs.braintrust.dev/loop.md#choose-a-provider-and-model), including scheduled automations. * [Loop automations](https://docs.braintrust.dev/loop/automations.md#create-a-loop-automation) now support timezone selection for custom cron schedules. New Pattern discovery automations run daily at 9:00 AM by default. * You can now copy a shareable link to a dashboard chart that opens it in fullscreen mode, and each chart's move and copy actions now sit directly in its more options menu (). See [Move and copy charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md#move-and-copy-charts). * Large traces now download faster from the trace viewer, with a progress indicator showing how many spans have been fetched and the percentage complete. * The [Braintrust MCP server](https://docs.braintrust.dev/integrations/developer-tools/mcp.md) now lists connected Slack workspaces and public channels with `list_slack_channels`, helping agents target the intended channel when configuring alerts and automations. * The [MCP server's `search_patterns` tool](https://docs.braintrust.dev/integrations/developer-tools/mcp#tools) now supports cursor pagination for listings and text searches, so agents can retrieve more than 100 matching patterns. * GPT-5.6 Luna now supports Max reasoning effort in [Loop](https://docs.braintrust.dev/loop.md#choose-a-provider-and-model). * You can now add filters through the search bar while viewing [live topic distributions](https://docs.braintrust.dev/observe/topics/review-insights.md#real-time-analysis), with autocomplete to help build filters. * The [**Evaluators** tab](https://docs.braintrust.dev/observe/examine-traces.md#test-and-apply-evaluators) in Logs now includes scorers and classifiers from other projects in your organization. * The [**Usage analytics**](https://docs.braintrust.dev/admin/billing/monitor-usage.md#view-usage) section now includes a **CSV** export for the **Entire organization** scope, alongside the existing project-level export. * GPT-6 Luna and GPT-6 Sol are now available in [Loop](https://docs.braintrust.dev/loop.md#choose-a-provider-and-model) and scheduled Loop automations. Both are multimodal reasoning models that support Max reasoning effort. * You can now create Scorer, Loop, and Topics automations from [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) without navigating to project settings. * You can now push code functions and scorers from Node.js 24, and they run in a matching `nodejs24.x` sandbox. See [Remote evals](https://docs.braintrust.dev/evaluate/remote-evals.md) for supported runtimes and async behavior requirements on Node.js 24. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) * The [span detail panel](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-hierarchy) now shows a **Context** section for spans with a `context` field, such as the span origin recorded by SDK integrations. * [Translating message content](https://docs.braintrust.dev/observe/examine-traces.md#translate-message-content) in the trace viewer is now faster. Translation uses a built-in model by default instead of the model selected in Loop. * SQL queries and dashboard charts that group by three or more keys now use Brainstore's faster columnar aggregation when every key qualifies for the fast path, and the [SQL sandbox](https://docs.braintrust.dev/reference/sql.md#sql-sandbox) and Loop no longer reject them. For a time series chart, the time bucket counts toward the three. See [Aggregate efficiently](https://docs.braintrust.dev/reference/sql/best-practices.md#aggregate-efficiently) for which keys qualify. Requires data plane v2.16.0+ (Self-hosted customers must upgrade to use this feature.) * [Loop](https://docs.braintrust.dev/loop.md) now recognizes HTTP and HTTPS URLs as links in your messages. Typing URLs no longer opens command or mention menus. * [Patterns](https://docs.braintrust.dev/observe/patterns/enable.md#common-issues) now surfaces failed automation runs with error messages and direct links to their Loop threads, so you can investigate failures from the Patterns page. * Clicking a span name in the trace panel now copies it to the clipboard instead of filtering the logs list. This works in every trace panel, not only in Logs. See [Copy a span name](https://docs.braintrust.dev/observe/examine-traces.md#copy-a-span-name) for details. * The [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) table now supports a **Dense** row height, so you can scan more rows at once. See [Change the table density](https://docs.braintrust.dev/observe/view-logs.md#change-the-table-density) for details. * [Patterns Slack digests](https://docs.braintrust.dev/observe/patterns/enable.md) now include active patterns and patterns closed during the run, excluding patterns that remained closed. * [Patterns evidence](https://docs.braintrust.dev/observe/patterns/review.md#examples) now previews up to three prioritized examples and shows saved frequency estimates when available. Copying a pattern for a coding agent includes all attached evidence and measurements. * You can now open or create a Loop automation directly from the [\*\* Loop\*\*](https://www.braintrust.dev/app/~/loop) page. See [Create a Loop automation](https://docs.braintrust.dev/loop/automations.md#create-a-loop-automation) for details. * You can now retry failed [Loop automation runs](https://docs.braintrust.dev/loop/automations.md#review-past-runs) to recover from transient failures without starting over or reconfiguring the automation. * The search dropdown now shows recent SQL filters that start with the text you type, so you can reuse a previous filter without clearing the search field. See [Filter and search logs](https://docs.braintrust.dev/observe/filter.md) for details. **August 2026** ### Dashboards Monitoring views are now dashboards, with a dedicated page per dashboard and a project-scoped list to browse, search, and star them. Start from an empty dashboard, clone the built-in **Cost and quality** dashboard, or duplicate an existing one, then rename or delete it from the **Dashboard actions** menu. Chart, filter, and grouping changes save automatically. See [Dashboards](https://docs.braintrust.dev/observe/dashboards.md) for details. ### Permissions for organization AI providers You can now use permissions to control who can access and manage organization-level AI providers. Learn more about [provider permissions](https://docs.braintrust.dev/admin/ai-providers.md#permissions-and-access). ### Coding-agent plugins trace through bt Tracing for [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md), [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md), [OpenCode](https://docs.braintrust.dev/integrations/developer-tools/opencode.md), and [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md) now runs through the [`bt` CLI](https://docs.braintrust.dev/reference/cli/trace). `bt trace setup` installs or updates the plugin, selects the Braintrust project, and writes the agent's tracing settings. [`bt trace run`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-run) and [`bt trace import`](https://docs.braintrust.dev/reference/cli/trace#bt-trace-import) cover one-off and saved-session workflows. **Breaking change**: `bt` now handles authentication and trace routing for these plugins. Older plugin-specific API key, project, tracing, and config-file settings must be migrated. See the [`bt` CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate#v0-16-0) and the upgrade notes for [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#migrate-from-an-older-plugin-version), [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md#upgrade), [OpenCode](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#upgrade), and [pi](https://docs.braintrust.dev/integrations/developer-tools/pi.md#upgrade). ### MCP write tools The Braintrust MCP server now exposes write tools, so coding agents can create and update Braintrust objects instead of only reading them. Agents can author prompts, scorers, and classifiers, configure the [Topics](https://docs.braintrust.dev/observe/topics.md) pipeline end to end, build dashboards, create alerts and scheduled jobs, run evals, and edit dataset rows. Write tools use the permissions of your authenticated Braintrust account, and several replace or remove objects, so configure your client to require confirmation before running one. See [Braintrust MCP](https://docs.braintrust.dev/integrations/developer-tools/mcp.md) for the full tool list. ### Kimi K3 and DeepSeek V4 Flash 0731 Kimi K3 and DeepSeek V4 Flash 0731 join GLM-5.2 as built-in open-source models served by Braintrust, with no AI provider setup required. Select them under the **Braintrust** provider in playgrounds, prompts, and scorers, or request `kimi-k3` and `deepseek-v4-flash-0731` through the Braintrust Gateway. Usage draws from your monthly [model credits](https://docs.braintrust.dev/plans-and-limits.md#model-credits), shared with Topics. See [Available models](https://docs.braintrust.dev/admin/ai-providers.md#available-models) for details. ### AWS Lambda extension The Braintrust Lambda Extension gives Python and TypeScript/JavaScript Lambda functions a local handoff path for traces, so the Braintrust SDK's `flush()` method spends less time in the request path. See [AWS Lambda](https://docs.braintrust.dev/integrations/runtime-environments/aws-lambda.md) for setup, compatibility, and considerations. > [!WARNING] > > The Braintrust Lambda extension is in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### Group scope for online scoring Online scoring rules now support **Group** scope, allowing you to evaluate a set of related, multi-turn traces as a single unit rather than one trace at a time. Groups are based on a session key of your choice, and don't require you to change your logging. Group scores can be kept on the first trace in the group and updated as the conversation grows, or written to every trace with prior turns as context. See [Score production traces](https://docs.braintrust.dev/evaluate/score-online.md) for details. Requires data plane v2.8.0+ (Self-hosted customers must upgrade to use this feature.) ### Pause and resume automations Log alerts, environment alerts, time window alerts, scheduled Loop jobs, and online scoring rules can now be paused and resumed instead of deleted, so you can stop one during an incident or a noisy deploy without losing its configuration. Each automation shows an **Active** or **Paused** status, along with when it last ran. Events and traces that arrive while an automation is paused aren't processed when it resumes. See [Pause an alert](https://docs.braintrust.dev/observe/alerts.md#pause-an-alert) and [Pause an automation](https://docs.braintrust.dev/evaluate/score-online.md#pause-an-automation) for details. Requires data plane v2.11.0+ (Self-hosted customers must upgrade to use this feature.) ### Summary table layout for the experiments list The experiments list now has a **Summary table** layout that compares every experiment in a project at a glance, with each experiment as a column and scores and metrics as rows. An **All scores (avg)** row shows the mean of the non-pairwise scores recorded for each experiment. Group the table to replace experiment columns with group values and compare aggregated scores across groups. See [Compare across all experiments](https://docs.braintrust.dev/evaluate/compare-experiments.md#compare-across-all-experiments) for details. ### Annotated version history Describe what changed as you save a new version of a prompt, parameter, or scorer, and the note stays pinned to that version in its history. Version history now also shows each version next to the one it replaced, so you can move directly between either side of a change. See [Version prompts](https://docs.braintrust.dev/evaluate/prompts/versions.md) for details. ### Trace groups in datasets A dataset row can now reference a group of traces instead of a single one, so a multi-turn session or a related set of logs becomes one example. Select several traces from your logs and add them as a group, up to 64 per row. The row renders each trace inline and flags any that are no longer available. See [Promote traces from logs](https://docs.braintrust.dev/annotate/datasets/create.md#promote-traces-from-logs) for details. ### Azure AI Gateway Braintrust now supports **Azure AI Gateway** as an AI provider, so you can use one provider for all backends behind an Azure API Management endpoint. It supports models that use the OpenAI Chat Completions, OpenAI Responses, or Anthropic Messages API. See [Azure AI Gateway](https://docs.braintrust.dev/integrations/ai-providers/azure-ai-gateway.md) for setup details. ### Query sidebar in the SQL sandbox The SQL sandbox now organizes your queries in a collapsible sidebar, where you can search by name, reorder by dragging, or jump to any query from the command bar. Each query has an menu to rename, duplicate, or delete it. Click **Copy share link** to hand a query to a teammate as a URL, which opens in their sandbox without running. See [SQL sandbox](https://docs.braintrust.dev/reference/sql.md#sql-sandbox) for details. ### Edit a member's permission groups You can now manage a member's permission groups directly from \*\* Settings\*\* > [\*\* Members\*\*](https://www.braintrust.dev/app/~/configuration/org/team), rather than opening each group in turn to find where that person belongs. The dialog shows the groups they belong to directly, the ones they inherit through another group's membership, and the ones you can add them to. See [Manage group membership](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#manage-group-membership) for details. ### Preprocessors for LLM-as-a-judge scorers Trace-level LLM-as-a-judge scorers can now select the preprocessor that turns the trace into `{{thread}}` and `{{preprocessed}}`. A new **Preprocessors** section in project settings lets you create, test, and manage the preprocessors that scorers, the Thread view, and Topics facets use. See [Choose a preprocessor](https://docs.braintrust.dev/evaluate/llm-as-a-judge.md#choose-a-preprocessor) and [Manage preprocessors](https://docs.braintrust.dev/admin/projects.md#manage-preprocessors) for details. Requires data plane v2.10.0+ (Self-hosted customers must upgrade to use this feature.) ### bt CLI releases * [v0.18.0](https://github.com/braintrustdata/bt/releases/tag/v0.18.0) - Added [`bt environments`](https://docs.braintrust.dev/reference/cli/environments) to manage deployment environments. Added prompt version listing and environment assignment to [`bt prompts`](https://docs.braintrust.dev/reference/cli/prompts), and environment-based version lookup to [`bt functions view`](https://docs.braintrust.dev/reference/cli/functions#bt-functions-view). Added [`--max-concurrency`](https://docs.braintrust.dev/reference/cli/eval#evaluator-concurrency) to limit how many evaluators run at once. [`bt scorers create`](https://docs.braintrust.dev/reference/cli/scorers) now validates model parameters before saving and returns actionable errors for invalid definitions. **Breaking change**: Removed the `--api-key` flag. Use `BRAINTRUST_API_KEY` or a saved profile instead. See the [v0.18.0 migration steps](https://docs.braintrust.dev/reference/cli/migrate.md#v0-18-0). * [v0.17.0](https://github.com/braintrustdata/bt/releases/tag/v0.17.0) - Added [`bt profiles`](https://docs.braintrust.dev/reference/cli/profiles) to list, rename, and delete saved login profiles. Added [`bt scorers create`](https://docs.braintrust.dev/reference/cli/scorers) to create prompt-based LLM scorers and classifiers from the CLI. `bt setup` is now deprecated in favor of the standalone setup wizard. Fixed `bt update` on Windows, fixed `bt init` project ID saving, and improved remote eval connection diagnostics. * [v0.16.2](https://github.com/braintrustdata/bt/releases/tag/v0.16.2) - Added automatic Braintrust instrumentation for JavaScript and TypeScript evals in [`bt eval`](https://docs.braintrust.dev/reference/cli/eval), so supported AI clients and instrumented libraries can emit spans before eval files load. Use `--no-auto-instrumentation` to opt out. This release also fixes `type: "model"` eval parameter serialization and improves `bt init` errors for unwritable `.bt` directories. * [v0.16.1](https://github.com/braintrustdata/bt/releases/tag/v0.16.1) - Improved [`bt trace`](https://docs.braintrust.dev/reference/cli/trace) so coding-agent tracing setup, per-run tracing, and session imports use the shared plugin runtime. `bt trace run` now waits briefly for trace events from the launched agent to flush before returning, making traced CI and scripted runs more reliable. * [v0.16.0](https://github.com/braintrustdata/bt/releases/tag/v0.16.0) - Added [`bt trace`](https://docs.braintrust.dev/reference/cli/trace) to install and configure supported coding-agent tracing plugins, trace individual agent runs, and import past sessions. `bt` now owns coding-agent tracing authentication through [`bt login`](https://docs.braintrust.dev/reference/cli/login), instead of each plugin reading its own credentials. Added [mise](https://docs.braintrust.dev/reference/cli/quickstart.md#install) as a supported installation method. Improved [`bt view`](https://docs.braintrust.dev/reference/cli/view) to correctly parse experiment links passed to `--url`. **Breaking changes**: the `bt auth` commands are replaced by top-level commands: `bt auth login` → [`bt login`](https://docs.braintrust.dev/reference/cli/login), `bt auth logout` → [`bt logout`](https://docs.braintrust.dev/reference/cli/logout), `bt auth profiles` → [`bt status --all`](https://docs.braintrust.dev/reference/cli/status), and `bt auth refresh` → [`bt login --refresh`](https://docs.braintrust.dev/reference/cli/login). The `--fresh` flag is renamed to `--force` on [`bt sync pull`](https://docs.braintrust.dev/reference/cli/sync), [`bt sync push`](https://docs.braintrust.dev/reference/cli/sync), and [`bt datasets pipeline push`](https://docs.braintrust.dev/reference/cli/datasets#bt-datasets-pipeline). Profiles are also [decoupled from organizations](https://docs.braintrust.dev/reference/cli/migrate.md#v0-16-0), so you log in once per account and select your organization with [`bt switch`](https://docs.braintrust.dev/reference/cli/switch). The `--api-key` flag is deprecated, hidden from help output, and will be removed soon; use `BRAINTRUST_API_KEY` or [`bt login`](https://docs.braintrust.dev/reference/cli/login) instead. See the [CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate.md) for migration steps. ### Python SDK releases * [v0.35.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.35.0) - Added `dataset_id` parameter to [`init_dataset()`](https://docs.braintrust.dev/sdks/python/api-reference#init_dataset) to initialize a dataset by its ID, taking precedence over `project`, `project_id`, and `name`. Fixed LiteLLM prompt caching metrics not being traced correctly. * [v0.34.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.34.0) - Added [Hugging Face Transformers](https://docs.braintrust.dev/integrations/ai-providers/huggingface.md#setup-python) auto-instrumentation for local pipelines, including text generation, summarization, translation, feature extraction, and question answering. The [Harbor](https://docs.braintrust.dev/integrations/sdk-integrations/harbor.md) plugin now defaults the Braintrust project name to `Harbor`, so passing `project_name` is optional. Eval case fields are now forwarded to scorer functions. Fixed Mistral cached prompt token tracking. Fixed [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) traces attributing query usage to the final LLM call and creating orphan spans for concurrently responding MCP tools. * [v0.33.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.33.0) - Added [Vercel AI SDK for Python](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) auto-instrumentation (enabled by default in `auto_instrument()`). Added [Cursor SDK](https://docs.braintrust.dev/integrations/agent-frameworks/cursor-sdk.md) Python instrumentation for tracing agent runs, model turns, and tool calls. Added native [Harbor](https://docs.braintrust.dev/integrations/sdk-integrations/harbor.md) job plugin for syncing Harbor evaluation results to Braintrust. * [v0.32.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.32.0) - Added [Pipecat](https://docs.braintrust.dev/integrations/agent-frameworks/pipecat.md) auto-instrumentation for tracing real-time voice AI pipelines, including LLM turns, STT, TTS, and tool calls. Added `invoke_async()` as an async counterpart to `invoke()`. **Breaking change**: [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md) audio attachments on `agent_speaking` spans are now disabled by default. Set `BRAINTRUST_CAPTURE_AGENT_AUDIO_ATTACHMENTS=true` to restore the previous behavior. * [v0.31.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.31.1) - Corrected Google token usage metrics so [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) and [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) traces report tool-use, reasoning, and audio and image tokens under standard metrics that reconcile with Google's reported totals. ### TypeScript SDK releases * [v3.29.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.29.0) - `braintrust eval` now applies auto-instrumentation, so supported AI clients and instrumented libraries can emit spans before eval files load. Added `datasetId` parameter to `initDataset()` to look up a dataset by ID without specifying a project. Added `apiKey` parameter to `loadPrompt()`. [Vercel AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) `generateImage` calls are now traced as LLM spans with generated images stored as Braintrust attachments. [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) thinking tokens and [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) reasoning tokens are now recorded as span metrics. Fixed `noTraceIO` being ignored in `wrapTraced` for plain functions. * [v3.28.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.28.0) - Added [Voyage AI](https://docs.braintrust.dev/integrations/ai-providers/voyageai.md) auto-instrumentation for embeddings, multimodal embeddings, reranking, and contextualized embeddings. Scorer functions in `Eval()` now receive the `id` and `tags` of each case. Fixed Claude Agent SDK subagent tool span parenting, and corrected per-call token and cost metrics so query-phase usage is no longer attributed to the final call. * [v3.27.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.27.0) - Added [Ollama](https://docs.braintrust.dev/integrations/ai-providers/ollama.md) and [`@cloudflare/think`](https://docs.braintrust.dev/integrations/agent-frameworks/cloudflare-think) instrumentation, Anthropic beta sessions tracing (`anthropic.beta.sessions.turn` and `anthropic.beta.sessions.thread.turn`), [Flue](https://docs.braintrust.dev/integrations/agent-frameworks/flue.md) v2 support, and a `vitest-evals` span input override via `meta.eval.input`. * [v3.26.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.26.0) - Added auto-instrumentation for [Cloudflare Agents](https://docs.braintrust.dev/integrations/agent-frameworks/cloudflare-agents.md), [Cloudflare AI Chat](https://docs.braintrust.dev/integrations/agent-frameworks/cloudflare-ai-chat.md), and [Hugging Face Transformers.js](https://docs.braintrust.dev/integrations/ai-providers/huggingface.md#transformers-typescript). Added system prompt capture for [Strands Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/strands-agent.md) spans. Fixed token counting for Google GenAI and Google ADK. * [`@braintrust/langchain-js` v0.2.5](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/%40braintrust%2Flangchain-js%400.2.5) - **Deprecated.** This is the final release of `@braintrust/langchain-js`. Import `BraintrustLangChainCallbackHandler` from `braintrust` instead. Fixed reasoning token recording as a metric on LangChain JS LLM spans. See the [LangChain integration guide](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) for migration details. ### Go SDK releases * [v0.12.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.12.0) - Adds remote eval support and a new `braintrust.NewEval` API for defining reusable evaluations. Define an eval once with its task, scorers, and `ParameterSchema`, then call `Run` against any dataset for local runs or register it with `evalrunner.RegisterEval` so the Braintrust playground can trigger it from your own infrastructure. Parameters declared in `ParameterSchema` become configurable controls in the playground and reach the task via `hooks.Parameters`. Also adds `eval.TaskWithHooks` for tasks that need to read parameters, and `DatasetQueryOpts.ProjectID` and `ProjectName` for scoped dataset lookups. **Go remote evals are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md).** See [Remote evals](https://docs.braintrust.dev/evaluate/remote-evals.md) and [Go API reference](https://docs.braintrust.dev/sdks/go/api-reference.md#evaluations) for details. * [v0.11.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.11.1) - Aligned [CloudWeGo Eino](https://docs.braintrust.dev/integrations/sdk-integrations/cloudwego-eino.md), [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md), and [Firebase Genkit](https://docs.braintrust.dev/integrations/sdk-integrations/firebase-genkit.md) instrumentation with the Braintrust spec. Firebase Genkit gains `WithProvider` and `WithModel` options on `NewMiddleware` for explicit model attribution, plus traced tool wrappers `DefineTool`, `DefineToolWithInputSchema`, and `DefineMultipartTool`. Auto-instrumentation now replaces `genkit.DefineTool`, `genkit.DefineToolWithInputSchema`, and `genkit.DefineMultipartTool` calls with their traced equivalents. Updates Orchestrion to v1.12.1. **Breaking change (Google GenAI)**: Provider metadata changed from `"gemini"` to `"google"`. Update trace queries that filter on the previous provider value. **Breaking change (Eino)**: ChatModel span output is now an OpenAI-compatible choices array (`[{"index": 0, "finish_reason": "...", "message": {...}}]`) instead of a flat message map. Embedding input is now `{"inputs": [{"content": "..."}]}` and output is `{"count": N}`, removing `embedding_length` and renaming `embeddings_count`. Provider metadata is now lowercase (for example, `"openai"` instead of `"OpenAI"`). Update trace queries that rely on the previous formats. * [v0.11.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.11.0) - Aligned [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md), [Bedrock Runtime](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md), and [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) instrumentation with the Braintrust spec. Anthropic spans now capture `prompt_cache_creation_5m_tokens` and `prompt_cache_creation_1h_tokens` for TTL-specific prompt caching. Bedrock Runtime spans now capture audio and video content blocks, and `InvokeModelWithResponseStream` is now fully instrumented for Anthropic Claude models. Google ADK spans now include reasoning and cached token metrics. **Breaking change (Anthropic)**: Span metadata no longer includes `endpoint`. The output field is now a single message object instead of an array. Non-streaming spans no longer emit `time_to_first_token`. **Breaking change (Bedrock)**: Span metadata renames `stop_sequences` to `stop` and removes `additional_model_request_fields`. Image, document, and tool block shapes align with Bedrock's native wire format. ### Java SDK releases * [v0.3.22](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.22) - [LangChain4j](https://docs.braintrust.dev/integrations/sdk-integrations/langchain4j.md) instrumentation now covers the OpenAI Responses API (`OpenAiResponsesChatModel`, `OpenAiResponsesStreamingChatModel`) for LangChain4j 1.14.0 or later. OpenAI and Anthropic LLM spans now capture vendor request IDs (`x-request-id` and `request-id` respectively) as span attributes. Fixes OpenAI audio transcription output capture, missing OpenAI and Anthropic generation parameters in span metadata, and Anthropic system prompts dropped from span input when using the prompt-caching array format. AWS Bedrock Converse spans now capture streaming tool-use and non-text content blocks, normalized reasoning (extended thinking) blocks, prompt caching metrics, and tool definitions. * [v0.3.21](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.21) - Web search, file search, code interpreter, and MCP calls in the OpenAI Responses API, and `server_tool_use` blocks in the Anthropic Messages API now appear as child `tool` spans nested under the LLM span. Anthropic spans also capture `server_tool_use_*` metrics (for example, `server_tool_use_web_search_requests`). **Breaking change**: The Google GenAI provider metadata field changed from `"gemini"` to `"google"`. Update trace queries that filter on the previous value. ### Ruby SDK releases * [v0.4.1](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.4.1) - All spans now carry [`braintrust.context_json`](https://docs.braintrust.dev/integrations/sdk-integrations/opentelemetry/attributes#braintrust-attributes) with SDK name, version, instrumentation scope, and detected runtime environment (set `BRAINTRUST_ENVIRONMENT_NAME` and `BRAINTRUST_ENVIRONMENT_TYPE` to override). **Breaking change**: Eval span attribute names and formats now conform to the Braintrust spec used by other SDKs: `braintrust.expected` is renamed to `braintrust.expected_json`, `braintrust.input_json` and `braintrust.output_json` now store the raw value rather than a wrapper object, and on task failure `braintrust.output_json` is omitted instead of set to `null`. `braintrust.expected_json` is now also written to task spans in addition to eval spans. Update any SQL queries or span-attribute handling that relied on the old names or format. ### C# SDK releases * [v0.2.12](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.12) - Added remote dataset support: call `FetchDatasetAsync("my-dataset")` on the `Braintrust` instance to pull eval cases directly from a Braintrust dataset, fetched a page at a time and linked back to their source records. `BraintrustOpenApiClient` is now the primary API client, replacing the deprecated `DefaultBraintrustApiClient`, `BraintrustApiClient`, and `IBraintrustApiClient`. Access it via `braintrust.OpenApiClient`. `Eval` now implements `IDisposable`, so use `using var eval = ...` to release its internal HTTP client. See [API reference](https://docs.braintrust.dev/sdks/csharp/api-reference.md) for details. * [v0.2.11](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.11) - Added `DefaultBraintrustApiClient.Api`, a generated client for the full Braintrust REST API. The client covers every REST endpoint and ships inside the existing `Braintrust.Sdk` package with no separate install. See [API client](https://docs.braintrust.dev/sdks/csharp/api-reference.md#api-client) for details. ### @braintrust/deepseek-harness releases * [`@braintrust/deepseek-harness` v1.0.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/%40braintrust%2Fdeepseek-harness%401.0.0) - Added [DeepSeek Harness](https://docs.braintrust.dev/integrations/agent-frameworks/deepseek-harness.md) tracing plugin. Install with `dsh plugin --profile add @braintrust/deepseek-harness` to trace user turns, LLM steps, tool calls, and nested child sessions in Braintrust. ### @braintrust/openai-agents releases * [`@braintrust/openai-agents` v0.1.6](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/%40braintrust%2Fopenai-agents%400.1.6) - `@braintrust/openai-agents` is deprecated and this is its final release. Use the `braintrust` package instead: prefer auto-instrumentation with Braintrust's import hook (`node --import braintrust/hook.mjs`), or import `OpenAIAgentsTraceProcessor` from `braintrust/instrumentation` for manual setup. See the [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) integration guide for setup examples. ### trace-claude-code releases * [v2.0.1](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v2.0.1) - `trace-claude-code` now sends Claude Code lifecycle events to the [`bt` CLI](https://docs.braintrust.dev/reference/cli/trace), which handles authentication, trace construction, and delivery. Authenticate with `bt login`, then configure tracing with [`bt trace setup claude`](https://docs.braintrust.dev/integrations/developer-tools/claude-code#trace-claude-code-sessions). **Breaking change**: this version uses `bt trace setup claude` for tracing configuration and breaks previous plugin-specific environment-variable setup. See [Claude Code migration instructions](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md#migrate-from-an-older-plugin-version) for details and the [CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate.md#v0-16-0) for related `bt` changes. * [v2.0.0](https://github.com/braintrustdata/braintrust-claude-plugin/releases/tag/v2.0.0) - Added cache token metrics, Git repository metadata, skill invocation tracking, and experiment attachment support to `trace-claude-code` traces. ### trace-codex releases * [v1.0.1](https://github.com/braintrustdata/braintrust-codex-plugin/releases/tag/v1.0.1) - `trace-codex` now sends Codex lifecycle events to the [`bt` CLI](https://docs.braintrust.dev/reference/cli/trace), which handles authentication, trace construction, and delivery. Authenticate with `bt login`, then configure tracing with [`bt trace setup codex`](https://docs.braintrust.dev/integrations/developer-tools/codex#trace-codex-sessions). **Breaking change**: this version uses `bt trace setup codex` for tracing configuration and breaks previous plugin-specific environment-variable and `config.json` setup. See [Codex upgrade notes](https://docs.braintrust.dev/integrations/developer-tools/codex.md#upgrade) for details and the [CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate.md#v0-16-0) for related `bt` changes. * [v1.0.0](https://github.com/braintrustdata/braintrust-codex-plugin/releases/tag/v1.0.0) - Tool spans now record `tool_approval: "approved"` metadata and surface tool output failures in the span `error` field. **Breaking change**: in this version, `config.json` takes precedence over environment variables. ### Google Antigravity tracing releases * [v0.1.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v0.1.0-antigravity) - Adds tracing for [Antigravity CLI sessions](https://docs.braintrust.dev/integrations/developer-tools/antigravity.md), including model calls and tool activity. ### @braintrust/trace-opencode releases * [v1.0.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v1.0.0-opencode) - `@braintrust/trace-opencode` now sends OpenCode lifecycle events to the [`bt` CLI](https://docs.braintrust.dev/reference/cli/trace), which handles authentication, trace construction, and delivery. Authenticate with `bt login`, then configure tracing with [`bt trace setup opencode`](https://docs.braintrust.dev/integrations/developer-tools/opencode#trace-opencode-sessions). **Breaking change**: this version moves authentication and trace delivery to `bt`, replacing plugin-owned credentials and endpoint configuration. See [OpenCode upgrade notes](https://docs.braintrust.dev/integrations/developer-tools/opencode.md#upgrade) for details and the [CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate.md#v0-16-0) for related `bt` changes. ### @braintrust/pi-extension releases * [v1.0.0](https://github.com/braintrustdata/braintrust-coding-agent-plugins/releases/tag/v1.0.0-pi) - `@braintrust/pi-extension` now sends pi lifecycle events to the [`bt` CLI](https://docs.braintrust.dev/reference/cli/trace), which handles authentication, trace construction, and delivery. Authenticate with `bt login`, then configure tracing with [`bt trace setup pi`](https://docs.braintrust.dev/integrations/developer-tools/pi#trace-pi-sessions). **Breaking change**: this version uses `bt trace setup pi` for tracing configuration and breaks previous extension-specific auth and settings. See [pi upgrade notes](https://docs.braintrust.dev/integrations/developer-tools/pi.md#upgrade) for details and the [CLI migration guide](https://docs.braintrust.dev/reference/cli/migrate.md#v0-16-0) for related `bt` changes. ### Improvements * The **Edit billing information** dialog now includes a **Billing address** section, so you can update your billing address without replacing the card on file. Click **Edit address** under **Billing address**. See [Update your billing address](https://docs.braintrust.dev/admin/billing/faq.md#how-do-i-update-my-billing-address) for details. * Custom views now support writing scores via `trace.update`, so you can capture numeric feedback without leaving the view. See [Add interactive controls](https://docs.braintrust.dev/annotate/custom-views.md#add-interactive-controls) for details. * The **Edit billing information** dialog now includes a **Purchase order** field (added to the memo on future invoices) and **Tax information** fields (country and tax ID type shown on invoices). See [Enable on-demand usage](https://docs.braintrust.dev/admin/billing/change-plan.md#enable-on-demand-usage) for details. * Starter and Pro plan subscribers can now redeem coupon codes directly from the billing page. Click **Redeem coupon** under **Current plan** on \*\* Settings\*\* > [\*\* Billing\*\*](https://www.braintrust.dev/app/~/configuration/org/billing). See [Redeem a coupon](https://docs.braintrust.dev/admin/billing/change-plan.md#redeem-a-coupon) for details. * Starter plan users can now disable on-demand billing immediately from the **Edit billing information** dialog. The card stays on file and plan limits apply from that point forward. See [Disable on-demand usage](https://docs.braintrust.dev/admin/billing/change-plan.md#disable-on-demand-usage) for details. * Images that point to your local or private network, or that use an unsafe URL scheme, now require approval before loading, even in **Auto-load images** mode. See [Image rendering controls](https://docs.braintrust.dev/admin/organizations.md#control-image-rendering) for details. * The Monitor chart editor now warns you when a measure, filter, or group-by is likely to make a chart slow, and asks you to confirm before saving a chart that still has warnings. See [Create custom charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md) for details. * The **Assigned to** filter in the **Basic** tab now lets you select permission groups in addition to individual members. See [Apply a filter](https://docs.braintrust.dev/observe/filter.md#apply-a-filter) for details. * The SQL sandbox now shows a performance warning on `LIKE` and `ILIKE` operators and offers a **Use MATCH** quick fix to convert them to the indexed `MATCH` operator. See [Pattern matching](https://docs.braintrust.dev/reference/sql/query-structure.md#pattern-matching) for details. * On the Starter plan, you can now use GLM-5.2 with your monthly [model credits](https://docs.braintrust.dev/plans-and-limits.md#model-credits) without first enabling on-demand usage. GLM-5.2 becomes unavailable once your credits are exhausted, until you enable on-demand usage or upgrade to Pro. See [AI providers](https://docs.braintrust.dev/admin/ai-providers.md#available-models) for details. * You can now add rows to a dataset in bulk from [\*\* Review\*\*](https://www.braintrust.dev/app/~/review). Select the rows, then click **Add to** and choose a dataset. See [Promote traces from logs](https://docs.braintrust.dev/annotate/datasets/create.md#promote-traces-from-logs) for details. * Online scoring automations now support bulk rewind, so you can re-process several rules from a single start time instead of rewinding each one separately. See [Rewind an automation](https://docs.braintrust.dev/evaluate/score-online.md#rewind-an-automation) for details. * The view API now accepts an optional `description` and returns an `updated_at` timestamp, set whenever a view is created or modified. Views not modified since the field was added report `updated_at` as `null`. See [Create view](https://docs.braintrust.dev/api-reference/views/create-view.md) for details. * **Group by** and **Cluster by** now live in the row type selector in the toolbar, and traces are not grouped by default, so you opt into grouping when you want to review multi-turn conversations end to end. See [Group related traces](https://docs.braintrust.dev/observe/view-logs.md#group-related-traces) for details. * You can now group related traces on the [\*\* Review\*\*](https://www.braintrust.dev/app/~/review) page when reviewing **Log spans**, so a conversation's traces stay together as you work through a review queue. See [Group related log traces](https://docs.braintrust.dev/annotate/human-review/manage-review-work.md#group-related-log-traces) for details. * Combining a single `ANY_SPAN()` with bare filter conditions now applies every condition to the same span, which lets the query planner evaluate these queries more efficiently. A query that previously matched its conditions against different spans returns fewer rows until you wrap each condition in its own `ANY_SPAN()` call. See [Single span filters](https://docs.braintrust.dev/reference/sql/query-structure.md#single-span-filters) for details. * You can now filter the logs list to a specific span name by clicking that span's name in the trace panel header, instead of building the filter by hand. See [Apply a filter](https://docs.braintrust.dev/observe/filter.md#apply-a-filter) for details. * If you self-host Braintrust on AWS, you can now run [Loop](https://docs.braintrust.dev/loop.md) in your own data plane by enabling the Loop runtime, an opt-in ECS service that executes each Loop session in an isolated MicroVM sandbox. It requires AWS Terraform module v6.5.2 or later and a [supported AWS region](https://docs.braintrust.dev/admin/self-hosting/configure/loop-runtime.md#supported-aws-regions). See [Loop runtime](https://docs.braintrust.dev/admin/self-hosting/configure/loop-runtime.md) for details. * The **Basic** filter tab's value picker is now searchable, and you can apply a value that isn't in the list by typing it directly. See [Apply a filter](https://docs.braintrust.dev/observe/filter.md#apply-a-filter) for details. * Hovering a row in the **List** view of a Topics facet card now reveals a **View logs** link that opens [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) filtered to that topic and every topic nested under it. See [Review insights](https://docs.braintrust.dev/observe/topics/review-insights.md#snapshot-view) for details. * Custom preprocessors whose output isn't a recognized message array now render in the thread layout as text labeled with the preprocessor's name, instead of failing. See [Write a custom preprocessor](https://docs.braintrust.dev/observe/topics/enable.md#write-a-custom-preprocessor) for details. * Service accounts now show the project permissions they inherit from their permission groups, and you can now manage a service account's group membership from the Service tokens page. See [Use service accounts](https://docs.braintrust.dev/admin/access-control/manage-permissions.md#use-service-accounts) for details. * Slack link previews now distinguish trace details by your Braintrust identity: if you open a shared trace's details without linking your account, you are prompted to connect it, and if you are linked but lack access to the trace's project, you see a restricted-access notice instead of a generic missing-trace error. See [Braintrust link previews in Slack](https://docs.braintrust.dev/admin/organizations.md#braintrust-link-previews-in-slack). **July 2026** ### Pairwise scoring When you compare two experiments in diff mode, you can now record which one produced the better output for each row. Braintrust aggregates these head-to-head judgments into a win rate, so you can run A-vs-B human evaluations without configuring a separate scoring function. See [Pairwise scoring](https://docs.braintrust.dev/evaluate/compare-experiments.md#pairwise-scoring) for details. ### Trace references in datasets When you add a trace to a dataset from [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs), you can now choose **Add full trace to dataset** to store a reference to the trace in the row's `input` instead of copying a single span. Braintrust renders these rows with a read-only **Trace** viewer that shows the referenced trace inline. Because the row stores a reference rather than a copy, it depends on the trace remaining available under your retention window. See [Promote traces from logs](https://docs.braintrust.dev/annotate/datasets/create.md#promote-traces-from-logs) for details. ### Interactive controls in custom views Custom views can now write back to your data instead of only displaying it. Add buttons, inputs, and other controls that update span metadata and tags in trace views, or metadata, input, expected output, and tags in dataset views. This turns a custom view into a purpose-built annotation surface, so reviewers can capture positive or negative feedback, correct dataset inputs and expected outputs, and apply tags without leaving the view. See [Add interactive controls](https://docs.braintrust.dev/annotate/custom-views.md#add-interactive-controls) for details. ### Slack notifications for review assignments When the Slack integration is enabled, assignees who have linked their Slack account now receive a Slack direct message when spans or dataset rows are assigned to them for review, in addition to the existing email notification. The message previews the assigned items and links directly to them. See [Assign rows for review](https://docs.braintrust.dev/annotate/human-review/manage-review-work.md#assign-rows-for-review) for details. ### API key controls Organizations have finer control over credential lifetime and creation. API keys and service tokens take an expiration date at creation, and organization owners can prevent members from creating new API keys without affecting keys that already exist. See [Manage API keys](https://docs.braintrust.dev/admin/organizations.md#manage-api-keys), [Disable API key creation](https://docs.braintrust.dev/admin/organizations.md#disable-api-key-creation), and [Create service tokens](https://docs.braintrust.dev/admin/organizations.md#create-service-tokens) for details. ### Trace search improvements Searching within a trace is faster and easier to navigate on large traces. **Full trace** searches default to whole-word matching, which is significantly faster than substring matching, and you can switch back with **Match whole words**. Navigation steps through results one occurrence at a time across span and field boundaries, with a per-field match count shown in the results panel. Clicking a result switches the span detail panel to the tab containing the matched field. See [Search within a trace](https://docs.braintrust.dev/observe/examine-traces.md#search-within-a-trace) for details. ### Search bar autocomplete The search bar suggests completions as you type. **Suggestions** offers field name completions, and **Recent** lists your most recent text searches and SQL filters, so you can re-run a previous query without retyping it. When Loop is enabled, select **Generate SQL with Loop** to open the SQL filter editor with your typed text as a prompt. Text search chips in the filter bar are editable, so you can revise a search term inline instead of clearing the filter and starting over. See [Filter and search logs](https://docs.braintrust.dev/observe/filter.md) for details. ### Extended data retention On the Pro plan, you can now keep logs and playground data, including each row's activity history, queryable for longer, with up to 180-day retention, for an additional storage charge. See [Configure data retention](https://docs.braintrust.dev/admin/data-management/retention.md#extend-the-retention-window) and [Plans and limits](https://docs.braintrust.dev/plans-and-limits.md#usage-limits) for details. ### GLM-5.2 Braintrust now offers [GLM-5.2](https://docs.braintrust.dev/admin/ai-providers.md#available-models) as a built-in model — no need to configure your own AI provider. In playgrounds, prompts, and scorers, it is available under the **Braintrust** model provider, and you can also call it through the Braintrust gateway. Usage draws from your monthly [credit](https://docs.braintrust.dev/plans-and-limits.md#model-credits), shared with Topics. For details, see [AI providers](https://docs.braintrust.dev/admin/ai-providers.md#available-models). ### Skip non-applicable cases in LLM-as-a-judge scorers LLM-as-a-judge scorers now support an **Allow skip** toggle. When enabled, the judge can return an implicit **Skip** choice that scores as `null`, so non-applicable test cases are excluded from score aggregates instead of lowering them. See [LLM-as-a-judge](https://docs.braintrust.dev/evaluate/llm-as-a-judge.md#skip-non-applicable-cases) for details. Requires data plane v2.7.0+ (Self-hosted customers must upgrade to use this feature.) ### Slack link previews When the Slack integration is enabled, Braintrust expands links to logs, traces, and experiments that you share in Slack, so your team gets context on a resource without opening it. Private resource details are only shown to Slack users who have linked their Slack account to a Braintrust account with access to the resource. See [Braintrust link previews in Slack](https://docs.braintrust.dev/admin/organizations.md#braintrust-link-previews-in-slack) for details. ### Query SQL data sources by name Most SQL table functions now accept object names in addition to IDs, so you can write queries like `experiment('my-experiment')` or `project_logs('my-project')` without first looking up a UUID. When the same name exists in multiple projects, disambiguate with a two-element array (`experiment(['my-project', 'my-experiment'])`). See [Data sources](https://docs.braintrust.dev/reference/sql/query-structure.md#data-sources) for the supported functions. Requires data plane v2.7.0+ (Self-hosted customers must upgrade to use this feature.) ### Python SDK releases * [v0.31.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.31.0) - The dev server now forwards dataset version and environment selection to remote evals. Fixed remote eval parameters without a `description` serializing `"description": null`, which failed playground validation and hid every eval on the endpoint. Fixed [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) stream cleanup being logged as errors during subagent delegation. Fixed [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) async runs inheriting ended spans. Fixed span metadata handling in [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) and [Strands](https://docs.braintrust.dev/integrations/agent-frameworks/strands-agent.md) integrations. Fixed error logging in the [Temporal](https://docs.braintrust.dev/integrations/sdk-integrations/temporal.md) integration. * [v0.30.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.30.1) - Fixed async streaming for the [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) integration: on async clients using `with_streaming_response`, `parse()` now returns an awaitable as expected. **Breaking change**: The OpenAI and [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md) integrations now allowlist span metadata, so internal request fields that were previously captured are no longer recorded. For example, OpenAI no longer records `extra_headers`. Update any queries, saved views, or automations that read these fields. * [v0.30.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.30.0) - [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) now resolves the actual underlying provider name (e.g., `"openai"`, `"anthropic"`) in span metadata instead of always reporting `"litellm"`. [LlamaIndex](https://docs.braintrust.dev/integrations/sdk-integrations/llamaindex.md) LLM calls are now typed as task spans instead of LLM spans, avoiding duplicate LLM spans when a provider integration is also active. [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md), [Instructor](https://docs.braintrust.dev/integrations/sdk-integrations/instructor.md), [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md), and LlamaIndex integrations now include `model` and `provider` fields in span metadata. Remote eval scorers that return multiple scores are now recorded correctly instead of erroring. **Breaking change**: Several integrations changed what appears in span metadata. Tool definitions are now recorded in `metadata.tools` for Google GenAI and LangChain spans (previously in `input` for Google GenAI, and under the span's invocation parameters in `metadata` for LangChain). The LangChain, [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md), and Google GenAI integrations now allowlist span metadata, so internal fields that were previously captured are no longer recorded; for example, LiveKit Agents no longer includes the full conversation context (`chat_ctx`). Update any queries, saved views, or automations that read these fields from their old locations. * [v0.29.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.29.0) - Spans now carry a per-integration source identifier in `context.span_origin.instrumentation.name` (e.g., `openai-auto`, `anthropic-auto`), so you can filter and query traces by the SDK integration that created each span. Improved span data quality for [CrewAI](https://docs.braintrust.dev/integrations/agent-frameworks/crew-ai.md) (provider metadata, tools routed to metadata), [DSPy](https://docs.braintrust.dev/integrations/sdk-integrations/dspy.md) (aligned span shape, removed nested LLM spans), [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md) (tools routed to metadata, media attachments materialized, exceptions propagated), [Bedrock Runtime](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md) (metrics, tools, and attachments), [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) (provider metadata, hook field allowlist), [ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) (provider, tools, errors in LLM span metadata), and [AgentScope](https://docs.braintrust.dev/integrations/agent-frameworks/agentscope.md). **Breaking change**: For CrewAI and Agno spans, tool definitions are now recorded in `metadata.tools` instead of `input`. Update any queries or saved views that read tool schemas from `input`. * [v0.28.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.28.0) - Added span origin provenance. Spans now automatically record the execution environment (CI provider, cloud platform, or custom `PYTHON_ENV`), overridable via `BRAINTRUST_ENVIRONMENT_TYPE` and `BRAINTRUST_ENVIRONMENT_NAME`. Fixed [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) span metadata to use an allowlist, preventing internal LiteLLM fields from leaking into spans. Fixed custom `PYTHON_ENV` values being dropped from span origin. * [v0.27.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.27.0) - Added tracing for anthropic `messages.parse` structured output calls, capturing `parsed_output` alongside standard message fields. Fixed function invocation errors on EU and self-hosted deployments. ### TypeScript SDK releases * [v3.25.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.25.0) - Expanded [Pi Coding Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md) support to v0.81.x. Added [`@openrouter/sdk`](https://docs.braintrust.dev/integrations/ai-providers/openrouter) v1 compatibility. Fixed eval tasks hanging until timeout when tasks complete early. Fixed [AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) `HarnessAgent` trace context propagation across suspended turns. * [v3.24.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.24.0) - Added tracing for the [Vercel AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) `HarnessAgent` class (`@ai-sdk/harness` v1.x), including `generate`, `stream`, `continueGenerate`, and `continueStream` turns. Fixed [Eve](https://docs.braintrust.dev/integrations/agent-frameworks/eve.md) LLM reasoning capture across workflow resumes and [LangChain JS](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) total token metrics. * [v3.23.1](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.23.1) - Cleaned up Claude Agent SDK span metadata to reduce trace noise. * [v3.23.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.23.0) - Added [LangSmith](https://docs.braintrust.dev/integrations/sdk-integrations/langsmith.md#tracing-typescript) auto-instrumentation and wrapper functions (`wrapLangSmithTraceable`, `wrapLangSmithRunTrees`, `wrapLangSmithClient`) for tracing LangSmith runs in Braintrust. Eve turns are now tracked as individual traces, and the Eve session ID is recorded on all spans. Strands agents now log inline document, image, and video content as Braintrust attachments rather than embedded bytes. Fixed Mastra `time_to_first_token` capture for streaming responses. * [v3.22.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.22.0) - Added `braintrustEveHook` and `braintrustEveInstrumentation` for tracing [Eve](https://docs.braintrust.dev/integrations/agent-frameworks/eve.md) agent sessions, turns, LLM steps, tool calls, and subagent interactions. Fixed Mastra `logFeedback` routing: the Braintrust row ID now matches the Mastra span ID so feedback attaches to the correct span instead of landing as a stray row. * [v3.21.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust%403.21.0) - Added `injectTraceContext` and `extractTraceContextFromHeaders` for [W3C trace-context propagation](https://docs.braintrust.dev/sdks/typescript/api-reference.md#injecttracecontext) across HTTP service boundaries, compatible with OpenTelemetry and other W3C-compliant systems. Added [`agentAssertionScorer`](https://docs.braintrust.dev/sdks/typescript/api-reference#agentassertionscorer), a trace scorer that evaluates agents through declarative assertions on tool calls and outputs. Added tracing for the Vercel AI SDK [`WorkflowAgent`](https://docs.braintrust.dev/integrations/sdk-integrations/vercel#trace-agents-typescript) class. The SDK now forwards the [`bt eval --sample N`](https://docs.braintrust.dev/reference/cli/eval) flag to the dataset loader, fetching only the sampled rows. Fixed AI SDK total token metrics synthesis, concurrent model-instance span parenting, AI SDK v7 parent span lifecycle, and AI SDK tool spans capturing the full message history in their input. ### Go SDK releases * [v0.10.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.10.0) - Added span origin provenance. Spans now automatically record the execution environment (CI provider or cloud platform), overridable via `WithEnvironment`, `BRAINTRUST_ENVIRONMENT_TYPE`, or `BRAINTRUST_ENVIRONMENT_NAME`. ### Java SDK releases * [v0.3.20](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.20) - Fixes streaming span output across OpenAI, LangChain4j, and Spring AI (OpenAI and Anthropic backends): tool-call argument fragments are now merged into the reconstructed span output instead of being dropped. Streaming Anthropic responses in Spring AI now also preserve `tool_use` blocks and `thinking` blocks. OpenAI Chat Completions spans now capture `completion_reasoning_tokens` from `completion_tokens_details` for reasoning models such as o1 and o3. * [v0.3.19](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.19) - Adds [Spring AI](https://docs.braintrust.dev/integrations/sdk-integrations/spring-ai.md) 2.x instrumentation for `OpenAiChatModel` and `AnthropicChatModel`, available through manual wrapping with `BraintrustSpringAI.wrap()` or auto-instrumentation. Async OpenAI and Anthropic clients are now supported, both when wrapped manually and auto-instrumented. Async and streaming calls now nest correctly under the span that started them. * [v0.3.18](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.18) - Adds span origin provenance. Spans now automatically record the execution environment (CI provider or cloud platform), overridable via `BraintrustConfig.Builder.environment(...)`, `BRAINTRUST_ENVIRONMENT_TYPE`, or `BRAINTRUST_ENVIRONMENT_NAME`. * [v0.3.17](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.17) - Eval spans record `input` and `output` as raw values (e.g., `"hello"` rather than `{"input": "hello"}`), and the task span carries its own `input`, `output`, and `expected`. **Breaking change**: if you read these span attributes directly, `braintrust.input_json` and `braintrust.output_json` now contain the raw value, `braintrust.expected` is renamed to `braintrust.expected_json`, and `braintrust.output_json` is absent when a task throws. * [v0.3.16](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.16) - Adds typed deserialization for datasets, remote evals, and custom scorers. `braintrust.fetchDataset(...)` and `RemoteEval.builder(...)` now take input and output `Class` arguments (or converter functions) so fetched cases deserialize into your own types instead of raw maps, and `braintrust.fetchScorer(...)` gains matching converter overloads. Custom scorer arguments now serialize with `BraintrustJsonMapper` (snake\_case field names, honoring `@JsonIgnore`) by default. OpenAI spans now capture `prompt_cached_tokens` from both the Chat Completions and Responses APIs. The untyped `fetchDataset` and no-arg `RemoteEval.builder()` overloads are deprecated. * [v0.3.15](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.15) - The dev server now supports [remote eval](https://docs.braintrust.dev/evaluate/remote-evals.md) experiment snapshot runs triggered from the Braintrust UI, creating a distinct experiment for each run. To support this, added the `Eval.Builder.ensureNew(boolean)` option, which forces `Eval.run()` to always create a new experiment even when one with the same name already exists (the backend deduplicates the name). `EvalResult` now also exposes `getExperimentId()` and `getExperimentName()` alongside `getExperimentUrl()`, with the URL now reflecting the backend-resolved experiment name. ### C# SDK releases * [v0.2.10](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.10) - Adds span origin provenance. Spans now automatically record the execution environment (CI provider, cloud platform, or deployment mode), overridable via `BRAINTRUST_ENVIRONMENT_TYPE` and `BRAINTRUST_ENVIRONMENT_NAME`. * [v0.2.9](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.9) - Eval spans record `input` and `output` as raw values (e.g., `"hello"` rather than `{"input": "hello"}`), and the task span carries its own `input`, `output`, and `expected`. **Breaking change**: if you read these span attributes directly, `braintrust.input_json` and `braintrust.output_json` now contain the raw value, and `braintrust.expected` is renamed to `braintrust.expected_json`. ### @braintrust/trace-opencode releases * [v0.1.0](https://github.com/braintrustdata/braintrust-opencode-plugin/releases/tag/v0.1.0) - Adds an `enable_tools` option to disable Braintrust-branded data access tools while keeping tracing active, captures Git repository metadata (`git_branch`, `git_commit_sha`, `git_origin_url`) in session spans, and traces skill loads as `skill: ` spans, adding skill metadata (`skill_name`, `tool_kind`) to tool spans and skill-load metadata (`loaded_skill_names`, `loaded_skills`) to turn spans. See the [OpenCode integration guide](https://docs.braintrust.dev/integrations/developer-tools/opencode.md) for details. ### trace-codex releases * [v0.1.0](https://github.com/braintrustdata/braintrust-codex-plugin/releases/tag/trace-codex-v0.1.0) - Session spans now capture Git attribution metadata (`git_origin_url` with credentials redacted, `git_branch`, and `git_commit_sha`). When Codex reads a `SKILL.md` file, the tool span is renamed to `skill: ` and carries `skill_name` and `skill_path` metadata. Prompts with explicit skill requests also record `loaded_skill_names` and `loaded_skills` on the turn span, and the skill span gains `skill_load_trigger: "explicit"`. Added `parentSpanId` and `rootSpanId` configuration to attach a Codex session under an existing Braintrust span. All spans now carry span origin provenance identifying them as `braintrust.plugin.codex`, with the execution environment auto-detected from standard CI and platform environment variables and overridable via `BRAINTRUST_ENVIRONMENT_TYPE` and `BRAINTRUST_ENVIRONMENT_NAME`. See the [Codex integration guide](https://docs.braintrust.dev/integrations/developer-tools/codex.md) for details. ### @braintrust/pi-extension releases * [v0.10.0](https://github.com/braintrustdata/braintrust-pi-extension/releases/tag/pi-extension-v0.10.0) - LLM spans now record model capability metadata (`model_context_window`, `model_max_tokens`, `model_supports_reasoning`, `supported_thinking_levels`, `deferred_tools_mode`) and provider request tool counts (`provider_request_tool_count`, `provider_request_deferred_tool_count`). All spans now carry span origin provenance identifying them as `braintrust.plugin.pi`, with the execution environment auto-detected from standard CI and platform environment variables and overridable via `BRAINTRUST_ENVIRONMENT_TYPE` and `BRAINTRUST_ENVIRONMENT_NAME`. See the [pi integration guide](https://docs.braintrust.dev/integrations/developer-tools/pi.md) for details. * [v0.9.0](https://github.com/braintrustdata/braintrust-pi-extension/releases/tag/pi-extension-v0.9.0) - Session spans now capture Git attribution metadata (`git_origin_url` with credentials redacted, `git_branch`, and `git_commit_sha`). When pi reads a `SKILL.md` file, the tool span is renamed to `skill: ` and carries `skill_name` and `skill_path` metadata. Prompts with explicit `/skill:` requests also record `loaded_skill_names` and `loaded_skills` on the turn span, and the skill span gains `skill_load_trigger: "explicit"`. See the [pi integration guide](https://docs.braintrust.dev/integrations/developer-tools/pi.md) for details. ### GitHub eval action releases * [v2.0.0](https://github.com/braintrustdata/eval-action/releases/tag/v2.0.0) - Added Go runtime support and upgraded the action runtime to Node 24. **Breaking change**: self-hosted runners must be updated to support Node 24. See [Run in CI/CD](https://docs.braintrust.dev/evaluate/run-in-ci.md) for details. ### Improvements * Spans ingested via OpenTelemetry now automatically include a `context.span_origin` field that records the span's provenance, including the SDK name, SDK version, and instrumentation scope. See [OpenTelemetry integration](https://docs.braintrust.dev/integrations/sdk-integrations/opentelemetry/attributes.md#braintrust-attributes) for details. * Loop now uses project-level AI provider secrets, so models configured as project-level providers are available in Loop sessions opened within that project. See [Configure AI providers](https://docs.braintrust.dev/admin/ai-providers.md#project-providers) for details. * You can now delete an AI provider directly while editing it. See [Delete a provider](https://docs.braintrust.dev/admin/ai-providers.md#delete-a-provider) for details. * GPT-5.6 Sol, Terra, and Luna are now available in [Loop](https://docs.braintrust.dev/loop.md), and GPT-5.6 Sol is the recommended default when OpenAI is configured as an organization AI provider. * You can now disable frontend Loop logging in \*\* Settings\*\* > [\*\* Loop\*\*](https://www.braintrust.dev/app/~/configuration/org/loop). See [Control Loop logging](https://docs.braintrust.dev/admin/organizations.md#control-loop-logging) for more info. * The tag filter dropdown on the experiments table now surfaces tags inferred from recent logged data alongside configured project tags, without requiring explicit registration. * Span errors now appear on both the **Messages** and **Details** tabs of the span detail panel, so errors are visible alongside the input and output that produced them. See [Inspect a trace](https://docs.braintrust.dev/observe/examine-traces.md#inspect-a-trace) for details. * Classifier results now appear as individual, filterable, and sortable columns in the experiments and logs tables and in playground results, with one column per classifier. In experiments and playgrounds, you can also group rows by a classifier's label. See [Classifiers](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) for details. * \*\* Settings\*\* > [\*\* Permission groups\*\*](https://www.braintrust.dev/app/~/configuration/org/groups) now paginates (50 groups per page) and includes a search box to filter by name or description. See [Manage permission groups](https://docs.braintrust.dev/admin/access-control/manage-permissions.md) for details. * The prompt editor now supports Anthropic cache control on individual messages and content blocks when using Anthropic or AWS Bedrock (Converse API) models, with TTL options of 5 minutes or 1 hour. See [Create prompts](https://docs.braintrust.dev/evaluate/prompts/create.md) for details. * The prompt editor now allows the `temperature` parameter for GPT-5.1 and later models when reasoning effort is set to `none`. Raising the reasoning effort removes the `temperature` setting, matching OpenAI's parameter support. See [Create prompts](https://docs.braintrust.dev/evaluate/prompts/create.md) for details. * You can link your Slack and Braintrust accounts on demand by running `/braintrust link` in any Slack channel, or from the prompt shown when you open the Slack App Home. If your Braintrust and Slack email addresses differ, the authorization page offers a **Switch account** option. See [Braintrust link previews in Slack](https://docs.braintrust.dev/admin/organizations.md#braintrust-link-previews-in-slack) for details. * You can now copy your organization ID or name directly from the organization switcher dropdown. See [Find your organization ID](https://docs.braintrust.dev/admin/organizations.md#find-your-organization-id) for details. * You can now delete a human review score directly from its edit panel, both in settings and while reviewing inline. See [Set up human review](https://docs.braintrust.dev/annotate/human-review.md#create-edit-and-delete-scores-inline) for details. * While reviewing experiment spans, you can now enable diff mode and select a comparison experiment to compare traces inline as you score. See [Score traces and datasets](https://docs.braintrust.dev/annotate/human-review/score-traces.md#change-the-trace-layout) for details. * The **Invoices** dropdown on the \*\* Settings\*\* > [\*\* Billing\*\*](https://www.braintrust.dev/app/~/configuration/org/billing) page now includes card validation fee receipts for pay-as-you-go users alongside regular usage invoices. See [Get an invoice or receipt](https://docs.braintrust.dev/admin/billing/faq.md#how-do-i-get-an-invoice-or-receipt) for details. * When **Reconcile with previous version** is enabled for a Topics facet, preserved topics now receive updated descriptions based on their current content rather than keeping the description from first creation. See [Tune clustering settings](https://docs.braintrust.dev/observe/topics/manage.md#tune-clustering-settings) for details. * Pausing and resuming Topics automations from the UI now requires organization owner permissions. See [Pause/resume automation](https://docs.braintrust.dev/observe/topics/manage.md#pause-resume-automation) for details. * The `sql_query` MCP tool now handles large results gracefully: results exceeding 1 MB are uploaded to object storage and returned as a signed `overflow_url` instead of inline data, preventing context overflow. Set `return_url: true` to request a URL even for results below the threshold. See [Braintrust MCP](https://docs.braintrust.dev/integrations/developer-tools/mcp.md#tools) for details. * Custom facet exclusion patterns and JSON schema `pattern`/`patternProperties` values are now validated for safety when you save them. Patterns that use lookahead, lookbehind, or backreferences are rejected with a clear error. See [Custom facets](https://docs.braintrust.dev/observe/topics/custom-facets.md#create-a-facet) and [Define schemas](https://docs.braintrust.dev/annotate/datasets/manage.md#define-schemas) for details. * The **Live tail** time range in [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) is now a distinct option separate from the **1 hour** fixed-window range. Live mode reliably delivers new rows, highlights newly arrived rows with a flash animation, and pauses polling when the browser tab is hidden. See [View your logs](https://docs.braintrust.dev/observe/view-logs.md#browse-logs) for details. * Experiment comparison diffs now let you choose the diff direction explicitly, so you can control which experiment is the baseline. Your last-used direction is saved for future comparisons. See [Use diff mode](https://docs.braintrust.dev/evaluate/compare-experiments.md#use-diff-mode) for details. * Custom columns are now available when reviewing **Log spans** on the [\*\* Review\*\*](https://www.braintrust.dev/app/~/review) page, so you can surface metadata, scores, or nested values alongside review work. See [Create custom columns](https://docs.braintrust.dev/annotate/human-review/manage-review-work.md#create-custom-columns) for details. * The **Assigned to** column is now a built-in column in the logs, experiment, and dataset tables, making reviewer assignments visible at a glance without a custom column workaround. See [Assign rows for review](https://docs.braintrust.dev/annotate/human-review/manage-review-work.md#assign-rows-for-review) for details. * Overriding code or graph (workflow) functions that have stored env secrets now requires `update` permission. Read-only callers that attempt to pass overrides to such functions receive an access denied error. See [Override function data per call](https://docs.braintrust.dev/deploy/functions.md#override-function-data-per-call) for details. * Monitor time series charts can now use `metrics.start` (event start time) instead of `created` (ingestion time) as the time dimension, which improves chart accuracy for projects that batch-ingest historical spans. See [Configure Monitor chart time dimension](https://docs.braintrust.dev/admin/projects.md#configure-monitor-chart-time-dimension) for details. * Custom views now support grouped sessions. When a view is open on a session grouped from multiple traces, the `trace` object's spans, field fetches, and updates operate across every trace in the session, up to a limit of 1,000 traces. See [Custom views](https://docs.braintrust.dev/annotate/custom-views.md) for details. * `openai.gpt-5.3` and later models on AWS Bedrock now route through Bedrock's Responses API instead of Chat Completions, so they work correctly through the Braintrust gateway. See [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md) for details. * A new [cookbook on supervising agent behavior with behavior specs](https://docs.braintrust.dev/cookbook/recipes/AgentBehavior.md) covers trajectory-level evals: defining expected agent behaviors in a spec and grading each one with an LLM judge. * The **Organization AI providers** and **Project AI providers** tables are now visible in read-only mode to members without permission to edit them, so they can see which providers are configured. Redacted key previews remain visible only to members who can edit each section. See [Configure AI providers](https://docs.braintrust.dev/admin/ai-providers.md#permissions-and-access) for details. * Project and organization automation configuration changes are now captured in the [audit log](https://docs.braintrust.dev/admin/audit-logs.md#events), with create, update, and delete events for each. Webhook action URLs are redacted. * Applied scorers and classifiers on the \*\* Evaluators\*\* tab now show their recorded rationale inline, so you can see why a score or label was assigned without opening the span that produced it. See [Test and apply evaluators](https://docs.braintrust.dev/observe/examine-traces.md#test-and-apply-evaluators) for details. **June 2026** ### Improved SDK documentation Braintrust's documentation now has a dedicated [SDKs](https://docs.braintrust.dev/sdks.md) tab, with sections for TypeScript, Python, Go, Java, Ruby, and C#. Each language has a quickstart, an install-and-instrument guide, and a curated API reference. See [SDKs](https://docs.braintrust.dev/sdks.md) to get started. ### Audit logging Braintrust now records administrative actions across your organization, such as creating projects, changing settings, granting permissions, managing members, and creating API keys. View recent activity in **Settings** > **Audit log** with time range, filter, and column controls, or query the full history with [SQL](https://docs.braintrust.dev/reference/sql.md) from the SQL sandbox, the [`bt sql`](https://docs.braintrust.dev/reference/cli/sql) CLI, or the API. Organizations with strict data access requirements can also enable auditing of data reads. See [Audit logging](https://docs.braintrust.dev/admin/audit-logs.md) for details. > [!NOTE] > > Audit logging is only available on the [Enterprise plan](https://docs.braintrust.dev/plans-and-limits.md#plans). Requires data plane v2.6.0+ (Self-hosted customers must upgrade to use this feature.) ### Gateway provider failover Gateway requests can now set `x-bt-fallback-providers` to retry the same request against fallback provider credentials when the primary provider returns a retryable provider error. Responses include headers showing which provider completed the request and, when failover succeeds, the original and final `model/provider` pairs. See [Enable provider failover](https://docs.braintrust.dev/deploy/gateway.md#enable-provider-failover) for details. ### Trace redesign Selecting a span now opens its detail panel to the **Messages** tab, focused on the highest-signal content first. When [topics](https://docs.braintrust.dev/observe/topics.md) are enabled, the matched topics and facet summaries appear at the top, followed by the span's input and output messages, tool calls, and annotations. Metadata, metrics, scores, and raw data remain available in the other tabs. See [Examine traces](https://docs.braintrust.dev/observe/examine-traces.md#inspect-a-trace) for details. ### Score visibility for human review Human review scores now support an optional visibility setting. Restrict a score to specific members or permission groups so only relevant reviewers see it during review, which keeps the review experience focused for large teams, or leave it unset to keep the score visible to everyone. See [Restrict score visibility](https://docs.braintrust.dev/annotate/human-review.md#restrict-score-visibility) for details. ### Conditional human review scores Human review scores now support filter conditions that control when a score appears in the review panel. Set a **Show when** condition on a score to surface it only when its filter expression evaluates to true for the span being reviewed — for example, show a detailed rubric only when a triage score is below a threshold, or a correction score only when the expected output matches a specific category. Conditions use SQL syntax and can reference other scores, expected values, and metadata. See [Show scores conditionally](https://docs.braintrust.dev/annotate/human-review.md#show-scores-conditionally) for details. ### Prompt and scorer versions in playground tasks When you add a saved prompt or scorer as a playground task, you can now build the task from a specific historical version instead of always getting the latest, making it easier to compare versions side by side and reproduce past results. See [Add tasks](https://docs.braintrust.dev/evaluate/playgrounds.md#add-tasks) for details. ### Dataset sampling in playgrounds Playground runs now support a sample rate control that evaluates a random subset of dataset rows, so you can iterate faster on large datasets before promoting a configuration to an experiment. See [Sample rate](https://docs.braintrust.dev/evaluate/playgrounds.md#sample-rate) for details. ### Rewind online scoring automations When you update a scorer or need to correct scoring results, you can now rewind an online scoring automation to re-process traces from a specific timestamp forward. This is useful for re-evaluating recent traces after scorer updates, fixing interrupted scoring runs, or addressing scoring errors. See [Rewind an automation](https://docs.braintrust.dev/evaluate/score-online.md#rewind-an-automation) for details. Requires data plane v2.3.0+ (Self-hosted customers must upgrade to use this feature.) ### Classifiers Classifiers are scorer-like evaluators that return a categorical label instead of a numeric score, so you can categorize by a dimension such as sentiment, issue type, or policy category. They run wherever scorers do, across eval cases in experiments and on production traces via online scoring. Classifications surface alongside scores and appear as columns you can sort and filter in experiment and trace tables. In logs, you can also promote a category's traces into a dataset, turning production patterns into evaluation test cases. See [Classifiers](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Multi-user human review Multiple reviewers can now score the same span independently without overwriting each other's work. Braintrust stores each reviewer's scores separately and automatically averages them on the parent span, with a breakdown of your own reviews, all reviews of the span, and all spans in the trace. For score types that can't be averaged, such as free-form text and categorical scores that write to expected output, you can choose which reviewer's value becomes the parent span's value. See [Add human feedback](https://docs.braintrust.dev/annotate/human-review.md) for details. ### Programmatic service token creation Organization owners can now create service tokens through the API. Call [`POST /v1/service_token`](https://docs.braintrust.dev/api-reference/servicetokens/create-service-token) and authenticate with a service token that has organization-owner permissions, or include `token_name` when creating a service account through [`PATCH /v1/organization/members`](https://docs.braintrust.dev/api-reference/organizations/modify-organization-membership). API key creation remains UI-only, and user API keys cannot be used to create service tokens. See [Create service tokens](https://docs.braintrust.dev/admin/organizations.md#create-service-tokens) for the supported flows. ### Run remote evals and sandboxes as experiments [Remote evals and sandboxes](https://docs.braintrust.dev/evaluate/remote-evals.md) can now run directly as experiments, not just from a playground. This lets you capture immutable, tracked results from custom agent and non-prompt code, comparable to any other experiment in your project, without going through the playground iteration loop first. ### Workload identity federation for Azure Connect Azure AI Foundry and Azure OpenAI to Braintrust using Microsoft Entra Workload Identity, which exchanges Braintrust-signed OIDC tokens for Microsoft Entra access tokens instead of storing long-lived Azure credentials in Braintrust. Available for Braintrust-hosted organizations through organization-level AI providers when the Braintrust Gateway is enabled. Project-level Azure providers continue to use API key or Entra API authentication. See [Azure AI Foundry](https://docs.braintrust.dev/integrations/ai-providers/azure.md#configure-the-integration) for setup instructions. ### Workload identity federation for OpenAI Connect OpenAI to Braintrust using workload identity federation, which exchanges Braintrust-signed JWTs for OpenAI access tokens instead of storing long-lived OpenAI API keys in Braintrust. Available for Braintrust-hosted organizations through organization-level AI providers when the Braintrust Gateway is enabled. Project-level OpenAI providers continue to use API key authentication. See [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md#add-openai-as-an-ai-provider) for setup instructions. ### Shingled search optimization Brainstore can now index multi-word shingles in bloom filters, so phrase and multi-word `search()` queries eliminate more log segments before scanning. Enable **Shingled search optimization** alongside log search optimization in project settings to speed up full-text search on high-volume logs. See [Speed up log filtering](https://docs.braintrust.dev/admin/projects.md#speed-up-log-filtering) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### Trace Codex CLI sessions The [trace-codex plugin](https://github.com/braintrustdata/braintrust-codex-plugin) (v0.0.2) traces [Codex](https://openai.com/codex) CLI sessions to Braintrust as session, turn, model call, and tool spans. Install the plugin, set `TRACE_TO_BRAINTRUST=true`, and Codex sessions appear in your Braintrust logs. See [Codex](https://docs.braintrust.dev/integrations/developer-tools/codex.md) for setup and CI instructions. ### Startup program Qualifying early-stage startups can apply for 6–12 months of free Pro plan access. To be eligible, you must be a new Braintrust customer at Series A or earlier and have raised at least \$100K in funding. Visit the [startups page](https://www.braintrust.dev/startups) to apply. See [Plans and limits](https://docs.braintrust.dev/plans-and-limits.md#plans) for plan details. ### bt CLI releases * [v0.15.0](https://github.com/braintrustdata/bt/releases/tag/v0.15.0) - Added [`bt experiments compare`](https://docs.braintrust.dev/reference/cli/experiments#bt-experiments-compare) to compare summary metrics for up to eight experiments side by side, by name or from a Braintrust comparison URL. Added [`bt view thread`](https://docs.braintrust.dev/reference/cli/view#bt-view-thread) to print a trace's collapsed LLM conversation transcript and [`bt view waterfall`](https://docs.braintrust.dev/reference/cli/view#bt-view-waterfall) (alias `timeline`) to render per-span timing, token, cost, and cache metrics. `bt eval --sample N` now propagates the sample rate into the SDK runtime, so evals that load a Braintrust dataset fetch only the sampled rows. Added `--web` to open datasets, prompts, projects, and experiments in the browser. * [v0.14.0](https://github.com/braintrustdata/bt/releases/tag/v0.14.0) - Fixed a bug where synchronous Python `Eval()` task functions ran sequentially even with `max_concurrency` set, because they were wrapped as async. Sync tasks now run in parallel across threads. Async tasks are unaffected. * [v0.13.0](https://github.com/braintrustdata/bt/releases/tag/v0.13.0) - Added [`bt functions view --version`](https://docs.braintrust.dev/reference/cli/functions#bt-functions-view) to inspect a specific historical version of a function. Improved the error message shown when a login session has expired, prompting you to re-authenticate with `bt auth login`. * [v0.12.0](https://github.com/braintrustdata/bt/releases/tag/v0.12.0) - Added [`bt datasets snapshots`](https://docs.braintrust.dev/reference/cli/datasets#bt-datasets-snapshots) to create, list, restore, and delete dataset snapshots from the CLI, with a preview of the rows affected before restoring. Added [`bt topics btmap`](https://docs.braintrust.dev/reference/cli/topics#bt-topics-btmap) to download the raw topic map artifact for a facet's topic map. Improved JSON parsing error messages to include more context about what failed and where. * [v0.11.1](https://github.com/braintrustdata/bt/releases/tag/v0.11.1) - Fixed the npm package build so the Windows binary is extracted to the correct path, completing npm distribution support across all platforms. * [v0.11.0](https://github.com/braintrustdata/bt/releases/tag/v0.11.0) - `bt` is now available through npm as an optional dependency of the [`braintrust` JavaScript SDK](https://docs.braintrust.dev/sdks/typescript/versions/latest) (v3.17.0 or later). After installing `braintrust`, run the CLI with `npx bt` or `pnpm exec bt`. See [Install with npm](https://docs.braintrust.dev/reference/cli/quickstart.md#install). * [v0.10.1](https://github.com/braintrustdata/bt/releases/tag/v0.10.1) - Fixed the global `--json` flag, which previously had no effect, to format command output as JSON. Fixed serialization of typed parameter schemas for remote evals. * [v0.10.0](https://github.com/braintrustdata/bt/releases/tag/v0.10.0) - Added [`bt datasets pipeline`](https://docs.braintrust.dev/annotate/datasets/pipelines) for transforming project logs into dataset rows, with one-shot (`run`) or staged (`pull`, `transform`, `push`) execution. Supports TypeScript and Python pipelines declared with `DatasetPipeline(...)`. `bt functions push` (Python) now discovers and pushes parameter definitions alongside functions and prompts. Windows binaries are now code-signed, removing SmartScreen "unknown publisher" warnings. Fixed `ERR_CLOSED_SERVER` errors when running evals via `vite-node`. ### Python SDK releases * [v0.26.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.26.0) - Added `load_prompt_async()` for non-blocking prompt loading. Added [pydantic-ai 2.0](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) compatibility. The SDK now honors the [`bt eval --sample N`](https://docs.braintrust.dev/reference/cli/eval) flag when loading a Braintrust dataset, fetching only the sampled rows instead of the full dataset. **Breaking change**: span and trace IDs now default to the OpenTelemetry-compatible format instead of UUIDs. If you store or parse Braintrust span or trace IDs outside the SDK and rely on the UUID format, set `BRAINTRUST_LEGACY_IDS=true` to keep it. Distributed tracing between Python services on v0.3.0 or later is unaffected, but if you use distributed tracing between Python and TypeScript services, either set `BRAINTRUST_LEGACY_IDS=true` on the Python service or enable compatibility mode on the TypeScript service to keep them interoperable. * [v0.25.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.25.0) - Added an [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md) integration that automatically traces `boto3` Bedrock Runtime `converse` and `invoke_model` calls, including their streaming variants, through `braintrust.auto_instrument()`. Added `project.classifiers.create()` for pushing [classifier](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) functions to Braintrust. The [AgentScope](https://docs.braintrust.dev/integrations/agent-frameworks/agentscope.md) integration now supports AgentScope 2.x, and the [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) integration captures inline moderation metadata. Fixed `base_experiment_id` propagation to experiment summaries, double-counting of [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) prompt cache tokens in [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) traces, and null handling in remote eval prompt parameter defaults. * [v0.24.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.24.0) - `invoke()` now accepts an `overrides` parameter that deep-merges configuration into the resolved [function](https://docs.braintrust.dev/deploy/functions.md#invoke-functions) data server-side for facet, code, global, and remote eval functions (no effect on prompt functions). The SDK now discovers `BRAINTRUST_API_KEY` from a `.env.braintrust` file, searching upward from the working directory, so you can set credentials per project without exporting shell environment variables. Organization-level [git metadata](https://docs.braintrust.dev/admin/organizations.md#set-git-metadata-logging) settings are now honored when creating experiments. Fixed [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) streaming spans to attach to the correct parent, and stopped the [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md) integration from emitting empty function tool spans. ### TypeScript SDK releases * [v3.20.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.20.0) - Added `braintrustFlueInstrumentation()` for cleaner [Flue v1 manual instrumentation](https://docs.braintrust.dev/integrations/agent-frameworks/flue.md), so Flue v1 apps can register Braintrust with Flue's `instrument()` API. Flue v0.8.x manual instrumentation continues to use `braintrustFlueObserver`. * [v3.19.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.19.0) - Added support for tracing [`@anthropic-ai/bedrock-sdk`](https://docs.braintrust.dev/integrations/ai-providers/anthropic#anthropic-bedrock-typescript), [`@aws-sdk/client-bedrock-runtime`](https://docs.braintrust.dev/integrations/ai-providers/bedrock#tracing-typescript), and [`@strands-agents/sdk`](https://docs.braintrust.dev/integrations/agent-frameworks/strands-agent#setup-typescript). The SDK now exposes a reporter for the [vitest-evals library](https://docs.braintrust.dev/integrations/sdk-integrations/vitest.md#report-vitest-evals-runs-to-braintrust). Added support for [Flue v1](https://docs.braintrust.dev/integrations/agent-frameworks/flue.md) via manual instrumentation, removing the previously erroring automatic instrumentation. Fixed Anthropic system-message ordering and dataset-backed eval origin tracking for copied dataset rows. * [v3.18.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.18.0) - Added support for tracing [AI SDK v7](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#manual-instrumentation-telemetry-registration-typescript) and the [Pi Coding Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/pi-coding-agent.md). Expanded support for tracing [Google GenAI Interactions API](https://docs.braintrust.dev/integrations/ai-providers/gemini.md#what-traced-typescript). Explicit `origin` metadata can now be passed into evals. `bt functions push` can upload project-scoped TypeScript [classifiers](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers). Fixed `BraintrustSpanProcessor` support for OpenTelemetry SDK v1 spans, Flue and Claude Agent SDK span nesting, AI SDK v6 time to first token metrics, and eval summaries for explicit base experiments. * [v3.17.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.17.0) - The [`bt` CLI](https://docs.braintrust.dev/reference/cli/quickstart) is now available as an optional dependency of the `braintrust` npm package, so JavaScript SDK users can pin `bt` to their SDK version and run it via `pnpm exec bt` without a separate install. Platform binaries are delivered through optional dependencies, with a postinstall download as a fallback when optional dependencies are skipped. Added `BT_BINARY_PATH` to override the resolved binary and `BT_SKIP_DOWNLOAD=1` to skip the postinstall download. * [v3.16.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.16.0) - Exported `LocalTrace`, the `Trace` object used by [trace-level scorers](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces), so you can construct one directly. `invoke()` now accepts an `overrides` parameter that deep-merges configuration into the resolved function data server-side for facet, code, global, and remote eval functions (no effect on prompt functions). Fixed `BraintrustStream` decoding of multi-byte UTF-8 characters split across chunk boundaries. The `DatasetPipelineRow` type no longer includes an `output` field. * [v3.15.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.15.0) - Updated [Flue](https://docs.braintrust.dev/integrations/agent-frameworks/flue.md) instrumentation to use the `@flue/runtime` 0.8.0 observe hooks, with new span names and metadata keys. Requires `@flue/runtime` 0.8.0 or later. **Breaking change**: removed the `wrapFlueContext` and `wrapFlueSession` exports. Migrate to the observe hooks API. Auto-instrumentation continues to work without changes. ### Go SDK releases * [v0.9.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.9.1) - Fixed Anthropic message tracing when calls are routed through the Braintrust Gateway or another proxy that adds a path prefix to the `/v1/messages` endpoint. * [v0.9.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.9.0) - Added `TrialCount` support to the eval runner: set `Opts.TrialCount` to run each eval case [multiple times](https://docs.braintrust.dev/evaluate/advanced-evaluations.md#run-trials), override it [per-case](https://docs.braintrust.dev/evaluate/advanced-evaluations.md#override-trial-count-per-case) with `Case.TrialCount`, and read the zero-based trial index in task hooks via `TaskHooks.TrialIndex`. * [v0.8.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.8.0) - Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) through the `Classifiers` option on the eval runner, with `eval.NewClassifier` and the `eval.Classification` type. The SDK now discovers `BRAINTRUST_API_KEY` from a `.braintrust.json` file, searching upward from the working directory when no key is set through the environment or the `WithAPIKey` option. `WithAPIKey` now ignores blank or whitespace-only values, so environment and file-based fallback remain in effect. * [v0.7.2](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.7.2) - Base64-encoded AI attachments (images, PDFs, and other binary data) in span inputs and outputs are now automatically uploaded as separate Braintrust attachment objects, reducing span payload sizes and the chance of exceeding OpenTelemetry span size limits. Supported for OpenAI, Anthropic, Google Gemini, and AWS Bedrock. Enabled by default. Disable with `BRAINTRUST_AUTO_CONVERT_AI_ATTACHMENTS=false` or the `WithAutoConvertAIAttachments(false)` option. ### Java SDK releases * [v0.3.14](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.14) - Fixed `DatasetCase.metadata()` returning an empty map for dataset rows fetched from Braintrust, such as a hosted dataset used in a [remote eval](https://docs.braintrust.dev/evaluate/remote-evals.md). Case-level metadata stored on those rows is now populated as expected. * [v0.3.13](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.13) - Fixed `BrainstoreTrace.getLLMConversationThread()` returning an empty conversation inside [trace-level scorers](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces). The method now reconstructs the conversation thread even when the trace's root span has not yet been ingested, which is the case during eval scoring. Also fixed remote evals so eval metadata omits the `parameters` field entirely when an eval defines no parameters. * [v0.3.12](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.12) - Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) in `Eval()` through the `classifiers` parameter. The SDK now also discovers `BRAINTRUST_API_KEY` from a `.env.braintrust` file, searching upward from the working directory. ### Ruby SDK releases * [v0.4.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.4.0) - Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) through the `classifiers:` keyword on `Braintrust::Eval.run` and `Braintrust::Eval::Evaluator`, with block-based (`Braintrust::Classifier.new`) and class-based (`include Braintrust::Classifier`) classifiers. The SDK also discovers `BRAINTRUST_API_KEY` from a `.braintrust.json` file, searching upward from the working directory when no key is set through the environment or an explicit `api_key:` argument. ### C# SDK releases * [v0.2.8](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.8) - Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) for categorical evaluation: implement `IClassifier`, use `FunctionClassifier` for inline single- or multi-label classifiers, or `ITracedClassifier` to inspect trace spans, and register them with `.Classifiers(...)` on the eval builder. The OpenAI and Anthropic integrations now tag LLM spans with their span type and capture request parameters such as model, temperature, max tokens, and tools as span metadata. The SDK now discovers `BRAINTRUST_API_KEY` from a `.env.braintrust` file, searching upward from the working directory, and defers API key validation until first use instead of at construction. ### @braintrust/trace-opencode releases * [v0.0.9](https://github.com/braintrustdata/braintrust-opencode-plugin/releases/tag/v0.0.9) - LLM spans now record cache token usage in [OpenCode](https://docs.braintrust.dev/integrations/developer-tools/opencode.md) traces. `prompt_tokens` follows Braintrust's inclusive convention (input + cache read + cache write), and `prompt_cached_tokens` and `prompt_cache_creation_tokens` are surfaced as separate metrics. ### @braintrust/pi-extension releases * [v0.8.0](https://github.com/braintrustdata/braintrust-pi-extension/releases/tag/pi-extension-v0.8.0) - Root spans are now named after the pi session name (falling back to the repository or directory name) and carry new `pi_version`, `pi_mode`, and `session_name` metadata for richer filtering and search. New **Branch Summary** spans trace branch summarization while navigating a session's history tree, capturing the entries being condensed and the resulting summary. **Compaction** spans now record `compaction_reason`, `will_retry`, `tokens_before`, and `estimated_tokens_after`. **Breaking change**: the compatibility shim for pi versions before 0.65.0 has been removed, so `session_switch` and `session_fork` events are no longer traced on pi versions before 0.65.0. See the [pi integration guide](https://docs.braintrust.dev/integrations/developer-tools/pi.md) for details. * [v0.7.0](https://github.com/braintrustdata/braintrust-pi-extension/releases/tag/pi-extension-v0.7.0) - Turn spans now capture input event metadata (`input_source`, `input_streaming_behavior`, `input_image_count`, `raw_input`) when pi fires an input event before a turn starts. Fixed ephemeral session key scoping to include the working directory, preventing session state conflicts when running pi in multiple directories simultaneously. ### Improvements * User impersonation now requires the impersonated user to belong to at least one organization. See [Impersonate users](https://docs.braintrust.dev/api-reference.md#impersonate-users) for the updated requirements. * Conditional human review scores now support **Trace** and **Subspan** condition scopes in addition to span-level conditions, so a score can show when any span in the trace, or any child span, matches its expression. See [Show scores conditionally](https://docs.braintrust.dev/annotate/human-review.md#show-scores-conditionally) for details. * Gateway requests can now set `x-bt-project-name` to specify a project by name instead of ID. The Gateway resolves the project automatically and logs the request to that project's logs without requiring `x-bt-parent`. See [Use project-level AI providers](https://docs.braintrust.dev/deploy/gateway.md#use-project-level-ai-providers) for details. * Timeline view now properly renders post-hoc scoring spans (such as AutoEval results) inline. By default, scorer spans are hidden to focus on the root execution timeline. Select and toggle **Include score spans in timeline** to view scoring duration alongside your trace execution. See [View as a timeline](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-timeline) for details. * Large text and object diffs can now be displayed in full by clicking **Show more** when comparing prompt versions, improving visibility for long prompts and complex data structures. * When Loop is enabled, the SQL filter editor for [log alerts](https://docs.braintrust.dev/observe/alerts.md#create-a-log-alert) now supports **Generate**, so you can write the filter from a natural-language description instead of by hand. * Added shareable links to specific saved [custom views](https://docs.braintrust.dev/annotate/custom-views.md). Selecting a saved trace or dataset view now records it in the URL, so copying the link opens that exact view. * The [custom view](https://docs.braintrust.dev/annotate/custom-views.md) empty state now includes a **Create default view** button that loads a pre-built template rendering your trace or dataset fields as formatted JSON, so you can start with a working view without writing a Loop prompt. * The [\*\* Permission groups\*\*](https://www.braintrust.dev/app/~/configuration/org/groups) dialog now lets you set organization-level, all-projects, and project-specific permissions inline while creating a group, and you can create a new permission group directly from the member assignment picker. See [Access control](https://docs.braintrust.dev/admin/access-control.md) for details. * Time series charts on the [Monitor](https://docs.braintrust.dev/observe/dashboards.md) page now support a **Target time interval** option (Auto, Week, Day, or Hour) for explicit control over time bucketing, with smoother automatic bucket sizes and query ranges aligned to full intervals. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * Git diff content is now opt-in. Braintrust no longer logs the **Diff** field of git metadata unless your organization explicitly enables it at [\*\* Settings\*\* > \*\* Logging\*\*](https://www.braintrust.dev/app/~/configuration/org/logging). Other git metadata fields are still collected by default. See [Set git metadata logging](https://docs.braintrust.dev/admin/organizations.md#set-git-metadata-logging) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * Enabling [Topics](https://docs.braintrust.dev/observe/topics.md) now requires organization owner permissions. Members without owner access see a prompt in the [Enable Topics](https://docs.braintrust.dev/observe/topics/enable.md) dialog to contact an org owner. * You can now tune topic clustering settings, including sample size, hierarchy threshold, reconciliation, clustering algorithm, and dimension reduction, directly from the Topics page in addition to the [`bt` CLI](https://docs.braintrust.dev/reference/cli/topics). See [Tune clustering settings](https://docs.braintrust.dev/observe/topics/manage.md#tune-clustering-settings) for details. * Trace-level scorers and prompts now support a [`{{thread_with_system}}`](https://docs.braintrust.dev/evaluate/llm-as-a-judge#score-traces) reserved variable that renders the full conversation including system messages. Scorer `{{thread}}` continues to omit system messages so judge rubrics aren't polluted by your application's system prompt. Requires data plane v2.3.0+ (Self-hosted customers must upgrade to use this feature.) * When creating an organization, the data plane region is now pre-selected based on your location: EU for users in the EU, EEA, UK, and Switzerland; US otherwise. You can still change it before creating the organization. See [Create an organization](https://docs.braintrust.dev/admin/organizations.md#create-an-organization) for details. * Log alerts with a webhook action now require a successful test before saving, so a stale test result can't authorize a changed webhook target. See [Set up alerts](https://docs.braintrust.dev/observe/alerts.md). * Automation webhook URLs are now validated when you save an alert. Malformed and non-http(s) URLs are rejected, and delivery to private or internal network addresses is blocked. See [Set up alerts](https://docs.braintrust.dev/observe/alerts.md). * [Spend alerts](https://docs.braintrust.dev/admin/billing/monitor-usage.md#set-up-spend-alerts) now support Slack channel notifications in addition to email. Select **Billing email**, **Slack channel**, or both when configuring your alert thresholds. * When you enter a US billing address while upgrading to Pro, Braintrust now verifies it against address records to ensure accurate tax calculation. * Webhook alert URLs must now use `http` or `https`, and Braintrust blocks delivery to private, internal, or otherwise reserved network addresses to protect against server-side request forgery (SSRF). See [Webhook payloads](https://docs.braintrust.dev/observe/alerts.md#webhook-payloads) for details. * MCP server URLs must now use `http` or `https` to protect against server-side request forgery (SSRF). See [Add MCP servers](https://docs.braintrust.dev/evaluate/prompts/create.md#add-mcp-servers) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * Bedrock custom models can now use the **OpenAI** format, routing through Bedrock's OpenAI-compatible Chat Completions endpoint, including streaming. See [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md#connect-bedrock-to-braintrust) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * Bedrock custom model format auto-detection now correctly handles geo inference profile IDs such as `eu.anthropic.claude-sonnet-4-6`: Anthropic profiles use the **Anthropic** format and all other providers use **Converse**, preventing routing errors when configuring geo-prefixed models in Loop and the playground. The Loop model list now shows geo qualifiers — **Bedrock (EU)**, **Bedrock (US)**, **Bedrock (Global)** — so you can distinguish geo profiles of the same model. See [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md#connect-bedrock-to-braintrust) for details. * Human review score cards now show a per-score review count with a breakdown of which reviewers match the base value and which diverge, and the review-span dropdown includes a **Base span** option to jump back to the parent span's scores. See [Review with multiple reviewers](https://docs.braintrust.dev/annotate/human-review/multiple-reviewers.md) for details. * The [Gateway](https://docs.braintrust.dev/deploy/gateway.md) now supports Google Gemini's native `embedContent` and `batchEmbedContents` endpoints, so you can route Gemini embedding requests, including multimodal text and image content, through the Gateway with logging and billing. See [Generate embeddings](https://docs.braintrust.dev/deploy/gateway.md#generate-embeddings) for details. * The [\*\* Experiments\*\*](https://www.braintrust.dev/app/~/experiments) list now shows an **Estimated cost** column alongside scores and token metrics, making it easier to compare total cost across experiment runs at a glance. * Custom columns in [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) now accept any valid non-aggregate SQL expression. In addition to simple field references, you can use functions (`concat`, `coalesce`, `lower`), `CASE` expressions, arithmetic operators, and string patterns (`LIKE`, `ILIKE`). See [Create custom columns](https://docs.braintrust.dev/observe/view-logs.md#create-custom-columns) for details. * The `len()` SQL function now returns `1` for scalar string, number, and boolean values instead of `null`. This preserves filter and sort behavior when row materialization collapses a single-element array into a scalar. See [SQL functions](https://docs.braintrust.dev/reference/sql/functions.md#sql-functions) for details. * **Loop** and [\*\* Playgrounds\*\*](https://www.braintrust.dev/app/~/playgrounds) now display actionable error messages when AI provider requests fail, with inline recovery actions such as editing your [\*\* AI providers\*\*](https://www.braintrust.dev/app/~/configuration/org/secrets) settings for invalid API key errors, starting a new session for context limit errors in Loop, or retrying for network errors. * [\*\* Topics\*\*](https://docs.braintrust.dev/observe/topics.md) is now accessible from the command palette (⌘K / Ctrl+K) for projects with Topics enabled. Settings shortcuts such as API keys and AI providers now appear above the project list in the command palette switcher. * The **Thread** () layout now includes an options menu with a **Refresh** action to reload thread data. See [View as a conversation](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-conversation) for details. * Trace action buttons are now available when expanding a trace from the [\*\* Review\*\*](https://www.braintrust.dev/app/~/review) page, matching the actions available in logs and experiments. See [Score traces and datasets](https://docs.braintrust.dev/annotate/human-review/score-traces.md#start-reviewing) for details. * [Monitor](https://docs.braintrust.dev/deploy/monitor.md) total-cost charts now use the [`estimated_cost()`](https://docs.braintrust.dev/reference/sql/functions#sql-functions) SQL function, which accounts for cache token pricing and fixes the negative cost values that could appear for some providers. Split-by-token-type cost views are unaffected. * The **Basic** filter on the [\*\* Logs\*\*](https://www.braintrust.dev/app/~/logs) and [\*\* Monitor\*\*](https://www.braintrust.dev/app/~/monitor) pages now includes a dedicated **Model** option that auto-detects the metadata field storing the model name (including OpenTelemetry's `gen_ai.request.model`) and lists available models with their trace counts, per-model cost, latency, and token usage. * Free-form human review scores can now write reviewer input to the `expected` field instead of `metadata`, configurable per score. See [Configure review scores](https://docs.braintrust.dev/annotate/human-review.md#configure-review-scores) for details. * Disabling built-in models now automatically pauses your topic automations instead of letting them fail silently. Re-enabling built-in models does not automatically resume them. Resume each automation manually from the [Topics](https://docs.braintrust.dev/observe/topics/manage.md#pause-resume-automation) page. * The Thread view preprocessor picker can now generate a custom preprocessor with a coding agent, which inspects your real trace data and writes the preprocessor for you. See [Write a custom preprocessor](https://docs.braintrust.dev/observe/topics/enable.md#write-a-custom-preprocessor) for details. * OpenTelemetry ingestion for [Pydantic AI v2](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) now correctly classifies `invoke_agent` wrapper spans as task spans instead of LLM spans, fixing inflated LLM call counts. Tool definitions, multimodal image inputs (`uri` and `blob` parts), and detailed token metrics (reasoning, cache, and audio) are also parsed from Pydantic AI v2 span attributes. * Scorers, prompts, tools, and workflows from other projects now appear in playground and automation dropdowns for users with project-level read access, without requiring organization-wide read. This supports a shared scorer library, where you grant read access to a central scorer project. See [Use scorers from another project](https://docs.braintrust.dev/evaluate/score-online.md#use-scorers-from-another-project) for details. * Creating, editing, and renaming [Monitor dashboards](https://docs.braintrust.dev/observe/dashboards/create.md#create-a-dashboard) now requires only the project-level **Projects > Update** permission, not organization-level permission. Setting an organization-wide default view still requires the organization-level **Update** permission. **May 2026** ### Topics pricing [Topics](https://docs.braintrust.dev/observe/topics.md) now uses monthly included credits across all plans: \$10/month on Starter and \$249/month on Pro as a launch promotion for the first 3 months, reverting to \$100/month after. Overage charges apply once credits are used up, at \$0.06/MTok input and \$0.40/MTok output, uniform across plans. Credits do not roll over month-to-month. [Custom facets](https://docs.braintrust.dev/observe/topics/custom-facets.md) are now available on all plans. See [Model credits](https://docs.braintrust.dev/plans-and-limits.md#model-credits) and [Billing FAQ](https://docs.braintrust.dev/admin/billing/faq.md) for details. ### Cached tokens view for traces The **Timeline** trace view adds a cached tokens view that scales LLM span bars by cached read tokens and breaks down input token usage by uncached input, cached read, and cache write, so you can assess prompt caching effectiveness across a trace at a glance. See [View as a timeline](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-timeline) for details. ### Topics for sessions and multi-turn conversations Topics automations can now classify groups of related traces, like sessions or multi-turn conversations, as a single unit instead of each trace independently. Pick a grouping key like `metadata.session_id`, set the interval and trace cap that defines each group, and choose whether the classification is written on the chronologically first trace in the group or on every trace with prior traces as context. See [Group traces into conversations](https://docs.braintrust.dev/observe/topics/manage.md#group-traces-into-conversations) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### Workload identity federation for Anthropic Connect Anthropic to Braintrust using workload identity federation, which exchanges Braintrust-signed JWTs for Anthropic access tokens instead of storing long-lived Anthropic API keys in Braintrust. Available for Braintrust-hosted organizations through organization-level AI providers. Project-level Anthropic providers continue to use API key authentication. See [Add Anthropic as an AI provider](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#add-anthropic-as-an-ai-provider) for setup instructions. ### Workload identity federation for Vertex AI Connect Google Vertex AI to Braintrust using workload identity federation, which exchanges Braintrust-signed OIDC tokens for Google access tokens instead of storing long-lived Google credentials in Braintrust. Available for Braintrust-hosted organizations through organization-level AI providers when the Braintrust Gateway is enabled. Project-level Vertex AI providers continue to use access token or service account key authentication. See [Google Vertex AI](https://docs.braintrust.dev/integrations/ai-providers/google.md#configure-the-integration) for setup instructions. ### Query patterns in the Infra dashboard For self-hosted deployments, the \*\* Infra dashboard\*\* now includes **UI queries** and **API queries** tables that group slow query shapes by object type, filter fields, source, and predicate types like `ILIKE`, `match()`, and inequalities. Each row shows count and p50/p95/p99 latency, helping you identify which query shapes are driving latency and where to focus indexing or query refactors. See [Infra dashboard](https://docs.braintrust.dev/admin/self-hosting.md#infra-dashboard) for details. ### Project-scoped default views Set a default view at the project level so everyone viewing a project lands on the same starting view, without affecting other projects or requiring organization-level permissions. Project admins (project-level **Update** permission) can now set defaults on Logs, Experiments, Datasets, Review, and Monitor pages, sitting between personal and organization defaults in the lookup order. See [Set default table views](https://docs.braintrust.dev/observe/view-logs.md#set-default-table-views) for details. ### Dataset filters for experiments Scope an [experiment run](https://docs.braintrust.dev/evaluate/run-in-ui.md#create-from-scratch) to a subset of a dataset instead of every record. Filters from a [filtered dataset view](https://docs.braintrust.dev/annotate/datasets/manage.md#filter-records) carry over when you create an experiment, so the run targets the same records. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### Dataset versioning in Loop [Loop](https://docs.braintrust.dev/loop/capabilities.md#curate-datasets) can now save, list, and restore dataset snapshots and tag versions with environments. Checkpoint a dataset before a batch of edits, roll back to a saved version, or promote a snapshot to `production` without leaving the chat. Available when Loop is opened from a dataset page. ### Comparison grade in experiment comparisons Experiment comparisons now label each experiment as improvement, regression, tradeoff, or tie relative to the base experiment, so you can tell at a glance whether a run is a clear win, a clear loss, or a mix of gains and tradeoffs. The grade appears as a row in \*\* Summary table\*\* layout and in column headers in \*\* Grid\*\* layout, including playground task columns. See [Compare experiments](https://docs.braintrust.dev/evaluate/compare-experiments.md#assess-overall-impact) for details. ### Assume role authentication for Amazon Bedrock Connect Amazon Bedrock to Braintrust using AWS STS `AssumeRole` with an IAM role in your AWS account, instead of storing long-lived AWS access keys in Braintrust. Available for Braintrust-hosted organizations. See [Connect Bedrock to Braintrust](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md#connect-bedrock-to-braintrust) for setup instructions. ### Self-hosted data plane v2.1.1 Data plane v2.1.1 is now the recommended target for self-hosted v2.x upgrades. Use Terraform AWS module v5.2.1 for AWS deployments, or Helm chart 6.2.1 for GCP and Azure deployments. Helm chart 6.2.1 disables code function execution by default. See [Upgrade to data plane v2.x](https://docs.braintrust.dev/admin/self-hosting/upgrade/v2.md) and [Self-hosting releases](https://docs.braintrust.dev/data-plane-changelog.md) for details. ### Secret previews and rotation tracking AI provider and env variable settings show a redacted preview of each saved value (for example, `abc...xyz`) and a **Last updated** timestamp that tracks when the secret value itself was last changed, along with the user who made the change. The same fields are also returned by the public API for `/v1/env_var` and `/v1/ai_secret`. See [Configure AI providers](https://docs.braintrust.dev/admin/ai-providers.md) and [Set environment variables](https://docs.braintrust.dev/admin/organizations.md#set-environment-variables) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### API key and service token creation restricted to the UI > [!WARNING] > > **Update (June 2026):** Service token creation through the API has since been re-enabled for organization owners. See [Programmatic service token creation](#programmatic-service-token-creation). > > **Breaking change**: API keys and service tokens can now be created only from the Braintrust UI. The REST API no longer supports creating either, and the `POST /v1/api_key`, `POST /v1/service_token`, and `PUT /v1/service_token` endpoints have been removed from the public API. The [`PATCH /v1/organization/members`](https://docs.braintrust.dev/api-reference/organizations/modify-organization-membership) endpoint can still create a service account, but its service token must be created by an organization owner via the Braintrust UI. See [Manage API keys](https://docs.braintrust.dev/admin/organizations.md#manage-api-keys) and [Create service tokens](https://docs.braintrust.dev/admin/organizations.md#create-service-tokens) for the supported flows. ### Auto-instrumentation for Java Braintrust now supports [auto-instrumentation](https://docs.braintrust.dev/instrument/trace-llm-calls.md#java) for Java, enabling zero-code tracing via the `braintrust-java-agent` JAR. Attach it at JVM startup to automatically trace OpenAI, Anthropic, Spring AI, LangChain4j, and Google GenAI calls. The agent also supports running alongside the Datadog Java agent. ### SQL `IF` and `COUNT_IF` for conditional expressions SQL queries now support `IF(condition, then_value, else_value)` as a shorter alternative to `CASE WHEN ... THEN ... ELSE ... END` for two-branch conditions, and `COUNT_IF(condition)` as a shorter alternative to `count(CASE WHEN condition THEN 1 ELSE NULL END)` for counting rows that match a predicate. See [Conditional expressions](https://docs.braintrust.dev/reference/sql/functions.md#conditional-expressions) for syntax and examples. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### SQL `TABLESAMPLE` for random sampling SQL queries now support `TABLESAMPLE` (with the shorter `SAMPLE` alias) to randomly sample rows from a table without scanning every row. Use `TABLESAMPLE n PERCENT` to sample a percentage of rows or `TABLESAMPLE n ROWS` to sample a fixed count. An optional `SEED` clause makes results deterministic across runs. See [SAMPLE](https://docs.braintrust.dev/reference/sql/query-structure.md#sample) for syntax and examples. Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) ### SQL `ROLLUP` and `GROUPING SETS` for multi-level aggregation SQL queries now support `ROLLUP` and `GROUPING SETS` for multi-level aggregation in a single query. `ROLLUP` produces subtotals and a grand total alongside individual groups, while `GROUPING SETS` lets you specify exactly which grouping combinations to compute. The `GROUPING()` function labels which dimensions are rolled up in each row, so you can distinguish detail rows from subtotals. See [ROLLUP and GROUPING SETS](https://docs.braintrust.dev/reference/sql/query-structure.md#rollup-and-grouping-sets) for details. Requires data plane v2.1.1+ (Self-hosted customers must upgrade to use this feature.) ### SQL `SETTINGS` clause for per-query options SQL queries now support a trailing `SETTINGS` clause for passing per-query hints to the Brainstore execution engine. The initial options are `max_bloom_terms`, which controls when bloom-filter segment elimination is skipped for filters with many disjunctive terms, and `preview_length`, which truncates preview fields (`input`, `output`, `expected`, `error`, `metadata`) to a chosen character length. See [SETTINGS](https://docs.braintrust.dev/reference/sql/query-structure.md#settings) for syntax and the full list of options. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) ### bt CLI releases * [v0.9.2](https://github.com/braintrustdata/bt/releases/tag/v0.9.2) - [`bt setup`](https://docs.braintrust.dev/reference/cli/setup) now opens a browser to sign in or sign up and then prompts in the terminal to pick an organization and project, replacing the previous API-key paste flow. When multiple [auth profiles](https://docs.braintrust.dev/reference/cli/overview.md#troubleshooting) are saved and no profile is selected by flag, env var, or org match, `bt` now prompts you to pick one interactively instead of failing with an ambiguous-profile error. * [v0.9.1](https://github.com/braintrustdata/bt/releases/tag/v0.9.1) - `bt` now warns once daily when any [AI provider](https://docs.braintrust.dev/admin/ai-providers.md) key is older than six months. `bt view trace` text output now suggests `--json` for fetching the full trace payload in one command. `bt sync push` reads input data more efficiently via byte-based buffering. Fixed `--verbose` defaulting to true for `bt sync pull`. Fixed `bt datasets view` row fetching. * [v0.9.0](https://github.com/braintrustdata/bt/releases/tag/v0.9.0) - Added `bt datasets` for creating, listing, viewing, updating (aliases: `add`, `refresh`), and deleting remote datasets from the CLI. Supports file, stdin, and inline JSON input. Fixed `--limit` being ignored when paginating `bt view` results. Fixed unicode handling in `bt sync pull`. `bt setup` now combines project creation and selection into a single interactive step. * [v0.8.0](https://github.com/braintrustdata/bt/releases/tag/v0.8.0) - Added `--matrix-param` to `bt eval` to specify multiple values for one or more parameters and run one experiment per combination. Added `bt topics config delete` to permanently remove a Topics automation. Added Copilot and Qwen support to `bt setup`, with skills installed at `.copilot/skills/braintrust/SKILL.md` and `.qwen/skills/braintrust/SKILL.md` respectively. Copilot MCP is configured via `copilot mcp add`. `bt setup` default mode is now ephemeral. Pass `--skills` to install agent skills or `--mcp` to configure MCP. Agent skills are no longer installed automatically by `bt setup`. Run `bt setup skills` to opt in. Added a post-success verification prompt to `bt setup`. ### Python SDK releases * [v0.23.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.23.0) - New [Hugging Face](https://docs.braintrust.dev/integrations/ai-providers/huggingface.md) Hub integration that traces `InferenceClient` calls (chat completion, text generation, feature extraction, and sentence similarity). [Instructor](https://docs.braintrust.dev/integrations/sdk-integrations/instructor.md) structured-output calls are now traced as `task` spans. **Breaking change**: when your org hasn't configured a \*\* Settings\*\* > [\*\* Logging\*\*](https://www.braintrust.dev/app/~/configuration/org/logging) policy, `Eval()` and `init()` no longer log a default set of [git metadata](https://docs.braintrust.dev/admin/organizations.md#set-git-metadata-logging) fields. Configure a policy in the UI, or pass a `git_metadata_settings` object specifying the fields to log. * [v0.22.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.22.1) - You can now pass [Pydantic](https://docs.pydantic.dev/) models (or any object that serializes to a dict) directly as `metadata` when logging, instead of converting to a dict first. [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) spans now break out Anthropic prompt cache creation tokens by TTL (`prompt_cache_creation_5m_tokens`, `prompt_cache_creation_1h_tokens`), so you can attribute prompt-cache spend to each pricing tier. Failed [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) runs now surface their error on the trace instead of being silently dropped, so you can see why a run failed without leaving Braintrust. * [v0.22.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.22.0) - [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) async moderation calls (`amoderation()`) are now traced, like the existing sync `moderation()`. Fixed inflated token counts in [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) traces for OpenAI-style providers using prompt caching, where cached input tokens were double-counted. [Custom scorers](https://docs.braintrust.dev/evaluate/write-scorers.md) that return `list[Score]` are no longer rejected by static type checkers. [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md) traces now capture end-of-utterance detection delay, so you can measure voice-turn responsiveness. * [v0.21.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.21.0) - Added [LiveKit Agents](https://docs.braintrust.dev/integrations/agent-frameworks/livekit-agents.md) integration for tracing real-time voice AI pipelines, including LLM turns, STT, TTS, audio output, and function tool calls. * [v0.20.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.20.0) - [AutoGen](https://docs.braintrust.dev/integrations/agent-frameworks/autogen.md) agents running inside teams (such as `RoundRobinGroupChat`) are now traced individually as child spans of the team. [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) `text_completion()` and `atext_completion()` calls are now traced as `Completion` spans. Default [git metadata](https://docs.braintrust.dev/admin/organizations.md#set-git-metadata-logging) collection for `Eval()`, `EvalAsync()`, and `init()` now uses a fixed 8-field set (`commit`, `branch`, `tag`, `dirty`, `author_name`, `author_email`, `commit_message`, `commit_time`) and excludes `git_diff`. **Breaking change**: orgs that relied on automatic diff collection must enable `git_diff` under [\*\* Settings\*\* > \*\* Logging\*\*](https://www.braintrust.dev/app/~/configuration/org/logging). Fixed [Autoevals](https://docs.braintrust.dev/evaluate/autoevals.md) scorers type error issue. [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md) v2 streaming responses now include citations in span output, and `chat.stream()` closes its span correctly when used as a context manager. [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) `beta.conversations` tool calls are now traced. The background logger no longer retries on HTTP 413. The [Strands](https://docs.braintrust.dev/integrations/agent-frameworks/strands-agent.md) integration correctly closes spans when running alongside a no-op OpenTelemetry provider. Anthropic cache tokens captured via [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) are now folded into `prompt_tokens` and `total_tokens`. Span metadata for [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) calls no longer includes entries for parameters you didn't pass. * [v0.19.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.19.0) - Auto-instrumentation now covers [Temporal](https://docs.braintrust.dev/integrations/sdk-integrations/temporal.md); call `braintrust.auto_instrument()` to trace workflows and activities. The integration moved to `braintrust.integrations.temporal`, and the old `braintrust.contrib.temporal` import is deprecated. [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) streaming audio transcriptions are now traced. [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md) and [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) tool calls now generate child tool spans. Fixed a [remote evals](https://docs.braintrust.dev/evaluate/remote-evals.md) bug where the Python SDK's `braintrust.devserver` module routed experiments to the wrong project when an `/eval` request omitted `project_id`. **Breaking change**: [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#python) per-TTL prompt cache write counts moved from span metadata to span metrics and were renamed: `cache_creation_ephemeral_5m_input_tokens` → `prompt_cache_creation_5m_tokens` and `cache_creation_ephemeral_1h_input_tokens` → `prompt_cache_creation_1h_tokens`. Update any references to the old names (for example, in SQL queries, saved filters, alerts, automations, or dashboards). ### TypeScript SDK releases * [v3.14.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.14.0) - Added auto-instrumentation for [Mastra](https://docs.braintrust.dev/integrations/agent-frameworks/mastra.md), tracing agents, workflows, steps, tool calls, and LLM calls with no code changes. Added the `BRAINTRUST_CACHE_LOCATION` [environment variable](https://docs.braintrust.dev/instrument/advanced-tracing.md#tune-performance) to select the prompt and parameter cache mode (`mixed`, `memory`, `disk`, or `none`). * [v3.13.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.13.0) - The Node.js SDK now searches parent directories for a `.env.braintrust` file containing `BRAINTRUST_API_KEY` at login. Added the experimental [`DatasetPipeline`](https://docs.braintrust.dev/annotate/datasets/pipelines) API for transforming project logs into dataset rows. * [v3.12.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.12.0) - Added auto-instrumentation for the [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) (`@openai/agents`), [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) and [LangGraph](https://docs.braintrust.dev/integrations/agent-frameworks/langgraph.md), and Flue (`@flue/runtime`). Added `wrapNextjsConfigWithBraintrust`, imported from `braintrust/next`, as the canonical Next.js setup helper, which selects the webpack plugin or Turbopack loader based on the active build. Renamed the bundler plugin exports to `braintrustVitePlugin`, `braintrustWebpackPlugin`, `braintrustEsbuildPlugin`, and `braintrustRollupPlugin` (the previous names are deprecated). Added the `braintrust/apply-auto-instrumentation` entrypoint so CommonJS and TypeScript-to-CommonJS projects can enable auto-instrumentation without the `--import` ESM hook. Traces can now span multiple projects. **Behavior change**: git metadata is no longer collected by default when your org has no [git metadata logging](https://docs.braintrust.dev/admin/organizations.md#set-git-metadata-logging) policy. Hardened `mergeDicts` against prototype pollution and added exponential backoff to request retries. * [v3.11.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.11.0) - Added auto-instrumentation for `@github/copilot-sdk`, `@openai/codex-sdk`, and [Firebase Genkit](https://docs.braintrust.dev/integrations/sdk-integrations/firebase-genkit.md). Instrumented [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) classification and moderation APIs, and added `projectId` support to `wrapVitest`. Captured reasoning content for [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md) and [Groq](https://docs.braintrust.dev/integrations/ai-providers/groq.md) reasoning models. Fixed duplicate LLM spans when multiple SDK instances load in one process, and corrected Google ADK agent naming. * [v3.10.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.10.0) - Added auto-instrumentation for [Groq](https://docs.braintrust.dev/integrations/ai-providers/groq.md) (`groq-sdk`) and Cursor (`@cursor/sdk`). Added dataset versioning support so experiments can pin a [dataset](https://docs.braintrust.dev/annotate/datasets.md) to a specific version, and experiment metadata now records the dataset filters used for a run. [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md) now captures extended thinking content and a `reasoning_tokens` metric. Expanded [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) auto-instrumentation to cover `@google/adk@1.0.0`. Fixed [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) reasoning capture, [Hugging Face](https://docs.braintrust.dev/integrations/ai-providers/huggingface.md) streamed tool calls, [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) built-in tool nesting, and bundler plugin export maps. ### Go SDK releases * [v0.7.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.7.0) - Upgraded the OpenTelemetry modules, gzipped OpenTelemetry span exports, and expanded OpenAI metadata capture to include request status, token usage, and details for incomplete or failed responses. **Breaking change**: requires Go 1.25.0 or later. ### Java SDK releases * [v0.3.10](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.10) - Automatic base64 attachment conversion (introduced in v0.3.9) now recognizes the Anthropic, Google Gemini, and AWS Bedrock Converse message formats in addition to OpenAI-style data URIs, so multimodal traces from those providers render as previewable [attachments](https://docs.braintrust.dev/instrument/attachments.md). * [v0.3.9](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.9) - OpenTelemetry span and log payloads are now gzip-compressed before export by default, reducing network bandwidth with no caller-side changes. Braintrust span size limits still apply to the uncompressed payload. Disable span compression with `BRAINTRUST_COMPRESS_OTEL_PAYLOAD=false` or `BraintrustConfig.builder().compressOtelPayload(false)`. Base64-encoded AI attachments (images, PDFs, and other binary data) in span inputs and outputs are now also automatically uploaded as separate Braintrust attachment objects, reducing span payload sizes and the chance of exceeding OpenTelemetry span size limits. Enabled by default. Disable with `BRAINTRUST_AUTO_CONVERT_AI_ATTACHMENTS=false` or `.autoConvertAIAttachments(false)`. * [v0.3.8](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.8) - Added [trace-level scoring](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces): implement `TracedScorer` instead of `Scorer` to score on intermediate LLM calls, tool invocations, and other spans produced during a task. The `Eval` framework dispatches to `score(TaskResult, BrainstoreTrace)`, where `BrainstoreTrace` gives cached access to task spans via `getSpans()` and `getSpans(type)` and a reconstructed conversation via `getLLMConversationThread()`. Existing `Scorer` implementations are unaffected. Opening a cursor on an empty Braintrust-hosted dataset no longer throws. * [v0.3.7](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.7) - [Auto-instrumentation](https://docs.braintrust.dev/instrument/trace-llm-calls.md#java) now requires `com.openai:openai-java` 2.15.0 or later. Streaming calls to the OpenAI Responses API (`client.responses().createStreaming(...)`) now capture `output`, token usage, and `time_to_first_token`. * [v0.3.6](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.6) - Internal publishing fix. No user-facing changes. * [v0.3.5](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.5) - [Anthropic instrumentation](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md#java) now captures per-TTL prompt cache metrics (`prompt_cache_creation_5m_tokens`, `prompt_cache_creation_1h_tokens`) in place of the aggregate `prompt_cache_creation_tokens`, and tracks cache reads as `prompt_cached_tokens`. `BraintrustConfig.sslContext()` is now applied to the API HTTP client, fixing API calls in custom-CA environments. `BraintrustApiClient` is deprecated in favor of `Braintrust.openApiClient()` and `BraintrustOpenApiClient` (generated from the OpenAPI spec). ### C# SDK releases * [v0.2.7](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.7) - Added [Azure OpenAI](https://docs.braintrust.dev/integrations/ai-providers/azure.md) instrumentation in a new `Braintrust.Sdk.AzureOpenAI` package, and upgraded the OpenTelemetry dependencies. ### Improvements * [AI provider](https://docs.braintrust.dev/admin/ai-providers.md) keys that have not been rotated in over six months now display a warning indicator on the secrets manager page, surfacing the recommendation to rotate secrets periodically. * New [SQL best practices](https://docs.braintrust.dev/reference/sql/best-practices.md) page covering query shapes, `ANY_SPAN()` filtering, subfield indexing, and performance warnings. * Score filters on `summary` shape queries now evaluate at the span level for better performance. This is correct for the common case where scorers run once per span. To filter on the averaged score across all spans in a trace, use `HAVING avg(scores.foo)`. See [Data shapes](https://docs.braintrust.dev/reference/sql/query-structure.md#data-shapes) for details. * The [Baseten](https://docs.braintrust.dev/integrations/ai-providers/baseten.md) provider now accepts an optional **API base URL** so you can point Braintrust at a custom Baseten deployment endpoint, such as a dedicated deployment for a Gemma model. * New models available through the Braintrust Gateway: GPT-5.5 Pro (`gpt-5.5-pro`) via OpenAI and Azure, and Claude 4.7 Opus (`claude-opus-4-7`) via Anthropic. See [Supported models](https://docs.braintrust.dev/deploy/supported-models.md) for the full list. * SQL queries now support `substring(text, start, length)` for extracting substrings. Positions are 1-based, and a start value of 0 or less is clipped to the first character. See [SQL functions](https://docs.braintrust.dev/reference/sql/functions.md#sql-functions) for the full function reference. Requires data plane v2.1.1+ (Self-hosted customers must upgrade to use this feature.) * `date_trunc` now accepts positive-integer multiples of `second(s)`, `minute(s)`, `hour(s)`, and `day(s)` (for example, `'15 minutes'` or `'2 hours'`) for finer-grained time bucketing in `SELECT` and `GROUP BY`. The existing single-unit intervals continue to work, including `'week'`, `'month'`, and `'year'`, which don't accept multiples because they have variable durations. See [SQL functions](https://docs.braintrust.dev/reference/sql/functions.md#sql-functions) for details. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * SQL queries now support `estimated_cost_breakdown()` and `estimated_cost_component(name)` for per-component cost analysis. Use these to separate prompt, cached prompt, cache creation, and completion costs. See [SQL functions](https://docs.braintrust.dev/reference/sql/functions.md#sql-functions) for available component names. Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) * Topics automations now let you select which Braintrust-served `brain-facet-*` model generates facet summaries, so you can pin a specific version or stay on `brain-facet-latest`. Configure under **Advanced > Facet model** in [automation settings](https://docs.braintrust.dev/observe/topics/manage.md#adjust-automation). * You can now [pause and resume the Topics automation](https://docs.braintrust.dev/observe/topics/manage.md#pause-resume-automation) to stop and restart log processing without losing your place. When resumed, the automation picks up from where it left off rather than reprocessing. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * Tightened validation for [custom query export automations](https://docs.braintrust.dev/admin/data-management/export.md#custom-sql-query): the `project_logs(...)` source must take a single string-literal project ID matching the automation's own project. Cross-project queries are now rejected. * The [SQL sandbox](https://docs.braintrust.dev/reference/sql.md) editor now autocompletes column names inferred from the `FROM` source (including nested fields), suggests the most common observed values when typing a filter comparison like `WHERE metadata.env =`, and completes `date_trunc` intervals and the `SETTINGS` clause. * The trace viewer now honors an explicit `metrics.estimated_cost` logged on any span, including parent spans. A parent's displayed cost is the sum of child span costs plus the parent's own logged cost, bringing the trace UI into alignment with the [`estimated_cost()`](https://docs.braintrust.dev/reference/sql/functions#sql-functions) SQL function. Previously, an explicit `estimated_cost` was only preserved on leaf spans. * When [testing a scorer](https://docs.braintrust.dev/evaluate/write-scorers.md#test-with-logs) with **Logs** as the source, the **Run** section now shows the full trace (tree and span detail) for each matching root span instead of a single JSON row, so you can drill into spans before running the scorer. * The [\*\* Billing\*\*](https://www.braintrust.dev/app/~/configuration/org/billing) page now shows a projected upcoming invoice broken down by Logs and Scores, so you can see on-demand usage spend in context before the invoice cycle closes. * Inline base64 payloads in [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md) Converse `image`, `video`, `audio`, and `document` content blocks are now automatically extracted into [Braintrust attachments](https://docs.braintrust.dev/instrument/attachments.md) at ingest, so multimodal Converse traces render as previewable attachments without any SDK changes. Requires data plane v2.2.1+ (Self-hosted customers must upgrade to use this feature.) * The [Braintrust Gateway](https://docs.braintrust.dev/deploy/gateway.md) now returns an `x-bt-error-origin` response header on error responses, with a value of `braintrust` for Gateway errors or the provider name (for example, `openai`) for upstream provider errors, so callers can distinguish Gateway failures from provider failures. * Timestamp columns (`created`, `start`, `end`, and any timestamp field) in tables can now be switched between absolute and friendly (relative) formats. Open the column header menu and select **Show friendly dates** or **Show absolute dates**. The choice is saved per column and per page. * Organization owners can now turn off automatic direct ownership grants for newly created objects with the **Create direct ownership grants** toggle in \*\* Settings\*\* > [\*\* Permission groups\*\*](https://www.braintrust.dev/app/~/configuration/org/groups), and optionally clear existing direct owner grants. See [Access control](https://docs.braintrust.dev/admin/access-control.md#how-permissions-work) for details. Requires data plane v2.2.0+ (Self-hosted customers must upgrade to use this feature.) **April 2026** ### Translate message content in traces You can now translate any message or string field in a trace without leaving the UI. This is useful when debugging multilingual agents or reviewing customer conversations in unfamiliar languages. Choose from English, Spanish, French, German, Japanese, and Chinese, or type any language name. See [Translate message content](https://docs.braintrust.dev/observe/examine-traces.md#translate-message-content) for details. ### Custom views for dataset rows Custom views, which let you build tailored interfaces for traces using Loop, are now also available for dataset rows. Use them to create annotation interfaces, side-by-side input and expected comparisons, or any visualization that helps your team review and label records more effectively. See [Create custom views](https://docs.braintrust.dev/annotate/custom-views.md) for details. ### Dataset snapshots Save named checkpoints of a dataset to mark stable states, compare before-and-after changes, and restore to a previous version if needed. See [Save snapshots](https://docs.braintrust.dev/annotate/datasets/manage.md#save-snapshots) for details. > [!NOTE] > > Snapshots are only available on [Pro and Enterprise plans](https://docs.braintrust.dev/plans-and-limits.md#plans). ### Dataset environments Datasets can now be assigned to [environments](https://docs.braintrust.dev/deploy/environments.md), alongside prompts and parameters. Pin a specific dataset version to production to ensure evals always run against a known-good baseline, and promote dataset versions through dev, staging, and production the same way you promote prompts. [Environment alerts](https://docs.braintrust.dev/observe/alerts.md#create-an-environment-alert) fire for dataset environment changes as well, and webhook payloads now include an `object_type` field (`"prompt"` or `"dataset"`) so handlers can branch on the type of change. Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) ### Monitor dashboard import and export Monitor dashboards can now be exported and imported across projects, making it easy to reuse dashboards without recreating them from scratch. Charts also support exporting their configuration or underlying data as CSV or JSON. See [Duplicate dashboards across projects](https://docs.braintrust.dev/observe/dashboards/create.md#copy-a-dashboard-to-another-project) and [Manage custom charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md#move-and-copy-charts) for details. ### Cloud storage export Cloud storage export now supports start-from and rewind controls and a Hive-partitioned file layout. See [Export to cloud storage](https://docs.braintrust.dev/admin/data-management/export.md) for details. ### Tag filtering for experiment runs Tags you apply to experiments are now available as a filter in the dataset **Runs** panel, making it easy to compare runs across a specific model version, prompt variant, or release candidate. See [Filter experiment runs](https://docs.braintrust.dev/annotate/datasets/track-performance.md#filter-experiment-runs) for details. ### Token distribution overview for traces The **Timeline** trace view now includes a token distribution overview above the timeline bars. It breaks down LLM span token usage by type — uncached input, cached read, cache write, and output — and shows cache hit rate per span, making it easy to spot where caching is and isn't working. The overview adjusts to the current **Scale by** selection. See [View as a timeline](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-timeline) for details. ### Summary table layout The experiments table now includes a **Summary table** layout that shows scores and metrics as rows with experiments as columns, making it easy to compare multiple experiments at a glance. Export results as a PDF to share with stakeholders who don't have Braintrust access. See [Compare experiments](https://docs.braintrust.dev/evaluate/compare-experiments.md#assess-overall-impact) for details. ### Infra dashboard for self-hosted deployments Organization owners and admins can now view infrastructure metrics for their self-hosted deployment directly in the Braintrust UI. The infra dashboard shows processing throughput, CPU and memory usage, object storage latency, realtime lag, and status checks. See [Monitor your infrastructure](https://docs.braintrust.dev/admin/self-hosting.md#monitoring) for details. ### Log indexing and full-text search You can now enable log search optimization to accelerate full-text queries across all fields, or add subfield indexes for fields you filter on frequently — such as `metadata.user_id` or `input.query`. Use the new [`search()`](https://docs.braintrust.dev/reference/sql/query-structure#full-text-search) SQL function to query all text fields in a single expression with automatic bloom filter acceleration when log search optimization is on. Braintrust suggests candidate fields based on your data and can backfill up to 3 days of history. See [Speed up log filtering](https://docs.braintrust.dev/admin/projects.md#speed-up-log-filtering) and [Full-text search](https://docs.braintrust.dev/reference/sql/query-structure.md#full-text-search) for details. Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) ### Data plane region selection When creating a new Braintrust organization, you can now choose to host its [data plane](https://docs.braintrust.dev/admin/organizations.md#data-plane-region) in the EU or the US. After creating an organization, you cannot change its region. For details, see [Create an organization](https://docs.braintrust.dev/admin/organizations.md#create-an-organization). ### `estimated_cost()` SQL function SQL now includes an `estimated_cost()` scalar function that returns the estimated cost of a span in US dollars. It resolves to the pre-computed `metrics.estimated_cost` value when available, and falls back to computing cost from token metrics and registered model pricing when it isn't. Unlike the raw `metrics.estimated_cost` field, `estimated_cost()` works across spans, traces, and summary shapes, and can be used inside aggregates like `sum(estimated_cost())`. See [SQL functions](https://docs.braintrust.dev/reference/sql/functions.md#sql-functions) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### SQL subqueries SQL queries now support subqueries in the `FROM` clause. Write `FROM () AS ` to use the result of one query as the data source for another. The alias is required. The most common pattern is aggregating in the inner query and filtering on the aggregated result in the outer query. Subqueries can be nested to multiple levels. See [Subqueries](https://docs.braintrust.dev/reference/sql/query-structure.md#subqueries) for syntax, constraints, and examples. Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) ### Sandboxes for agent evals Use sandboxes when your eval runs custom agent code that can't be expressed as a playground prompt. Unlike remote evals — which require running a dev server — sandboxes let you push your eval once and then run it from the playground on demand. Supports AWS Lambda (Python and TypeScript) and Modal (TypeScript, custom container). See [Test complex agents](https://docs.braintrust.dev/evaluate/remote-evals.md) for details. > [!WARNING] > > Sandboxes are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md). The API, configuration, and behavior are likely to change before general availability. Requires a [Pro or Enterprise plan](https://docs.braintrust.dev/plans-and-limits.md). Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### pi integration Braintrust now integrates with [pi](https://pi.dev), a minimal terminal coding harness. Install the [`@braintrust/pi-extension`](https://github.com/braintrustdata/braintrust-pi-extension) package to automatically trace pi sessions, turns, LLM calls, and tool executions to Braintrust. The extension also supports attaching pi sessions under existing parent traces for end-to-end workflow observability. See the [pi integration guide](https://docs.braintrust.dev/integrations/developer-tools/pi.md) for setup instructions. ### OTel attribute deduplication OTel ingestion previously stored each recognized OTel attribute twice on a span: once as the raw attribute on `metadata` (such as `gen_ai.input.messages`, `ai.prompt.messages`, or `llm.input_messages`), and again in the structured field Braintrust mapped it into (such as `input`, `output`, or `metrics`). Braintrust now keeps only the structured copy. This reduces cost for OTel-heavy workloads. For Braintrust-hosted organizations, this change is enabled by default. Self-hosted deployments can opt in via the `STRIP_OTEL_ATTRIBUTES_FROM_METADATA` environment variable. In either case, you can preserve raw attributes on a specific span by setting the `braintrust.otel.preserve_attributes` attribute to `true` on it. For details, see [Strip OTel attributes from metadata](https://docs.braintrust.dev/kb/strip-otel-attributes-from-metadata). > [!WARNING] > > **Breaking change**: this change can break SQL queries, automations, or dashboards that read raw OTel attributes (like `gen_ai.input.messages`) directly from `metadata`. Update them to read from the structured field instead (`input`, `output`, `metrics`, etc.). If you need raw attributes preserved on specific spans, set the `braintrust.otel.preserve_attributes` attribute to `true` on those spans in your OTel instrumentation. ### `bt` CLI releases * [v0.7.1](https://github.com/braintrustdata/bt/releases/tag/v0.7.1) - Added [`--ca-cert `](https://docs.braintrust.dev/reference/cli/overview#global-flags) global flag (env: `BRAINTRUST_CA_CERT`) for custom PEM CA bundles, useful for self-hosted deployments with private TLS certificates. The Braintrust SDK (`braintrust`, `autoevals`, `@braintrust/*`) is now [bundled into function archives](https://docs.braintrust.dev/reference/cli/functions.md) by default for both JS and Python, so deployed functions are self-contained. Added `--disable-reconciliation` flag to [`bt topics config topic-map set`](https://docs.braintrust.dev/reference/cli/topics) to force fresh topic map generation without referencing the previously saved report. * [v0.7.0](https://github.com/braintrustdata/bt/releases/tag/v0.7.0) - Added [`--param key=value`](https://docs.braintrust.dev/reference/cli/eval) flag to `bt eval` for passing runtime parameter values into evaluators that declare a parameters schema. Multi-evaluator commands filter params per-evaluator so unrecognized keys are silently dropped. [`bt setup`](https://docs.braintrust.dev/reference/cli/setup) now supports `gemini` as a coding agent target, symlinking `.gemini/skills` to `.agents/skills/braintrust/SKILL.md`. `bt setup` now displays the created API key after setup completes. Minimum Node.js version for TypeScript evals updated to 18.19.0+ or 20.6.0+ (Bun 1.0+ and Deno with Node compat also supported). * [v0.6.0](https://github.com/braintrustdata/bt/releases/tag/v0.6.0) - Added [`--first N` and `--sample N [--sample-seed S]`](https://docs.braintrust.dev/reference/cli/eval) flags to `bt eval` for running a subset of eval data as a non-final smoke run. Summary output now includes `runMode`, `isFinal`, and `runLabel` fields. **Breaking change:** `bt` no longer automatically loads `.env` files. Set environment variables explicitly before running `bt` commands. [`bt setup`](https://docs.braintrust.dev/reference/cli/setup) now handles authentication and API key creation interactively, simplifying first-time onboarding. Improved `bt setup mcp` experience for MCP server configuration. * [v0.5.0](https://github.com/braintrustdata/bt/releases/tag/v0.5.0) - Added [`bt topics`](https://docs.braintrust.dev/reference/cli/topics) command group for managing Topics automations, including `status`, `poke`, `rewind`, `open`, and `config` subcommands. Auth resolution order updated: explicit `--profile` now takes top priority, followed by `--api-key`/`BRAINTRUST_API_KEY`, then `BRAINTRUST_PROFILE` as a distinct step. `bt setup` now detects non-interactive environments (CI, agents) and falls back gracefully when no TTY is available. `bt functions push` now rejects Python bundle paths containing whitespace characters. * [v0.4.0](https://github.com/braintrustdata/bt/releases/tag/v0.4.0) - Added [`bt functions push`](https://docs.braintrust.dev/reference/cli/functions) for uploading TypeScript and Python function definitions to Braintrust, with esbuild bundling for TypeScript and source collection for Python. Added [`bt functions pull`](https://docs.braintrust.dev/reference/cli/functions) for downloading function definitions from Braintrust to local files. Enhanced [`bt setup instrument`](https://docs.braintrust.dev/reference/cli/setup) with a full agentic SDK installation workflow — language detection, exact version pinning, LLM client instrumentation, app verification, and Braintrust permalink output — supporting Python, TypeScript, Go, Java, Ruby, and C#. The eval runner's SSE `start` event now uses camelCase keys (`projectName`, `experimentName`, `projectId`, `experimentId`, `projectUrl`, `experimentUrl`); the previous snake\_case keys are no longer emitted. Added `parent` field support in the eval request payload for attaching eval results to an existing trace span. ### Python SDK releases * [v0.18.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.18.0) - Added native [LlamaIndex integration](https://docs.braintrust.dev/integrations/sdk-integrations/llamaindex.md) (`llama-index-core>=0.13.0`); call `braintrust.auto_instrument()` to trace LLM calls, embeddings, and query engine runs automatically. Added `cached_tokens` to OpenAI Agents SDK span metrics. Fixed `wrap_openai()` compatibility with Datadog `ddtrace`. Broadened `EvalResult.scores` from `dict[str, float | None]` to `Mapping[str, float | None]`, allowing scorers to return `MappingProxyType` or custom `Mapping` subclasses without type errors. Code that relies on `scores` being a mutable `dict` may need updating. **Breaking change**: Requires Python 3.10 or later. * [v0.17.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.17.0) - Added automatic tracing for [CrewAI](https://docs.braintrust.dev/integrations/agent-frameworks/crew-ai.md), [AutoGen](https://docs.braintrust.dev/integrations/agent-frameworks/autogen.md), and [Strands Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/strands-agent.md). Expanded [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md) tracing to cover audio transcription calls and [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) tracing to cover rerank and async rerank calls. [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) now traces beta conversations API calls. Fixed [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) tool span output preservation across back-to-back `tool_use` messages and simplified [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) wrapper span output. * [v0.16.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.16.0) - Added tracing for [Cohere](https://docs.braintrust.dev/integrations/ai-providers/cohere.md) `auto_instrument()` instrumenting v1/v2 chat, streaming, embed, and rerank calls. The [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) integration captures `TaskSpanData` and `TurnSpanData` span types (openai-agents ≥ v0.14.0). Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) in `Eval()` through the `classifiers` parameter. Fixed Python typing issues around exports and Mapping types. * [v0.15.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.15.0) - Added per-input [`trial_count`](https://docs.braintrust.dev/evaluate/advanced-evaluations#override-trial-count-per-case) override on `EvalCase`. Expanded [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) tracing to cover audio speech, transcription, and OCR calls, and expanded [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) tracing to cover beta managed agents APIs and server-side tool content blocks (e.g., web search) as child tool spans. The [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) Responses API now emits child tool spans for function calls and web search, and [LiteLLM](https://docs.braintrust.dev/integrations/sdk-integrations/litellm.md) now traces audio speech, transcription, and image generation calls. Speech audio and streaming audio from OpenAI and LiteLLM are captured as attachments, streaming refusals from OpenAI are preserved, and a `CachedSpanFetcher` bug that permanently cached empty span fetches was fixed. * [v0.14.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.14.0) - Added automatic tracing for the [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) Images and Audio APIs (sync and async). [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) `.text_stream` now captures structured output metadata and `time_to_first_token`, and the [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) now logs `ResultMessage` metrics. Fixes landed for scorer and task callback default parameter handling and a `braintrust push` whitespace bug. * [v0.13.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.13.0) - Added native [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) tracing, added [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) tracing, and expanded Python Auto-Instrumentation to support `mistral` and `openai_agents`. [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) now traces `interactions` and Live API calls, [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) now logs base64 multimodal inputs as attachments, and [DSPy](https://docs.braintrust.dev/integrations/sdk-integrations/dspy.md) now traces adapter callbacks. * [v0.12.1](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.12.1) - Fixed nested usage metrics and metadata capture for [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md), fixed parameter handling for Python sandboxes, and encoded request bodies as UTF-8 bytes to prevent Latin-1 corruption. * [v0.12.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.12.0) - Added [LangChain](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md) auto-instrumentation and moved the Python LangChain integration into the main `braintrust` package, added native [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md) tracing with `wrap_openrouter()`, and added [AgentScope](https://docs.braintrust.dev/integrations/agent-frameworks/agentscope.md) tracing with `setup_agentscope()`. `auto_instrument()` now supports `langchain`, `openrouter`, and `agentscope`. [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) now traces `generate_images`, and fixes landed for LiteLLM async embeddings, Anthropic, remote prompt parameter rehydration, and Temporal. ### TypeScript SDK releases * [v3.9.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/braintrust@3.9.0) - Added tracing for [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md), [Cohere SDK](https://docs.braintrust.dev/integrations/ai-providers/cohere.md), and [Hugging Face Inference SDK](https://docs.braintrust.dev/integrations/ai-providers/huggingface.md). Expanded [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) coverage with `embedContent` and grounding metadata. Added reranking instrumentation for [AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) and [OpenRouter](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md). Added [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) `beta.messages.toolRunner` support, with full tool-loop tracing and an `anthropic_tool_runner_iterations` metadata field for the iteration count. Anthropic server-side tool use metrics (e.g., `server_tool_use_web_search_requests`) are now captured. Per-input [`trialCount`](https://docs.braintrust.dev/evaluate/advanced-evaluations#override-trial-count-per-case) override now supported on `EvalCase` in `Eval()`. Improved [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) lifecycle handling and subagent tool span nesting. Added [classifier support](https://docs.braintrust.dev/evaluate/write-scorers.md#classifiers) in `Eval()` through the `classifiers` parameter. Fixed AI SDK prompt cache metrics (`prompt_cached_tokens` and `prompt_cache_creation_tokens`) and OpenAI streaming `logprob` and `refusals` capture. * [v3.8.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.8.0) - Added tracing for [Mistral](https://docs.braintrust.dev/integrations/ai-providers/mistral.md) and [OpenRouter Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openrouter-agent.md), expanded AI SDK instrumentation to cover `embed()` and `embedMany()`, and added OpenAI `responses.compact()` instrumentation. SDK tracing now works in Cloudflare Workers and Vercel Edge Runtime via an automatic polyfill. Fixed AI SDK streaming instrumentation (corrected sync/async handling for `streamText` / `streamObject` across v3–v6 and fixed `Agent.stream` tracing in v5), corrected Claude Agent SDK tool-call nesting, fixed span-context propagation in OTel compat mode, and allowed `undefined` as a source for remote eval params. * [v3.7.1](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.7.1) - Stability updates and fixes. * [v3.7.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.7.0) - Added the `braintrust/webpack-loader` API for Next.js Turbopack support. See [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for usage. ### Go SDK releases * [v0.6.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.6.1) - Fixed: org names containing spaces or special characters are now correctly percent-encoded in experiment permalink URLs generated by the eval runner. Previously, such org names produced malformed URLs. * [v0.6.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.6.0) - Added a new [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md#go) tracing integration (`trace/contrib/bedrockruntime`) with Orchestrion auto-instrumentation support. Added embeddings tracing across existing integrations: [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) (both `openai-go` and `sashabaranov/go-openai`), [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md), [LangChainGo](https://docs.braintrust.dev/integrations/sdk-integrations/langchain.md#go), [CloudWeGo Eino](https://docs.braintrust.dev/integrations/sdk-integrations/cloudwego-eino.md#tracing-embeddings-go), and [Firebase Genkit](https://docs.braintrust.dev/integrations/sdk-integrations/firebase-genkit.md#go). * [v0.5.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.5.0) - The [Gemini](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) integration now instruments `streamGenerateContent` calls, with reconstructed output, token usage metrics, and `time_to_first_token` (previously only non-streaming `generateContent` produced spans). Gemini `ThinkingConfig` is captured in span metadata, and `thoughtsTokenCount` is mapped to `completion_reasoning_tokens` in span metrics. The [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) middleware now preserves document citations from streaming `citations_delta` events. Restored support for the `BRAINTRUST_ENABLE_TRACE_CONSOLE_LOG` environment variable (broken since v0.1) and added a `WithEnableTraceConsoleLog(bool)` option for programmatic configuration. * [v0.4.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.4.1) - The [OpenAI](https://docs.braintrust.dev/integrations/ai-providers/openai.md) integration now captures `max_completion_tokens` and `reasoning_effort` in span metadata, giving visibility into reasoning model (e.g. `o4-mini`) configuration in traces. The OpenAI integration also now captures the `text` parameter for the responses API in span metadata. * [v0.4.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.4.0) - Moved tracing integrations to separate Go modules and added LangChainGo to `trace/contrib/all`. **Breaking change**: users of `trace/contrib/*` integrations must now `go get` each module explicitly. See the [migration guide](https://docs.braintrust.dev/sdks/go/migrations/v0-3-to-v0-4.md). * [v0.3.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.3.1) - Anthropic tracing now records `time_to_first_token` for both streaming and non-streaming requests and correctly captures streaming extended thinking blocks. Eval trace structure also changed: each scorer now runs in its own named child span under the eval span, with scorer-specific `span_attributes` and full task context attached. Failed eval tasks now keep `input`, `expected`, `metadata`, and `origin` on the eval span, and omit `output_json` instead of writing `null`. This is a breaking change for teams that inspect raw eval span structure. * [v0.3.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.3.0) - Added [CloudWeGo Eino](https://docs.braintrust.dev/integrations/sdk-integrations/cloudwego-eino.md) and [Firebase Genkit](https://docs.braintrust.dev/integrations/sdk-integrations/firebase-genkit.md) tracing integrations with Orchestrion auto-instrumentation support. ### Java SDK releases * [v0.3.4](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.4) - The `opentelemetry-sdk-extension-autoconfigure` and `opentelemetry-sdk-extension-autoconfigure-spi` modules are no longer transitive dependencies of the main `braintrust-sdk-java` artifact. Applications that used the SDK without the Braintrust Java agent and relied on these being on the classpath should either add them explicitly, depend on the [`braintrust-otel-extension`](https://central.sonatype.com/artifact/dev.braintrust/braintrust-otel-extension) artifact, or run the [`braintrust-java-agent`](https://docs.braintrust.dev/instrument/trace-llm-calls#auto-instrumentation). * [v0.3.3](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.3) - Fixed grandchild spans being dropped when `braintrust-java-agent` ran alongside the Datadog Java agent with `DD_TRACE_OTEL_ENABLED=true`. Datadog detection now uses JVM JAR manifest inspection, making it reliable on Linux. The [`braintrust-otel-extension`](https://central.sonatype.com/artifact/dev.braintrust/braintrust-otel-extension) artifact now ships its OTel autoconfigure implementation, enabling Braintrust tracing under the OpenTelemetry Java agent without the Braintrust agent. * [v0.3.2](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.2) - Added [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md#java) tracing via `BraintrustAWSBedrock.wrap()` for `converse` and `converseStream`, with auto-interception under `braintrust-java-agent`. Requires `software.amazon.awssdk:bedrockruntime` 2.30.0 or later. The Braintrust agent now coexists with the OpenTelemetry Java agent. For full library auto-instrumentation in that setup, use the [`braintrust-otel-extension`](https://central.sonatype.com/artifact/dev.braintrust/braintrust-otel-extension) artifact. * [v0.3.1](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.1) - Spring AI instrumentation rewritten on HTTP-level interceptors (RestClient + WebClient) instead of Micrometer observation handlers. Streaming responses now capture `time_to_first_token`, and image and file attachments in Spring AI messages are captured as `braintrust_attachment` objects in span inputs. Fixed `time_to_first_token` being recorded as `0` for non-streaming LangChain calls. * [v0.3.0](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.3.0) - Added `ParameterDef` for typed parameter definitions configurable from the Playground UI: use `ParameterDef.data(name, defaultValue)` for string, number, boolean, array, and POJO types, and `ParameterDef.model(name, defaultValue)` for a model picker. Added `Parameters` runtime container with typed `get(key, Class)` accessors and numeric coercion. `Task.apply(DatasetCase, Parameters)` is now the primary method to implement; the single-arg form becomes a backward-compatible default. `TaskResult` now carries a `Parameters` field — use `new TaskResult<>(output, datasetCase, parameters)` to forward parameters to remote scorers. `Eval.Builder` gains `.parameters()` and `.parameterValues()` for declaring and overriding parameter values in offline runs. **Breaking change**: `RemoteEval.Parameter` and `RemoteEval.ParameterType` are removed; replace with `ParameterDef.data(...)` and `ParameterDef.model(...)`, and update `RemoteEval.Builder.parameter()` calls to accept `ParameterDef`. * [v0.2.11](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.11) - The Java auto-instrumentation agent now supports running alongside the Datadog Java agent. List the Braintrust `-javaagent` after the Datadog `-javaagent`. Added `BRAINTRUST_FILTER_AI_SPANS` environment variable to export only AI-related spans and drop generic instrumentation spans. See [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for setup details. ### Ruby SDK releases * [v0.3.2](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.3.2) - Relaxed the `openssl` dependency constraint to allow `openssl` 4.x for compatibility with newer Ruby environments. * [v0.3.1](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.3.1) - Added a `parameters:` keyword argument to `Eval.run` for passing runtime configuration to tasks and scorers, which powers [remote evals](https://docs.braintrust.dev/evaluate/remote-evals.md) triggered from the playground. Tasks and scorers that declare `parameters:` receive the values automatically, and those that don't are unaffected. `Functions#list` and `Prompt.load` now accept `project_id:` as an alternative to a project name. `Prompt` instances expose a new `#version` accessor returning the loaded version's transaction ID. Added support for Ruby 4.0. ### C# SDK releases * [v0.2.5](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.5) - Added a `Braintrust.Sdk.AgentFramework` package that traces apps built on the [Microsoft Agent Framework](https://docs.braintrust.dev/integrations/agent-frameworks/microsoft-agent-framework.md), with agent-level (`WithBraintrustAgentTracing`), LLM-level (`UseBraintrustLLMTracing`), and function-level (`UseBraintrustFunctionTracing`) tracing, plus a combined `UseBraintrustTracing` that instruments the whole pipeline in one call. ### Improvements * Playground table cells now render YAML and Markdown content with syntax highlighting, using colors consistent with the prompt editor. * Lambda sandbox evals are no longer limited to 15 minutes end-to-end. Braintrust manages dataset iteration outside the Lambda invocation, so only individual per-case invocations are subject to the Lambda execution limit. * Modal sandbox evals now have a 60-minute lifetime by default, up from the previous 5-minute Modal default, so longer-running agent evals complete without hitting the sandbox lifetime cap. Self-hosted deployments can override the default with the `MODAL_SANDBOX_TIMEOUT_S` environment variable (in seconds). Requires data plane v2.1.0+ (Self-hosted customers must upgrade to use this feature.) * Project descriptions now render as Markdown. Use headings, bullet points, links, and code blocks to format your project description. * When you run `Eval()` with [`loadParameters()`](https://docs.braintrust.dev/evaluate/write-parameters#use-in-evaluations), the experiment's **Details** sidebar now shows a **Parameters** section with a clickable link back to the saved parameters object used. The experiments table also includes a **Parameters** column so you can see which parameter version each experiment used at a glance. * SQL now supports null-safe equality operators: `<=>` (null-safe equal) and `` (null-safe not-equal). Unlike `=` and `!=`, these treat `null` as a comparable value rather than propagating `null` in the result. `coalesce(field, 'x') != 'y'` patterns are also automatically rewritten to `field 'y'` for better index performance. See [SQL operators](https://docs.braintrust.dev/reference/sql/functions.md#sql-operators) for details. * `ANY_SPAN()` now supports one level of nesting. See [`ANY_SPAN()`](https://docs.braintrust.dev/reference/sql/query-structure#matching-spans-filters) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) * [Experiments created in the Braintrust UI](https://docs.braintrust.dev/evaluate/run-in-ui.md) now run without a time limit on cloud and on self-hosted deployments running data plane v2.0 or later. Previously, UI-originated experiments timed out after 15 minutes. * When the Braintrust Slack app is updated with new permissions, a **Refresh permissions** button now appears next to connected workspaces on the Integrations page, making it easy to re-authorize and enable the latest features. See [Enable Slack integration](https://docs.braintrust.dev/admin/organizations.md#enable-slack-integration) for details. * Monitor charts now support **count distinct** as an aggregator, making it easy to plot unique value counts (e.g. distinct users or models) as a time series. See [Create custom charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md) for details. * The [custom chart editor](https://docs.braintrust.dev/observe/dashboards/build-charts.md) now accepts complex SQL aggregate expressions as measures, such as `sum(tokens) / count(id)` and `100 * sum(errors) / count(id)`. Invalid expressions display a validation error on hover. * The [Compare experiments](https://docs.braintrust.dev/evaluate/compare-experiments.md) guide has been significantly updated: restructured around common workflows, verified against the source code, and expanded with new sections on setting baselines, diff mode, layout options, sharing results, and comparing trials. * You can now create an [online scoring rule](https://docs.braintrust.dev/evaluate/score-online.md#create-scoring-automation-rules) directly from the **Scorers** list page by selecting one or more scorers. * The scorer documentation has been reorganized into dedicated pages for each scorer type: [Autoevals](https://docs.braintrust.dev/evaluate/autoevals.md), [LLM-as-a-judge](https://docs.braintrust.dev/evaluate/llm-as-a-judge.md), and [Custom code](https://docs.braintrust.dev/evaluate/custom-code.md). The [Scorers overview](https://docs.braintrust.dev/evaluate/write-scorers.md) covers where to define scorers, how to test them, and best practices. * The custom facet panel now lets you test the preprocessor on a selected trace independently of the prompt, so you can verify your transformation before iterating on the prompt. See [Create a facet](https://docs.braintrust.dev/observe/topics/custom-facets.md#create-a-facet) for details. **March 2026** ### Tag and star datasets Tags can now be applied to datasets themselves, not just to individual records within them. Use dataset tags to organize datasets in the list and star datasets to pin them to the top of the table and dataset picker dropdowns. See [Tag and star datasets](https://docs.braintrust.dev/annotate/datasets/manage.md#tag-and-star-datasets) for details. ### Estimated cost in trace tree The trace tree now shows estimated LLM cost inline on each span by default, alongside duration and total tokens. Cost is propagated from child spans to parent spans, making it easy to see which parts of a multi-step workflow are consuming most of your cost budget. Use **Display** > **Display metric types** to customize which metrics appear. See [View as a hierarchy](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-hierarchy) for details. ### Timeline scaling by tokens and cost The **Timeline** trace view now lets you scale span bars by metrics other than duration — including total tokens, prompt tokens, completion tokens, or estimated cost. This makes it easy to diagnose context bloat by seeing at a glance which spans consume the most tokens. See [View as a timeline](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-timeline) for details. ### Assign prompts to environments via API The [`POST /v1/prompt`](https://docs.braintrust.dev/api-reference/prompts/create-prompt) and [`PUT /v1/prompt`](https://docs.braintrust.dev/api-reference/prompts/create-or-replace-prompt) endpoints now accept an `environment_slugs` parameter, letting you assign a prompt to one or more environments in a single atomic request. See [Assign to environments](https://docs.braintrust.dev/deploy/environments.md#assign-to-environments) for details. ### Organization-wide default views Admins can now set organization-wide default views on any page that supports them, including Logs, Monitor, Review, Playgrounds, Experiments, Datasets, Prompts, Scorers, Parameters, and Tools. See [Set default table views](https://docs.braintrust.dev/observe/view-logs.md#set-default-table-views) and [Set default dashboards](https://docs.braintrust.dev/observe/dashboards.md) for details. ### Annotate playground outputs Annotate multiple outputs across many prompts using quick thumbs up/down reactions and free text comments, and then let Loop suggest prompt improvements based on your feedback. See [Annotate outputs](https://docs.braintrust.dev/evaluate/playgrounds.md#annotate-outputs) for details. > [!NOTE] > > Playground annotations are only available on [Pro and Enterprise plans](https://docs.braintrust.dev/plans-and-limits.md#plans). ### Preprocessor-powered thread view The **Thread** tab now has a settings control that lets you apply a preprocessor to the thread view. Choose the built-in **Thread** preprocessor to format the trace as a readable conversation, or use a [custom preprocessor](https://docs.braintrust.dev/observe/topics/custom-facets.md) to control how messages are rendered. See [View as a conversation](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-conversation) for details. ### Trace-level filters in charts The **Monitor** page now supports trace-level filters in charts and at the page level. Page-level filters, which previously matched at the span level, now match traces where any span satisfies the conditions. This is useful for filtering by metadata you may have set, like `metadata.email` or `metadata.org`. See [Create custom charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Search scope in trace views Trace search (`Cmd/Ctrl+F`) now includes a scope selector to search within just the currently selected span or across the full trace. See [Search within a trace](https://docs.braintrust.dev/observe/examine-traces.md#search-within-a-trace) for details. ### Monitor topic trends The **Monitor** page now includes a built-in chart to help you track topic distribution changes over time. See [Track trends over time](https://docs.braintrust.dev/observe/topics/review-insights.md#track-trends-over-time) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Duplicate custom views across projects Custom trace views can now be duplicated to other projects and organizations, enabling you to reuse custom interfaces across different contexts. See [Duplicate views](https://docs.braintrust.dev/annotate/custom-views.md#duplicate-views) for details. ### Custom view version history Custom trace views now maintain version history, enabling you to track changes over time and revert to previous versions. See [Access version history](https://docs.braintrust.dev/annotate/custom-views.md#access-version-history) for details. ### Chrome Local Network Access for self-hosted deployments Self-hosted deployments on private networks can now enable the **Data plane is on a private network** checkbox in organization settings. When enabled, Braintrust will detect Chrome Local Network Access permission issues and display instructions to resolve them. See [Grant browser permissions](https://docs.braintrust.dev/admin/self-hosting/configure/security.md#grant-browser-permissions) for details. ### Python SDK releases * [v0.11.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.11.0) - [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) integration now instruments hooks, creating dedicated spans for each hook invocation. [Anthropic](https://docs.braintrust.dev/integrations/ai-providers/anthropic.md) integration now instruments `messages.batches.create` and `beta.messages.batches.create` (sync and async), and captures `thinking` config in span metadata when extended thinking is enabled. [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) integration now instruments `embed_content` and `aembed_content` for embedding calls. [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) integration now correctly traces nested subagent tool calls with proper span parenting. Fixed OpenAI traced stream wrappers to support the context manager protocol (`with`/`async with`). `Eval()` now emits a warning when it receives an empty dataset. * [v0.10.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.10.0) - Added saved-parameter loading in Python via `load_parameters()`, traced OpenAI `with_raw_response` calls, added `jsonschema` as a required dependency, and fixed noisy Pydantic serializer warnings. * [v0.9.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.9.0) - Improved [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md) tracing, including CLI and parallel child-span handling, and added `tags` support to Python SDK builder classes. * [v0.8.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.8.0): Added [pytest integration](https://docs.braintrust.dev/integrations/sdk-integrations/pytest.md) for running tracked evals inside pytest suites. Expanded [Agno](https://docs.braintrust.dev/integrations/agent-frameworks/agno.md) tracing to include workflow execution paths, including workflow spans and nested agent traces. Fixed installation of the Python SDK `performance` extra. ### TypeScript SDK releases * [v3.6.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.6.0) - Added OpenRouter SDK instrumentation support. See [OpenRouter SDK](https://docs.braintrust.dev/integrations/ai-providers/openrouter.md). * [v3.5.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.5.0) - Agent tool call tracing for [AI SDK v5/v6](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md): individual tool calls in agentic workflows are now captured as separate `function`-type spans. `prompts.create()` now accepts an `environment` field. Tool definitions are now recorded in span metadata for AI SDK and [Google GenAI](https://docs.braintrust.dev/integrations/ai-providers/gemini.md) wrappers. Fixed `AsyncLocalStorage` resolution in Cloudflare Workers environments. * [v3.4.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.4.0) - Added [Node.js test runner integration](https://docs.braintrust.dev/integrations/sdk-integrations/node-test-runner.md), support for `tags` in functions, model parameter support, and a `BRAINTRUST_DEBUG_LOG_LEVEL` [environment variable](https://docs.braintrust.dev/instrument/advanced-tracing.md#tune-performance) for internal SDK troubleshooting output. * [v3.3.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.3.0) - Improved eval runtime performance by up to 8x through smarter flush backpressure. Added `BRAINTRUST_FLUSH_BACKPRESSURE_BYTES` [environment variable](https://docs.braintrust.dev/instrument/advanced-tracing.md#tune-performance) to configure the flush threshold. Deprecated `BRAINTRUST_LOG_FLUSH_CHUNK_SIZE`. ### Java SDK releases * [v0.2.10](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.10) - Added the Braintrust Java auto-instrumentation agent (`braintrust-java-agent`): attach it at JVM startup with `-javaagent` to automatically trace OpenAI, Anthropic, Spring AI, LangChain4j, and Google GenAI calls with no code changes. The OpenAI instrumentation now also covers the `/v1/responses` endpoint. Spring AI instrumentation now supports the OpenAI and Anthropic model backends. Fixed: score spans in `Eval` runs now correctly include the `purpose` tag. See [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for setup instructions. * [v0.2.9](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.9) - Added Anthropic beta service instrumentation. * [v0.2.8](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.8) - Support Gateway header in devserver CORS. ### Ruby SDK releases * [v0.3.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.3.0) - Added a [Rails engine](https://docs.braintrust.dev/evaluate/remote-evals.md#run-as-a-rails-engine) for mounting the eval server into existing Rails 8.x applications. Scorers can now return [multiple named scores](https://docs.braintrust.dev/evaluate/custom-code.md#return-multiple-scores) and attach metadata to scores. Trace structure aligned with other SDKs. * [v0.2.1](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.2.1) - Added `trace:` keyword for [scoring full eval traces](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces). Scorer and Task blocks now use keyword arguments. `Braintrust::Scorer.new` and `Braintrust::Task.new` are now available as top-level classes. * [v0.2.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.2.0) - Added a [dev server](https://docs.braintrust.dev/evaluate/remote-evals.md) for running evaluations from the Braintrust UI against code on your own infrastructure. Define evaluators using `Braintrust::Eval::Evaluator` (inline or by subclassing), mount them with `Braintrust::Server::Rack.app`, and start a Rack-compatible server to expose them to the playground. ### C# SDK releases * [v0.2.3](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.3) - Added trace-level scoring: implement `ITracedScorer` to score based on intermediate trace spans, such as individual LLM calls in a multi-step task. The eval runner automatically flushes spans before traced scorers run. The new `EvalTrace` provides cached access to task spans via `GetSpansAsync()` and a reconstructed conversation via `GetThreadAsync()`. Existing `IScorer` implementations are unaffected. * [v0.2.2](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.2) - Eval tasks and scorers now handle errors gracefully. When a task throws, each scorer's new `ScoreForTaskException` method is called as a fallback. When a scorer throws, its new `ScoreForScorerException` method is called as a fallback. Both default to a score of `0.0` and can be overridden on `IScorer`. * [v0.2.1](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.1) - `Score` now accepts an optional `Metadata` dictionary for attaching structured data to individual scores. * [v0.2.0](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.2.0) - Added Anthropic instrumentation (`Braintrust.Sdk.Anthropic`). Moved OpenAI instrumentation into a separate `Braintrust.Sdk.OpenAI` package. Added automatic git repo info on experiments, with opt-out and field-level control via `RepoInfo()` and `GitMetadataSettings()` on `Eval.Builder`. ### Improvements * Documented how to [add metadata and tags to spans](https://docs.braintrust.dev/instrument/trace-application-logic.md#add-metadata-and-tags) at creation time or from within a span. * Loop is now available on the projects overview page, enabling natural language analysis of project metrics, usage trends, and patterns across all your projects. See [Analyze projects](https://docs.braintrust.dev/loop/capabilities.md#analyze-projects) for details. * Loop is now available on individual project overview pages, with access to the project's logs, experiments, datasets, prompts, and score progress chart. See [Analyze projects](https://docs.braintrust.dev/loop/capabilities.md#analyze-projects) for details. * Added alerts management to the \*\* Monitor\*\* page. * Topics clustering now supports configurable sample size (100-50,000 summaries) for faster processing of large datasets. See [Re-generate topics](https://docs.braintrust.dev/observe/topics/manage.md#re-generate-topics) for details. * Added `TS_API_KEEP_ALIVE_TIMEOUT_SECONDS` environment variable for self-hosted deployments to [set the HTTP keep-alive timeout](https://docs.braintrust.dev/admin/self-hosting/configure/networking.md#set-the-http-keep-alive-timeout) when running behind load balancers. * [Preset monitor charts](https://docs.braintrust.dev/observe/dashboards/build-charts.md) (Spans, Latency, Total LLM cost, Token count, and Time to first token) now automatically exclude internal scorer spans generated by online scoring automations such as [Topics](https://docs.braintrust.dev/observe/topics.md) and [online scorers](https://docs.braintrust.dev/evaluate/score-online.md), so metrics reflect actual production traffic. **February 2026** ### TypeScript auto-instrumentation Braintrust now supports [auto-instrumentation](https://docs.braintrust.dev/instrument/trace-llm-calls.md) for TypeScript, enabling automatic tracing of all LLM calls with minimal setup. ### Topics for automated log insights Topics automatically analyze and classify logs to surface patterns and insights without manual review. Create topic maps that combine preprocessors (which transform trace data) and AI prompts (which extract summaries) to analyze your logs. Summaries are clustered into meaningful topics using machine learning, then used to automatically classify new and existing traces. Built-in topic maps include Task (user intents), Sentiment (emotional tone), and Issues (agent problems). Custom topic maps enable domain-specific analysis. See [Discover insights with Topics](https://docs.braintrust.dev/observe/topics.md) for details. > [!NOTE] > > **Self-hosted deployments**: Topics requires data plane v2.0+ and has additional eligibility requirements. See [Enable Topics](https://docs.braintrust.dev/admin/self-hosting/upgrade/v2.md#enable-topics) in the v2.0 upgrade guide. ### Braintrust Gateway The Braintrust Gateway provides a production-grade, unified API to access LLM models from OpenAI, Anthropic, Google, AWS, Mistral, and third-party providers with automatic caching, observability, and multi-provider support. Use any supported provider's SDK to call any provider's models—standardize on one SDK while accessing all available models. See [Use the Braintrust Gateway](https://docs.braintrust.dev/deploy/gateway.md) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Braintrust CLI The `bt` command-line interface provides terminal access to Braintrust features including running evaluations, querying logs with SQL, managing prompts and projects, syncing data to local NDJSON files, and configuring coding agents. The CLI supports interactive log browsing, multiple authentication profiles for switching between organizations, and integration with CI/CD pipelines via API key authentication. See [CLI reference](https://docs.braintrust.dev/reference/cli/quickstart.md) for installation and available commands. ### Evaluation parameters Parameters let you create reusable, versioned configurations for your evaluations. Create parameters once in Braintrust and load them across multiple evaluations, enabling centralized management and environment-specific values. In remote evals, parameters automatically become editable controls in the playground UI, letting you experiment with different configurations without changing code. See [Create parameters](https://docs.braintrust.dev/evaluate/write-parameters.md) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Render attachments in custom views Custom trace views now support rendering images, videos, audio, and other attachments directly in the interface. When span data is fetched, Braintrust automatically converts attachment references to signed URLs ready for display. This enables building rich annotation interfaces for multimodal evaluations, visual inspection workflows, and media-heavy applications. See [Render attachments](https://docs.braintrust.dev/annotate/custom-views.md#render-attachments) for examples and implementation details. ### Navigate to trace origins Navigate from traces in logs back to their originating prompt or dataset row, enabling rapid iteration between analyzing results and refining the prompts or test data that generated them. See [Navigate to trace origins](https://docs.braintrust.dev/observe/examine-traces.md#navigate-to-trace-origins) for details. ### Loop improvements Loop now supports `@` mention autocomplete for quickly adding data sources to your queries. Type `@` in the Loop chat input to bring up a searchable menu of available datasets, experiments, project logs, playgrounds, and SQL queries. Loop can also help you file support tickets directly from the chat when you need assistance from the Braintrust support team, automatically including relevant context from your conversation. See [Select data sources](https://docs.braintrust.dev/loop.md#start-a-thread) and [Request support](https://docs.braintrust.dev/loop/capabilities.md#request-support) for details. ### Trace-level scorers LLM-as-a-judge and custom code scorers can now access the entire execution trace to evaluate multi-step workflows and agent behavior. This enables scoring based on tool usage patterns, workflow steps, operation counts, and multi-turn interactions. See [Trace-level scorers](https://docs.braintrust.dev/evaluate/custom-code.md#score-traces) for details. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### LangSmith integration Braintrust now provides an experimental LangSmith wrapper to integrate LangSmith with Braintrust. The wrapper can either send tracing and evaluation calls to both LangSmith and Braintrust in parallel, or route them solely to Braintrust, with minimal code changes. See [LangSmith integration](https://docs.braintrust.dev/integrations/sdk-integrations/langsmith.md) for details. ### Braintrust MCP server improvements The Braintrust MCP server now includes a resource to auto-install the Braintrust SDK, streamlining initial setup. Also, the `btql_query` tool has been replaced with `sql_query` for querying logs using standard SQL syntax, and long tool descriptions have been moved into MCP resources to reduce context window usage while keeping detailed documentation accessible when needed. See [Braintrust MCP](https://docs.braintrust.dev/integrations/developer-tools/mcp.md) for more details. ### OpenCode integration Braintrust now integrates with [OpenCode](https://opencode.ai/), an open-source AI coding assistant. Install the Braintrust plugin to automatically configure the Braintrust MCP server and enable tracing, giving OpenCode access to query logs, fetch experiment results, and log evaluation data using natural language. Sessions are automatically traced with detailed span tracking for workflows, turns, and tool executions. See [OpenCode integration](https://docs.braintrust.dev/integrations/developer-tools/opencode.md) for setup instructions. ### Cursor integration Braintrust now integrates with [Cursor](https://cursor.sh/), the AI-powered code editor. Install the Braintrust extension to automatically configure the Braintrust MCP server, enabling Cursor to query logs, fetch experiment results, and log data using natural language. This streamlines AI-assisted development workflows with direct access to your Braintrust data from within Cursor. See [Cursor integration](https://docs.braintrust.dev/integrations/developer-tools/cursor.md) for setup instructions. ### Claude Code integration updates The `braintrust` Claude Code plugin now automatically configures the Braintrust MCP server for seamless integration. Also, the `trace-claude-code` plugin now supports embedding Claude Code traces inside parent traces, enabling better observability when Claude Code is used as part of a larger workflow. See [Claude Code](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md) for details. ### Image rendering security controls Braintrust now provides configurable image rendering modes to prevent sensitive data leaks from malicious image URLs. Organizations can control when images load in logs with three modes: auto-load images (default), click-to-load (requires user approval), or block all external images. This protection applies organization-wide to all logs, experiments, and playgrounds. See [Image rendering controls](https://docs.braintrust.dev/admin/organizations.md#control-image-rendering) for configuration details. Requires data plane v1.1.31+ (Self-hosted customers must upgrade to use this feature.) ### Single span filters with aggregations Single span filters (`any_span()`) can now be combined with `GROUP BY` to aggregate traces based on span-level conditions. This enables analyzing patterns across traces that contain specific types of spans, such as counting traces with both errors and production tags grouped by model, or calculating average costs for traces with specific span characteristics. This works with both `traces` and `summary` shapes. See [Analyze traces with span filters](https://docs.braintrust.dev/reference/sql/examples.md#analyze-traces-with-span-filters) for examples. Requires data plane v1.1.31+ (Self-hosted customers must upgrade to use this feature.) ### LIMIT with aggregations SQL queries now support LIMIT with GROUP BY aggregations to restrict the number of grouped results returned. This enables top-N queries and paginating through aggregated data. When combined with ORDER BY, rows are sorted before limiting. See [LIMIT](https://docs.braintrust.dev/reference/sql/query-structure.md#limit) for examples. Requires data plane v2.0+ (Self-hosted customers must upgrade to use this feature.) ### Python SDK releases * [v0.7.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.7.0): Added support for passing tags at experiment and eval creation time. The [Google ADK](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) integration is now auto-enabled when using `braintrust.auto_instrument()` automatic instrumentation. Fixed a bug with custom `git_metadata_settings` in `braintrust.init()` crashing the SDK. * [v0.6.0](https://github.com/braintrustdata/braintrust-sdk-python/releases/tag/py-sdk-v0.6.0): Added automatic tool call span creation for [Pydantic AI](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai.md) agents with detailed timing, inputs, and outputs. Fixed compatibility with Agno v2.5.0 and resolved Pydantic AI patching issues when using logfire co-instrumentation. * [v0.5.6](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.6): Added public `name` property to the Span interface, support for retrieving threads in Python, fixed LiteLLM wrapper metric handling for booleans, included root span ID in `update_span()`, and added `with_raw_response` to `responses.parse()` for OpenAI clients. * [v0.5.5](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.5): Improved OpenAI Agents integration to handle all span types and added the `review` span type. * [v0.5.4](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.6): Fixed empty tool call arguments from string concatenation with None. * [v0.5.3](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.6): Added classifications field, improved thread safety for Python context vars, added support for overflowing payloads to S3 at upload time, and removed duplicate retry logic in HTTPConnection. ### TypeScript SDK releases * [v3.2.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.2.0) - Added [Vitest integration](https://docs.braintrust.dev/integrations/sdk-integrations/vitest.md) for running evaluations directly within Vitest test suites. Adds JavaScript [auto-instrumentation](https://docs.braintrust.dev/instrument/trace-llm-calls.md) support. * [v3.1.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.1.0) - Added support for passing tags at experiment creation time. Fixed token double counting between parent and child spans in [Vercel AI SDK](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md) integration. * [v3.0.0](https://github.com/braintrustdata/braintrust-sdk-javascript/releases/tag/js-sdk-v3.0.0) - Improved compatibility with Cloudflare Workers, Next.js Edge, and other restricted environments. Added cache write token metrics for OpenAI Agents. **Breaking changes**: Nunjucks templating is now a separate package. If you use `templateFormat: "nunjucks"` with prompts, you must install `@braintrust/templates-nunjucks-js` and register the plugin at startup. Mustache templating continues to work without any changes. See [v2.x to v3.x migration guide](https://docs.braintrust.dev/sdks/typescript/migrations/v2-to-v3.md) for details. * [v2.2.2](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.2.2) - Initialize parameter values to defaults. Include root span ID in span updates for better trace linking. * [v2.2.1](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.2.1) - Added sub-agent nesting for [Claude Agent SDK](https://docs.braintrust.dev/integrations/agent-frameworks/claude-agent-sdk.md). Includes internal improvements for AsyncIterable input handling, trace scorer capabilities, and S3 overflow support for large payloads. ### Go SDK releases * [v0.2.1](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.2.1) - Added langchaingo support to autoinstrumentation and new tracing library for the ADK (Agent Development Kit). Pinned orchestrion dependency to version 1.6.1 for stability. ### Java SDK releases * [v0.2.7](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.7) - Upgraded OpenTelemetry dependency versions for improved observability and tracing capabilities. * [v0.2.6](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.6) - Added task and scorer error handling for evaluations. Fixed experiment dataset linking to properly connect experiments with remote datasets. * [v0.2.5](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.5) - Added support for custom JSON object mapper configuration, top-level tags and metadata for experiments, and tagging metadata for eval cases. * [v0.2.4](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.4) - Added distributed tracing for remote scorers with per-scorer span creation. Fixed issue where score spans were incorrectly flagged as missing a score. ### Ruby SDK releases * [v0.1.4](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.1.4) - Improved scorer API to handle objects in addition to strings, fixed HTTP redirect handling for Evals API calls, and added proper dataset linking for experiments using remote dataset sources. * [v0.1.3](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.1.3) - Added origin tagging for eval datasets, elevated datasets to first-class API, refactored Eval API to use `api` objects instead of `state`, and fixed Anthropic instrumentation to properly record system prompts. * [v0.1.2](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.1.2) - Added support for prompts. ### Improvements * Added [troubleshooting](https://docs.braintrust.dev/instrument/trace-application-logic.md#span-names-must-be-strings) to prevent validation failures that silently hide logs from dashboards. * Added [Nunjucks templating](https://docs.braintrust.dev/evaluate/prompts/templating.md#nunjucks) support when invoking prompts via SDK. * Updated permission groups interface to use a **Members** button for adding and removing users from groups, improving discoverability. * Added SQL cursor pagination support using `OFFSET ''` (with `_pagination_key` sorting), alongside BTQL `cursor:` pagination. * Added `FILTER_SPANS()` function for SQL queries to return only matching spans from traces, complementing `ANY_SPAN()` which returns entire traces. See [Filter to matching spans only](https://docs.braintrust.dev/reference/sql/query-structure.md#matching-spans-filters) for details. * Added support for unpivoting nested fields from arrays of objects in SQL queries. See [Array of objects unpivot](https://docs.braintrust.dev/reference/sql/query-structure.md#unpivot) for examples. * Added Claude Opus 4.6 support. * Added search functionality to JSON attachment viewer. * Simplified trace views in diff mode by disabling Timeline, Thread, and custom views during experiment comparison for a more focused side-by-side comparison experience. * Added support for viewing span instead of traces in the table on the experiments page. * Added support for changing a project's creator via the API. * Added support for setting a default limit for listing out experiments via the API. * Added support for input cache read and write cost fields in [custom AI provider configuration](https://docs.braintrust.dev/integrations/ai-providers/custom.md#model-metadata), enabling accurate cost estimation for models that support prompt caching. * Added API key authentication option for [AWS Bedrock](https://docs.braintrust.dev/integrations/ai-providers/bedrock.md) as an alternative to IAM credentials. * Improved trace detail panel to display custom column values and the create custom column button on all spans, not just root spans. * Added invite members option to the projects page dropdown for quick access to organization member invitations. **January 2026** ### Auto-instrumentation for Python, Ruby, and Go Braintrust now supports [auto-instrumentation](https://docs.braintrust.dev/instrument/trace-llm-calls.md) for Python, Ruby and Go, enabling zero-code tracing for most providers. For any other providers, use [manual wrapping](https://docs.braintrust.dev/instrument/trace-llm-calls.md#manual-instrumentation). ### Temporal integration Braintrust now integrates with [Temporal](https://temporal.io/), a durable execution platform for building reliable distributed applications. The integration automatically traces Temporal workflows and activities, capturing execution spans, metadata, and distributed traces across workers. This provides full observability across workflow executions with parent-child relationships between workflows and activities. Available for TypeScript, Python, and Go. See [Temporal integration](https://docs.braintrust.dev/integrations/sdk-integrations/temporal.md) for setup instructions. ### TrueFoundry integration Braintrust now integrates with [TrueFoundry](https://www.truefoundry.com/), an AI Gateway that provides a unified interface for accessing multiple AI providers. TrueFoundry exports LLM traces to Braintrust using OpenTelemetry, automatically capturing all interactions including chat completions, agent responses, embeddings, token usage, costs, and performance metrics. Configure the integration through the TrueFoundry dashboard by setting up the Braintrust OpenTelemetry endpoint and authentication headers. See [TrueFoundry integration](https://docs.braintrust.dev/integrations/sdk-integrations/truefoundry.md) for setup instructions. ### Kanban layout for reviews The **Review** page now supports a kanban layout for managing flagged spans. Drag and drop cards to update status, and click to open the full trace. See [Kanban layout](https://docs.braintrust.dev/annotate/human-review/manage-review-work.md#use-kanban-layout) for details. ### Streamlined online scoring setup Create online scoring rules directly from scorers or the logs browser with automatic prepopulation of scorers and filters, enabling rapid iteration from production logs to scoring rules. See [Create scoring rules](https://docs.braintrust.dev/evaluate/score-online.md#create-scoring-automation-rules) for details. ### Loop on trace pages **Loop** is now available when viewing individual traces. Select a trace and open it in fullscreen or a separate page, then use Loop to summarize trace execution, identify errors, search project logs for similar patterns, and generate custom visualizations. Loop on trace pages provides a focused set of tools optimized for single-trace analysis. See [Analyze individual traces](https://docs.braintrust.dev/observe/debug-traces.md) for details. ### View raw trace and span data You can now view and search the complete JSON representation of individual spans or entire traces. This gives you access to all span fields including metadata and internal properties that aren't visible in other views, making it easier to debug issues, verify exact values, and export data for reproduction. See [View raw span data](https://docs.braintrust.dev/observe/examine-traces.md#view-raw-span-data) for details. ### HAVING clause for SQL queries SQL queries now support the `HAVING` clause for filtering aggregated results after `GROUP BY` operations. Use `HAVING` to filter based on aggregate values like counts, averages, or sums. This enables queries like finding models with average scores above a threshold or identifying patterns with a minimum number of occurrences. See [HAVING for filtering aggregations](https://docs.braintrust.dev/reference/sql/query-structure.md#having-for-filtering-aggregations) for details. ### Implicit aliasing in SQL queries SQL queries now support implicit aliasing for multi-part identifiers. When you reference nested fields like `metadata.category`, you can now use the short form `category` in all SQL operations when unambiguous. This makes queries more concise while maintaining clarity. For more details, see [Implicit aliasing](https://docs.braintrust.dev/reference/sql/query-structure.md#implicit-aliasing). ### Project-level AI providers You can now configure AI provider API keys at the project level to override organization-level keys. This allows you to isolate API usage, manage separate billing, or use different credentials per project. Project-level keys work across playgrounds, experiments, and the AI Proxy. See [Configure AI providers](https://docs.braintrust.dev/admin/ai-providers.md#project-providers) for setup instructions. ### Flexible tags across spans You can now add tags to any span in a trace (not just root spans) through the SDK and UI. Tags from all spans are automatically aggregated at the trace level for filtering. When you log additional tags, they are merged (union) rather than replaced, allowing you to add contextual tags throughout your workflow. See [Add tags](https://docs.braintrust.dev/instrument/trace-application-logic.md#add-metadata-and-tags) for details. ### Thread view search You can now search within the **Thread** view of a trace to quickly find specific content in long conversations. This is especially helpful when debugging multi-turn LLM conversations, finding where specific tools were called, or locating particular scores in lengthy traces. See [thread view](https://docs.braintrust.dev/observe/examine-traces.md#view-as-a-conversation) for details. ### New docs structure Our docs now follow a systematic workflow for using Braintrust to measure, understand, and improve your AI applications. This goal-based structure makes it easier to find the right guidance at the right time: * **Instrument** - Capture traces from your application * **Observe** - Find patterns and issues in your data * **Annotate** - Review and improve with human feedback * **Evaluate** - Test and validate improvements * **Deploy** - Ship changes and monitor impact The new structure includes: * [Braintrust workflow overview](https://docs.braintrust.dev/workflow.md) * Improved [observability quickstart](https://docs.braintrust.dev/tracing-quickstart.md) * New [Integrations tab](https://docs.braintrust.dev/integrations.md) for all AI provider and framework integrations * New [Reference tab](https://docs.braintrust.dev/reference.md) for SDKs, API, and other reference content ### Python SDK releases * [v0.5.2](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.2) - Added [`braintrust.auto_instrument()`](https://docs.braintrust.dev/instrument/trace-llm-calls) for one-line automatic instrumentation, cache control options for evals, and automatic cache cleanup after span export. Fixed hanging evals issue. * [v0.5.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.5.0) - Added trace argument support in scorers, dataset initialization with ID parameter, and force tag replacement option. Improved Claude Agent SDK async iterable support and git metadata collection. * [v0.4.3](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.4.3) - The "agent" function type has been renamed to "workflow" to better reflect its purpose. * [v0.4.2](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.4.2) - OpenTelemetry enhancements, improved span tags, and `patch_litellm()` now patches `responses` and `aresponses`. * [v0.4.1](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.4.1) - Fixes Pydantic AI wrapper context loss, improves serialization, and adds Claude Agent SDK Query property forwarding. * [v0.4.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.4.0) - Bug fixes for global scoping issues. **Breaking change**: Requires Python 3.10+. * [v0.3.15](https://github.com/braintrustdata/braintrust-sdk/releases/tag/py-sdk-v0.3.15) - Smaller logs with `wrap_openai`, new [`setup_pydantic_ai()`](https://docs.braintrust.dev/integrations/agent-frameworks/pydantic-ai) integration. ### TypeScript SDK releases * [v2.2.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.2.0) - Added cache control options for evals and automatic cache cleanup after span export. * [v2.1.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.1.0) - Added [Temporal](https://docs.braintrust.dev/integrations/sdk-integrations/temporal.md) integration, `wrapAgentClass` for agent tracing, trace argument support in scorers, and bug fixes. * [v2.0.1](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.0.1) - Bug fixes and improvements. * [v2.0.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v2.0.0) - Span purpose tracking for scorers, bug fixes. **Breaking change**: Zod is now a peer dependency (`npm install zod`). * [v1.1.1](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.1.1) - Fixes async generator handling. * [v1.1.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.1.0) - Smaller logs with `wrapOpenAI`, Vercel AI SDK v6 support, Nunjucks template format for prompts. * [v1.0.3](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.0.3) - Browser support for Eval and SDK integrations, plain text eval output, type safety improvements. ### Go SDK releases * [v0.2.0](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.2.0) - Adds automatic instrumentation using [Orchestrion](https://github.com/DataDog/orchestrion) for zero-code compile-time tracing. **Breaking change**: Requires Go 1.24+. * [v0.1.2](https://github.com/braintrustdata/braintrust-sdk-go/releases/tag/v0.1.2) - Adds time to first token support for OpenAI responses, aligns Gemini span naming with other SDKs. ### Java SDK releases * [v0.2.3](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.3) - Added support for remote scorers in evals and devserver. * [v0.2.2](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.2) - Extended [Langchain4j](https://github.com/langchain4j/langchain4j) instrumentation to AI Services. * [v0.2.1](https://github.com/braintrustdata/braintrust-sdk-java/releases/tag/v0.2.1) - Adds remote evals devserver support and [Langchain4j](https://github.com/langchain4j/langchain4j) integration for OpenAI chat models. ### Ruby SDK releases * [v0.1.1](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.1.1) - Added OpenAI moderations API support, Anthropic Messages beta API support, and automatic trace flushing at process end for ephemeral environments. * [v0.1.0](https://github.com/braintrustdata/braintrust-sdk-ruby/releases/tag/v0.1.0) - Introduces automatic instrumentation via `require: "braintrust/setup"` in Gemfile for Rails and Ruby applications. Adds new integration API for custom instrumentation. **Breaking change**: Deprecates `.wrap(client)` method in favor of new integration API. ### C# SDK releases * [v0.0.2](https://github.com/braintrustdata/braintrust-sdk-dotnet/releases/tag/v0.0.2) - Added support for enriching spans with [custom metadata and tags](https://docs.braintrust.dev/instrument/trace-application-logic.md#add-metadata-and-tags), enhanced OpenAI instrumentation with additional span attributes. **Breaking change**: APIs are now async-first. Update `eval.Run()` to `await eval.RunAsync()` and change `static void Main` to `static async Task Main`. Cursor types replaced with `IAsyncEnumerable`. Some interfaces renamed. ### Improvements * In-product Python and Typescript editors now lint and autocomplete while you type. * Added default data view preferences in personal settings. You can now set your preferred format (Pretty, JSON, YAML, or Tree) for viewing span fields across all traces, and reset all manually configured view overrides with one click. See [Personal settings](https://docs.braintrust.dev/admin/personal-settings.md#default-data-display-format) for details. * Self-hosted deployments now automatically provision service tokens when configuring data plane URLs, eliminating a manual setup step and immediately enabling features like data retention. See [Configure API URLs](https://docs.braintrust.dev/admin/organizations.md#configure-api-urls-self-hosted) for details. * You can now create log alerts directly from filtered views on the **Logs** or **Monitor** pages. See [Set up alerts](https://docs.braintrust.dev/observe/alerts.md) for details. * Added inline dataset creation from experiment and playground workflows. You can now create datasets directly when running experiments or adding datasets to playgrounds without navigating to the datasets page. * Documented testing scorers with manual input, datasets, and production logs. Each method is optimized for different stages of scorer development, from prototyping to validation against real-world data. See [Test scorers](https://docs.braintrust.dev/evaluate/write-scorers.md#test-scorers-and-classifiers) for details. * Slack channel selection now uses a searchable typeahead interface instead of manual channel ID entry. The channel list automatically refreshes every 7 days and includes a manual refresh option for newly created channels. * Added client-side SQL filtering for experiment runs at the dataset level. You can now filter the list of experiments associated with a dataset using SQL queries based on scores, timestamps, experiment names, and other fields. Filter states persist in URLs for easy bookmarking and sharing. See [Filter experiment runs](https://docs.braintrust.dev/annotate/datasets/track-performance.md#filter-experiment-runs). * Improved MCP server authentication with refresh token support for longer-lasting sessions without re-authentication. * Added chart-based time filtering for dataset experiment runs. See [Filter experiment runs](https://docs.braintrust.dev/annotate/datasets/track-performance.md#filter-experiment-runs). * The "agent" function type has been renamed to "workflow" to better reflect its purpose. * The data plane's realtime service now contains only metadata, with no visibility into underlying data. * Dataset imports now display a live table preview that updates as you categorize columns. CSV columns now default to the `input` field for faster setup, and tag fields are automatically created in your project configuration. See [Upload CSV](https://docs.braintrust.dev/annotate/datasets/create.md#upload-csv-json) for details. * Native GCS authentication is now available for self-hosted deployments on Google Cloud Platform. You can use Workload Identity instead of HMAC keys for more secure authentication. See [GCS authentication options](https://docs.braintrust.dev/admin/self-hosting/deploy.md#gcs-authentication-options) for configuration details. * The version dialog on prompt and scorer activity views now allows diffing any version against each other. * Moved toolbar controls (columns, row height, layout toggles) into a consolidated **Display** menu. * Added explicit `function_type` parameter for global function invocation, replacing naming-based type inference. * Restricted file parts in playgrounds and prompts to formats supported by the selected provider. * The user feedback modal now supports adding up to 5 image attachments. * Added [limits documentation page](https://docs.braintrust.dev/plans-and-limits.md) covering usage quotas, rate limits, and system constraints. * Improved [evaluation quickstart](https://docs.braintrust.dev/evaluation-quickstart.md) with inline datasets, iteration examples showing prompt improvements, and troubleshooting. **December 2025** ### Claude Code integration You can now use Braintrust with [Claude Code](https://code.claude.com/docs/en/overview), Anthropic's agentic coding tool. The integration automatically traces Claude Code sessions to give you insight into LLM calls, tool usage, and performance, while enabling Claude to query logs, fetch experiment results, and log data using natural language, especially useful when writing and iterating on evals. For setup instructions and usage examples, see the [Claude Code integration guide](https://docs.braintrust.dev/integrations/developer-tools/claude-code.md). ### New SDKs: Java, Go, Ruby, and C\# Braintrust now offers native SDKs for Java, Go, Ruby, and C#/.NET that provide tools for evaluating and tracing AI applications in Braintrust. See the SDK documentation for setup instructions and examples: [Java](https://github.com/braintrustdata/braintrust-sdk-java), [Go](https://github.com/braintrustdata/braintrust-sdk-go), [Ruby](https://github.com/braintrustdata/braintrust-sdk-ruby), [C#](https://github.com/braintrustdata/braintrust-sdk-dotnet/tree/main). ### Nunjucks templating syntax for prompts You can now use [Nunjucks](https://mozilla.github.io/nunjucks/templating.html) as an advanced templating syntax for prompts in the UI. Nunjucks provides features like loops, conditionals, and filters for sophisticated prompt engineering workflows. For more details, see [Use templating](https://docs.braintrust.dev/evaluate/prompts/templating.md). ### Track dataset performance across experiments You can now see which experiments used your dataset and how each row performed. This helps you identify problematic test cases and understand your evaluation data quality. For more details, see [Track dataset performance](https://docs.braintrust.dev/annotate/datasets/track-performance.md). ### Slack integration for alerts You can now post alerts to Slack channels when conditions are met. For more details, see [Alerts](https://docs.braintrust.dev/observe/alerts.md). Requires data plane v1.1.29+ (Self-hosted customers must upgrade to use this feature.) ### SQL syntax support BTQL now supports standard SQL syntax as an alternative to the native clause-based syntax. The parser automatically detects whether your query is SQL or BTQL. For more details, see [SQL](https://docs.braintrust.dev/reference/sql.md). Requires data plane v1.1.29+ (Self-hosted customers must upgrade to use this feature.) ### MCP servers in prompts You can now use public MCP (Model Context Protocol) servers to give your prompts access to external tools and data. This is useful for evaluating complex tool calling workflows, experimenting with external APIs and services, and tuning public MCP servers. For more details, see [Add MCP servers](https://docs.braintrust.dev/evaluate/prompts/create.md#add-mcp-servers) Requires data plane v1.1.28+ (Self-hosted customers must upgrade to use this feature.) ### Custom trace views Using Loop, you can now use natural language to create custom views of traces. This helps you highlight specific parts of a trace or visualize the trace in a way that is specific to your use case. For more details, see [Create custom trace views](https://docs.braintrust.dev/annotate/custom-views.md). ### Pass/fail thresholds for scorers You can now define a minimum score (between 0 and 1) that a scorer must achieve for a result to be considered passing. This helps you quickly identify which evaluations meet your quality standards. For more details, see [Pass/fail thresholds](https://docs.braintrust.dev/evaluate/custom-code.md#set-pass-thresholds). Requires data plane v1.1.28+ (Self-hosted customers must upgrade to use this feature.) ### TypeScript SDK releases * [v1.0.0](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.0.0) - This release moves OpenTelemetry functionality to the separate `@braintrust/otel` package. This solves ESM build issues in Next.js (edge), Cloudflare Workers, Bun, and TanStack applications, and adds support for both OpenTelemetry v1 and v2. If you are using OpenTelemetry functionality, this is a **breaking change**. See [TypeScript SDK upgrade guide](https://docs.braintrust.dev/sdks/typescript/migrations/v0-to-v1.md) for migration instructions. If you are not using OpenTelemetry, upgrade as usual with `npm install braintrust@latest`. * [v1.0.1](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.0.1) - This release makes URLs clickable in supported terminals when running evaluations and enables project-level AI secrets to work correctly in remote evaluations. * [v1.0.2](https://github.com/braintrustdata/braintrust-sdk/releases/tag/js-sdk-v1.0.2) - This release improves tracing for Vercel AI SDK applications, adding complete visibility into [multi-step tool interactions](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#multi-round-tool-interactions-typescript) and automatic tracing for [AI SDK Agents](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#trace-agents-typescript). ### Improvements * Added `json_extract` function to BTQL for extracting values from JSON strings using path expressions with support for dot notation, array indexing, and nested paths. * Added progress bar for UI-triggered experiments. * Online scores now always show up in the list of scores in the summary view. * Online scores now always show up in the trace table, even if they haven't been run yet. * Added support for extracting last turn prompts and scorer-format dataset inputs in iterate in playground modals. * Updated BTQL filters to automatically extract filter statements from broader input. * Added docs on [attaching custom metadata to traces](https://docs.braintrust.dev/integrations/sdk-integrations/vercel.md#add-metadata-typescript) when using the Vercel AI SDK. * Expanded deep search docs with setup instructions, query examples, and filtering workflows. * Improved Loop chart type and unit selection for data visualizations * Added ability to [filter logs and experiments by comments](https://docs.braintrust.dev/observe/filter.md#apply-a-filter). * Added project description field to projects. This can be used to provide additional context to teammates and when using AI features. * Consolidate project configuration and organization settings pages into a single page. **November 2025** * Custom metric columns on the experiments list page * Aggregate table column headers on the experiments list page * Resizable trace timeline sidebar * Tag from prompt/scorer pages * Add option to maintain hierarchy in trace tree view while filtering span types * Dataset schemas with visual schema builder and form-based editing with validation * Aggregate table column headers on the projects list page * Added support for Loop to make btql queries with arbitrary time range * Added logs and dataset browsers to scorer detail page * Added support for Loop to generate monitoring chart in monitor page's edit chart dialog * BTQL now supports using `dimensions` and `measures` with the `summary` shape to group and aggregate traces. This enables analyzing patterns, monitoring performance trends, and comparing metrics across models or time periods. See [Aggregated trace analytics](https://docs.braintrust.dev/reference/sql/query-structure.md#aggregated-trace-analytics). * BTQL queries issued through the API are now rate limited at 20 requests per object per minute. * Added automatic context mechanism to Loop to automatically add currently viewed trace as context in logs and experiments page * Added Grok 4.1 support * Added Claude Opus 4.5 support * If you are self-hosted, requests to [https://api.braintrust.dev](https://api.braintrust.dev) will now fail * Fix max tokens and reasoning token budget settings for Gemini models ### Python SDK version 0.3.8 * Fixes logging very deep objects or with circular recursion ### Python SDK version 0.3.7 * Added time to first token for Anthropic wrapper * Fixes nesting for OpenAI agents wrapper ### TypeScript SDK version 0.4.9 * SDK integration rewrite. Based on customer feedback we rewrote the integration to be simpler and more robust. Now officially supports v3 up to v6 of the library. All users are recommended to switch to `wrapAISDK` instead of now deprecated `wrapAISDKModel` and `BraintrustMiddleware`. BREAKING CHANGE: spans have a different input/output and metadata and the do\* spans are no longer needed. ### SDK Integrations: Google ADK 0.2.3 * Support MCP agent tracing ### SDK Integrations: LangChain / LangGraph JS 0.2.1 * Added time to first token metric ### SDK Integrations: LangChain / LangGraph Python 0.1.5 * Added time to first token metric **October 2025** * Enabled editing and resending Loop chat messages * Document how to integrate Apollo GraphQL and Braintrust for automatic tracing * Add support for Grok 4 Fast (Reasoning & Non-Reasoning) * Add support for Groq gwen/gwen3-32 & moonshotai/kimi-k2-instruct-0905 * Deprecate Anthropic Claude 3.5 models as they are no longer supported by Anthropic * Modify Apply filter button in btql tool to be more prominent * Added AI-assisted generation to run data box in scorer details page * Added message queuing to Loop * Added a button to extract filter clause from a btql query in filter btql editor * Start a Loop conversation from the CMD-K menu * Move Loop button to the bottom right of the screen * Use case examples when creating a playground * Java SDK * Scope collapse state for span fields by the span type * Collapse/expand all button for LLM data view * By default, collapse all messages in LLM data view besides the last turn * Generate scorer spans when applying scores to logs * Added support for scoring experiment rows * Added AI-assisted generation in tools form, btql filter form and online scoring form * Increased default maximum agentic tool use roundtrips from 5 to 100 * Added support for Gemini tracing * Added support for Claude 4.5 Haiku * Added Loop to the prompt and scorer detail pages * **Refreshed OpenAI Realtime Audio proxy support** - Updated AI proxy to support the latest OpenAI SDK (v6.0+) for realtime audio interactions * Added support for both `OpenAIRealtimeWebSocket` (browser/Cloudflare Workers) and `OpenAIRealtimeWS` (Node.js with ws library) * Updated event types to match the current OpenAI Realtime API specification (`response.output_audio.delta`, `response.output_text.delta`, etc.) * Added header-based authentication and logging with `x-bt-parent` and `x-bt-compress-audio` headers * Improved audio logging with automatic format detection and optional MP3 compression for reduced storage costs * Added "Pretty" span field display option that optimizes for object value readability and renders object values in markdown * The Pretty display option replaces the Markdown option since Pretty renders markdown by default * Added support for viewing spans in the table on the logs page * Added GPT-5 Pro support * Added **Review** page to see spans marked for review in logs, experiments and datasets across a project * Fixed Loop prompt optimization of remote evals * Fix issue with thinking events coming from Mistral * Added Toplist and Big number monitor chart types * Support for [JSON attachments](https://docs.braintrust.dev/instrument/attachments.md) * Improve "Raw span data" and new buttons to download a span or entire trace as JSON from the trace viewer ### SDK Integrations: LangChain (Python) v0.1.2 * Bug fix to ignore async context changed error ### Python SDK version 0.3.6 * Fixed remote evals bug where experiments were not properly marked as completed on the backend * Fixed dataset `_internal_btql` parameter to properly override default BTQL settings (e.g., custom limit values) ### TypeScript SDK version 0.4.8 * Added OpenTelemetry distributed tracing helpers (`contextFromSpanExport()` and `spanContextFromSpanExport()`) for seamless trace propagation between Braintrust and OpenTelemetry across service boundaries ### Python SDK version 0.3.5 * Added DSPy integration with `wrap_dspy` wrapper for automatic tracing of DSPy applications * Added OpenTelemetry distributed tracing helpers (`context_from_span_export()` and `span_context_from_span_export()`) for seamless trace propagation between Braintrust and OpenTelemetry across service boundaries ### Python SDK version 0.3.4 * Added support for `GEMINI_API_KEY` environment variable ### TypeScript SDK version 0.4.6 * Properly support querying versioned datasets ### TypeScript SDK version 0.4.3 * Improved LangChain integrations with simplified parsing for both TypeScript and Python * Added JSON attachment SDK support ### TypeScript SDK version 0.4.2 * Add OpenTelemetry compatibility mode for TypeScript. This allows OTel spans to work with Evals ### TypeScript SDK version 0.4.1 * Added Google GenAI wrapper support * Updated Mastra wrapper methods from `generateVNext`/`streamVNext` to `generate`/`stream` * Moved langchain-js braintrust dependency to peer dependencies * Fixed handling of attachments for Anthropic to avoid large base64 strings in UI * Fixed preservation of result object when returning from `wrappedStreamObject` in AI SDK * Fixed `LanguageModelV1#supportsUrl` being a function, not a property ### Python SDK version 0.3.3 * Properly support querying versioned datasets **September 2025** * Added Anthropic Claude 4.5 Sonnet support * Fixed Gemini schema support to enable proper function calling and structured outputs when using Google's Gemini models through Braintrust and the AI proxy * Added Claude Agent SDK Integration support * Added Gemini Flash and Lite Preview (Sept 2025) support * Improved prompt detail chat logging and added link to corresponding trace * Fixed bugs with parallel tool calling in Loop * Enabled Loop to write BTQL queries against arbitrary data sources on non-BTQL-sandbox pages * Added support for creating datasets and scorers with Loop from the experiment, dataset, and logs pages * Resolved excessive `localStorage` usage in Loop and BTQL sandbox * Improved Loop's `from` clause handling in the BTQL sandbox * Fixed cross-tab syncing and session restoration bugs in Loop * Prompt/scorer activity view UI updates * Before: selecting a version showed a diff vs. the current editor content, where the selected version is the base of the diff * After: prompt versions can be viewed without diffing vs. editor. When diff is enabled, version is shown as incoming, to indicate what would occur when reverting to that version * Added support for updating the email associated with billing data * Added support for iterating on logs in playgrounds * Added support for scoring existing logs * Trace tree is now visible in human review mode * BTQL sandbox improvements * Loop is now on the page and can write queries, debug errors and answer syntax questions * Tabs * Simple charts * Improved auto-complete * Updated UI color palette * Custom charts added to the monitor page (requires data plane 1.1.22) * View state changes for non-saved views * Before: We would attempt to restore any previous edited view state to the URL * After: With a few exceptions, edited view state for non-saved views is only represented in the URL * Loop can search through Braintrust's docs and blog posts to help you answer questions about how to use Braintrust, including generating sample code ### Python SDK version 0.3.1 * Ensure experiments use SpanComponentsV3 by default ### Python SDK version 0.3.0 * Added OpenTelemetry compatibility mode for seamless integration between Braintrust and OTEL tracing * Added `setup_claude_agent_sdk` for automatic tracing of Claude Agent SDK applications * Improved Anthropic wrapper to log consistent input/output format * Added `strict` parameter to `Prompt.build` for strict schema validation * Added SpanComponentsV4 support ### TypeScript SDK version 0.4.0 * Added `wrapClaudeAgentSDK` for automatic tracing of Claude Agent SDK applications * Improved Anthropic wrapper to log consistent input/output format * Fixed AI SDK model detection in `wrapGenerate` callback * Added SpanComponentsV4 support **August 2025** * Traces in the trace viewer on the logs page can now show all associated traces based on a metadata field or tag * Monitor page layout changed to be more responsive to screen size * Various UX improvements to prompt dialog * Improved onboarding experience * Trace timeline layout improvements * Pro plan organizations can now downgrade to the Starter plan via the settings page without contacting support * Prevent read-only users from downloading data from the UI * @mention team members in comments to notify them via email. To mention someone, type "@" and a team member's name or email in any comment input * You can now assign users to rows in experiments, logs, and datasets. Once assigned, you can filter rows by a specific user or a group of users * View configuration has been changed to no longer auto-save changes. It now shows a dirty state and you have the option of saving or resetting those changes back to the base view ### TypeScript SDK version 0.3.7 * Support locking down remote evals via `--dev-org-name` to only accept users from your org * Fixed parent span precedence issues for better trace hierarchy * Improved propagation of parentSpanId into parentSpanContext for OpenTelemetry JS v2 compatibility * Fold the `@braintrust/core` package into `braintrust`. This package consists of a small set of utility functions that is more easily-managed as part of the main `braintrust` package. After version `0.3.7`, you should no longer need a dependency on `@braintrust/core` ### Python SDK version 0.2.6 * Python SDK now correctly nests spans logged from inside tool calls in OpenAI Agents ### Python SDK version 0.2.5 * Support data masking (see [docs](https://docs.braintrust.dev/instrument/advanced-tracing.md#redact-sensitive-data)) * Remote evals in Python SDK * Support tags in Eval hooks * Validate attachment file readability at creation time ### Python SDK version 0.2.4 * Allow non-batch span processors in `BraintrustSpanProcessor` ### Python SDK version 0.2.3 * Fix openai-agents to inherit the right tracing context ### TypeScript SDK version 0.3.6 * OpenAI responses wrapper no longer filters out span data fields when logging * Fixed `withResponse` and `wrapOpenAI` interaction to not hide response data ### TypeScript SDK version 0.2.5 * Support data masking (see [docs](https://docs.braintrust.dev/instrument/advanced-tracing.md#redact-sensitive-data)) * Support tags in Eval hooks * Validate attachment file readability at creation time ### TypeScript SDK version 0.2.4 * Support OpenAI Agents SDK ### SDK Integrations: Google ADK (Python) (version 0.1.1) * Added integration with [Google Agent Development Kit (ADK)](https://docs.braintrust.dev/integrations/agent-frameworks/google.md) ### SDK Integrations: OpenAI Agents (TS) (version 0.0.2) * Fix openai-agents to inherit the right tracing context ### Python SDK version 0.2.2 * Added `environment` parameter to `load_prompt` * The Otel SpanProcessor now keeps `traceloop.*` spans by default * Experiments can now be run without sending results to the server * Span creation is significantly faster in Python ### TypeScript SDK version 0.2.3 * Added `environment` parameter to `load_prompt` * The Otel SpanProcessor now keeps `traceloop.*` spans by default * Experiments can now be run without sending results to the server * Fix `npx braintrust pull` for large prompts ### TypeScript SDK version 0.2.2 * Fix ai-sdk tool call formatting in output * Log OpenAI Agents input and output to root span * Wrap OpenAI responses.parse * Add wrapTraced support for generator functions ### Python SDK version 0.2.1 * Fix langchain-py integration tracing when users use a @traced method * Wrap OpenAI responses.parse * Add @traced support for generator functions ### Autoevals PY (version 0.0.130) * Fold the `braintrust_core` external package into the `autoevals` package, since it is the only user of `braintrust_core`. Future braintrust packages will not depend on the `braintrust_core` py package **July 2025** * New improved UI for trace tree * Token and cost metrics are computed per sub-tree in the trace viewer * Download BTQL sandbox results as JSON or CSV * Moved monitor chart legends to the bottom and increased chart heights * Fixed a monitor chart issue where the series toggle selector would filter the incorrect series * Improved monitor fullscreen experience: charts now open faster and retain their series filter state * Loop is now available in the experiments page and has a new ability to render interactive components inside the chat that will help you find the UI element that Loop is referencing * You can now use remote evals with the "+Experiment" button to create a new experiment. Previously, they were only available in the playground * Add monitor page UTC timezone toggle * Improved trace view loading performance for large traces * Loop can now create custom code scorers in playgrounds * Schema builder UI for structured outputs * Sort datasets when the `Faster tables` feature flag is enabled * Change LLM duration to be the sum, not average, of LLM duration across spans * Add support for Grok 4 and Mistral's Devstral Small Latest ### TypeScript SDK version 0.2.1 * Fix support for the `openai.chat.completions.parse` method when used with `wrapOpenAI` * Added support for ai-sdk\@beta with new `BraintrustMiddleware` * Support running remote evals as full experiments ### TypeScript SDK version 0.2.0 * When running multiple trials per input (`trial_count > 1`), you can now access the current trial index (0-based) via `hooks.trialIndex` in your task function * Added `BraintrustExporter` in addition to `BraintrustSpanProcessor` * Bound max ancestors in git to 1,000 ### Python SDK version 0.2.0 * When running multiple trials per input (`trial_count > 1`), you can now access the current trial index (0-based) via `hooks.trial_index` in your task function * New LiteLLM `wrap_litellm` wrapper * Increase max ancestors in git to 1,000 ### Python SDK version 0.1.8 * Added `BraintrustSpanProcessor` to simplify Braintrust's integration with OpenTelemetry ### Python SDK version 0.1.7 * Added support for loading prompts by ID via the `load_prompt` function. You can now load prompts directly by their unique identifier ### TypeScript SDK version 0.1.1 * Added `BraintrustSpanProcessor` to simplify integration with OpenTelemetry ### TypeScript SDK version 0.1.0 * Fix a bug where large experiments would drop spans if they could not flush data fast enough * Fix bug in attachment uploading in evals executed with `npx braintrust eval` * Upgrading zod dependency from `^3.22.4` to `^3.25.3` * Added support for loading prompts by ID via the `loadPrompt` function **June 2025** * Time range filters on the logs page * Add support for multi-factor authentication * Fix a bug with Vertex AI calls when the request includes the anthropic-beta header * Add Zapier integration to trigger Zaps when there's a new automation event or a new project * Add OpenAI's [o3-pro](https://platform.openai.com/docs/models/o3-pro) model to the playground and AI proxy * View parameters are now present in the url when viewing a default view * Experiments charting controls have been added into views * Experiment objects now support tags through the API and on the experiments view * Add support for Gemini 2.5 Pro, Gemini 2.5 Flash, and Gemini 2.5 Flash Lite * Correctly propagate `expected` and `metadata` values to function calls when running `invoke` * Chat-like thread layout that simplifies thread display to LLM and score data * Enable all agent nodes to access dataset variables with the mustache variable `{{dataset}}` * Improve reliability of online scoring when logging high volumes of data to a project * Tags can now be sorted in the project configuration page which will change their display order in other parts of the UI * System-only messages are now supported in Anthropic and Bedrock models * Logs page UI can now filter nested data fields in `metadata`, `input`, `output`, and `expected` * Support reasoning params and reasoning tokens in streaming and non-streaming responses in the [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) and across the product * New [braintrust-proxy](https://pypi.org/project/braintrust-proxy/) Python library to help developers integrate with their IDEs to support new reasoning input and output types * New `@braintrust/proxy/types` module to augment OpenAI libraries with reasoning input and output types * New streaming protocol between Brainstore and the API server speeds up queries * Time brushing interaction enabled on Monitor page charts * Can create user-defined views in the monitoring page * Live updating time mode added to the monitoring page * The `anthropic` package is now included by default in Python functions * Audit log queries must now specify an `id` filter for the set of rows to fetch * (Beta) continuously export logs, experiments, and datasets to S3 * Enable passing `metadata` and `expected` as arguments to the first agent prompt node ### Autoevals.js v0.0.130 * Remove dependency on `@braintrust/core` ### TypeScript SDK version 0.0.209 * Ensure SpanComponentsV3 encoding works in the browser ### TypeScript SDK version 0.0.208 * Ensure running remote evals (i.e. `runDevServer`) works without the CLI wrapper * Add span + parent ids to `StartSpanArgs` ### TypeScript SDK version 0.0.207 * The SDK's under-the-hood queue for sending logs now has a default size of 5000 logs * You can configure the max size by setting `BRAINTRUST_LOG_QUEUE_MAX_SIZE` in your environment * Improvements to the logging of parallel tool calls * Attachments are now converted to base64 data URLs, making it easier to work with image attachments in prompts ### TypeScript SDK version 0.0.206 * Add support for `project.publish()` to directly `push` prompts to Braintrust (without running `braintrust push`) * The OpenAI and Anthropic wrappers set `provider` metadata ### Python SDK version 0.1.5 * The SDK's under-the-hood log queue will not block when full and has a default size of 25000 logs * You can configure the max size by setting `BRAINTRUST_LOG_QUEUE_MAX_SIZE` in your environment * Improvements to the logging of parallel tool calls * Attachments are now converted to base64 data URLs, making it easier to work with image attachments in prompts ### Python SDK version 0.1.4 * Add `project.publish()` to directly `push` prompts to Braintrust (without running `braintrust push`) * `@traced` now works correctly with async generator functions * The OpenAI and Anthropic wrappers set `provider` metadata ### Python SDK version 0.1.3 * Improve retry logic in the control plane connection (used to create new experiments and datasets) **May 2025** * The "Faster tables" flag is now the default. You should notice experiments, datasets, and the logs page load much faster * Add Claude 4 models in Bedrock and Vertex to the AI proxy and playground * Braintrust now incorporates cached tokens into the cost calculations for experiments and logs * The monitor page also now includes separate lines so you can track costs and counts for uncached, cached, and cache creation tokens * Native support for thinking parameters in the playground * Improved playground prompt editor stability and performance * Capture cached tokens from OpenAI and Anthropic models in a unified format and surface them in the UI * Create experiments from the experiments list page using saved prompts/agents * New BTQL sandbox page and editor with autocomplete * Fullscreen-able monitor charts * Added a 'Copy page' button to the top of every docs page * Brainstore now supports vacuuming data from object storage to reclaim space * Organization owners can manage API keys for all users in their organization in the UI * Add endpoint for admins to list all ACLs within an org * Collapsible sidebar navigation * Command bar (CMD/CTRL+K) to quickly navigate and between pages and projects * View monitor page logs across all projects in an organization * Added Mistral Medium 3 and Gemini 2.5 Pro Preview to the AI proxy and playground * Self-hosted builds now log in a structured JSON format that is easier to parse ### Python SDK version 0.1.2 * Added support for `metadata` and `tags` arguments to `invoke` * The SDK now gracefully handles OpenAI's `NotGiven` parameter * Added `span.link()` to synchronously generate permalinks ### Python SDK version 0.1.1 * Update cached token accounting in `wrap_anthropic` to correctly capture cached tokens * Pull additional metadata in `braintrust pull` for prompts and functions to improve tracing ### SDK (version 0.1.0) * Allow custom model descriptions in Braintrust * Improve support for PDF attachments to multimodal OpenAI models * The Python library no longer has a dependency on `braintrust_core` ### TypeScript SDK version 0.0.206 * Add support for `metadata` and `tags` arguments to `invoke` ### TypeScript SDK version 0.0.205 * Make the `_xact_id` field in `origin` optional * Added `span.link()` as a synchronous means of generating permalinks ### TypeScript SDK version 0.0.204 * Update cached token accounting in `wrapAnthropic` to correctly capture cached tokens ### SDK (version 0.0.203) * Add new reasoning to OpenAI messages ### SDK (version 0.0.202) * Gracefully handle experiment summarization failures in Eval() * Fix a bug where `wrap_openai` was breaking `pydantic_ai run_stream` func * Add tracing to the `client.beta.messages` calls in the TypeScript Anthropic library * Fix some deprecation warnings in the Python SDK **April 2025** * Permission groups settings page now allows admins to set group-level permissions * Automations alpha: trigger webhooks based on log events * Preview attachments in playground input cells * Playground now support list mode which includes score and metric summaries * Handle structured outputs from OpenAI's responses API in the "Try prompt" experience * Allow users to remove themselves from any organization they are part of using the `/v1/organization/members` REST endpoint * Group monitor page charts by metadata path * Download playground contents as CSV * Add pending and streaming state indicators to playground cells * Distinguish per-row and global playground progress * Added GPT-4.1, o4-mini and o3 to the AI proxy and playground * On the monitor page, add aggregate values to chart legends * Add Gemini 2.5 Flash Preview model to the AI proxy and playground * Add support for audio and video inputs for Gemini models in the AI proxy and playground * Add support for PDF files for OpenAI models * Native tracing support in the proxy has finally arrived! Read more in [the docs](https://docs.braintrust.dev/deploy/ai-proxy.md#enable-logging) * Upload attachments directly in the UI in datasets, playgrounds, and prompts * Playground option to append messages from a dataset to the end of a prompt * A new toggle that lets you skip tracing scoring info for online scoring * GIF and image support in comments * Add embedded view and download action for inline attachments of supported file types ### SDK (version 0.0.201) * Support OpenAI `client.beta.chat.completions.parse` in the Python wrapper ### SDK (version 0.0.200) * Ensure the prompt cache properly handles any manner of prompt names * Ensure the output of `anthropic.messages.create` is properly traced when called with `stream=True` in an async program ### SDK (version 0.0.199) * Fix a bug that broke async calls to the Python version of `anthropic.messages.create` * Store detailed metrics from OpenAI's `chat.completion` TypeScript API ### SDK (version 0.0.198) * Trace the `openai.responses` endpoint in the Typescript SDK * Store the `token_details` metrics return by the `openai/responses` API ### SDK (version 0.0.197) * Fix a bug in `init_function` in the Python SDK which prevented the `input` argument from being passed to the function correctly when it was used as a scorer * Support setting `description` and `summarizeScores`/`summarize_scores` in `Eval(...)` **March 2025** * Many improvements to the playground experience: * Fixed many crashes and infinite loading spinner states * Improved performance across large datasets * Better support for running single rows for the first time * Fixed re-ordering prompts * Fixed adding and removing dataset rows * You can now re-run specific prompts for individual cells and columns * You can now do "does not contain" filters for tags in experiments and datasets * When you `invoke()` a function, inline base64 payloads will be automatically logged as attachments * Add a strict mode to evals and functions which allows you to fail test cases when a variable is not present in a prompt * Add Fireworks' DeepSeek V3 03-24 and DeepSeek R1 (Basic), along with Qwen QwQ 32B in Fireworks and Together.ai, to the playground and AI proxy * Fix bug that prevented Databricks custom provider form from being submitted without toggling authentication types * Unify Vertex AI, Azure, and Databricks custom provider authentication inputs * Add Llama 4 Maverick and Llama 4 Scout models to Together.ai, Fireworks, and Groq providers in the playground and AI proxy * Add Mistral Saba and Qwen QwQ 32B models to the Groq provider in the playground and AI proxy * Add Gemini 2.5 Pro Experimental and Gemini 2.0 Flash Thinking Mode models to the Vertex provider in the playground and AI proxy * Add OpenAI's [o1-pro](https://platform.openai.com/docs/models/o1-pro) model to the playground and AI proxy * Support OpenAI Responses API in the AI proxy * Add support for the Gemini 2.5 Pro Experimental model in the playground and AI proxy * Option to disable the experiment comparison auto-select behavior * Add support for Databricks custom provider as a default cloud provider in the playground and AI proxy * Allow supplying a base API URL for Mistral custom providers in the playground and AI proxy * Support pushed code bundles larger than 50MB * The OTEL endpoint now understands structured output calls from the Vercel AI SDK * Added support for `concat`, `lower`, and `upper` string functions in BTQL * Correctly propagate Bedrock streaming errors through the AI proxy and playground * Online scoring supports sampling rates with decimal precision * Added support for OpenAI GPT-4o Search Preview and GPT-4o mini Search Preview in the playground and AI proxy * Add support for making Anthropic and Google-format requests to corresponding models in the AI proxy * Fix bug in model provider key modal that prevents submitting a Vertex provider with an empty base URL * Add column menu in grid layout with sort and visibility options * Enable logging the `origin` field through the REST API * Add support for "image" pdfs in the AI proxy * Fix issue in which code function executions could hang indefinitely * Add support for custom base URLs for Vertex AI providers * Add dataset column to experiments table * Add python3.13 support to user-defined functions * Fix bug that prevented calling Python functions from the new unified playground ### SDK (version 0.0.196) * Adding Anthropic tracing for our TypeScript SDK. See `braintrust.wrapAnthropic` * The SDK now paginates datasets and experiments, which should improve performance for large datasets and experiments * Add `strict` flag to `invoke` which implements the strict mode described above * Raise if a Python tool is pushed without defined parameters, instead of silently not showing the tool in the UI * Fix Python OpenAI wrapper to work for older versions of the OpenAI library without `responses` * Set time\_to\_first\_token correctly from AI SDK wrapper ### SDK (version 0.0.195) * Improve the metadata collected by the Anthropic client * Anthropic client can now be run with `braintrust.wrap_anthropic` * Fix a bug when `messages.create` was called with `stream=True` ### SDK (version 0.0.194) * Add Anthropic tracing to the Python SDK with `wrap_anthropic_client` * Fix a bug calling `braintrust.permalink` with `NoopSpan` ### SDK (version 0.0.193) * Fix retry bug when downloading large datasets/experiments from the SDK * Background logger will load environment variables upon first use rather than when module is imported ### SDK (version 0.0.192) * Improve default retry handler in the python SDK to cover more network-related exceptions ### SDK (version 0.0.190) * Fix `prompt pull` for long prompts * Fix a bug in the Python SDK which would not retry requests that were severed after a connection timeout ### SDK (version 0.0.189) * Added integration with [OpenAI Agents SDK](https://docs.braintrust.dev/integrations/agent-frameworks/openai-agents-sdk.md) ### SDK (version 0.0.188) * Deprecated `braintrust.wrapper.langchain` in favor of the new `braintrust-langchain` package ### SDK (version 0.0.187) * Always bundle default python packages when pushing code with `braintrust push` * Fix bug in the TypeScript SDK where `asyncFlush` was not correctly defaulted to false * Fix a bug where `span_attributes` failed to propagate to child spans through propagated events * Added support for handling score values when an Eval has errored * Improve support for binary packages in `npx braintrust eval` * Support templated structured outputs * Fix dataset summary types in Typescript ### Autoevals (version 0.0.124) * Added `init` to set a global default client for all evaluators (Python and Node.js) * Added `client` argument to all evaluators to specify the client to use * Improved the Autoevals docs with more examples ### Autoevals (version 0.0.123) * Swapped `polyleven` for `levenshtein` for faster string matching ### SDK Integrations: LangChain (Python) (version 0.0.2) * Add a new `braintrust-langchain` integration with an improved `BraintrustCallbackHandler` and `set_global_handler` to set the handler globally for all LangChain components ### SDK Integrations: LangChain.js (version 0.0.6) * Small improvement to avoid logging unhelpful LangGraph spans * Updated peer dependencies with LangChain core that fixes the global handler for LangGraph runs ### SDK Integrations: Val Town * New `val.town` integration with example vals to quickly get started with Braintrust **February 2025** * Add support for removing all permissions for a group/user on an object with a single click * Add support for Claude 3.7 Sonnet model * Add [llms.txt](https://www.braintrust.dev/llms.txt) for docs content * Enable spellcheck for prompt message editors * Add support for Anthropic Claude models in Vertex AI * Add support for Claude 3.7 Sonnet in Bedrock and Vertex AI * Add support for Perplexity R1 1776, Mistral Saba, Gemini LearnLM, and more Groq models * Support system instructions in Gemini models * Add support for Gemini 2.0 Flash-Lite * Add support for default Bedrock cross-region inference profiles in the playground and AI proxy * Move score distribution charts to the experiment sidebar * Add support for OpenAI GPT-4.5 model in the playground and AI proxy * Add deprecation warning for `_parent_id` field in the REST API * Add support for stop sequences in Anthropic, Bedrock, and Google models * Resolve JSON Schema references when translating structured outputs to Gemini format * Add button to copy table cell contents to clipboard * Add support for basic Cache-Control headers in the AI proxy * Add support for selecting all or none in the categories of permission dialogs * Respect Bedrock providers not supporting streaming in the AI proxy * Store table grouping, row height, and layout options in the view configuration * Add the ability to set a default table view * Add support for Google Cloud Vertex AI in the playground and proxy * Add default cloud providers section to the organization AI providers page * Support streaming responses from OpenAI o1 models in the playground and AI proxy * Add complete support for Bedrock models in the playground and AI proxy * Fix model provider configuration issues in which custom models could clobber default models * Fix bug in streaming JSON responses from non-OpenAI providers * Supported templated structured outputs in experiments run from the playground * Support structured outputs in the playground and AI proxy for Anthropic models, Bedrock models, and any OpenAI-flavored models that support tool calls * Support templated custom headers for custom AI providers * Added and updated models across all providers in the playground and AI proxy * Support tool usage and structured outputs for Gemini models in the playground and AI proxy * Simplify playground model dropdown by showing model variations in a nested dropdown ### SDK (version 0.0.187) * Added support for handling score values when an Eval has errored * Improve support for binary packages in `npx braintrust eval` * Support templated structured outputs * Fix dataset summary types in Typescript **January 2025** * Add support for duplicating prompts, scorers, and tools * Fix pagination for the `/v1/prompt` REST API endpoint * "Unreviewed" default view on experiment and logs tables to filter out rows that have been human reviewed * Add o3-mini to the AI proxy and playground * Scorer dropdown now supports using custom scoring functions across projects * Drag and drop to reorder span fields in experiment/log traces and dataset rows * Small convenience improvement to the BTQL Sandbox * Add an attachments browser to view all attachments for a span in a sidebar * Add support for setting a baseline experiment for experiment comparisons * UI updates to experiment and log tables * Trace audit log now displays granular changes to span data * Start/end columns shown as dates/times * Non-existent trace records display an error message instead of loading indefinitely * Creating an experiment from a playground now correctly renders prompts with `input`, `metadata`, `expected`, and `output` mapped fields * The [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) now includes `x-bt-used-endpoint` as a response header * Add support for deeplinking to comments within spans * In Human Review mode, display all scores in a form * Experiment table rows can now be sorted based on score changes and regressions for each group * The OTEL endpoint now converts attributes under the `braintrust` namespace directly to the corresponding Braintrust fields * New OTEL attributes that accept JSON-serialized values have been added for convenience * Experiment tables and individual traces now support comparing trial data between experiments ### SDK Integrations: LangChain.js (version 0.0.5) * Less noisy logging from the LangChain.js integration * You can now pass a `NOOP_SPAN` to the `BraintrustCallbackHandler` to disable logging * Fixes a bug where the LangChain.js integration could not handle null/undefined values in chain inputs/outputs ### SDK Integrations: LangChain.js (version 0.0.4) * Support logging spans from inside evals in the LangChain.js integration ### SDK (version 0.0.184) * `span.export()` will no longer throw if braintrust is down * Improvement to the Python prompt rendering to correctly render formatted messages, LLM tool calls, and other structured outputs ### SDK (version 0.0.183) * Fix a bug related to `initDataset()` in the Typescript SDK creating links in `Eval()` calls * Fix a few type checking issues in the Python SDK ### SDK (version 0.0.182) * Improved logging for moderation models from the SDK wrappers ### SDK (version 0.0.181) * Add `ReadonlyAttachment.metadata` helper method to fetch a signed URL for downloading the attachment metadata ### SDK (version 0.0.179) * New `hook.expected` for reading and updating expected values in the Eval framework * Small type improvements for `hook` objects * Fixed a bug to enable support for `init_function` with LLM scorers in Python * Support nested attachments in Python * Add support for imports in Python functions pushed to Braintrust via `braintrust push` ### SDK (version 0.0.178) * Cache prompts locally in a two-layered memory/disk cache * Support for using custom functions that are stored in Braintrust in evals * Add support for running traced functions in a `ThreadPoolExecutor` in the Python SDK * Improved formatting of spans logged from the Vercel AI SDK's `generateObject` method * Default to `asyncFlush: true` in the TypeScript SDK ### SDK integrations: LangChain.js (version 0.0.2) * Add support for initializing global LangChain callback handler to avoid manually passing the handler to each LangChain object **December 2024** * Add support for free-form human review scores (written to the `metadata` field) * Add support for structured outputs in the playground * Sparkline charts added to the project home page * Better handling of missing data points in monitor charts * Clicking on monitor charts now opens a link to traces filtered to the selected time range * Add `Endpoint supports streaming` flag to custom provider configuration * Experiments chart can be resized vertically by dragging the bottom of the chart * BTQL sandbox to explore project data using [Braintrust Query Language](https://docs.braintrust.dev/reference/sql.md) * Add support for updating span data from custom span iframes * Significantly speed up loading performance for experiments and logs, especially with lots of spans * Searches inside experiments will only work over content in the tabular view, rather than over the full trace * While searching on the logs page, realtime updates are disabled * Starring rows in experiment and dataset tables now supported * "Order by regression" option in experiment column menu can now be toggled on and off without losing previous order * Add expanded timeline view for traces * Added a 'Request count' chart to the monitor page * Add headers to custom provider configuration which the [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) will include in the request to the custom endpoint * The logs viewer now supports exporting the currently loaded rows as a CSV or JSON file * Experiment columns can now be reordered from the column menu * You can now customize legends in monitor charts ### Autoevals (version 0.0.110) * Python Autoevals now support custom clients when calling evaluators ### SDK (version 0.0.179) * Add support for imports in Python functions pushed to Braintrust via `braintrust push` ### SDK (version 0.0.178) * Cache prompts locally in a two-layered memory/disk cache * Support for using custom functions that are stored in Braintrust in evals * Add support for running traced functions in a `ThreadPoolExecutor` in the Python SDK * Improved formatting of spans logged from the Vercel AI SDK's `generateObject` method * Default to `asyncFlush: true` in the TypeScript SDK ### SDK (version 0.0.177) * Support for creating and pushing custom scorers from your codebase with `braintrust push` ### SDK (version 0.0.176) * New `hook.metadata` for reading and updating Eval metadata when using the `Eval` framework ### SDK (version 0.0.175) * Fix bug with serializing ReadonlyAttachment in logs ### SDK (version 0.0.174) * AI SDK fixes: support for image URLs and properly formatted tool calls so "Try prompt" works in the UI ### SDK (version 0.0.173) * Attachments can now be loaded when iterating an experiment or dataset ### SDK (version 0.0.172) * Fix a bug where `braintrust eval` did not respect certain configuration options, like `base_experiment_id` * Fix a bug where `invoke` in the Python SDK did not properly stream responses ### SDK integrations: LangChain.js (version 0.0.1) * New LangChain.js integration to export traces from `langchainjs` runs **November 2024** * The Traceloop OTEL integration now uses the input and output attributes to populate the corresponding fields in Braintrust * The monitor page now supports querying experiment metrics * Removed the `filters` param from the REST API fetch endpoint * New experiment summary layout option, a url-friendly view for experiment summaries that respects all filters * Add a default limit of 10 to all fetch and `/sql` requests for project\_logs * You can now export your prompts from the playground as code snippets and run them through the [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) * Support for creating and pushing custom Python tools and prompts from your codebase with `braintrust push` * You can now view grouped summary data for all experiments by selecting **Include comparisons in group** from the **Group by** dropdown inside an experiment * The experiments page now supports downloading as CSV/JSON * Downloading or duplicating a dataset in the UI now properly copies all dataset rows * You can now view a score data as a bar chart for your experiments data by selecting **Score comparison** from the X axis selector * Trials information is now shown as a separate column in diff mode in the experiment table * Cmd/Ctrl + S hotkey to save from prompts in the playground and function dialogs * The Braintrust [AI Proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) now supports the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) * Add "Group by" functionality to the monitor page * The experiment table can now be visualized in a [grid layout](https://docs.braintrust.dev/evaluate/interpret-results.md#adjust-table-layout) * 'Select all' button in permission dialogs * Create custom columns on dataset, experiment and logs tables from `JSON` values in `input`, `output`, `expected`, or `metadata` fields * The Braintrust [AI Proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) can now [issue temporary credentials](https://docs.braintrust.dev/deploy/ai-proxy.md#configure-api-keys) to access the proxy for a limited time * Move experiment score summaries to the table column headers * You now receive a clear error message if you run out of free-tier capacity while running an experiment from the playground * Filters on JSON fields now support array indexing, e.g. `metadata.foo[0] = 'bar'` ### SDK (version 0.0.171) * Add a `.data` method to the `Attachment` class, which lets you inspect the loaded attachment data ### SDK (version 0.0.170) * Support uploading [file attachments in the Python SDK](https://docs.braintrust.dev/sdks/python/api-reference.md#attachments) * Log, feedback, and dataset inputs to the Python SDK are now synchronously deep-copied for more consistent logging ### SDK (version 0.0.169) * The Python SDK `Eval()` function has been split into `Eval()` and `EvalAsync()` * Improved type annotations in the Python SDK ### SDK (version 0.0.168) * A new `Span.permalink()` method allows you to format a permalink for the current span * `braintrust push` support for Python tools and prompts * `initDataset()`/`init_dataset()` used in `Eval()` now tracks the dataset ID and links to each row in the dataset properly ### SDK (version 0.0.167) * Support uploading [file attachments in the TypeScript SDK](https://docs.braintrust.dev/instrument/attachments.md) * Log, feedback, and dataset inputs to the TypeScript SDK are now synchronously deep-copied for more consistent logging * Address an issue where the TypeScript SDK could not make connections when running in a Cloudflare Worker **October 2024** * The Monitor page now shows an aggregate view of log scores over time * Improvement/Regression filters between experiments are now saved to the URL * Add `max_concurrency` and `trial_count` to the playground when kicking off evals * Show a button to scroll to a single search result in a span field when using trace search * Indicate spans with errors in the trace span list * After using "Copy to Dataset" to create a new dataset row, the audit log of the new row now links back to the original experiment, log, or other dataset * Tools now stream their `stdout` and `stderr` to the UI * Fix prompt, scorer, and tool dropdowns to only show the correct function types * The [Github action](https://docs.braintrust.dev/evaluate/run-in-ci.md#github-actions) now supports Python runtimes * Add support for [Cerebras](https://cerebras.ai/) models in the proxy, playground, and saved prompts * You can now create [span iframe viewers](https://docs.braintrust.dev/instrument/advanced-tracing.md#customize-span-rendering) to visualize span data in a custom iframe * `NOT LIKE`, `NOT ILIKE`, `NOT INCLUDES`, and `NOT CONTAINS` supported in BTQL * Add "Upload Rows" button to insert rows into an existing dataset from CSV or JSON * Add "Maximum" aggregate score type * The experiment table now supports grouping by input (for trials) or by a metadata field * Gemini models now support multimodal inputs * Preview [file attachments](https://docs.braintrust.dev/instrument/attachments.md) in the trace view * View and filter by comments in the experiment table * Add table row numbers to experiments, logs, and datasets ### SDK (version 0.0.166) * Allow explicitly specifying git metadata info in the Eval framework ### SDK (version 0.0.165) * Support specifying dataset-level metadata in `initDataset/init_dataset` ### SDK (version 0.0.164) * Add `braintrust.permalink` function to create deep links pointing to particular spans in the Braintrust UI ### SDK (version 0.0.163) * Fix Python SDK compatibility with Python 3.8 ### SDK (version 0.0.162) * Fix Python SDK compatibility with Python 3.9 and older ### SDK (version 0.0.161) * Add utility function `spanComponentsToObjectId` for resolving the object ID from an exported span slug **September 2024** * Basic monitor page that shows aggregate values for latency, token count, time to first token, and cost for logs * Create custom tools to use in your prompts and in the playground * Pull your prompts to your codebase using the `braintrust pull` command * Select and compare multiple experiments in the experiment view using the `compared with` dropdown * The playground now displays aggregate scores (avg/max/min) for each prompt and supports sorting rows by a score * Compare span field values side-by-side in the trace viewer when fullscreen and diff mode is enabled * The tag picker now includes tags that were added dynamically via API * You can now create server-side online evaluations for your logs * New member invitations now support being added to multiple permission groups * Move datasets and prompts to a new Library navigation tab, and include a list of custom scorers * Clean up tree view by truncating the root preview and showing a preview of a node only if collapsed * Automatically save changes to table views * You can now upload typescript evals from the command line as functions, and then use them in the playground * Click a span field line to highlight it and pin it to the URL * Copilot tab autocomplete for prompts and data in the Braintrust UI * Basic filter UI (no BTQL necessary) * Add to dataset dropdown now supports adding to datasets across projects * Add REST endpoint for batch-updating ACLs: `/v1/acl/batch_update` * Cmd/Ctrl click on a table row to open it in a new tab * Show the last 5 basic filters in the filter editor * You can now explicitly set and edit prompt slugs * Fixed comment deletion * You can now use `%` in BTQL queries to represent percent values ### Autoevals (version 0.0.86) * Add support for Azure OpenAI in node ### SDK (version 0.0.160) * Fix a bug with `setFetch()` in the TypeScript SDK ### SDK (version 0.0.159) * In Python, running the CLI with `--verbose` now uses the `INFO` log level * Create and push custom tools from your codebase with `braintrust push` * You can now pull prompts to your codebase using the `braintrust pull` command ### SDK (version 0.0.158) * A dedicated `update` method is now available for datasets * Fixed a Python-specific error causing experiments to fail initializing when git diff encounters invalid repositories * Token counts have the correct units when printing `ExperimentSummary` objects ### SDK (version 0.0.157) * Enable the `--bundle` flag for `braintrust eval` in the TypeScript SDK ### SDK (version 0.0.155) * The client wrappers `wrapOpenAI()`/`wrap_openai()` now support [Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs) **August 2024** * You can now create custom LLM and code (TypeScript and Python) evaluators in the playground * Fullscreen trace toggle * Datasets now accept JSON file uploads * When uploading a CSV/JSON file to a dataset, columns/fields named `input`, `expected`, and `metadata` are now auto-assigned to the corresponding dataset fields * Full text search UI for all span contents in a trace * New metrics in the UI and summary API: prompt tokens, completion tokens, total tokens, and LLM duration * Switching organizations via the header navigates to the same-named project in the selected organization * Errors now show up in the trace viewer * New cookbook recipe on [benchmarking LLM providers](https://docs.braintrust.dev/cookbook/recipes/ProviderBenchmark.md) * Viewer mode selections will no longer automatically switch to a non-editable view if the field is editable * Show `%` in diffs instead of `pp` * Add rename, delete and copy current project id actions to the project dropdown * Playgrounds can now be shared publicly * Duration now reflects the "task" duration not the overall test case duration * Duration is now also displayed in the experiment overview table * Add support for Fireworks and Lepton inference providers * "Jump to" menu to quickly navigate between span sections * Speed up queries involving metadata fields using the columnstore backend if it is available * Update to include the latest Mistral models in the proxy/playground * Categorical human review scores can now be re-ordered via Drag-n-Drop * Human review row selection is now a free text field, enabling a quick jump to a specific row ### Autoevals (version 0.0.85) * LLM calls used in autoevals are now marked with `span_attributes.purpose = "scorer"` so they can be excluded from metric and cost calculations ### Autoevals (version 0.0.84) * Fix a bug where `rationale` was incorrectly formatted in Python * Update the `full` docker deployment configuration to bundle the metadata DB (supabase) inside the main docker compose file ### SDK (version 0.0.151) * `Eval()` can now take a base experiment. Provide either `baseExperimentName`/`base_experiment_name` or `baseExperimentId`/`base_experiment_id` ### SDK (version 0.0.148) * While tracing, if your code errors, the error will be logged to the span ### SDK (version 0.0.147) * `project_name` is now `projectName`, etc. in the `invoke(...)` function in TypeScript * `Eval()` return values are printed in a nicer format * [`updateSpan()`/`update_span()`](https://docs.braintrust.dev/instrument/advanced-tracing#update-spans) allows you to update a span's fields after it has been created ### SDK (version 0.0.146) * Add support for `max_concurrency` in the Python SDK * Hill climbing evals that use a `BaseExperiment` as data will use that as the default base experiment **July 2024** * In preparation for auth changes, we are making a series of updates that may affect self-deployed instances * Human review scores are now sortable from the project configuration page * Streaming support for tool calls in Anthropic models through the proxy and playground * The playground now supports different "parsing" modes: `auto`, `parallel`, `raw`, `raw_stream` * Table views [can now be saved](https://docs.braintrust.dev/reference/views), persisting the BTQL filters, sorts, and column state * Add support for the new `window.ai` model into the playground * Use push history when navigating table rows to allow for back button navigation * In the experiments list, grouping by a metadata field will group rows in the table as well * Allow the trace tree panel to be resized * Port the log summary query to BTQL for improved speed * Update the experiment progress and experiment score distribution chart layouts * Format table column headers with icons * Move active filters to the table toolbar * Enable RBAC for all users * Use btql to power the datasets list, making it significantly faster if you have multiple large datasets * Experiments list chart supports click interactions * Jump into comparison view between 2 experiments by selecting them in the table and clicking "Compare" * Add support for labeling [expected fields using human review](https://docs.braintrust.dev/annotate/labels.md#update-expected-values) * Create and edit descriptions for datasets * Create and edit metadata for prompts * Click scores and attributes (tree view only) in the trace view to filter by them * Highlight the experiments graph to filter down the set of experiments * Add support for new models including Claude 3.5 Sonnet * Improved empty state and instructions for custom evaluators in the playground * Show query examples when filtering/sorting * [Custom comparison keys](https://docs.braintrust.dev/evaluate/compare-experiments.md#set-a-comparison-key) for experiments * New model dropdown in the playground/prompt editor that is organized by provider and model type ### Autoevals (version 0.0.80) * New `ExactMatch` scorer for comparing two values for exact equality ### Autoevals (version 0.0.77) * Officially switch the default model to be `gpt-4o` * Support claude models ### Autoevals (version 0.0.76) * New `.partial(...)` syntax to initialize a scorer with partial arguments like `criteria` in `ClosedQA` * Allow messages to be inserted in the middle of a prompt ### SDK (version 0.0.140) * New `wrapTraced` function allows you to trace javascript functions in a more ergonomic way ### SDK (version 0.0.138) * The TypeScript SDK's `Eval()` function now takes a `maxConcurrency` parameter * `braintrust install api` now sets up your API and Proxy URL in your environment * You can now specify a custom `fetch` implementation in the TypeScript SDK ### Deployment * The proxy service now supports more advanced functionality which requires setting the `PG_URL` and `REDIS_URL` parameters **June 2024** * You can now collapse the trace tree. It's auto collapsed if you have a single span * Improvements to the experiment chart including greyed out lines for inactive scores and improved legend * Show diffs when you save a new prompt version * You can now see which users are viewing the same traces as you are in real-time * Improve whitespace and presentation of diffs in the trace view * Show markdown previews in score editor * Show cost in spans and display the average cost on experiment summaries and diff views * Published a new [Text2SQL eval recipe](https://docs.braintrust.dev/cookbook/recipes/Text2SQL-Data.md) * Add groups view for RBAC * Deprecate the legacy dataset format (`output` in place of `expected`) in a new version of the SDK * Improve the UX for saving and updating prompts from the playground * New hide/show column controls on all tables * New [model comparison](https://docs.braintrust.dev/cookbook/recipes/ModelComparison.md) cookbook recipe * Add support for model / metadata comparison on the experiments view * New experiment picker dropdown * Markdown support in the LLM message viewer * Support copying to clipboard from `input`, `output`, etc. views * Improve the empty-state experience for datasets * New multi-dimensional charts on the experiment page for comparing models and model parameters * Support `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` environment variables in the API containers * Support infinite scroll in the logs viewer and remove dataset size limitations * Denser trace view with span durations built in * Rework pagination and fix scrolling across multiple pages in the logs viewer * Make BTQL the default search method * Add support for Bedrock models in the playground and the proxy * Add "copy code" buttons throughout the docs * Automatically overflow large objects (e.g. experiments) to S3 for faster loading and better performance * Show images in LLM view * Send an invite email when you invite a new user to your organization * Support selecting/deselecting scores in the experiment view * Roll out [Braintrust Query Language](https://docs.braintrust.dev/reference/sql.md) (BTQL) for querying logs and traces * Smart relative time labels for dates (`1h ago`, `3d ago`, etc.) * Added double quoted string literals support * Jump to top button in trace details for easier navigation * Fix a race condition in distributed tracing **May 2024** * Incremental support for roles-based access control (RBAC) logic within the API server backend * Changed the semantics of experiment initialization with `update=True` * Added support for new multimodal models * Introduced [REST API for RBAC](https://docs.braintrust.dev/api-reference.md) * Improved AI search and added positive/negative tag filtering in AI search * Added functionality for distributed tracing * Introduce multimodal support for OpenAI and Anthropic models in the prompt playground and proxy * The REST API now gzips responses * You can now return dynamic arrays of scores in `Eval()` functions * Launched Reporters * New coat of paint in the trace view * Added support for Clickhouse as an additional storage backend * Implemented realtime checks using a WebSocket connection * Introduced an API version checker tool * Faster optimistic updates for large writes in the UI * "Open in playground" now opens a lighter weight modal instead of the full playground * Can create a new prompt playground from the prompt viewer * Shipped support for [prompt management](https://docs.braintrust.dev/evaluate/prompts.md) * Moved playground sessions to be within projects * Allowed customizing proxy and real-time URLs through the web application * Improved documentation for Docker deployments * Improved folding behavior in data editors * Support custom models and endpoint configuration for all providers * New add team modal with support for multiple users * New information architecture to enable faster project navigation * Experiment metadata now visible in the experiments table * Improve UI write performance with batching * Log filters now apply to *any* span * Share button for traces * Images now supported in the tree view * Show auto scores before manual scores (matching trace) in the table * New logo is live! * Any span can now submit scores, which automatically average in the trace * Improve sidebar scrolling behavior * Add AI search for datasets and logs * Add tags to the SDK * Support viewing and updating metadata on the experiment page **April 2024** * Add support for tags * Score fields are now sorted alphabetically * Add support for Groq ModuleResolutionKind * Improve tree viewer and XML parser * New experiment page redesign * Support duplicate `Eval` names * Fallback to `BRAINTRUST_API_KEY` if `OPENAI_API_KEY` is not set * Throw an error if you use `experiment.log` and `experiment.start_span` together * Add keyboard shortcuts (j/k/p/n) for navigation * Increased tooltip size and delay for better usability * Support more viewing modes: HTML, Markdown, and Text **March 2024** * Tons of improvements to the prompt playground * Cloudformation now supports more granular RDS configuration * Support optional slider params * Lots of style improvements for tables * Deleting a prompt takes you back to the prompts tab **February 2024** * New [REST API](https://docs.braintrust.dev/api-reference.md) * [Cookbook](https://docs.braintrust.dev/cookbook.md) of common use cases and examples * Support for [custom models](https://docs.braintrust.dev/evaluate/playgrounds.md) in the playground * Search now works across spans, not just top-level traces * Show creator avatars in the prompt playground * Improved UI breadcrumbs and sticky table headers * UI improvements to the playground * Added an example of closed QA / extra fields * New YAML parser and new syntax highlighting colors for data editor * Added support for enabling/disabling certain git fields from collection * Added new GPT-3.5 and 4 models to the playground * Fixed scrolling jitter issue in the playground * Made table fields in the prompt playground sticky **January 2024** * Added ability to download dataset as CSV * Added YAML support for logging and visualizing traces * Added JSON mode in the playground * Added span icons and improved readability * Enabled shift modifier for selecting multiple rows in Tables * Improved tables to allow editing expected fields and moved datasets to trace view * Added ability to manually score results in the experiment UI * Added comments and audit log in the experiment UI * Added ability to upload dataset CSV files in prompt playgrounds * Published new [guide for tracing and logging your code](https://docs.braintrust.dev/instrument/trace-application-logic.md) * Added support to download experiment results as CSVs **December 2023** * Dropped the official 2023 Year-in-Review dashboard * Improved ergonomics for the Python SDK * The `@traced` decorator will automatically log inputs/outputs * You no longer need to use context managers to scope experiments or loggers * Enable skew protection in frontend deploys * Added syntax highlighting in the sidepanel to improve readability * Add `jsonl` mode to the eval CLI to log experiment summaries in an easy-to-parse format * Released new trials feature to rerun each input multiple times * Added ability to run evals in the prompt playground * Added support for Gemini and Mistral Platform in AI proxy and playground * Enabled the prompt playground and datasets for free users * Added Together.ai models including Mixtral to AI Proxy * Turned prompts tab on organization view into a list * Removed data row limit for the prompt playground * Enabled configuration for dark mode and light mode in settings * Added automatic logging of a diff if an experiment is run on a repo with uncommitted changes * API keys are now scoped to organizations * You can now search for experiments by any metadata, including their name, author, or even git metadata * Filters are now saved in URL state so you can share a link to a filtered view * Improve performance of project page by optimizing API calls **November 2023** * Added experiment search on project view to filter by experiment name * Upgraded AI Proxy to support tracking Prometheus metrics * Modified Autoevals library to use the [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md) * Upgraded Python braintrust library to parallelize evals * Optimized experiment diff view for performance improvements * Added support for new Perplexity models to playground * Released [AI proxy](https://docs.braintrust.dev/deploy/ai-proxy.md): access many LLMs using one API w/ caching * Added [load balancing endpoints](https://docs.braintrust.dev/deploy/ai-proxy.md#load-balance-across-providers) to AI proxy * Updated org-level view to show projects and prompt playground sessions * Added ability to batch delete experiments * Added support for Claude 2.1 in playground * Made experiment column resized widths persistent * Fixed our libraries including Autoevals to work with OpenAI's new libraries * Added support for function calling and tools in our prompt playground * Added tabs on a project page for datasets, experiments, etc * Improved selectors for diffing and comparison modes on experiment view * Added support for new OpenAI models (GPT4 preview, 3.5turbo-1106) in playground * Added support for OS models (Mistral, Codellama, Llama2, etc.) in playground using Perplexity's APIs **October 2023** * Improved experiment sidebar to be fully responsive and resizable * Improved tooltips within the web UI * Multiple performance optimizations and bug fixes * Improved prompt playground variable handling and visualization * Added time duration statistics per row to experiment summaries * [Launched new tracing feature: log and visualize complex LLM chains and executions](https://docs.braintrust.dev/instrument/trace-application-logic.md) * Added a new "text-block" prompt type in the playground * Increased default # of rows per page from 10 to 100 for experiments * UI fixes and improvements for the side panel and tooltips * The experiment dashboard can be customized to show the most relevant charts * Performance improvements related to user sessions * All experiment loading HTTP requests are 100-200ms faster * The prompt playground now supports autocomplete * Dataset versions are now displayed on the datasets page * Projects in the summary page are now sorted alphabetically * Long text fields in logged data can be expanded into scrollable blocks **September 2023** * The Eval framework is now supported in Python! * Onboarding and signup flow for new users * Switch product font to Inter * Big performance improvements for registering experiments (down from \~5s to \<1s) * New graph shows aggregate accuracy between experiments for each score * Throw errors in the prompt playground if you reference an invalid variable * A significant backend database change which significantly improves performance while reducing costs * No more record size constraints (previously, strings could be at most 64kb long) * New autoevals for numeric diff and JSON diff * You can duplicate prompt sessions, prompts, and dataset rows in the prompt playground * You can download prompt sessions as JSON files * You can adjust model parameters (e.g. temperature) in the prompt playground * You can publicly share experiments * Datasets now support editing, deleting, adding, and copying rows in the UI **August 2023** * The prompt playground is now live! * A new chart shows experiment progress per score over time * The eval CLI now supports `--watch`, which will automatically re-run your evaluation * You can now edit datasets in the UI * Introducing datasets! You can now upload datasets to Braintrust and use them in your experiments * Fix several performance issues in the SDK and UI * Complex data is now substantially more performant in the UI * The UI updates in real-time as new records are logged to experiments * Ergonomic improvements to the SDK and CLI * The JS library is now Isomorphic and supports both Node.js and the browser * The Evals CLI warns you when no files match the `.eval.[ts|js]` pattern **July 2023** * You can now break down scores by metadata fields * Improve performance for experiment loading (especially complex experiments) * Support for renaming and deleting experiments * When you expand a cell in detail view, the row is now highlighted * A new [framework](https://docs.braintrust.dev/evaluate/run-evaluations.md) for expressing evaluations in a much simpler way * `inputs` is now `input` in the SDK (>= 0.0.23) and UI * Improved diffing behavior for nested arrays * SDK updates that allow you to update an existing experiment `init(..., update=True)` and specify an id in `log(..., id='my-custom-id')` * Tables with lots and lots of columns are now visually more compact in the UI * A new Node.js SDK which mirrors the Python SDK * You can now swap the primary and comparison experiment with a single click * You can now compare `output` vs. `expected` within an experiment * Version 0.0.19 is out for the SDK * Support for real-time updates, using Redis * New settings page that consolidates team, installation, and API key settings * The experiment page now shows commit information for experiments run inside of a git repository **June 2023** * Experiments track their git metadata and automatically find a "base" experiment to compare against * The Python SDK's `summarize()` method now returns an `ExperimentSummary` object with score differences * Organizations can now be "multi-tenant" * New scatter plot and histogram insights to quickly analyze scores and filter down examples * API keys that can be set in the SDK and do not require user login * Improved performance for event logging in the SDK * Auto-merge experiment fields with different types * Tutorial guide + notebook * Automatically refresh cognito tokens in the Python client * New filter and sort operators on the experiments table * SQL query explorer to run arbitrary queries against one or more experiments This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.