> ## Documentation Index > Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt > Use this file to discover all available pages before exploring further. # API reference > Functions and configuration options found in the Braintrust Go SDK. This page covers the key APIs in the Braintrust Go SDK. For setup, see the [Quickstart](https://docs.braintrust.dev/sdks/go/quickstart.md). For the complete reference, see [pkg.go.dev](https://pkg.go.dev/github.com/braintrustdata/braintrust-sdk-go). ## Tracing Tracing records what your application does as spans you can inspect in Braintrust. The recommended way to capture AI calls is auto-instrumentation: use the `trace/contrib` packages to instrument supported provider libraries, either at build time with Orchestrion or with runtime middleware (see [Go SDK integrations](https://docs.braintrust.dev/sdks/go/sdk-integrations.md)). Tracing is built on OpenTelemetry, so you trace your own code with the standard OpenTelemetry API. The APIs below create the client, trace your own code, and link to your traces. ### `braintrust.New` Creates a Braintrust client and configures the OpenTelemetry pipeline that exports spans to Braintrust. Call it once on startup, passing your `TracerProvider` and any [options](#configuration). ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import ( "github.com/braintrustdata/braintrust-sdk-go" "go.opentelemetry.io/otel" "go.opentelemetry.io/otel/sdk/trace" ) tp := trace.NewTracerProvider() otel.SetTracerProvider(tp) client, err := braintrust.New(tp, braintrust.WithProject("My project")) if err != nil { log.Fatal(err) } ``` Returns: `(*braintrust.Client, error)`. `braintrust.New` reads `BRAINTRUST_API_KEY` from the environment. Configure the rest with functional options or environment variables (see [Configuration](#configuration)). Because tracing is built on OpenTelemetry, you trace your own application code with the standard OpenTelemetry API, and traced AI calls nest under your spans. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} ctx, span := otel.Tracer("my-app").Start(ctx, "process-request") defer span.End() ``` ### `Client.Permalink` Builds a Braintrust UI URL for a span, so you can link straight to a trace from your own logs or app. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} url := client.Permalink(span) ``` Returns: `string`. ### `config.SpanCustomizer` Span customizers modify each span after it ends and before the SDK exports it to Braintrust. Register them with `braintrust.WithSpanCustomizers`. They run in registration order on every exported span, including spans you create with the OpenTelemetry API. See [Span customizers](https://docs.braintrust.dev/sdks/go/redact-sensitive-data.md) for examples and behavior. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import ( sdktrace "go.opentelemetry.io/otel/sdk/trace" "github.com/braintrustdata/braintrust-sdk-go" "github.com/braintrustdata/braintrust-sdk-go/config" ) client, err := braintrust.New(tp, braintrust.WithSpanCustomizers(config.SpanCustomizer{ OnSpanExport: func(span sdktrace.ReadOnlySpan) (sdktrace.ReadOnlySpan, error) { return span, nil }, }), ) ``` `config.SpanCustomizer` fields: * **`OnSpanExport`** (`func(span sdktrace.ReadOnlySpan) (sdktrace.ReadOnlySpan, error)`): receives a completed span and returns it or a replacement that embeds `sdktrace.ReadOnlySpan`. Key rules: * Don't change the trace ID, span ID, parent trace ID, or parent span ID. Attributes can change. * If a customizer returns an error, panics, returns `nil`, or changes an ID, the SDK skips the remaining customizers for that span and exports it with its content stripped. Other spans in the batch export normally. See [When a customizer fails](https://docs.braintrust.dev/sdks/go/redact-sensitive-data.md#when-a-customizer-fails). * Customizers run before the SDK converts and uploads [attachments](https://docs.braintrust.dev/instrument/attachments.md), so removing inline base64 data keeps the file out of Braintrust. * The SDK can call customizers from background goroutines, so make them safe for concurrent use. ## Evaluations An evaluation runs your task over a set of cases, scores each output, and logs the results to an experiment, which is how you measure quality and catch regressions as you change prompts or models. The recommended pattern is to define an eval once with `braintrust.NewEval`, then call `Run` with any dataset. The same definition works for local runs, `bt eval ` runs from the command line, and [remote eval](https://docs.braintrust.dev/evaluate/remote-evals.md) runs triggered from the playground. ### `braintrust.NewEval` Creates a runnable `*eval.Eval` by combining a client with an eval definition. Call `Run` on it to execute the evaluation. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import ( "context" "github.com/braintrustdata/braintrust-sdk-go" "github.com/braintrustdata/braintrust-sdk-go/eval" ) e := braintrust.NewEval(client, &eval.Eval[string, string]{ Name: "classify", Task: eval.T(func(ctx context.Context, input string) (string, error) { return classify(input), nil }), Scorers: []eval.Scorer[string, string]{exactMatch}, ProjectName: "my-project", }) _, err := e.Run(ctx, eval.RunOpts[string, string]{ Dataset: eval.NewDataset([]eval.Case[string, string]{ {Input: "apple", Expected: "fruit"}, }), }) ``` Returns: `*eval.Eval[I, R]`. `Run` returns `(*eval.Result, error)`. `eval.Eval[I, R]` fields: * **`Name`** (`string`, required): eval name, used as the default experiment name and as the registration key for remote evals. * **`Task`** (`eval.TaskFunc[I, R]`, required): the function under evaluation. Wrap a plain `func(ctx, input) (output, error)` with `eval.T`, or use `eval.TaskWithHooks` if the task needs to read parameters or other run metadata. * **`Scorers`** (`[]eval.Scorer[I, R]`, required): scoring functions applied to each case. To use classifiers, use [`braintrust.NewEvaluator`](#braintrust-newevaluator) instead. * **`ParameterSchema`** (`eval.ParameterSchema`): declares parameters the eval accepts. Each entry becomes a configurable control in the Braintrust playground (see [Parameters](#parameters)). * **`Dataset`** (`eval.Dataset[I, R]`): default dataset for the eval. A playground run supplies its own dataset and overrides this. `bt eval ` and a direct `Run` call use it when no dataset is supplied at call time. * **`ProjectName`** (`string`): Braintrust project for this eval. Falls back to the client's configured project. `eval.RunOpts[I, R]` fields (passed to `Run`): * **`Dataset`** (`eval.Dataset[I, R]`): test cases for this run. Defaults to `Eval.Dataset`; required if the eval defines none. * **`Experiment`** (`string`): experiment name. Defaults to `Eval.Name`. * **`ProjectName`** (`string`): overrides the project for this run. * **`ProjectID`** (`string`): identifies the project by ID instead of by name. Takes precedence over `ProjectName` when set. * **`Tags`** (`[]string`): tags to apply to the experiment. * **`Metadata`** (`eval.Metadata`): metadata to attach to the experiment. * **`Update`** (`bool`): append to an existing experiment with the same name. Defaults to `false`. * **`Parallelism`** (`int`): number of goroutines. Defaults to `1`. * **`Quiet`** (`bool`): suppress result output. Defaults to `false`. * **`Parameters`** (`eval.Parameters`): parameter values for this run, merged over the defaults in `Eval.ParameterSchema` and passed to the task through `hooks.Parameters`. When unset, the task gets the declared defaults. * **`NoSendLogs`** (`bool`): run the eval locally without sending anything to Braintrust. Cases still run and are scored, but no experiment is created and no eval spans are uploaded, so the result has no experiment ID or permalink. Defaults to `false`. Requires v0.17.2 or later. ### `braintrust.NewEvaluator` Creates an evaluator for input type `I` and result type `R`, bound to a client. Call `Run` on it with the cases, task, and scorers to execute the evaluation and log an experiment. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import ( "context" "github.com/braintrustdata/braintrust-sdk-go" "github.com/braintrustdata/braintrust-sdk-go/eval" ) evaluator := braintrust.NewEvaluator[string, string](client) _, err := evaluator.Run(context.Background(), eval.Opts[string, string]{ Experiment: "answers-v1", Dataset: eval.NewDataset([]eval.Case[string, string]{ {Input: "How do I reset my password?", Expected: "Use the account recovery flow."}, {Input: "How do I export my data?", Expected: "Open Settings and choose Export."}, }), Task: eval.T(answerQuestion), Scorers: []eval.Scorer[string, string]{ eval.NewScorer("exact_match", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) { v := 0.0 if r.Output == r.Expected { v = 1.0 } return eval.S(v), nil }), }, }) ``` Returns: `*eval.Evaluator[I, R]`. `Run` returns `(*eval.Result, error)`. `eval.Opts[I, R]` fields: * **`Experiment`** (`string`, required): experiment name. * **`Dataset`** (`eval.Dataset[I, R]`, required): the cases to run. Build an in-memory one with `eval.NewDataset` (see [Datasets](#datasets)). * **`Task`** (`eval.TaskFunc[I, R]`, required): the function under test. Wrap a plain `func(ctx, input) (output, error)` with `eval.T`. * **`Scorers`** (`[]eval.Scorer[I, R]`): scorers to apply to each case. Provide `Scorers`, `Classifiers`, or both. * **`Classifiers`** (`[]eval.Classifier[I, R]`): classifiers to apply to each case. Provide `Scorers`, `Classifiers`, or both. * **`ProjectName`** (`string`): project to log to. Defaults to the client's configured project. * **`ProjectID`** (`string`): project ID. When set, takes precedence over `ProjectName` and uses the project as-is rather than creating it. * **`Tags`** (`[]string`): tags to apply to the experiment. * **`Metadata`** (`eval.Metadata`): metadata to attach to the experiment. * **`Update`** (`bool`): append to an existing experiment with the same name. Defaults to `false`. * **`Parallelism`** (`int`): number of goroutines. Defaults to `1`. * **`TrialCount`** (`int`): number of times to run each case. Defaults to `1`. * **`Quiet`** (`bool`): suppress result output. Defaults to `false`. * **`NoSendLogs`** (`bool`): run the eval locally without sending anything to Braintrust. Cases still run and are scored, but no experiment is created and no eval spans are uploaded, so the result has no experiment ID or permalink. Defaults to `false`. Requires v0.17.2 or later. ### `eval.NewScorer` Creates a scorer from a function. A scorer measures how good the task's output is, returning one or more named scores per case. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} scorer := eval.NewScorer("exact_match", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Scores, error) { if r.Output == r.Expected { return eval.S(1.0), nil } return eval.S(0.0), nil }) ``` Returns: `eval.Scorer[I, R]`. The score function receives an `eval.TaskResult[I, R]` (with `Input`, `Output`, `Expected`, and `Metadata`) and returns `eval.Scores`. Use `eval.S` to build a single score. ### `eval.NewClassifier` Creates a classifier from a function. Use a classifier to categorize output instead of scoring it numerically. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} classifier := eval.NewClassifier("topic", func(_ context.Context, r eval.TaskResult[string, string]) (eval.Classifications, error) { return eval.Classifications{{ID: "billing", Label: "Billing"}}, nil }) ``` Returns: `eval.Classifier[I, R]`. ### `eval.TaskWithHooks` Creates a task function that receives `*eval.TaskHooks`, which gives access to parameters, metadata, tags, and the current spans. Use it when your task needs to read parameter values that were configured in the playground or passed via `RunOpts.Parameters`. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} task := eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) { model := hooks.Parameters.String("model") return classify(input, model), nil }) ``` Returns: `eval.TaskFunc[I, R]`. The hooks give access to `Parameters`, `Metadata`, `Tags`, `TrialIndex`, `TaskSpan`, and `EvalSpan`. Use `eval.T` instead when the task doesn't need hooks. ## Workflow evaluations A workflow evaluation is an evaluation whose task or scorers submit requests to an asynchronous provider API, such as the [OpenAI Batch API](https://developers.openai.com/api/docs/guides/batch), and collect the results later. The run saves its progress to a store, so your application can resume it by polling or from a webhook. With a shared store such as Redis, a different process can resume it. See [Workflow evaluations](https://docs.braintrust.dev/sdks/go/workflow-evaluations.md) for a complete OpenAI example. > [!WARNING] > > Go workflow evaluations are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. > [!NOTE] > > Requires Go SDK v0.18.0 or later. The Go SDK saves run state in its own format, so start and resume a run with the Go SDK. ### `braintrust.NewWorkflowEval` Attaches a client to an `*eval.WorkflowEval` definition, so its runs create an experiment and log results to Braintrust. Every process that starts or resumes a run that logs to Braintrust must use a definition attached this way. Returns: `*eval.WorkflowEval[I, R]`. Defining an evaluation does not start a run. To start one, call [`Start`](#workflow-eval-methods). `eval.WorkflowEval[I, R]` fields: * **`Name`** (`string`, required): evaluation name, used as the default experiment name and to identify the run's records in the store. * **`Store`** (`eval.WorkflowEvalStore`, required): persistence backend shared by every process that starts or resumes the run. See [Workflow stores](#workflow-stores). * **`Task`** (`eval.WorkflowTaskDefinition[I, R]`, required): an ordinary `eval.TaskFunc[I, R]` such as one built with `eval.T`, an [`eval.WorkflowTask`](#eval-workflowtask-and-eval-workflowscorer), or an [`eval.WorkflowBatchTask`](#eval-workflowbatchtask-and-eval-workflowbatchscorer). Task outputs are saved to the store, so they must be JSON-serializable. An ordinary task that returns `eval.TaskOutput` with `UserData` fails for that case. Use metadata instead. * **`Scorers`** (`[]eval.Scorer[I, R]`): ordinary scorers, which run in your process after a case's task output is collected. * **`WorkflowScorers`** (`[]eval.WorkflowScorerDefinition[I, R]`): asynchronous scorers. Each is an `eval.WorkflowScorer` or an `eval.WorkflowBatchScorer`. * **`Classifiers`** (`[]eval.Classifier[I, R]`): classifiers, which run in your process after a case's task output is collected. * **`ParameterSchema`** (`eval.ParameterSchema`): declares parameters the evaluation accepts (see [Parameters](#parameters)). Pass runtime values to `Start`. * **`ProjectName`** (`string`): Braintrust project for this evaluation. Falls back to the client's configured project. * **`MaxConcurrency`** (`int`): how many of your callbacks the SDK runs at the same time during each `Start`, `Poll`, or `ProcessSubmissionResult` call. Defaults to `10`. Must not be negative. Scorer and classifier names must be nonempty and unique across `Scorers`, `WorkflowScorers`, and `Classifiers`, and can't be `task`. Keep the definition, including scorer names and order, unchanged while runs are active. ### `WorkflowEval` methods Each method returns `(*eval.WorkflowResult, error)` with the run's current state. * **`Start(ctx, opts eval.WorkflowStartOpts[I, R])`**: creates the experiment, saves the dataset to the store, and submits the initial task work. It returns without waiting for the provider. With an ordinary `eval.TaskFunc` task, `Start` runs the task and the ordinary scorers and classifiers for every case before it returns. If a callback fails after the run is saved, `Start` still returns the run ID. Resume that run with `Poll` instead of starting a second one. * **`Poll(ctx, runID string)`**: checks each pending poll-mode submission once, collects the completed ones, and advances the run. It also retries submissions that failed to submit. Call it on a schedule until the run completes. * **`ProcessSubmissionResult(ctx, ref eval.WorkflowSubmissionResult)`**: collects one completed submission and advances its run. Call it after your application verifies a provider webhook. Repeated notifications for the same submission are harmless. * **`Status(ctx, runID string)`**: reads the run's progress without submitting, polling, or collecting. Serialize calls that advance the same run, including across processes. The store operations are atomic, but the SDK doesn't provide a distributed lock. Different runs can advance independently. `eval.WorkflowStartOpts[I, R]` fields: * **`Dataset`** (`eval.Dataset[I, R]`, required): test cases for this run. Inputs, expected values, and metadata must be JSON-serializable. * **`Experiment`** (`string`): experiment name. Defaults to `WorkflowEval.Name`. * **`ProjectID`** (`string`): identifies the project by ID instead of by name. * **`Tags`** (`[]string`): tags to apply to the experiment. * **`Metadata`** (`eval.Metadata`): metadata to attach to the experiment. * **`TrialCount`** (`int`): number of times to run each case. Defaults to `1`. A case's `TrialCount` overrides this value. * **`Parameters`** (`eval.Parameters`): parameter values for this run, merged over the defaults in `ParameterSchema`. * **`NoSendLogs`** (`bool`): run without creating an experiment or logging results to Braintrust, so no client is required. Run state is still saved to the store. Tracing that your own callbacks do is not disabled. `eval.WorkflowSubmissionResult` fields: * **`ExternalID`** (`string`): the provider's ID for the completed operation, as returned by `GetExternalID`. Sufficient on its own unless the same ID belongs to several runs. In that case, also set `RunID`. * **`RunID`** (`string`): the run that owns the submission. * **`SubmissionID`** (`string`): the SDK's submission ID. Requires `RunID`. Use it instead of `ExternalID`. `eval.WorkflowResult` fields: * **`RunID`** (`string`): run ID to pass to `Poll`, `Status`, or `ProcessSubmissionResult`. * **`Status`** (`eval.WorkflowRunStatus`): `eval.WorkflowRunWaiting` (`"waiting"`) while cases or submissions are outstanding, and `eval.WorkflowRunCompleted` (`"completed"`) once every case has finished, including failed cases. * **`Completed`** (`int`): number of finished cases, including failed cases. * **`Failed`** (`int`): number of cases with a failed task, scorer, or classifier. A completed run can have failures. * **`PendingPoll`** and **`PendingWebhook`** (`int`): number of submitted operations not yet collected, by completion mode. * **`Scores`** (`map[string]float64`): average of the successful values for each score name. * **`ExperimentID`** and **`Permalink`** (`string`): the experiment and its URL. Empty for `NoSendLogs` runs. Each trial is logged as its own experiment row. The row's `metadata.workflow_eval` records `run_id`, `case_id`, and `trial_index`, plus `batch_submission_id` when an `eval.WorkflowBatchTask` produced the output. `case_id` is the dataset case's `ID` when set, otherwise an ID generated when the run starts. A failed task's error appears in the row's `error` field, and failed scorers and classifiers appear in `metadata.scorer_errors` and `metadata.classifier_errors`. ### `eval.WorkflowTask` and `eval.WorkflowScorer` `eval.WorkflowTask` and `eval.WorkflowScorer` each submit one asynchronous provider operation for each case and trial. Use `eval.WorkflowTask` for the task and `eval.WorkflowScorer` for a scorer. `eval.WorkflowTask` and `eval.WorkflowScorer` fields: * **`Name`** (`string`, required, `WorkflowScorer` only): scorer name. * **`Submit`** (required): sends the request to the provider and returns a `json.RawMessage` handle, such as the provider's job ID, that later callbacks receive. The handle must be valid JSON and must not contain clients or credentials. A task's `Submit` receives an `eval.WorkflowTaskItem[I, R]`, and a scorer's receives an `eval.WorkflowScorerItem[I, R]`. * **`Completion`** (`eval.WorkflowCompletion`, required): how the SDK learns the operation finished. See [`eval.WorkflowCompletion`](#eval-workflowcompletion). * **`Collect`** (required): reads the finished operation from its handle. A task's `Collect` returns an `eval.WorkflowTaskResult[R]`, and a scorer's returns `eval.Scores`. If `Collect` returns an error, the case stays pending. The SDK calls `Collect` again on the next `Poll`, or for webhook completion, on the next `ProcessSubmissionResult` call for that submission. Every callback also receives an `eval.WorkflowSubmissionContext` with `RunID` and `SubmissionID`. `SubmissionID` is stable across retries, so use it as the provider's idempotency key. A crash after the provider accepts a request but before the SDK saves the handle can call `Submit` again. `eval.WorkflowTaskItem[I, R]` fields: `ID` (unique for each trial), `CaseID` (shared by a case's trials), `Input`, `Expected`, `Metadata`, `Tags`, `Parameters`, and `TrialIndex`. `eval.WorkflowTaskResult[R]` fields: `Output`, and optional `Metadata` and `Tags`. `Tags` replaces the case's tags. `eval.WorkflowScorerItem[I, R]` embeds `eval.TaskResult[I, R]` (with `Input`, `Output`, `Expected`, and `Metadata`) and adds `ID`, `CaseID`, `Tags`, and `TrialIndex`. ### `eval.WorkflowBatchTask` and `eval.WorkflowBatchScorer` `eval.WorkflowBatchTask` and `eval.WorkflowBatchScorer` group many cases into one provider operation, such as one OpenAI batch. They have the same `Name` (scorer only) and `Completion` fields as `eval.WorkflowTask` and `eval.WorkflowScorer`, plus: * **`Batching`** (`eval.WorkflowBatching`, required): batch size limits. See below. * **`Submit`** (required): receives a slice of `eval.WorkflowBatchItem` values, each with a `CustomID` and the `Item`. Send `CustomID` as the provider's per-request ID, such as OpenAI's `custom_id`. Treat it as opaque. * **`Collect`** (required): reads the finished operation and calls its `yield` function once for each result with an `eval.WorkflowBatchResult` (`CustomID`, `Result`, and a per-item `Error`). Return immediately when `yield` returns an error. `eval.WorkflowBatching` fields: * **`MaxSize`** (`int`, required): maximum number of cases in a batch. Must be positive. * **`MaxWait`** (`time.Duration`): batch scorers only. How long a partial batch can wait before it's submitted. The SDK checks it only when the run advances, with no background timer. Defaults to `0`, which waits for a full batch or for the last remaining cases. A batch task submits immediately and rejects a nonzero `MaxWait`. Collection is checkpointed for each result, so a failed `Collect` can run again, and results already saved are ignored. A duplicate or unknown `CustomID` is an error. Cases with no result when `Collect` succeeds are recorded as failed. Failed tasks skip their scorers and classifiers, and a failed scorer keeps the case's other scores. ### `eval.WorkflowCompletion` Selects how the SDK learns that a submission finished. Set exactly one field: * **`Poll`**: checks the provider and returns an `eval.WorkflowPollResult` with a `Status` of `eval.WorkflowPending`, `eval.WorkflowComplete`, or `eval.WorkflowFailed`, and an optional `Error`. `WorkflowEval.Poll` calls it. * **`GetExternalID`**: returns the provider's ID for the submission, which the provider's webhook later delivers. Pass that ID to `ProcessSubmissionResult` as `ExternalID`. > [!WARNING] > > A submission that reports `eval.WorkflowFailed` stays pending, and `Poll` returns its error. `Poll` checks the submission again on every later call, so a permanent provider failure makes every later `Poll` return an error, and the run never completes. Start a new run to recover. To finish a run despite a failed provider batch, report `eval.WorkflowComplete` instead and omit the missing results from `Collect`, so those cases are recorded as failed. ### Workflow stores A store persists the run's dataset, outputs, submission handles, and progress. Use the same store and key prefix for every call that starts or resumes a run. * **`eval.NewWorkflowEvalMemoryStore()`** → `*eval.WorkflowEvalMemoryStore`: keeps state in memory for single-process runs. State is lost when the process exits. * **`eval.NewWorkflowEvalRedisStore(command eval.WorkflowRedisCommand, prefix string, ttl time.Duration)`** → `(*eval.WorkflowEvalRedisStore, error)`: stores state in Redis. `command` runs a Redis command and returns its decoded result, so the SDK doesn't depend on a Redis client. With go-redis, wrap `client.Do(ctx, args...).Result()` and return `nil, nil` for `redis.Nil`. The command must support `EVAL`. An empty `prefix` defaults to `braintrust-eval:`, and a zero `ttl` defaults to seven days. Writes refresh the TTL and reads don't, so set it longer than the full run and your webhook retry window. An expired run can't be resumed. For another backend, implement the `eval.WorkflowEvalStore` interface: `Read`, `Write`, `GetOrSet`, `AddToSet`, and `GetSetSize`. `Read` returns `nil, nil` for a missing key, and `GetOrSet` and `AddToSet` must be atomic. ## Parameters Parameters let you declare configurable options on an eval. When the eval runs from the Braintrust playground via a [remote eval](https://docs.braintrust.dev/evaluate/remote-evals.md), each declared parameter becomes a control in the UI. For a local run, the task receives the declared defaults. Declare parameters with `eval.ParameterSchema` on the `eval.Eval` definition: ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} e := braintrust.NewEval(client, &eval.Eval[string, string]{ Name: "classify", ParameterSchema: eval.ParameterSchema{ "model": { Type: eval.ParameterTypeModel, Default: "gpt-5-mini", Description: "Model to use for classification", }, "threshold": { Type: "number", Default: 0.5, }, }, Task: eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) { model := hooks.Parameters.String("model") threshold := hooks.Parameters.Float64("threshold") return classify(input, model, threshold), nil }), // ... }) ``` `eval.ParameterSchema` is `map[string]eval.ParameterDef`. Each `eval.ParameterDef` has: * **`Type`** (`string`): `"model"` renders a model picker. `"prompt"` renders a prompt editor. `"string"`, `"number"`, `"integer"`, `"boolean"` render plain inputs. Use the `eval.ParameterTypeModel` and `eval.ParameterTypePrompt` constants for the two dedicated controls. * **`Default`** (`any`): used when no value is supplied for this run. This is also what a local run sees. * **`Description`** (`string`): shown alongside the control in the playground. `eval.Parameters` (the resolved values, available as `hooks.Parameters`) has typed accessors that never panic on a type mismatch: * **`Parameters.String(name)`** → `string` * **`Parameters.Int(name)`** → `int` * **`Parameters.Float64(name)`** → `float64` * **`Parameters.Bool(name)`** → `bool` * **`Parameters.Prompt(name)`** → `(*prompt.Prompt, bool)`: returns the value for `name` as a renderable prompt, and whether one was found. Use this for `"prompt"` parameters. Returns `false` when the parameter is absent or cannot be converted. * **`Parameters.Get(name)`** → `(any, bool)` * **`Parameters.Has(name)`** → `bool` ## Remote evals Remote evals let the Braintrust playground trigger your Go eval code on your own infrastructure. The `evalrunner` package turns a Go binary into a target that the [`bt` CLI](https://docs.braintrust.dev/reference/cli/eval) can drive. > [!WARNING] > > Go remote evals are in [public preview](https://docs.braintrust.dev/feature-lifecycle.md) and can change before reaching general availability. ### `evalrunner.New` and `evalrunner.RegisterEval` Create a runner and register your evals. The runner reads the `bt` environment variables, dispatches the right eval, and streams results back. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} package main import ( "context" "github.com/braintrustdata/braintrust-sdk-go/eval" "github.com/braintrustdata/braintrust-sdk-go/evalrunner" ) func main() { r := evalrunner.New() evalrunner.RegisterEval(r, &eval.Eval[string, string]{ Name: "classify", Task: eval.TaskWithHooks(func(ctx context.Context, input string, hooks *eval.TaskHooks) (string, error) { model := hooks.Parameters.String("model") return classify(input, model), nil }), Scorers: []eval.Scorer[string, string]{exactMatch}, ParameterSchema: eval.ParameterSchema{ "model": {Type: eval.ParameterTypeModel, Default: "gpt-5-mini"}, }, ProjectName: "my-project", }) evalrunner.Main(r) } ``` Then run from the command line: ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} # Run all evals locally bt eval ./cmd/evals # Run all evals without logging in or sending results to Braintrust (v0.17.2 or later) bt eval --no-send-logs ./cmd/evals # Start the dev server so the playground can trigger runs bt eval --dev --language go ./cmd/evals ``` `evalrunner.New` accepts `evalrunner.Option` values: * **`evalrunner.WithLogger(l logger.Logger)`**: custom logger. It must write to stderr, because stdout is reserved for the `bt` protocol. The default logger writes to stderr. * **`evalrunner.WithTracerProvider(tp *sdktrace.TracerProvider)`**: shared OpenTelemetry provider. Supply one when instrumented code (LLM clients, custom spans) should appear in the same trace as eval spans. When nil, a per-run provider is created and shut down on exit. `evalrunner.RegisterEval[I, R any](r *Runner, ev *eval.Eval[I, R])` registers an eval by its `Name`. Registering two evals under the same name replaces the first. `evalrunner.Main(r *Runner)` dispatches and exits. Use `evalrunner.Run(ctx, r)` instead if you need to handle the error yourself. `r.Mode()` returns the current `evalrunner.Mode`: `ModeList`, `ModeEval`, `ModeBatch`, `ModeInspect`, or `ModeUnknown`. `ModeUnknown` means `bt` requested a mode the runner doesn't recognize, and `evalrunner.Run` returns an error. Use it to skip expensive setup when `bt` only wants the list of registered evals: ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} if r.Mode() != evalrunner.ModeList { warmCaches() } ``` ## Prompts Prompts saved in Braintrust carry a template, a model, and parameters. The Go SDK loads them from the API, renders their variables, and returns a provider-agnostic result you hand to any LLM client. A prompt is fetched every time. Nothing is cached. ### `client.LoadPrompt` Loads a prompt from Braintrust by slug and returns it ready to render. The client's configured project is used when no project is specified in `opts`. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import "github.com/braintrustdata/braintrust-sdk-go/prompt" p, err := bt.LoadPrompt(ctx, prompt.LoadOpts{Slug: "summarizer"}) if err != nil { return err } built, err := p.Build(map[string]any{"input": article}) ``` Returns: `(*prompt.Prompt, error)`. `prompt.LoadOpts` fields: * **`Slug`** (`string`): prompt slug. Required unless `ID` is set. * **`Project`** (`string`): project name. Filled in from the client's project when empty. * **`ProjectID`** (`string`): project ID. Takes precedence over `Project` when set. * **`ID`** (`string`): load by Braintrust prompt ID instead of slug. * **`Version`** (`string`): pin to a specific prompt version. Defaults to the latest. * **`Environment`** (`string`): load the prompt deployed to an environment (for example, `"production"`). Ignored when `Version` is set. For lower-level access, use `prompt.LoadWithClient(ctx, apiClient, opts)` with an `*openapi.Client` from [`openapi.NewClient`](#api-client), configured with a base URL and `Authorization` header (v0.19.0 or later). `prompt.Load`, which takes an `*api.API`, is deprecated but remains available for existing code. ### `prompt.Definition` Declares a prompt in Go code. Use it as the `Default` for a prompt eval parameter: ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import "github.com/braintrustdata/braintrust-sdk-go/prompt" eval.ParameterSchema{ "summary_prompt": { Type: eval.ParameterTypePrompt, Default: prompt.Definition{ Model: "gpt-5-mini", Messages: []prompt.Message{ prompt.System("You summarize articles in one sentence."), prompt.User("Summarize this:\n\n{{input}}"), }, Params: map[string]any{"temperature": 0}, }, }, } ``` `prompt.Definition` fields: * **`Model`** (`string`, required): model the prompt targets. * **`Messages`** (`[]prompt.Message`): chat messages. Mutually exclusive with `Prompt`. * **`Prompt`** (`string`): completion template. Mutually exclusive with `Messages`. * **`Params`** (`map[string]any`): model parameters such as `temperature` or `max_tokens`. * **`Tools`** (`[]prompt.Tool`): tool definitions offered to the model. Chat prompts only. * **`TemplateFormat`** (`string`): `"mustache"` (default) or `"none"`. Helper constructors for `prompt.Message`: `prompt.System(text)`, `prompt.User(text)`, `prompt.Assistant(text)`. ### `prompt.Prompt.Build` Renders the prompt's template variables and returns a `*prompt.Built` ready to send to a model. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} built, err := p.Build(map[string]any{"input": article}) if err != nil { return err } // built is provider-agnostic. built.Map() returns an OpenAI-shaped map you can // send to any client: body, err := json.Marshal(built.Map()) ``` Returns: `(*prompt.Built, error)`. Build fails if the prompt has no body or no model, the template is malformed, or the format is not supported (`"nunjucks"` is not rendered locally). For the OpenAI Go client, `traceopenai.ChatCompletionParams(built)` (from `github.com/braintrustdata/braintrust-sdk-go/trace/contrib/openai`) converts a chat prompt directly into `openai.ChatCompletionNewParams`. `prompt.Built` fields: * **`Model`** (`string`): model to call. * **`Messages`** (`[]prompt.Message`): rendered chat messages, for a chat prompt. * **`Prompt`** (`string`): rendered completion text, for a completion prompt. * **`Params`** (`map[string]any`): model parameters. * **`Tools`** (`[]prompt.Tool`): rendered tool definitions. * **`Metadata`** (`*prompt.Metadata`): prompt identity for tracing. Nil when the prompt has no Braintrust identity. `built.Map()` returns the built prompt as an OpenAI-shaped `map[string]any` (model, messages or prompt, tools, and every model parameter at the top level) for clients this SDK has no adapter for. `Build` options: * **`prompt.WithDefaults(defaults map[string]any)`**: default model parameters, used for any parameter the prompt does not set. * **`prompt.WithTemplateFormat(format string)`**: override the template format saved on the prompt. * **`prompt.WithStrict()`**: fail when the template references a variable that was not supplied. By default a missing variable renders as an empty string. ### `built.AnnotateSpan` Records the prompt's identity and rendered variables on a span, linking the model call back to the prompt in Braintrust. Call it before ending the span that covers the model call. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} built, err := p.Build(map[string]any{"input": input}) if err != nil { return err } built.AnnotateSpan(hooks.TaskSpan) ``` Does nothing when `Metadata` is nil (the prompt has no Braintrust identity) or the span is not recording. ## Datasets A dataset is the set of cases an evaluation runs against. Define cases inline in memory, or manage datasets in Braintrust through the [API client](#api-client). ### `eval.NewDataset` Groups cases into an in-memory dataset you pass to `Evaluator.Run`, as an alternative to loading one from Braintrust. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} dataset := eval.NewDataset([]eval.Case[string, string]{ {Input: "How do I reset my password?", Expected: "Use the account recovery flow."}, {Input: "How do I export my data?", Expected: "Open Settings and choose Export."}, }) ``` Returns: `eval.Dataset[I, R]`. Each `eval.Case[I, R]` has an `Input` and optional `Expected`, `Tags`, `Metadata`, and `TrialCount`. ## Attachments When your traces involve binary content like images or PDFs, log it as an attachment so it appears in Braintrust instead of as an opaque blob. When you trace AI calls, Braintrust automatically converts base64 attachments in provider messages into uploaded attachments, so you rarely need the APIs below for instrumented calls. Reach for them when you're attaching binary content to a span yourself. ### `attachment.From*` Creates an attachment from bytes, a file, or a URL. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import "github.com/braintrustdata/braintrust-sdk-go/trace/attachment" att, err := attachment.FromFile("image/png", "chart.png") ``` Constructors: * **`attachment.FromBytes(contentType string, data []byte)`** → `*attachment.Attachment`: from raw bytes. * **`attachment.FromFile(contentType string, path string)`** → `(*attachment.Attachment, error)`: reads a file. * **`attachment.FromURL(url string)`** → `(*attachment.Attachment, error)`: fetches a URL and uses the response content type. * **`attachment.FromReader(contentType string, r io.Reader)`** → `*attachment.Attachment`: from an `io.Reader`. ## API client The `openapi` package gives direct access to the Braintrust REST API. Its client, methods, and models are generated from the Braintrust OpenAPI specification. It covers projects, experiments, datasets, prompts, functions, project automations, function invocation, and other REST resources, beyond what the higher-level APIs above cover. Requires v0.19.0 or later. ### `openapi.NewClientWithResponses` Creates a REST API client for a base URL. The client doesn't read environment variables or add authentication, so set the base URL and the `Authorization` header yourself. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import ( "context" "fmt" "log" "net/http" "os" "github.com/braintrustdata/braintrust-sdk-go/openapi" ) ctx := context.Background() key := os.Getenv("BRAINTRUST_API_KEY") client, err := openapi.NewClientWithResponses("https://api.braintrust.dev", openapi.WithRequestEditorFn(func(_ context.Context, req *http.Request) error { req.Header.Set("Authorization", "Bearer "+key) return nil }), ) if err != nil { log.Fatal(err) } limit := 5 resp, err := client.GetProjectWithResponse(ctx, &openapi.GetProjectParams{Limit: &limit}) if err != nil { log.Fatal(err) } if resp.StatusCode() != http.StatusOK || resp.JSON200 == nil { log.Fatalf("list projects: HTTP %d: %s", resp.StatusCode(), resp.Body) } for _, project := range resp.JSON200.Objects { fmt.Println(project.Id, project.Name) } ``` Returns: `(*openapi.ClientWithResponses, error)`. Use `openapi.WithHTTPClient` to supply a custom `*http.Client`, for example with a timeout or transport. Key rules: * An HTTP error response isn't a Go error. Check `StatusCode()` and the decoded field for the expected status, such as `JSON200`, as well as `err`. `Body` holds the raw response bytes, including error details. * Methods ending in `WithResponse` read and close the response body. If you use the raw methods from `openapi.NewClient`, close each response body yourself. * List and fetch operations return one page. Pass the pagination parameters or fetch cursor to request the next page. The client doesn't paginate automatically. * Optional fields use pointers or `nullable.Nullable[T]`. For a nullable field, the zero value omits the field, `nullable.NewNullableWithValue(value)` sends a value, and `nullable.NewNullNullable[T]()` sends JSON `null`. The difference matters for PATCH requests. For a complete example, including dataset and automation create, update, and delete flows, see the [API client guide](https://github.com/braintrustdata/braintrust-sdk-go/blob/v0.19.0/docs/api-client.md) in the SDK repository. ### `api.NewClient` (deprecated) > [!NOTE] > > The `api` package and `Client.API()` are deprecated as of v0.19.0. They remain available and keep their existing behavior, but they won't cover new REST resources. Use the [`openapi` package](#openapi-newclientwithresponses) for new code. Creates a REST API client from an API key. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} import "github.com/braintrustdata/braintrust-sdk-go/api" client := api.NewClient(os.Getenv("BRAINTRUST_API_KEY")) ``` Returns: `*api.API`. Namespaces: * **`client.Projects()`**: project management. * **`client.Experiments()`**: experiment management. Methods include `Create`, `Register`, `List`, `Get`, `Delete`, and `InsertEvents(ctx, experimentID, events)`, which inserts `experiments.Event` rows (v0.18.0 or later). To update fields of an existing row instead of replacing it, set the event's `ID` to the row's ID and `IsMerge` to `true`. * **`client.Datasets()`**: dataset management. Methods include `Create`, `Insert`, `InsertEvents`, `Delete`, `Fetch`, and `Query`. * **`client.Functions()`**: function management, including `Invoke(ctx, functionID, input)` to call a deployed function. * **`client.Prompts()`**: prompt management. Methods include `Query(ctx, params)` to search for prompts and `Get(ctx, id, params)` to fetch one by ID. ## Configuration Configure the client with functional options passed to `braintrust.New`, or with environment variables. ```go #skip-compile theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}} client, err := braintrust.New(tp, braintrust.WithProject("My project"), braintrust.WithBlockingLogin(true), ) ``` Client options: * **`braintrust.WithAPIKey(apiKey string)`**: API key. Defaults to `BRAINTRUST_API_KEY`. * **`braintrust.WithAPIURL(apiURL string)`**: Braintrust API URL. * **`braintrust.WithAppURL(appURL string)`**: Braintrust app URL, used for permalinks. * **`braintrust.WithOrgName(orgName string)`**: organization name, useful when credentials can access multiple orgs. * **`braintrust.WithProject(projectName string)`**: project that receives exported spans. * **`braintrust.WithProjectID(projectID string)`**: project ID. Takes precedence over the project name. * **`braintrust.WithBlockingLogin(enabled bool)`**: log in synchronously during `New` instead of in the background. * **`braintrust.WithExporter(exporter trace.SpanExporter)`**: supply a custom span exporter. Intended for testing. * **`braintrust.WithEnableTraceConsoleLog(enabled bool)`**: print spans to the console. * **`braintrust.WithFilterAISpans(enabled bool)`**: export only AI-related spans. * **`braintrust.WithSpanCustomizers(customizers ...config.SpanCustomizer)`**: register [span customizers](#span-customizers) that modify spans before export. Repeated calls append to the list. * **`braintrust.WithEnvironment(environmentType string, name ...string)`**: set span-origin environment provenance. Overrides auto-detection from CI and server environment variables. ### Environment variables * **`BRAINTRUST_API_KEY`** (required): Braintrust API key. * **`BRAINTRUST_API_URL`**: Braintrust API URL. Defaults to `https://api.braintrust.dev`. * **`BRAINTRUST_APP_URL`**: Braintrust app URL, used for permalinks. Defaults to `https://www.braintrust.dev`. * **`BRAINTRUST_DEFAULT_PROJECT`**: project that traced spans route to. Defaults to `default-go-project`. * **`BRAINTRUST_DEFAULT_PROJECT_ID`**: project UUID. Takes precedence over the project name. * **`BRAINTRUST_ORG_NAME`**: organization name, useful when credentials can access multiple orgs. * **`BRAINTRUST_OTEL_FILTER_AI_SPANS`**: set to `true` to export only AI-related spans. * **`BRAINTRUST_ENABLE_TRACE_CONSOLE_LOG`**: set to `true` to print spans to the console. * **`BRAINTRUST_OTEL_ENABLE_BUILTIN_ADK_TRACES`**: set to `true` to export spans from Google ADK's built-in telemetry. Defaults to `false`. * **`BRAINTRUST_AUTO_CONVERT_AI_ATTACHMENTS`**: scan spans for base64 attachments and upload them to object storage. Defaults to `true`; set to `false` to disable. * **`BRAINTRUST_DEBUG`**: set to `true` to enable SDK debug logging. Defaults to `false`. * **`BRAINTRUST_BLOCKING_LOGIN`**: set to `true` to log in synchronously at startup. * **`BRAINTRUST_ENVIRONMENT_TYPE`**: environment type for span-origin provenance (for example, `ci` or `server`). Detected automatically from common CI and server runtimes when unset. * **`BRAINTRUST_ENVIRONMENT_NAME`**: environment name for span-origin provenance (for example, `github_actions` or `aws_lambda`). Detected automatically when unset. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.