> ## Documentation Index
> Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# LangSmith

> Trace LangSmith applications in Braintrust to observe and evaluate LLM calls, chains, and tools

[LangSmith](https://smith.langchain.com/) is LangChain's platform for tracing, evaluation, and monitoring of LLM applications. Braintrust traces LangSmith applications from your code.

**TypeScript**

## Setup

Install the packages, then set your API key and enable LangSmith tracing.

**1. Install braintrust and langsmith**

```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add braintrust langsmith
```

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install braintrust langsmith
```

**2. Set your API key and enable tracing**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
export BRAINTRUST_API_KEY=your-braintrust-api-key
export LANGSMITH_TRACING=true
```

> [!NOTE]
>
> Braintrust intercepts LangSmith's tracing pipeline, so `LANGSMITH_TRACING=true` must be set for `langsmith` to produce the runs that Braintrust captures. The legacy `LANGCHAIN_TRACING_V2=true` also works. To also send runs to LangSmith, set `LANGSMITH_API_KEY`.

## Tracing

Braintrust supports two modes for intercepting LangSmith calls:

* **Auto-instrumentation** (recommended): Run your app with `node --import braintrust/hook.mjs` to patch `langsmith` at startup without code changes.
* **Manual instrumentation**: Wrap `langsmith` namespaces explicitly using Braintrust's wrapper functions.

> [tab] Auto-instrumentation
>
> Initialize Braintrust, then run your app with Braintrust's import hook to patch `langsmith` at runtime. Requires `langsmith` v0.3.30 or later.
>
> **1. Initialize Braintrust and call LangSmith**
>
> ```typescript title="trace-langsmith-auto.ts" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
> import { initLogger } from "braintrust";
> import { traceable } from "langsmith/traceable";
>
> initLogger({
>   projectName: "langsmith-example", // Replace with your project name
>   apiKey: process.env.BRAINTRUST_API_KEY,
> });
>
> const retrieveContext = traceable(
>   async ({ question }: { question: string }) => [
>     { content: "Refunds accepted within 30 days.", title: "Refund policy" },
>   ],
>   { name: "retrieve-context", run_type: "retriever" },
> );
>
> const result = await retrieveContext({ question: "What is the refund window?" });
> console.log(result);
> ```
>
> **2. Run with the import hook**
>
> ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
> node --import braintrust/hook.mjs trace-langsmith-auto.ts
> ```
>
> The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects. Compile your TypeScript to JavaScript and run the compiled file with the import hook, or [run TypeScript directly](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) with a TypeScript loader.
>
> > [!NOTE]
> >
> > If you're using a bundler, see [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for plugin and loader setup.

> [tab] Manual instrumentation
>
> Wrap `langsmith` namespaces with Braintrust's wrapper functions before calling `traceable`, `RunTree`, or `Client`. Each wrapper intercepts the corresponding LangSmith API and forwards spans to Braintrust while still calling LangSmith normally.
>
> ```typescript title="trace-langsmith-manual.ts" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
> import { initLogger, wrapLangSmithTraceable } from "braintrust";
> import * as traceableNs from "langsmith/traceable";
>
> initLogger({
>   projectName: "langsmith-example", // Replace with your project name
>   apiKey: process.env.BRAINTRUST_API_KEY,
> });
>
> const { traceable } = wrapLangSmithTraceable(traceableNs);
>
> const retrieveContext = traceable(
>   async ({ question }: { question: string }) => [
>     { content: "Refunds accepted within 30 days.", title: "Refund policy" },
>   ],
>   { name: "retrieve-context", run_type: "retriever" },
> );
>
> const result = await retrieveContext({ question: "What is the refund window?" });
> console.log(result);
> ```
>
> `wrapLangSmithRunTrees` and `wrapLangSmithClient` are also available from `braintrust` to intercept `RunTree` and `Client` usage respectively.

## What Braintrust traces

Braintrust captures:

* `traceable`-wrapped function calls as spans, with inputs and outputs
* `RunTree` operations and `Client` API calls (`createRun`, `updateRun`, `batchIngestRuns`) as spans
* Run type mapped to span type: `llm` and `embedding` → LLM spans, `tool` and `retriever` → Tool spans, other run types → Task spans
* Token usage (`prompt_tokens`, `completion_tokens`, `total_tokens`, `prompt_cached_tokens`, `prompt_cache_creation_tokens`)
* `time_to_first_token` from `new_token` events in run history
* Model name and provider from `ls_model_name` and `ls_provider` metadata fields
* LLM settings (`temperature`, `top_p`, `max_tokens`, and related fields) from run metadata
* Tags and errors

> [!NOTE]
>
> LangChain runs (identified by `serialized.lc === 1`) are skipped by default to avoid double-tracing when both LangChain and LangSmith instrumentation are active.

## Resources

* [LangSmith documentation](https://docs.smith.langchain.com/)
* [braintrust on npm](https://www.npmjs.com/package/braintrust)

**Python**

> [!WARNING]
>
> This is an experimental feature. The API may change based on user feedback.

Braintrust provides an experimental wrapper to integrate LangSmith with Braintrust. The wrapper can either send tracing and evaluation calls to both LangSmith and Braintrust in parallel, or route them solely to Braintrust, with minimal code changes.

The wrapper supports two modes:

* **Parallel** (default): Send traces and evaluations to both LangSmith and Braintrust simultaneously. Use this to compare services, maintain existing workflows, or run both long-term.
* **Standalone**: Send traces and evaluations only to Braintrust. Use this when you want to use Braintrust exclusively.

## Setup

Install the packages, then set your Braintrust and LangSmith environment variables.

**1. Install braintrust and langsmith**

Requires Braintrust Python SDK v0.4.3 or later.

```bash uv theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
uv add braintrust langsmith
```

```bash pip theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust langsmith
```

**2. Set your Braintrust API key**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
export BRAINTRUST_API_KEY=your-braintrust-api-key
```

**3. Set your LangSmith environment variables**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=your-langsmith-api-key
export LANGSMITH_PROJECT=your-project-name
```

> [!NOTE]
>
> These are the current LangSmith variable names. The legacy `LANGCHAIN_TRACING_V2`, `LANGCHAIN_API_KEY`, and `LANGCHAIN_PROJECT` equivalents also work.

## Tracing

The wrapper automatically redirects:

* Functions decorated with LangSmith's `@traceable` to Braintrust's [`@traced`](https://docs.braintrust.dev/instrument/trace-application-logic#trace-function-calls)
* Nested span hierarchies with inputs and outputs
* Complete execution traces with metadata

### Parallel tracing

By default, traces are sent to both LangSmith and Braintrust simultaneously.

To use the wrapper, call `setup_langsmith()` **before** importing from LangSmith modules:

```python trace_parallel.py {3-9} theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

# Setup Braintrust wrapper BEFORE importing from langsmith
from braintrust.wrappers.langsmith_wrapper import setup_langsmith

setup_langsmith(
    project_name="langsmith-integration",
    api_key=os.environ.get("BRAINTRUST_API_KEY"),
)

# Now import and use LangSmith as usual - these are patched to use Braintrust
from langsmith import traceable
from openai import OpenAI

client = OpenAI()

@traceable(name="chat_completion")
def chat_completion() -> str:
    """Single traced call."""
    result = client.responses.create(
        model="gpt-5-mini",
        input=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "What is machine learning?"},
        ],
    )
    return result.output_text

if __name__ == "__main__":
    print(chat_completion())
```

> [!NOTE]
>
> When you don't pass `project_name` to `setup_langsmith()`, the wrapper falls back to the `LANGCHAIN_PROJECT` environment variable for the Braintrust project name. Pass `project_name` explicitly (as the examples do) to set it directly.

### Standalone tracing

With standalone mode, traces are sent only to Braintrust. Set `standalone=True` when calling `setup_langsmith()`; the rest of the [parallel tracing](#parallel-tracing-python) example is unchanged.

```python trace_standalone.py theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
setup_langsmith(
    project_name="langsmith-integration",
    api_key=os.environ.get("BRAINTRUST_API_KEY"),
    standalone=True,  # Only Braintrust will receive traces
)
```

You can also enable standalone mode via environment variable:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
export BRAINTRUST_STANDALONE=1
```

## Evals

The wrapper automatically redirects:

* `evaluate()` calls to Braintrust's [`Eval()`](https://docs.braintrust.dev/evaluate/run-evaluations) function
* `aevaluate()` calls to Braintrust's `EvalAsync()` function

Just like for tracing, you can send evaluation calls to both LangSmith and Braintrust in parallel, or only to Braintrust.

### Parallel evals

Evaluators follow the LangSmith signature: `(inputs, outputs, reference_outputs) -> bool | dict`. The wrapper automatically converts these to Braintrust scorers.

> [!NOTE]
>
> When LangSmith evals are instrumented with `@traceable`, scores show up both in experiments and logging.

```python eval_langsmith_parallel.py {3-9} theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

# Setup Braintrust wrapper BEFORE importing from langsmith
from braintrust.wrappers.langsmith_wrapper import setup_langsmith

setup_langsmith(
    project_name="langsmith-integration",
    api_key=os.environ.get("BRAINTRUST_API_KEY"),
)

# Now import and use LangSmith as usual - these are patched to use Braintrust
from langsmith import Client, traceable

# Define a target function (the function being evaluated)
# LangSmith requires the parameter to be named 'inputs' (or 'attachments'/'metadata')
def multiply(inputs: dict, **kwargs) -> int:
    """Multiply two numbers.

    Args:
        inputs: Dictionary with 'x' and 'y' keys
        **kwargs: Additional arguments (e.g., langsmith_extra from LangSmith)
    """
    return inputs["x"] * inputs["y"]

# Define LangSmith-style evaluators
# LangSmith evaluators use signature: (inputs, outputs, reference_outputs) -> bool | dict
# When target returns a plain value, LangSmith wraps it as {"output": value}
def exact_match_evaluator(inputs: dict, outputs: dict, reference_outputs: dict) -> dict:
    """
    LangSmith-style evaluator that checks for exact match.
    """
    expected = reference_outputs["output"]
    actual = outputs["output"]
    return {
        "key": "exact_match",
        "score": 1.0 if actual == expected else 0.0,
    }

def range_evaluator(inputs: dict, outputs: dict, reference_outputs: dict) -> dict:
    """
    LangSmith-style evaluator that checks if result is in expected range.
    """
    actual = outputs["output"]
    expected = reference_outputs["output"]
    # Check if within 10% of expected
    if expected == 0:
        score = 1.0 if actual == 0 else 0.0
    else:
        diff = abs(actual - expected) / abs(expected)
        score = 1.0 if diff <= 0.1 else 0.0
    return {
        "key": "within_range",
        "score": score,
        "metadata": {"actual": actual, "expected": expected},
    }

def main():
    print("LangSmith to Braintrust Evaluation Example")
    print("=" * 50)
    print()

    # Create a LangSmith client (patched to use Braintrust)
    client = Client()

    # Create a dataset in LangSmith (proper LangSmith API usage)
    dataset_name = "multiply-dataset-example"

    # Try to get or create the dataset
    try:
        dataset = client.read_dataset(dataset_name=dataset_name)
        print(f"Using existing dataset: {dataset_name}")
    except Exception:
        # Create new dataset if it doesn't exist
        dataset = client.create_dataset(dataset_name=dataset_name, description="Multiplication test dataset")
        print(f"Created new dataset: {dataset_name}")

        # Create examples in the dataset (proper LangSmith API)
        client.create_examples(
            dataset_id=dataset.id,
            examples=[
                {"inputs": {"x": 2, "y": 3}, "outputs": {"output": 6}},
                {"inputs": {"x": 5, "y": 5}, "outputs": {"output": 25}},
                {"inputs": {"x": 10, "y": 0}, "outputs": {"output": 0}},
                {"inputs": {"x": 7, "y": 8}, "outputs": {"output": 56}},
            ],
        )
        print(f"Created {4} examples in dataset")

    print()
    print("Running evaluation...")
    print()

    # Run evaluation using LangSmith's API (redirects to Braintrust)
    # Pass the dataset name - this is valid LangSmith API usage
    client.evaluate(
        multiply,  # Target function
        data=dataset_name,  # Dataset name (valid LangSmith API)
        evaluators=[exact_match_evaluator, range_evaluator],
        experiment_prefix="multiply-test",
        description="Testing multiplication function",
        metadata={"version": "1.0", "migrated_from": "langsmith"},
    )
    print()
    print("=" * 50)
    print("✓ Evaluation completed!")
    print("Check Braintrust to see the experiment results.")

if __name__ == "__main__":
    main()
```

### Standalone evals

With standalone mode, evaluation results are sent only to Braintrust. Set `standalone=True` when calling `setup_langsmith()`; the rest of the [parallel evals](#parallel-evals-python) example is unchanged.

```python eval_langsmith_standalone.py theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
setup_langsmith(
    project_name="langsmith-integration",
    api_key=os.environ.get("BRAINTRUST_API_KEY"),
    standalone=True,  # Only Braintrust will receive evals
)
```

## Resources

* [LangSmith documentation](https://docs.smith.langchain.com/)
* [Braintrust evaluation guide](https://docs.braintrust.dev/evaluate.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.