> ## Documentation Index
> Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Hugging Face

> Trace Hugging Face Inference, Transformers.js, and local Transformers pipeline calls in Braintrust to debug prompts, evaluate models, and monitor production usage

[Hugging Face Inference](https://huggingface.co/docs/inference-providers/) provides a unified interface to LLMs, embeddings, and other models hosted on Hugging Face and routed providers. Braintrust traces chat completions, text generation, and feature extraction calls.

**TypeScript**

## Setup

Install the Braintrust and `@huggingface/inference` packages, then set your API keys. Requires `@huggingface/inference` v2.0.0 or later (any 2.x, 3.x, or 4.x release).

**1. Install packages**

```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add braintrust @huggingface/inference
```

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install braintrust @huggingface/inference
```

**2. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
BRAINTRUST_API_KEY=<your-braintrust-api-key>
HUGGINGFACE_API_KEY=<your-huggingface-token>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

## Auto-instrumentation

To trace Hugging Face Inference SDK calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust's import hook to patch the SDK at runtime.

**1. Initialize Braintrust and call Hugging Face**

```javascript title="trace-huggingface-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger } from "braintrust";
import * as huggingFace from "@huggingface/inference";

initLogger({
  projectName: "huggingface-example",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const client = new huggingFace.InferenceClient(
  process.env.HUGGINGFACE_API_KEY,
);

const response = await client.chatCompletion({
  model: "meta-llama/Llama-3.1-8B-Instruct",
  provider: "featherless-ai",
  messages: [
    {
      role: "user",
      content: "Reply with exactly OK.",
    },
  ],
  max_tokens: 16,
  temperature: 0,
});

console.log(response.choices?.[0]?.message?.content);
```

**2. Run with the import hook**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
node --import braintrust/hook.mjs trace-huggingface-auto.js
```

The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects. Compile your TypeScript to JavaScript and run the compiled file with the import hook, or [run TypeScript directly](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) with a TypeScript loader.

> [!NOTE]
>
> If you're using a bundler, see [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for plugin and loader setup.

## Manual instrumentation

To trace Hugging Face clients manually, wrap them yourself with `wrapHuggingFace()`. Use this when you want to trace selected clients, or wrap the module directly before constructing clients.

```javascript title="trace-huggingface-wrap.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger, wrapHuggingFace } from "braintrust";
import * as huggingFace from "@huggingface/inference";

initLogger({
  projectName: "huggingface-example",
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const hf = wrapHuggingFace(huggingFace);
const client = new hf.InferenceClient(process.env.HUGGINGFACE_API_KEY);

const embedding = await client.featureExtraction({
  inputs: "Paris France",
  model: "thenlper/gte-large",
  provider: "hf-inference",
});

console.log(embedding);
```

`wrapHuggingFace()` can wrap either:

* The module import itself, including `InferenceClient`, `InferenceClientEndpoint`, `HfInference`, and `HfInferenceEndpoint`.
* An already-constructed client instance.

If you use routed or custom endpoints via `client.endpoint(...)`, Braintrust records the endpoint URL in span metadata.

## What Braintrust traces

Braintrust traces these `@huggingface/inference` SDK calls, capturing:

* Chat completions, including streaming chat with first-token timing.
* Text generation, including streaming text generation with first-token timing.
* Feature extraction, summarized as embedding count, length, and batch count.
* Token usage, including prompt, completion, and total tokens.
* Request metadata, including model, provider, and selected request parameters.
* Response identifiers, including ID, model, object, created, and finish reason.
* Routed endpoint URL when calling `client.endpoint(...)`.

## Resources

* [Hugging Face Inference SDK](https://github.com/huggingface/huggingface.js/tree/main/packages/inference)
* [Hugging Face Inference Providers](https://huggingface.co/docs/inference-providers/)
* [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md)

## Transformers.js

[Transformers.js](https://huggingface.co/docs/transformers.js/) (`@huggingface/transformers`) runs models locally in Node.js or the browser without a remote API call. Braintrust traces text generation, text-to-text generation, summarization, feature extraction, and question answering pipelines. Requires `@huggingface/transformers` v3.0.0 or later (any 3.x or 4.x release).

### Setup

Install Braintrust alongside `@huggingface/transformers`, then set your API key.

**1. Install packages**

```bash pnpm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pnpm add braintrust @huggingface/transformers
```

```bash npm theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
npm install braintrust @huggingface/transformers
```

**2. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

### Auto-instrumentation

To trace Transformers.js pipeline calls without modifying your application code, initialize Braintrust normally, then run your app with Braintrust's import hook to patch the SDK at runtime.

**1. Initialize Braintrust and call a pipeline**

```javascript title="trace-transformers-auto.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger } from "braintrust";
import { pipeline } from "@huggingface/transformers";

initLogger({
  projectName: "transformers-example", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const generator = await pipeline(
  "text-generation",
  "Xenova/distilgpt2",
);

const result = await generator("Hello, world!", { max_new_tokens: 20 });
console.log(result);
```

**2. Run with the import hook**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
node --import braintrust/hook.mjs trace-transformers-auto.js
```

The auto-instrumentation example uses plain JavaScript so `node --import` can run the file directly. The Braintrust APIs work the same in TypeScript projects. Compile your TypeScript to JavaScript and run the compiled file with the import hook, or [run TypeScript directly](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) with a TypeScript loader.

> [!NOTE]
>
> If you're using a bundler, see [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md#auto-instrumentation) for plugin and loader setup.

### Manual instrumentation

To trace Transformers.js calls manually, wrap the module or a pipeline instance with `wrapHuggingFaceTransformers()`.

```javascript title="trace-transformers-wrap-module.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger, wrapHuggingFaceTransformers } from "braintrust";
import * as transformers from "@huggingface/transformers";

initLogger({
  projectName: "transformers-example", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const tracedTransformers = wrapHuggingFaceTransformers(transformers);

const generator = await tracedTransformers.pipeline(
  "text-generation",
  "Xenova/distilgpt2",
);

const result = await generator("Hello, world!", { max_new_tokens: 20 });
console.log(result);
```

```javascript title="trace-transformers-wrap-pipeline.js" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import { initLogger, wrapHuggingFaceTransformers } from "braintrust";
import { pipeline } from "@huggingface/transformers";

initLogger({
  projectName: "transformers-example", // Replace with your project name
  apiKey: process.env.BRAINTRUST_API_KEY,
});

const rawPipeline = await pipeline("text-generation", "Xenova/distilgpt2");
const generator = wrapHuggingFaceTransformers(rawPipeline);

const result = await generator("Hello, world!", { max_new_tokens: 20 });
console.log(result);
```

`wrapHuggingFaceTransformers()` can wrap either:

* The module import itself, so all pipelines created from it are traced.
* An already-constructed pipeline instance.

### What Braintrust traces

Braintrust traces these `@huggingface/transformers` pipeline calls, capturing:

* Text generation (`text-generation`) calls, with the prompt as input and the generated text as output.
* Text-to-text generation (`text2text-generation`) calls, with the input text and the generated text.
* Summarization (`summarization`) calls, with the input passage and the summary.
* Feature extraction (`feature-extraction`) calls, summarized as embedding count and embedding length.
* Question answering (`question-answering`) calls, with context and question as input and the answer as output.
* Request metadata, including model identifier and `provider: "huggingface"` on every span.

### Resources

* [Transformers.js documentation](https://huggingface.co/docs/transformers.js/)
* [`@huggingface/transformers` on npm](https://www.npmjs.com/package/@huggingface/transformers)
* [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md)

**Python**

[Hugging Face Transformers](https://huggingface.co/docs/transformers/) runs models locally without a remote API call. Braintrust traces text generation, text-to-text generation, summarization, translation, feature extraction, and question answering pipelines. Requires `transformers` 4.42.0 or later.

## Setup

Install the Braintrust SDK and `transformers`, then set your API key.

**1. Install packages**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust transformers torch
```

> [!NOTE]
>
> Running Transformers pipelines requires a deep learning backend such as PyTorch (`torch`), TensorFlow, or JAX. Install at least one alongside `transformers`.

**2. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
BRAINTRUST_API_KEY=<your-braintrust-api-key>

# For organizations on the EU data plane, use https://api-eu.braintrust.dev
# For self-hosted deployments, use your data plane URL
# BRAINTRUST_API_URL=<your-braintrust-api-url>
```

## Auto-instrumentation

To trace Transformers pipeline calls without modifying your application code, call `auto_instrument()` once at startup. The `transformers` integration is enabled by default.

**1. Initialize Braintrust and call a pipeline**

```python title="trace_transformers_auto.py" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import os

import braintrust

braintrust.auto_instrument()
braintrust.init_logger(
    api_key=os.environ["BRAINTRUST_API_KEY"],
    project="transformers-example",  # Replace with your project name
)

from transformers import pipeline

generator = pipeline("text-generation", model="gpt2")  # Replace with your model
result = generator("Hello, world!", max_new_tokens=20)
print(result)
```

**2. Run your app**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
python trace_transformers_auto.py
```

## Manual instrumentation

To trace specific pipelines manually, wrap them with `wrap_transformers()`. Wrapping applies to the pipeline class and is idempotent.

```python title="trace_transformers_wrap.py" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import braintrust
from braintrust.integrations.transformers import wrap_transformers
from transformers import pipeline

braintrust.init_logger(project="transformers-example")  # Replace with your project name

generator = pipeline("text-generation", model="gpt2")  # Replace with your model
generator = wrap_transformers(generator)

result = generator("Hello, world!", max_new_tokens=20)
print(result)
```

`wrap_transformers()` accepts a pipeline instance or class. Unsupported pipeline types are returned unchanged.

## What Braintrust traces

Braintrust instruments local Transformers pipeline calls and creates an LLM-typed span per call:

* Text generation (`text-generation`) calls, with the prompt as input and the generated text as output.
* Text-to-text generation (`text2text-generation`) calls, with input text and the generated text.
* Summarization (`summarization`) calls, with the input passage and the summary.
* Translation calls (`translation` and language-specific variants such as `translation_en_to_fr`), with the source text and the translation.
* Feature extraction (`feature-extraction`) calls, with output summarized as tensor shape.
* Question answering (`question-answering`) calls, with context and question as input and the answer as output.
* Request metadata including model identifier, task, device, dtype, and generation parameters (temperature, top\_p, max\_tokens, and more).
* Streaming calls (those using a `streamer` argument) are not traced and pass through unchanged.

## Resources

* [Hugging Face Transformers documentation](https://huggingface.co/docs/transformers/)
* [transformers on PyPI](https://pypi.org/project/transformers/)
* [Trace LLM calls](https://docs.braintrust.dev/instrument/trace-llm-calls.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.