LiteLLM

​

Setup

uv add braintrust litellm

pip install braintrust litellm

BRAINTRUST_API_KEY=<your-braintrust-api-key>
OPENAI_API_KEY=<your-openai-api-key>

​

Auto-instrumentation

import os

import braintrust

braintrust.auto_instrument()
braintrust.init_logger(
    api_key=os.environ["BRAINTRUST_API_KEY"],
    project="litellm-example",  # Replace with your project name
)

import litellm

response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)

from braintrust.integrations.litellm import patch_litellm

patch_litellm()

import litellm
from braintrust import init_logger

# Initialize Braintrust
logger = init_logger(project="litellm-example")

# Use LiteLLM as normal - all calls are automatically traced
response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)

​

Manual instrumentation

import os

import litellm
from braintrust import init_logger
from braintrust.integrations.litellm import wrap_litellm

init_logger(
    api_key=os.environ["BRAINTRUST_API_KEY"],
    project="litellm-example",  # Replace with your project name
)

wrap_litellm(litellm)

response = litellm.completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
)
print(response.choices[0].message.content)

​

What Braintrust traces

• Completion spans (Completion) for litellm.completion, litellm.acompletion, litellm.text_completion, and litellm.atext_completion, with messages or prompt, model, and request parameters. Responses include choices, token usage, and time-to-first-token for streaming. Base64 data URL images in messages are captured as attachments.
• Responses API spans (Response) for litellm.responses / litellm.aresponses, with input and request parameters; response output, token usage, and time-to-first-token for streaming.
• Decision spans (Decision) for litellm.decisions, litellm.adecisions, and System One calls through litellm.systemone and litellm.asystemone (LiteLLM v1.105.0 or later), with the input state, the questions (each with an id, type, instructions, and criteria for choice and score questions), and model. Output records an answers list keyed by question id, with probabilities, scores, Noul values, and confidence, plus token usage. Images in the input are captured as attachments. Images, audio, and video passed in images, assets, audio, or videos are also captured as attachments (LiteLLM v1.106.0 or later).
• Image generation spans (Image Generation) for litellm.image_generation / litellm.aimage_generation, with prompt and request parameters; output capturing per-image data (attachment for base64 responses or URL reference for URL responses) plus metadata like output format, size, quality, and image count, plus timing and token usage when reported.
• Embedding spans (Embedding) for litellm.embedding / litellm.aembedding, with input text and request parameters; output summarized as the embedding vector dimension (length of the first embedding), plus token usage.
• Moderation spans (Moderation) for litellm.moderation / litellm.amoderation, with input and request parameters; classification results and token usage when reported.
• Speech spans (Speech) for litellm.speech / litellm.aspeech, with text input and request parameters; generated audio captured as an attachment, plus timing.
• Transcription spans (Transcription) for litellm.transcription / litellm.atranscription, with the input audio captured as an attachment plus model and request parameters; transcribed text and token usage.
• Rerank spans (Rerank) for litellm.rerank / litellm.arerank, with query, documents, and request parameters (plus auto-derived document_count); results as a list of {index, relevance_score} items (capped at 100, with documents intentionally dropped); token metrics (prompt, completion, total), plus Cohere-style billed-unit metrics (search units, classifications) when the response includes them.
• Token usage metrics (prompt, completion, total, plus cached and reasoning tokens when the provider reports them).
• Errors captured on every call.

​

Resources

• LiteLLM documentation
• LiteLLM supported providers
• DSPy integration, which combines LiteLLM tracing with DSPy-specific callbacks