> ## Documentation Index
> Fetch the complete documentation index at: https://docs.braintrust.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Roast

> Trace Roast AI workflows and individual cogs in Braintrust to debug step-by-step execution, inspect LLM calls, and monitor workflow failures

[Roast](https://github.com/Shopify/roast) is a Ruby framework for building structured AI workflows from steps called cogs, such as `chat`, `agent`, `ruby`, and `cmd`. Braintrust traces each workflow run and each cog in it, including the LLM calls that `chat` cogs make.

**Ruby**

## Setup

Install the Braintrust and Roast gems, then configure your API keys. Roast integration requires Braintrust Ruby SDK v0.6.0 or later and `roast-ai` v1.0.0 or later, which requires Ruby 3.3 or later.

**1. Install gems**

Add the gems to your Gemfile:

```ruby title="Gemfile" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
gem "roast-ai"
gem "braintrust"
```

Then install them:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
bundle install
```

**2. Set environment variables**

```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
BRAINTRUST_API_KEY=your-api-key
BRAINTRUST_DEFAULT_PROJECT=your-project-name  # Project that spans are logged to

# API key for the provider your chat cogs use (OpenAI by default)
OPENAI_API_KEY=your-openai-key
```

## Auto-instrumentation

To trace Roast workflows without modifying your workflow files, run the `roast` command through `braintrust exec`. Braintrust patches Roast when it loads, along with [RubyLLM](https://docs.braintrust.dev/integrations/sdk-integrations/ruby-llm.md), which Roast's `chat` cog uses to call models.

**1. Create workflow\.rb**

This workflow passes a question from a `ruby` cog to a `chat` cog:

```ruby title="workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
config do
  chat(:answer) { no_show_stats! }
end

execute do
  ruby(:question) { "What is the capital of France? Answer in three words or fewer." }
  chat(:answer) { ruby!(:question).value }
end
```

**2. Run the workflow**

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
bundle exec braintrust exec -- roast execute workflow.rb
```

`braintrust exec` initializes Braintrust and logs to the project set by `BRAINTRUST_DEFAULT_PROJECT`.

To run workflows from your own Ruby code instead, load `braintrust/setup` before you require `roast`. When you call `Roast::Workflow.from_file` inside a span, the workflow's span nests under it.

```ruby title="run_workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require "braintrust/setup"
require "roast"

Roast::Workflow.from_file("workflow.rb", Roast::WorkflowParams.new([], [], {}))
```

## Manual instrumentation

To trace Roast manually, initialize Braintrust with `auto_instrument: false` and enable instrumentation yourself with `Braintrust.instrument!(:roast)`. Also call `Braintrust.instrument!(:ruby_llm)` to trace the LLM calls that `chat` cogs make.

```ruby title="run_workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
require "braintrust"
require "roast"

Braintrust.init(default_project: "My Project", auto_instrument: false)

Braintrust.instrument!(:roast)
Braintrust.instrument!(:ruby_llm)

Roast::Workflow.from_file("workflow.rb", Roast::WorkflowParams.new([], [], {}))
```

## What Braintrust traces

Braintrust captures:

* Workflow spans (`roast.workflow`), with the workflow file name in metadata (`contrib.roast.workflow.name`), plus the workflow's targets, positional arguments, and keyword arguments as input when you pass them.
* Cog spans (`roast.cog.<name>`), nested under the workflow span, with the cog's type, name, and outcome (`completed`, `failed`, or `skipped`) in metadata. Cogs in a scope that a `call` cog runs nest under that cog's span.
* Cog output for `ruby` cogs that return a string, and the model response for `chat` cogs.
* LLM call spans (`ruby_llm.chat`) for `chat` cogs, nested under the cog span, with the messages, model, response, and token usage that the [RubyLLM integration](https://docs.braintrust.dev/integrations/sdk-integrations/ruby-llm.md#what-traced-ruby) records.
* Errors, recorded on the cog span when a cog fails.

## Resources

* [Roast on GitHub](https://github.com/Shopify/roast)
* [Roast tracing example](https://github.com/braintrustdata/braintrust-sdk-ruby/blob/main/examples/contrib/roast/basic.rb)
* [Braintrust Ruby SDK](https://github.com/braintrustdata/braintrust-sdk-ruby)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.