Skip to main content
Pattern rules control what Loop investigates, how often it runs, and where it sends results. After you enable Patterns and review its findings, you can tune discovery to your project’s needs.

Tune a pattern rule

To edit a rule, go to Patterns, open the rules menu in the page header, and select the rule. The editor has these fields, in order:
  • Name: Choose a name that describes the rule’s focus. It appears on the patterns the rule finds.
  • Traces: Click Filter to limit discovery to traces that match a SQL filter, such as traces with specific metadata or tags.
  • Watch for: Add specific problems to investigate, or select suggestions from recent traces. Loop can still report other problems in the matching traces.
  • Custom instructions: Add guidance, such as what to exclude. Ask Loop to find or update patterns. Asking only for a summary produces a summary, not pattern records.
  • Model: The model and provider for the rule’s runs. See How Patterns selects a provider.
  • Schedule: Choose Continuous, Daily, Weekly, Interval, or Custom (cron). A project with steady traffic rarely needs more than the default daily run. For how each option works, see Set the schedule and query range.
  • Looks back: Choose 12 hours, 1 day, 3 days, 7 days, 14 days, 30 days, or a custom number of days up to 30. Each run inspects the whole range, including traces that earlier runs already checked. A longer range than the concern needs increases cost.
  • Alerts: Add a Slack, Webhook, or PagerDuty destination. Slack receives per-pattern alerts. Webhook and PagerDuty destinations receive the run output. For each destination’s settings, see Choose destinations.
  • Preview prompt: Expand to read the full instruction that Braintrust builds from these fields and sends to Loop.
Click Save to apply your changes. To run a rule without waiting for its schedule, hover over it in the rules menu and click , or click Run now in the rule editor. A paused rule doesn’t run. The default rule starts with Traces, Watch for, and Custom instructions empty, so Loop looks for any recurring problem in the project’s traces. It can create and update patterns, and its Alerts contains the Slack channel selected during setup, if any.

Add another pattern rule

Add another pattern rule when a concern needs its own scope, schedule, or destination, such as tool-call errors, cost growth, latency for specific customers, or quality regressions sent to a different team’s channel. To change what a single run finds, tune the existing rule instead.
  1. On the Patterns page, click Pattern rule. A New pattern rule draft opens with the default Patterns settings and no alerts.
  2. Enter a Name that describes the rule’s focus. The name appears on the patterns it discovers.
  3. Narrow the rule to the concern with Traces, Watch for, and Custom instructions.
  4. Adjust Schedule and Looks back for the concern.
  5. Optional: Add a destination under Alerts.
  6. Click Create.
All pattern rules in a project write to the same patterns list. Before saving a finding, Loop checks every pattern in the project, including inactive ones, and updates a matching pattern instead of creating a duplicate. Each rule runs its own investigation and adds to your cost. To limit cost, keep each rule’s scope narrow and its Looks back range no longer than the concern requires.

Customize severity and categories

Loop gives each pattern a severity and categories so your team can see what matters most and what kind of problem it is. A rubric defines what each severity level means and which categories Loop can assign. Every pattern rule uses the project’s default rubric unless you customize it, for example to add categories specific to your product or to change what counts as High severity. The default rubric checks the categories in their listed order and assigns at most one. When none clearly fits, Loop leaves the category empty. Ask Loop in a Loop thread to change the rubric. Specify whether the change applies to the project’s default or to a named pattern rule. Customizing either requires the project’s Update permission. For example, to customize one rule:
The rule then uses its own rubric and no longer follows changes to the project default. Rules with existing custom rubrics keep using them. Changes to the project default apply to rules without their own rubric and to Loop conversations. To restore inheritance, ask Loop to return the named rule to the project’s rubric.
Restoring a rule to the project’s rubric discards that rule’s custom rubric.
The whole rubric can be up to 32,000 characters:
  • Severity: The levels are always High, Medium, Low, and Informational, but you can redefine what each one means.
  • Categories: List them under a Categories heading, each once, with a backtick-quoted name followed by a colon and description, such as - `billing`: charges are wrong. A rubric needs at least one category and can have up to 64, with names up to 64 characters each.
The default rubric also gives each category Look for, Read, One pattern, and Count instructions. These guide searches, how much context Loop reads, and how it groups and measures findings. You can include these instructions in custom rubrics. Without them, Loop derives searches from the category descriptions.

Configure impact measurement

User and session IDs let Patterns measure how many people or conversations are affected, rather than counting every trace separately. Loop also uses these IDs to analyze users and read neighboring turns of a conversation. Each project can configure one field of each kind. Both must be in metadata or span_attributes, such as span_attributes."session.id". Setting, replacing, or clearing them requires the project’s Update permission. Loop looks for these IDs during pattern analysis and saves fields it can identify without replacing existing configuration. It leaves ambiguous session fields unset and doesn’t infer sessions from time gaps or use user, request, or trace IDs as session IDs. If an ID appears only in input or output, Loop recommends logging it in metadata or span_attributes. To set either field explicitly, ask Loop or your MCP-connected assistant. Substitute the paths your traces log:
The assistant uses set_project_user_id_field and set_project_session_id_field. By default, these tools set a field only if none is configured. See Impact for how the report chooses between users, sessions, and traces.
Without only_if_unset, PATCH /v1/project/{id} replaces all project settings with the settings you send. Include only_if_unset and send only one ID field setting per request to preserve other project settings.
Send settings.user_id_fields or settings.session_id_fields in PATCH /v1/project/{id}. Use only_if_unset: true to initialize an unset field without replacing an existing choice.Set the user ID field:
Set the session ID field in a separate request:
Explicitly ask Loop or your assistant to replace or clear the field. Through MCP, this requires only_if_unset: false. Pass field: null to clear it. Through the REST API, use only_if_unset: false and send the replacement path, or an empty list to clear the field. Follow the API precautions above.Changing or clearing a field invalidates counts measured with the old field until Braintrust measures the pattern again. Open patterns are measured again when a rule runs or the pattern changes.Until then, the corresponding list cells are empty and the default visible column can fall back to another unit. The measurement below the pattern’s name falls back to traces when its user or session counts are invalid.

Troubleshoot

When an enabled pattern rule fails four runs in a row, the rules menu in the page header shows the failure. The menu’s button shows the rule’s name followed by failed, or the number of failed rules. A single failure, or a few in a row, doesn’t appear, because a later run often succeeds.
  1. Open the menu. Each failed rule shows Failed and its error message.
  2. Select the rule to open it, then select the Past runs tab to read the failed runs.
Enabling Patterns doesn’t test your provider’s connection before it creates the rule. For provider errors, check that the configured credentials are valid and that the provider can serve the rule’s Model. After you fix the error, run the rule again with in the rules menu or Run now in the rule editor.
Loop records a pattern only when it can attach supporting evidence, so a run that records nothing hasn’t failed.To see why, open the rule, select the Past runs tab, and select the run to read the transcript. Select Fork to chat to continue in an editable thread and ask Loop why it recorded nothing.If runs are consistently empty, the Looks back range, the Traces filter, or the watched problems are usually too narrow.
Only rule runs send alerts. Patterns that Loop records during an interactive chat appear in the list but don’t trigger Slack or webhook destinations.A run sends Slack alerts only when it creates or reactivates patterns. A run that only updates existing patterns sends no Slack message.Check that the rule has a Slack destination under Alerts, and that the Slack workspace is still connected.

Next steps