Invoke saved preprocessors from custom code scorers

Applies to:

Plan - Any
Deployment - Any
Apply saved preprocessors in Python and TypeScript custom code scorers, with the correct input reference for trace or group scope.

Summary

Python and TypeScript custom code scorers cannot select a saved preprocessor by passing its slug to trace.get_thread() or trace.getThread(). To fix this, call the saved preprocessor by function ID using invoke() inside your scorer. Alternatively, set it as the project default and call the thread method without an override. For custom conversation assembly, fetch the raw spans and extract the messages inside the scorer’s code.

What is happening

The preprocessor override in Python’s trace.get_thread() and TypeScript’s trace.getThread() is sent as a global preprocessor name. It does not look up saved project functions or execute JavaScript source passed as a string. This causes the “Unsupported global preprocessor” error.

Fix or suggestion

Option 1: Invoke the saved preprocessor directly

Call the saved preprocessor by function ID from inside your scorer. This lets each scorer choose its own preprocessor without changing the project default. For trace-scoped and group-scoped scorers, use the reference matching the scoring scope:

For a group of related traces, pass group_ref.
For a single trace, pass trace_ref.

See Online scoring configuration for the differences between span, trace, and group scopes. The following snippets assume trace is available inside your scorer handler. Replace the function ID with your saved preprocessor’s ID. See Score traces with custom code for scorer setup and the available trace methods.

import { invoke } from "braintrust";

const ref = trace.getConfiguration();
const payload =
  "root_span_ids" in ref
    ? { group_ref: ref }
    : { trace_ref: ref };

const thread = await invoke({
  function_id: "<preprocessor-function-id>",
  input: payload,
  mode: "json",
});

import braintrust

ref = trace.get_configuration()
payload = (
    {"group_ref": ref}
    if "root_span_ids" in ref
    else {"trace_ref": ref}
)

thread = braintrust.invoke(
    function_id="<preprocessor-function-id>",
    input=payload,
    mode="json",
)

Use the returned thread in your scoring logic, then return the scorer’s score. These snippets retrieve the preprocessed data. They do not calculate a score. In Python, braintrust.invoke() is synchronous and does not use await. The trace.get_thread() and trace.get_spans() methods in Options 2 and 3 are asynchronous; call them with await inside an async def scorer handler.

Option 2: Use the project default preprocessor

Set the saved preprocessor as the project default, then call the thread method without an override:

const thread = await trace.getThread();

thread = await trace.get_thread()

This simplifies the scorer, but also changes preprocessing for other project features that use the project default.

Option 3: Assemble the conversation from spans

Fetch the spans and build the conversation in your scorer when you need control over span selection, ordering, or deduplication:

const spans = await trace.getSpans();

spans = await trace.get_spans()

Extract the relevant messages, order them for your application, and remove duplicates before applying your scoring logic. This gives you full control, but you maintain the conversation assembly yourself. See Score traces with custom code for examples of inspecting and filtering spans.

Confirm it worked

Confirm that the preprocessed result contains the conversation messages your scorer needs. See Write a custom preprocessor for input fields and message-format examples.
For group-scoped rules, confirm that it receives group_ref and reconstructs the conversation from the traces included in the group.
For single-trace runs, confirm that it receives trace_ref and returns the expected messages.
Confirm that the scorer uses the preprocessed data and returns the intended score.

Notes

Group-scoped preprocessing remains limited to the traces included in the scoring group. See Score groups for how grouped context is provided to the scorer.
Use a small synthetic example to validate the returned structure before applying the scorer more broadly.

​Summary

​What is happening

​Fix or suggestion

​Option 1: Invoke the saved preprocessor directly

​Option 2: Use the project default preprocessor

​Option 3: Assemble the conversation from spans

​Confirm it worked

​Notes

Summary

What is happening

Fix or suggestion

Option 1: Invoke the saved preprocessor directly

Option 2: Use the project default preprocessor

Option 3: Assemble the conversation from spans

Confirm it worked

Notes