Skip to main content
Applies to:
  • Plan - Pro, Enterprise
  • Deployment - Braintrust-hosted, Self-hosted
  • Use case - Automated online scoring with scheduled alerts for production logs
Summary Online scoring runs asynchronously on a schedule, so scores may not be immediately visible in the UI when viewing logs, and alerts can trigger before scores.your_scorer_name appears on spans. This occurs because online scoring executes based on its configured schedule rather than in real-time, causing a delay between when logs are generated and when scores are attached. To resolve this, add scores.your_scorer_name IS NOT NULL to alert BTQL filters to ensure alerts only fire after scoring completes.

​
Resolution Steps

​
Step 1: Configure online scoring rule

Go to Settings → Automations and, under Online scoring, click + Online scoring rule. Configure scorers, sampling rate, BTQL filter, and span types.

​
Step 2: Update alert BTQL filter

Modify your alert’s BTQL filter to include a score existence check:

​
Step 3: Force immediate scoring (optional)

To see scores immediately in the UI, use “Score existing logs” under Automations. This forces synchronous execution rather than waiting for the scheduled run.