Skip to main content
Applies to:
  • Plan - Any
  • Deployment - Any
  • Use case - Debugging Braintrust webhook alerts that do not appear in incident.io

​
Summary

If a Braintrust alert does not appear in incident.io, check whether Braintrust sent the webhook, which destination received it, and how that destination responded. Before you begin, check the Braintrust alert documentation, incident.io HTTP alert-event API reference, and Custom HTTP source guide for current payload requirements and configuration options. Compare these with your alert and source configuration, as either product may have changed since this guide was written.

​
1. Confirm the destination and source type

Compare the webhook URL configured in Braintrust with the URL provided by the intended incident.io HTTP alert source. Check that authentication matches that source’s configuration. In incident.io, check whether Type of HTTP source is set to Default or Custom:
  • Default expects incident.io’s standard payload.
  • Custom lets you transform Braintrust’s incoming payload into that format.
If results differ between tests, check whether the URL, authentication, or source configuration changed. For authentication and endpoint details, see the incident.io HTTP alert-event API reference.

​
2. Identify where the request failed

For a log alert using a webhook action, click Test beside the webhook URL in Braintrust and read the result shown in the UI. If the test fails, use the error message to identify the next check. See Test an alert for supported alert types. If incident.io rejects the request with a validation error, use the response body to identify the rejected fields and check the required fields and mapping below. If you use a service to forward alerts to incident.io, check its logs to confirm incident.io accepted the request. Braintrust’s successful test may only confirm that the forwarding service received it.

​
3. Check required fields and mapping

If the error identifies missing or invalid fields, compare your mapping with the Braintrust webhook payload and incident.io’s expected format. For the standard incident.io HTTP alert-event body: See the incident.io API reference for current requirements and optional fields. Braintrust’s log alert payload includes nested fields such as automation.name and details.message. These need to be mapped to the fields incident.io expects. For example:
  • automation.name → title
  • details.message → description
  • automation.url → source_url
For log alerts, set status to "firing" in your incident.io transform or forwarding service.
When mapping log alerts to "firing", configure how they resolve:
  • Set up automatic resolution after a delay. Check whether alerts in active incidents are excluded. This clears alerts on a timer, regardless of recovery.
  • Use a forwarder to check for recovery and send "resolved" to the same incident.io source with the same deduplication key.
For time window alerts, map details.transition explicitly: Handle scheduled, evaluation_failed, and evaluation_recovered separately. These describe scheduled execution or evaluation health, rather than whether the monitored condition is firing or resolved. Do not map them to resolved by default.

​
If using a Custom HTTP source

Check that your incident.io transform maps the incoming Braintrust payload to the required fields. Configure deduplication in incident.io using the Deduplication key path setting. This selects a field from the incoming Braintrust payload that incident.io uses to identify notifications belonging to the same alert. Check this setting and your transform against the Custom HTTP source guide. A custom-transform error is different from an HTTP validation rejection. According to incident.io documentation, fallback alert creation occurs when a transform fails, so an empty alert feed alone does not prove a mapping error.

​
If using a relay or forwarder

Check that the relay maps the Braintrust payload into incident.io’s expected fields and forwards it using the intended source’s endpoint and authentication.

​
4. Confirm the fix

Run the Braintrust alert test again. Confirm it succeeds and that the expected alert is created or updated in incident.io with the correct title and status. If you use a forwarding service, also confirm incident.io accepted its request. If the issue persists, share the Braintrust alert link, UTC test timestamp, exact UI result or error, incident.io source type, and a redacted destination URL with support. Include any available relay errors and note whether the URL or source configuration changed between tests. Do not include authentication tokens.