Remote evals firewall egress and auth troubleshooting

​

Summary

A remote eval server can return HTTP 401 on GET /list when its outbound login request fails.

​

What is happening

The eval server and the bt CLI perform an outbound login to the app URL to validate session tokens. If that POST fails, the server can report the failure as 401 for GET /list.

• CloudFront backs api-eu.braintrust.dev. Do not rely on a fixed set of IPs resolved once for this hostname, because CloudFront address ranges can change.
• If requests reset during the TLS handshake, ask your network team to investigate connectivity from the failing pod.
• The remote eval server requires the x-bt-org-name header. If both Authorization and x-bt-auth-token are present, it uses the token from x-bt-auth-token

​

Fix or suggestion

​

Option 1: Allow hostname-based HTTPS egress (recommended)

1. Add hostname-based HTTPS allow rules for api-eu.braintrust.dev and www.braintrust.dev

​

Option 2: If hostname rules are not possible

1. Work with your network team to establish reliable HTTPS connectivity from the pod to www.braintrust.dev and api-eu.braintrust.dev
2. If your network team uses an IP allowlist for api-eu.braintrust.dev, keep the published CloudFront address ranges current.

​

How to confirm it worked

After either egress approach:

• Verify from the affected pod that HTTPS requests to www.braintrust.dev and api-eu.braintrust.dev complete without TLS errors.
• Run the login test from the affected pod using the same current token received by the remote eval server. If both auth headers are present, use the token from x-bt-auth-token. Keep the token local. Repeat the test to confirm HTTP 200, an org_info array containing your organization name, and no TLS resets.
  curl -X POST 'https://www.braintrust.dev/api/apikey/login' \
    -H "Content-Type: application/json" \
    -H 'Authorization: Bearer <same-current-token-received-by-the-server>' \
    -d '{}' \
    --write-out '\nHTTP %{http_code}\n'
  
• Repeat the previously failing GET /list requests from the affected pod and confirm they continue to return 200. Confirm that matching pod logs with the same request_id show status=200

​

Notes

• See the Remote evals docs for architecture and URLs: Remote evals.
• When reproducing the failed GET /list request, capture its UTC time, URL and method, response body, request or response ID, matching pod logs, and received auth and org headers. Redact token values.