BTQL timeouts caused by oversized row fields

​

Summary

Your export requests to POST /btql time out or return 502 because a single page can contain very large rows. A page with a fixed row LIMIT can still produce hundreds of MiB of internal response when individual rows are large, causing the server to take longer than your client timeout. Drop heavy fields, run narrow diagnostic queries, or use cloud storage exports or async queries to avoid client timeouts.

​

What is happening

A fixed LIMIT controls row count, not bytes per page. Individual rows can be much larger than earlier pages, which increases server processing and internal response bytes. Your client times out and disconnects. You may see an HTML 502 from the Google load balancer even though the server completes the request and returns HTTP 200 after your client has already disconnected. The measured internal response size can be hundreds of MiB while the load-balancer payload is small.

​

Fix or suggestion

​

Option 1: Project away heavy fields

1. Run a narrow projection at the failing cursor without changing your checkpoint:
   SELECT id, _xact_id, created
   FROM project_logs('<project_id>', shape => 'spans')
   WHERE _xact_id > '...watermark...'
   LIMIT 50 OFFSET '...cursor...'
   
2. If the narrow query returns fast, add estimated_cost() and then add other columns one by one to find the heavy field.
3. Remove or exclude the heavy field from your export query. If you need the heavy field, test a small cloud storage export containing your largest full records, as described in Option 2.
4. Instead of only lowering LIMIT, split work by transaction or time ranges using _xact_id or created filters so each request touches fewer large rows.

​

Option 2: Use server-side export or async queries

Cloud storage exports and async queries require Enterprise.

1. Use Braintrust cloud storage exports to S3 on Braintrust-hosted deployments or to GCS on self-hosted GCP deployments. This avoids the API response-processing path.
2. When exporting large rows to BigQuery, validate per-row limits. Google documents an approximately 100 MB maximum per complete JSON row. Export a small sample containing your largest full records to GCS and confirm BigQuery can load them before scaling.

​

How to confirm it worked

• For a modified synchronous export query, re-run the failing page at the original cursor and confirm the expected projected rows return within your client timeout without ReadTimeout or HTML 502 errors.

​

Notes

• For server-side export details, see Data management exports.