Export cursor and service account handoff guidance

​

Summary

Export automations run under an internal project-scoped identity. The recorded user_id is metadata and does not change the runtime credentials. Inspect the aggregated cursor summary with POST /brainstore/automation/get-cursors. The checks below do not establish a gap-free handoff.

​

What is happening

btql_export automations keep separate cursor state per automation and per storage segment. A deleted automation ID does not transfer its cursor to a new automation even if both use the same S3 export_path. For exports, the top-level status shortcut is not reliable. Pause state is returned under config.status and an omitted value means active.

​

Fix or suggestion

​

Register a replacement export

1. Pause the existing automation. Update the complete automation config with config.status: "paused".
2. Verify that GET /v1/project_automation/{id} shows config.status: "paused".
3. Inspect the aggregated cursor summary with POST /brainstore/automation/get-cursors using automation_id and project_id. The summary does not return individual per-segment cursor values.
4. Wait for any inflight exports to finish. An unchanged cursor summary is not proof that none are running.
5. Use a service token to authenticate creation of the replacement automation with config.status: "paused".
6. Register the replacement with POST /brainstore/automation/reset-cursors, passing its automation_id, object_id: "project_logs:<project_id>", and the intended start_xact_id.
7. Inspect the replacement’s aggregated cursor summary with POST /brainstore/automation/get-cursors before activation.
8. Activate the replacement by updating its complete config with config.status: "active".

​

S3 checkpoint limitation

Do not use the largest _xact_id in S3 as a guaranteed gap-free handoff boundary. It may not represent a complete checkpoint across segments.

​

How to confirm it worked

• Review the replacement’s Export status dialog to confirm successful runs.

​

Notes

• Specify start_xact_id when setting a starting position. Omitting it resets the starting value to zero.
• Restarting from an earlier position can write overlapping rows alongside existing files in the export path.