AWS access and security

Deployment automation and Braintrust engineers use separate roles in your AWS account. Review their capabilities, the guardrails you apply, and the records you can audit before applying the bootstrap.

​

Roles and access

Braintrust uses three roles in your account, which separate automated deployment and standing inspection from temporary access that can reach your data. <BOOTSTRAP_NAME> is your bootstrap name. All three roles use the /braintrust-byoc/ IAM path, which is part of their ARNs, not their role names.

​

What each role can do

The following limits and exceptions apply:

• Deployment scope. The deployment role’s permissions cover all data planes under its bootstrap. Separate Terraform state keys and resource prefixes don’t provide complete IAM isolation between data planes. Some infrastructure permissions and discovery APIs apply across the account, and credentials aren’t restricted to the resources involved in each operation.
• EKS permissions. Cluster creation requires a BraintrustDeploymentName tag matching the reserved prefix. Cluster updates and deletion, and node group and add-on management, are scoped to cluster names under that prefix. The deployment role can create access entries for Deployment and Observer, but not Diagnostics. These permissions don’t provision an EKS cluster or install Kubernetes role bindings.
• Diagnostics operational changes. Activated Diagnostics is limited to ECS services and tasks, Brainstore scaling and instance health, Lambda concurrency, the module’s scheduled jobs, database and cache reboots, and database parameter tuning. Instance types, database capacity, and network changes go through the deployment workflow, which also reverts temporary Diagnostics changes or makes them permanent. These changes can affect availability and cost: ECS service updates can change more than task counts, and ECS Application Auto Scaling permissions apply across the account in the configured Region. Monitor them alongside deployment operations.
• No role chaining. Observer can’t assume other roles.
• Diagnostics credential exposure. Activated Diagnostics can also read secrets through workloads and their runtime credentials. Its ECS task-definition access is a separate AWS API path that can expose credentials without opening a shell.
• Visible metadata. Observer can inspect resource names, tags, descriptions, stack outputs, task overrides, and schedule inputs, even though it can’t read application data directly. Keep secrets and customer payloads out of this metadata.

​

Trust and credentials

Braintrust doesn’t use IAM users or long-lived access keys in your account. Every session uses short-lived AWS Security Token Service (STS) credentials, and each role has a maximum session duration of one hour.

• Deployment role. Trusts one exact Braintrust role dedicated to your bootstrap, not the whole Braintrust AWS account. Each session must present your unique External ID and a source identity of the form byoc-operation-<operation-id>, which ties recorded AWS API calls to a specific Braintrust operation.
• Observer and Diagnostics roles. Each trusts one exact Braintrust source role used by Braintrust’s single sign-on. Each session must carry a source identity that Braintrust’s identity provider binds to the individual engineer, so CloudTrail attributes recorded AWS API activity to a named person. CloudTrail doesn’t record shell contents. Diagnostics shell commands and output go to the separate transcript log group.

The deployment role can’t edit its own role, trust policy, or attached policies. Increasing its access requires a bootstrap update that you apply. Data-plane module upgrades don’t change the bootstrap or grant additional permissions.

​

Temporary Diagnostics access

Diagnostics provides temporary access for troubleshooting and operational changes, separate from the standing Observer role. Braintrust engineers use activated Diagnostics for every operational change, including routine maintenance, whether or not they open a shell.

• Inactive by default. The Diagnostics role has no policies attached until Braintrust activates it. Activation attaches two policies, BraintrustInspectionPolicy-<BOOTSTRAP_NAME> and BraintrustDiagnosticsPolicy-<BOOTSTRAP_NAME>, and closure removes them. Observer uses the same inspection policy.
• Time-limited. An activation lasts up to two hours. Extensions are explicit and recorded.
• Verified cleanup. At expiry or closure, Braintrust removes both Diagnostics policies and closes the associated sessions. An expired deadline or removed policies alone don’t prove access has ended. Outages can delay cleanup. Braintrust retries incomplete cleanup and confirms closure only after verifying that the policies and associated sessions are gone.
• Recorded. Braintrust records the requesting engineer, the reason, the scope, the expiry, any extensions, a reference to your approval, and the cleanup result in your operation log bucket.
• Logged sessions only. When engineers need a shell, they connect through Session Manager shells on Brainstore instances and ECS Exec in containers. SSH and port forwarding aren’t used. Session commands and output are written to the diagnostics CloudWatch log group in your account.
• Individual sessions. One activation covers all data planes under the bootstrap. Each engineer uses a session tied to their identity.
• Logging checks. Activation verifies the protected session document, transcript destination, and delivery configuration. Missing or unexpected logging configuration blocks activation, or triggers closure and session cleanup during active access.

Do not grant the Diagnostics role or its sessions independent access through resource policies. That access would not be removed by detaching the two activation policies. Approval follows your agreed process rather than an automated check on your side. Braintrust records the approval reference agreed for each activation and extension, which can be a standing approval for an investigation. Confirm your approval process with your account representative. A privileged shell can interfere with transcript capture, so transcripts aren’t guaranteed to be a complete record of every action. For stronger separation, copy the diagnostics log group to a protected archive you control, outside Braintrust’s management access. You can use your existing logging and archive setup, such as CloudWatch Logs centralization. The diagnostics log group in the BYOC account stays the destination for session transcripts.

​

Bootstrap security controls

The bootstrap’s IAM policies grant access. AWS Organizations service control policies (SCPs) set limits on that access but don’t grant permissions themselves. Braintrust provides three reference SCPs as JSON with the bootstrap package. These safeguards are required: apply all three, or use existing organization controls that provide the same safeguards. The JSON doesn’t need to be copied exactly, but you must validate the combined controls before production use. The role policies restrict which AWS services can receive passed roles. Access SCP

• Requires the runtime boundary on every workload role the deployment creates, and limits workload role creation and PassRole to the reserved prefix. The exception is AWS service-linked roles: the deployment role can create them only for a fixed set of AWS services, and they don’t receive the runtime boundary or fall under SCPs.
• Protects the bootstrap roles and policies, your audit controls, and account-level S3 Block Public Access.
• Denies direct reads of application objects and secrets outside documented exceptions, and caps KMS decryption.
• Denies log export and routing by the deployment role.
• Denies Braintrust roles access to S3 buckets owned outside the account.

Retained data SCP

• Protects the resources kept when you deprovision: Brainstore buckets and objects, KMS keys and aliases, final RDS snapshots, and operation records.
• Reinforces the data, log, and interactive-access restrictions on Observer.

Diagnostics SCP

• Protects the Session Manager document and diagnostics log group.
• Prevents workloads from starting shell or command sessions.

When you deploy the SCPs:

• Your administrators keep control of account settings. The administration deny applies only to Braintrust’s management and workload roles. Your administrative roles outside the reserved IAM namespaces remain in control of account settings. Other explicit SCP denies still apply to your roles when they match a statement.
• Use the ready-to-apply files. Braintrust supplies each SCP as a ready-to-apply policy file with your deployment values filled in, plus a readable copy for review. Apply them with your own policy deployment process.
• Don’t bypass the limits with resource policies. Don’t grant permissions directly to workload role sessions through resource policies. Those grants can bypass implicit identity-policy and permissions-boundary limits, although explicit SCP restrictions still apply.

New data-plane features can require updates to the deployment policies, the runtime boundary, or the SCPs. Braintrust versions these changes and tells you what to update. Routine upgrades don’t change the trust restrictions, runtime boundary, SCPs, audit ownership, or data-access restrictions.

​

Access outside the BYOC account

Braintrust’s three roles manage the BYOC account. The deployment role also reads deployment artifacts from one named Braintrust-hosted S3 bucket. The access SCP blocks the roles from other S3 buckets owned outside the account. Data-plane workloads use resources in the BYOC account. Some features you turn on need access to a role or resource outside it. For example, the Bedrock integration can require the gateway to assume an IAM role in another account, and S3 export writes to a bucket you designate. These features stay unavailable until you configure that access.

​

State and audit records

These records stay in storage you control:

• CloudTrail records AWS API activity by Braintrust roles, for the event types you enable. Each event carries the role session’s source identity, which identifies the Braintrust operation or the individual engineer. CloudTrail doesn’t record shell commands or output.
• The diagnostics CloudWatch log group stores the commands and output of Diagnostics sessions.
• The operation log bucket stores a record of each Braintrust operation, such as a deployment’s outcome or a Diagnostics activation. The deployment role can write records but not read or delete them. Keep versioning enabled, configure retention controls, and consider S3 Object Lock for operation records.
• The state bucket stores Terraform state. State can contain generated credentials, so treat it as sensitive.

Deployment, Observer, and activated Diagnostics can read regional CloudTrail event history through cloudtrail:LookupEvents. IAM cannot restrict this API to a single event name. For Diagnostics session cleanup, the deployment service requests a bounded ExecuteCommand history and discards command bodies and transport tokens. These are application checks, not IAM restrictions.

​

Monitor Braintrust activity

You own CloudTrail, alerting, and the audit archive, and you can use your existing trails, pipelines, and archive. You don’t need to create new ones for BYOC. In addition to the bootstrap security controls, Braintrust recommends that your audit collection covers:

• All Regions, with events delivered to an archive that Braintrust roles can’t access, such as a log archive account.
• Read and write management events.
• S3 data events for the Brainstore, Terraform state, and operation log buckets.
• Data events for the deployed Lambda functions, DynamoDB tables, and SQS queues.

Send alerts to a destination you control that Braintrust can’t access, such as your existing SIEM or an EventBridge bus. Evaluate both successful calls and authorization failures. A denied attempt at a sensitive action can indicate a compromised or misconfigured session. At minimum, alert on the following: Braintrust can provide the full alert matrix and request-parameter checks with the bootstrap package. The deployment role manages Brainstore lifecycle configuration during normal operation, and IAM can’t distinguish a safe lifecycle change from one that adds an expiration rule. Alert on every PutBucketLifecycleConfiguration call, and rely on bucket versioning and recovery testing.

​

Revoke access

You can revoke Braintrust’s access at any time without the Braintrust API or Braintrust support. The deployment role can’t restore trust that you remove. Blocking new sessions and ending existing access are separate steps:

1. Block new sessions. Remove Braintrust from the trust policy of the affected role, or of all three roles.
2. End Diagnostics access. Detach BraintrustInspectionPolicy-<BOOTSTRAP_NAME> and BraintrustDiagnosticsPolicy-<BOOTSTRAP_NAME> from BraintrustDiagnosticsRole-<BOOTSTRAP_NAME>. These managed policies use the /braintrust-byoc/ IAM path.
3. Revoke issued credentials. Revoke active sessions for each affected role. Credentials issued before you changed trust stay valid until they expire, for up to one hour, unless you revoke them.
4. Close open connections. Terminate any active Session Manager or ECS Exec sessions.

​

Security review materials

Ask your Braintrust account team for these security review materials:

• The bootstrap package and every policy it applies, for review before you apply it.
• SOC 2 reports and other materials in the Braintrust Trust Center.
• Penetration test summaries, under NDA.
• The Data Processing Agreement and subprocessor list.

​

Next steps

• Set up BYOC on AWS: Prepare your account and apply the reviewed bootstrap.
• BYOC operations and lifecycle: Review upgrades, incident response, and offboarding.