Skip to content

feat(terraform): support long running applies - #200

Merged
cdelfabbro merged 4 commits into
masterfrom
feat/long-running-terraform-applies
Aug 28, 2026
Merged

cdelfabbro merged 4 commits into
masterfrom
feat/long-running-terraform-applies

Conversation

@cdelfabbro

@cdelfabbro cdelfabbro commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Problem

A terraform apply on the sencrop-preproduction-mysql8 Blue/Green deployment failed after one hour:

Error: updating RDS DB Instance (sencrop-preproduction-mysql8): creating Blue/Green Deployment:
waiting for Green environment: operation error RDS: DescribeDBInstances,
StatusCode: 403, api error ExpiredToken: The security token included in the request is expired

This is not an RDS error: the STS session expired while terraform was still waiting. Nothing in the workflow allowed a run to go past one hour.

What was in the way

Limit Before
STS session duration 1h — configure-aws-credentials never passed role-duration-seconds, which defaults to 3600
Job timeout 6h — no timeout-minutes on terraform-apply-v2
Runner github hosted runners kill the job at 6h regardless

Changes

  • actions/configure-aws-credentials: new role_duration_seconds input, forwarded as role-duration-seconds. The AWS action does not infer the duration from the role, it always asks for one hour unless told otherwise, even on a role that allows twelve.
  • terraform-apply-v2: role_duration_seconds and timeout_minutes inputs, the latter wired to the job timeout-minutes.
  • README: a "Long running applies" section with a full example and the prerequisites.

The defaults (3600 and 360) match the current behaviour, so no existing caller is affected.

IAM prerequisite

role_duration_seconds cannot exceed the max_session_duration of the assumed role, otherwise AssumeRoleWithWebIdentity is rejected and the job fails while configuring the credentials. Our roles are created at 3600, so a caller asking for a longer session needs either a raised max_session_duration on github-actions-<repository>, or a dedicated role passed through the existing ci_iam_role input. The README documents when each one applies.

12h is the AWS maximum for a role assumed through OIDC.

with:
  self_hosted: true
  timeout_minutes: 600
  role_duration_seconds: 43200

Related PR: https://github.com/sencrop/infra-ci/pull/403

@cdelfabbro
cdelfabbro force-pushed the feat/long-running-terraform-applies branch 6 times, most recently from 02147c9 to 9c26949 Compare August 27, 2026 12:42

This comment was marked as resolved.

@PaulArnaud PaulArnaud left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you test this PR by using
sencrop/github-workflows/.github/workflows/terraform-apply-v2.yml@feat/long-running-terraform-applies in one of your workflow somewhere ? Just to be sure to not break any CI

@cdelfabbro

Copy link
Copy Markdown
Contributor Author

Did you test this PR by using sencrop/github-workflows/.github/workflows/terraform-apply-v2.yml@feat/long-running-terraform-applies in one of your workflow somewhere ? Just to be sure to not break any CI

Third try of the b/g deployment will be a good test :D

Comment thread actions/configure-aws-credentials/action.yml
Comment thread .github/workflows/terraform-apply-v2.yml
Comment thread README.md Outdated
@cdelfabbro

cdelfabbro commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

FYI, this version is currently being tested in this run.

Edit: It was using the master version of configure-aws-credentials instead of the PR's version... I pushed a commit to temporarily use the PR's version.

⚠️ do not forget to put back master.

@cdelfabbro

Copy link
Copy Markdown
Contributor Author

@cdelfabbro
cdelfabbro merged commit 0c55591 into master Aug 28, 2026
1 check passed
@cdelfabbro
cdelfabbro deleted the feat/long-running-terraform-applies branch August 28, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants