Skip to content

(cli): #1745 removed the express-mode replacement guard, so --express deploys replacements with rollback disabled #1931

Description

@deeheber

Describe the bug

cdk deploy --express no longer checks whether the change set contains a replacement before deploying it with rollback disabled. Express shipped with a purpose-built clause for this, and #1745 removed it.

2.1129.0 (the first release with express) through 2.1132.0:

const rollback = this.options.rollback ?? true;
const expressNoRollback = this.options.express && this.options.rollback !== true;
...
if ((!rollback || expressNoRollback) && replacement) {
  return { type: 'replacement-requires-rollback' };
}

#1745 deleted expressNoRollback and added an early return above the guards:

// Route express mode deployments directly to executeChangeset, express mode stacks cannot use rollback API
if (this.options.express) {
  return this.executeChangeSet(changeSetDescription);
}

#1785 later restructured that into if (!this.options.express) { ...guards... }, same behavior. As of 2.1138.0 an express deploy reaches executeChangeSet with no replacement check.

This matters because express sends {Mode: "EXPRESS"} with no DisableRollback, so the service disables rollback, and CloudFormation refuses replacement updates in that mode.

The consequence is not a failed deploy you retry. Our first cdk deploy --express on a healthy stack failed with Replacement type updates not supported on stack with disable-rollback on three AWS::ECS::TaskDefinition resources, and the stack could not be recovered: express retries hit the same replacement error, --rollback and non-express deploys are rejected with This stack is currently in a non-terminal [UPDATE_FAILED] state, and cdk rollback is refused because RollbackStack is not supported for stacks that were last updated using EXPRESS deployment mode and are in a failed state (the same dead end as #1739). We deleted the stack.

Worth flagging separately: the recovery path #1745 added does not cover this shape. It lives in checkAndExecuteChangeSet, which only runs on a successfully created change set. CloudTrail for our --express --rollback attempt shows CreateChangeSet succeeding, then 13 DescribeChangeSet polls over roughly 40s, then no ExecuteChangeSet. The change set fails on its own, so that code never runs.

Regression Issue

  • Select this option if this issue appears to be a regression.

Introduced in 2.1133.0 by #1745.

Last Known Working CDK Version

2.1132.0

Expected Behavior

cdk deploy --express without an explicit --rollback should return replacement-requires-rollback when the change set contains a replacement, and prompt, as it did through 2.1132.0. The --rollback help text already documents that express defaults rollback to false, which is what expressNoRollback encoded.

Current Behavior

No prompt, and a stack that cannot be recovered:

$ cdk deploy --express
...
<stack> failed: The following resource(s) failed to update: [<resources>]
Replacement type updates not supported on stack with disable-rollback

$ cdk deploy --express --rollback
<stack> failed: ValidationError: This stack is currently in a non-terminal [UPDATE_FAILED] state.
To update the stack from this state, please use the disable-rollback parameter with update-stack API.
To rollback to the last known good state, use the rollback-stack API

$ aws cloudformation rollback-stack --stack-name <stack>
An error occurred (ValidationError) when calling the RollbackStack operation:
RollbackStack is not supported for stacks that were last updated using EXPRESS deployment
mode and are in a failed state. To recover this stack, submit an UpdateStack request with
the last known stable template.

Reproduction Steps

  1. Deploy a stack containing an AWS::ECS::TaskDefinition and get it to UPDATE_COMPLETE. Every property change replaces a task definition.
  2. Change a property on it.
  3. On 2.1132.0, cdk deploy --express returns replacement-requires-rollback and prompts.
  4. On 2.1133.0 or later, the same command deploys without prompting and fails with Replacement type updates not supported on stack with disable-rollback.

We observed step 4 only. Our project went straight from 2.1114.1, which predates express, to 2.1138.0, so we never ran a version where express had the guard. Step 3 is from reading packages/@aws-cdk/toolkit-lib/lib/api/deployments/deploy-stack.ts at the 2.1132.0 tag, where express is an ordinary change-set deployment that reaches the guard and expressNoRollback is true whenever --rollback is absent.

Possible Solution

Restore the express case, which is your own code from 2.1132.0:

const expressNoRollback = this.options.express && this.options.rollback !== true;
...
if ((!rollback || expressNoRollback) && replacement) {
  return { type: 'replacement-requires-rollback' };
}

#1745 needed express deploys to avoid the RollbackStack API, and the two failpaused-need-rollback-first returns are the ones that lead there. replacement-requires-rollback does not call RollbackStack; it prompts and then redeploys with rollback enabled, so it can stay in the express path.

Additional Information/Context

Region us-east-1. Happy to share account and request IDs privately.

CDK CLI Version

2.1138.0

Framework Version

aws-cdk-lib 2.266.0

Node.js Version

v24.18.0

OS

macOS 26.6.2

Language

TypeScript

Language Version

TypeScript 6.0.2

Other information

Related: #1739, #1745, #1785.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions