shiichan

CloudFormation updates finally get smarter with Instance Refresh!

Hi, I'm Shii-chan! Today I found news that might sound small at first, but it's actually a big deal if you run Auto Scaling groups through CloudFormation. Let me walk you through it!

AWS What's New aws.amazon.com

What was announced?

According to AWS What's New, Amazon EC2 Auto Scaling now supports Instance Refresh as a new AWS CloudFormation update policy. When you set the new AutoScalingInstanceRefresh update policy in your template and then update a property that requires instance replacement (like LaunchTemplate or VPCZoneIdentifier), CloudFormation automatically triggers an Instance Refresh for you.

This feature is available in all AWS Regions at no additional cost.

The story so far

Until now, if you wanted CloudFormation to replace instances in an Auto Scaling group during a stack update, the go-to option was AutoScalingRollingUpdate. But rolling updates come with some real limitations. As the CloudFormation documentation puts it directly:

Rolling updates don't honor the instance maintenance policy configured on your Auto Scaling group. During a rolling update, healthy capacity can drop below the MinHealthyPercentage set on the group, and CloudFormation doesn't launch new instances before terminating existing ones, regardless of MaxHealthyPercentage.

In other words, whatever instance maintenance policy you'd carefully configured on your Auto Scaling group got ignored, and launch-before-terminate behavior wasn't possible. On top of that, you often had to use SuspendProcesses to manually pause Auto Scaling processes (like health checks or scaling policies) so they wouldn't interfere with the update. Features like termination policies and scale-in protection simply weren't available during a CloudFormation rolling update either.

What changes

With the AutoScalingInstanceRefresh policy, CloudFormation updates now ride on top of Auto Scaling's native Instance Refresh mechanism, so you get access to:

  • Root volume replacement (via the ReplaceRootVolume strategy) for in-place updates
  • Launch-before-terminate, so new instances come up before old ones are terminated
  • Alarm-based monitoring, where CloudWatch alarms can trigger an automatic rollback if a threshold is breached
  • Checkpoints with bake time, for controlled, staged rollouts

Best of all, Auto Scaling features like scaling policies and health checks stay active throughout the update, so your service health isn't at risk during the deployment. You no longer need to suspend processes manually with SuspendProcesses either. Rollback is now handled through CloudFormation's own stack rollback mechanism, which simplifies what happens when something goes wrong.

Dive Deep

Configuring this is as simple as adding AutoScalingInstanceRefresh under UpdatePolicy, something like this:

UpdatePolicy:
  AutoScalingInstanceRefresh:
    Strategy: Rolling
    Preferences:
      MinHealthyPercentage: 90
      MaxHealthyPercentage: 120
      InstanceWarmup: 300
      CheckpointPercentages:
        - 50
        - 100
      CheckpointDelay: 3600
      BakeTime: 600
      SkipMatching: true
      ScaleInProtectedInstances: Wait
      StandbyInstances: Wait
      AlarmSpecification:
        Alarms:
          - my-error-rate-alarm

Digging into the CloudFormation Template Reference, here are the concrete details worth knowing:

  • Strategy is required, and must be either Rolling or ReplaceRootVolume
  • You can't specify both AutoScalingInstanceRefresh and AutoScalingRollingUpdate on the same Auto Scaling group — doing so fails the stack update
  • CloudFormation only triggers an Instance Refresh when you change one of: LaunchTemplate, MixedInstancesPolicy, VPCZoneIdentifier, AvailabilityZones, AvailabilityZoneIds, or PlacementGroup
  • The gap between MinHealthyPercentage and MaxHealthyPercentage can't exceed 100 — a wider range lets more instances be replaced at once
  • Both BakeTime (wait time before the refresh is considered complete) and CheckpointDelay (wait time after a checkpoint) can go up to 172,800 seconds (48 hours)
  • CheckpointPercentages is a list of thresholds, and the last value must be 100 if you want all instances replaced
  • ScaleInProtectedInstances and StandbyInstances each accept Refresh, Ignore, or Wait, and both default to Wait (waiting one hour before failing if nothing changes)
  • An Auto Scaling group can only run one Instance Refresh at a time, so starting a stack update with this policy while a manual Instance Refresh is already running can cause the update to fail
  • The classic cfn-signal helper script isn't supported here — instead, use an autoscaling:EC2_INSTANCE_LAUNCHING lifecycle hook if you need to wait for instance readiness
  • Both the forward update and any rollback are bounded by CloudFormation's 36-hour resource timeout, so for long-running refreshes it's recommended to configure a service role on the stack to avoid issues from expiring temporary credentials

Wrap-up

  • CloudFormation has a new update policy, AutoScalingInstanceRefresh, that lets stack updates use Auto Scaling's native Instance Refresh mechanism
  • You get root volume replacement, launch-before-terminate, alarm-based monitoring, and checkpoints with bake time, all while scaling policies and health checks stay active
  • Rollback is now unified under CloudFormation's stack rollback, and it's available in every region at no extra cost
  • The older AutoScalingRollingUpdate policy can't be combined with it, and still carries its old constraints like SuspendProcesses

If you manage Auto Scaling groups through CloudFormation and have ever winced during an update, this one is worth checking out!