AWS Fargate & RDS Update Strategy

prisma

#1

I’m using both of Prisma’s templates (fargate.yml & aurora.yml) to create my AWS infrastructure. They work really well, and made the setup a breeze. But, they leave me a little worried about performing upgrades. Normally, with a self-built architecture, I’d make changes to tasks and service definitions, etc. But I know that doing so will massively upset CloudFormation, as all changes should be done through CF ideally.

So I’m in a situation such as this: Prisma v1.9 was just released, but my fargate.yml file only went up to v1.8.3 & so the deployment is also on v1.8.3.

Assuming I’m upgrading a production environment, I clearly need to do the upgrade with as little downtime as possible (ideally, none).

What is the recommended way of performing these upgrades for both the Fargate and Aurora/RDS stacks?

So far I’ve managed to do the upgrade by:

  • Grabbing the latest copy of the fargate.yml file
  • Doing an “Update Stack” operation
  • Providing it with the newly updated template
  • Changing the Prisma version selection in the drop-down
  • Submitting the changes

It seems to have worked but, (if I’m honest) I’m not sure whether it caused any downtime, or what exactly happened as a result of it.

Under the hood I can see that a new task definition was created and the service was updated to point at this new definition.

But I’m not sure whether this will cause downtime? And whether it causes the Prisma deployment info to be lost? Ie. Do I need to re-prisma deploy after an update such as this?

With the rapid release cycle of Prisma, I’d really like to make sure that I’ve got a handle on the update process, so that updates can be deployed shortly after they are released.


#2

To be sure to avoid a downtime, you can switch the AZ(availibilty zone).

So I guess it would be possible to :

  • Create an AZ B
  • Switch the AZ A to AZ B
  • Update AZ A
  • Test AZ A
  • Switch B to A
  • Remove or update AZ B

#3

I’m pretty certain it can be achieved easier than that…?

If the service settings allow the maximum healthy percentage to be 200%, then when a new task definition is deployed it should be able to spin up a second task, using the new task definition, in parallel with the old version. Then the load balancer will add it, wait until it becomes healthy, at which point it removes the old version… etc.


#4

This is a great question, and even though @AsPrO 's solution would probably work, I agree that it’s a lot of work for what should ideally be fairly easy.


#5

I believe what I’ve written is actually what will happen when updating the Prisma image in an update stack. But, I just wanted to make sure! And, I think it would be good to think about it & document it :wink: