[LRUG] Idempotency vs the cloud

[Frederick Cheung]

On 17 Jul 2013, at 14:55, David Nolan <dave at textgoeshere.org.uk> wrote:
> 
> I'd love my servers to be truly ephemeral. Sadly our Chef recipes are still littered with only_if/not_if conditions to ensure idempotence. The main reasons being
> 
> - Data gets harder to shift about the more of it you have
> - If you do several deploys a day to a cloud you pay for in blocks of 1 full hour, it gets expensive to be merciless
> - It's slow to boot a fresh node/server relative to converging an existing one

I do a halfway house for app servers for the last 2 reasons, so a deploy of the app is just a 'normal' deploy. However the app is deployed to a separate volume than the boot volume which we then snapshot. On EC2 this means we can then bring up new instances quickly and simply: bake a new instance from an AMI and create a new app volume from the latest snapshot (and this works with autoscaling groups and all that sort of stuff). We use this so that autoscaling can replace dead instances/respond to traffic increases recently and we also do a lot of our overnight processing on temporary instances.

Fred