[LRUG] Idempotency vs the cloud

[Paul Battley]

I've found working with both Chef and Puppet to be a dispiriting
exercise. There's a lot of (what to me feels like arbitrary)
complexity[0] to learn and fight through, and it seems very hard to
test configuration in any way except empirically, which means that you
never really get the full benefits of idempotency. In fact, I suspect
that the massive inherent complexity of an operating system's state is
the insoluble problem here.

The solution to massively complex software is to break it into
smaller, more understandable parts, and I don't see that this is
necessarily any different. Virtualisation solves *some* of the
problems, but I suspect that tools that facilitate a smaller unit of
computing than the server - CoreOS[1], LXC, Docker[2], CDE[3] etc. -
are probably a more fruitful direction. And in that world, idempotent
system configuration does hopefully become much less important.

Except - whilst you can provision and terminate web servers with
reckless abandon, you can't quite do the same with data storage.
Networks are slow; well, they're never fast enough, anyway. I think
I'd rather have to fight Puppet than clone databases. Moving data
around seems like a problem that's going to be with us for a long
time, at least until we all move to magic self-healing distributed
databases, i.e. never.

Paul.

[0]: Not least the kitchen/LOLCAT-inspired nomenclature of Chef. WTF
even is a data bag?
[1]: http://dev.coreos.com/
[2]: http://dotcloud.github.io/docker/
[3]: http://www.pgbovine.net/cde.html

On 17 July 2013 12:13, Tom Stuart <tom at codon.com> wrote:
> Hi LRUG,
>
> I enjoyed Simon's talk about Chef on Monday. I'm an occasional user of both Puppet and Chef, and I like the idea of configuration management in principle, but I've never achieved full proficiency with either and I'm interested in why I find it such a struggle to properly integrate them into my life.
>
> One recurring thought I've had, which the talk reinforced, is that some of the complexity (and concomitant notional benefit) of these tools is their ability to convert a declarative description of a desirable state into an idempotent series of imperative actions. And okay, that's great, but is it a bit of a historical accident?
>
> The assumption is that we have persistent servers whose state needs to be carefully massaged over the course of geological time, and that's usually true, but would we have even bothered evolving these tools if some constraint or trend had made us more accustomed to deploying updates by spinning up a fresh EC2 instance instead of frobbing an existing one? If things begin to move more in that direction, can we stop caring about idempotent configuration management, and can it therefore become drastically simpler?
>
> This question appeals to me because of its connection with the imperative vs functional programming situation. Imperative programming is easier (in the Hickey[0] sense) because it more closely matches our intuitive stateful view of the world, but as any fule kno we eventually get into trouble because of the complexity which emerges from over-reliance on state. Functional programming offers an appealing remedy: stop worrying about state, treat everything as a mapping instead of a mutation, and concentrate on how to turn inputs into outputs. Running a one-shot shell script to provision a fresh instance feels simpler than getting Chef or Puppet to massage an existing one; it's not quite functional programming, but it's a gesture in that direction, if only because it doesn't concern itself with how to keep the plates spinning.
>
> tl;dr: If you don't care about running your recipes/manifests/whatever repeatedly, do Chef and Puppet still add enough value to justify their complexity?
>
> Cheers,
> -Tom
>
> [0] http://www.infoq.com/presentations/Simple-Made-Easy
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> http://lists.lrug.org/listinfo.cgi/chat-lrug.org