[LRUG] What's the best way to go from 1 server to 2

[Adrian Sevitz]

Hi Andy,

We went through this a few years ago. When you start off it's easiest to have the whole stack on one box, but then as you grow you start getting more concerned with scaling, single points of failure etc etc.

We're on AWS. I love AWS, and if you start planning around it, you can build a lot of this in without too much work (although it does take some work in the beginning).

Our tendency has been around using small instances and growing those to around 8 instances. Then going form 8 small to 4 medium. Then when you grow up to 8 medium shrinking down again to 4 large. Using this philosophy I would start with 2 instances of as small as will run your platform. This gives you then redundancy from the off, but bakes in scaling.

We use Chef with our build and autoscaling along with ELB (elastic load balancing) to spin up and down servers as you need them. Even if you don't need more than 2 servers for performance, this does allow autoscaling to pull out unhealthy instances and spin in new ones. (Note: this can cause problems with only two instances as we did have issues where it would pull out a marginally healthy instance and before the replacement was spun up it overloaded the single remaining healthy instance and brought down the system. This can be mitigated with how you manage your autoscaling)

I would also pull your database out into RDS. RDS just makes it a lot easier to manage and you can scale vertically as you need. To add redundancy you can sit with an single RDS in the beginning  and make sure you have backups and snapshots to manually recover from a failure. Then as you become more mission critical you can go to multi-AZ replicas.

So my minimum would be (from the get go)

ELB → 2x EC2 with autoscaling running on chef → RDS

Start with the smallest EC2 and RDS instances you can get away with.

Then as you grow move to MultiAZ RDS.

Advantage of this is you a lot of easy scaling (vertically and horizontally) baked in.

AWS itself is pretty solid. Yes they have high profile melt down occasionally. But it's low risk, and I can't recommend them enough. 

Building with this in mind also saves you down the road from plenty of hassle.

We keep breaking off bits of our architecture into new smaller/dumber services following the same architecture as above. Once something gets big enough to run off 4 (sometimes 2, but normally 4) small instances all on it's own, we simplify our code, and break off logical function areas into their own units.

YMMV, usual caveats apply.


> Message: 3
> Date: Tue, 16 Apr 2013 12:50:42 +0200
> From: Andrew Stewart <boss at airbladesoftware.com>
> 
> Good afternoon El Rug,
> 
> What's the best way to increase from one server to two?
> 
> Currently I have everything for my webapp ? code, database, background jobs, etc ? on one server.  Performance is fine but it's a single point of failure (see this morning's email thread).  Off the top of my head I'm thinking:
> 
> - Use a different host in a different city from my current server.
> - Install same operating system as current server and set up identically via Chef/whatever.
> - Deploy all code changes to both servers with Capistrano but have second server serving Rails maintenance page (just in case anybody finds it).
> - Ideally set up live (mysql) replication...somehow.
> - If/when first server croaks, manually fail over to second server via changing DNS.
> 
> I'm sure it's more complicated than that, particularly the switching from one server to the other (and back).  Does anybody have any tips?
> 
> Thanks again,
> 
> Andy Stewart


_____________
Adrian Sevitz


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20130416/a1461c5f/attachment-0003.html>