[LRUG] Compiling native extensions during deployment?

[Thom May]

On Sun, Oct 23, 2011 at 10:06, gareth rushgrove
<gareth.rushgrove at gmail.com> wrote:
> On 20 October 2011 14:41, Thom May <thom at may.lt> wrote:
>> We run bundle install --deployment as part of our CI process, and then
>> push out the results of CI as a tarball to our app servers.
>> I'm being hounded from various quarters to write a blog post about
>> this, so I might do that shortly.
>
> Definitely interested in that blog post.
>
Heh, you've mentioned that :)

> Building on that, have you thought about wrapping up the package as a
> deb or rpm rather than just a tarball?
>
Yes, and consciously rejected it. I miss-spoke a little when i said
"push the results to our app servers", actually, so let's go into a
bit of detail.

- jenkins is triggered to build by Github

- the last stage of our standard builds produces a tarball and
notifies our management webapp (octobutler) of the existence of the
tarball. The tarball is placed in a standardised location and made
available through http

- Octobutler queries mcollective (using a plugin called 'trebuchet')
to get the current deployed state of a given app, filtering on an ohai
fact to get the list of nodes running the app. (This gives some side
benefits like knife ssh 'trebuchet:app_name')

- When the user wishes to deploy, octobutler does the moral equivalent
of 'mc-rpc -F trebuchet:app_name trebuchet install app_name branch
sha', which causes the plugin to download the tarball, extract it and
then run chef-solo to do the final setup. We then set up a 'current'
symlink. For management, we use foreman
(http://ddollar.github.com/foreman) to generate upstart jobs.

So, why not packages? The state of the machine is managed by chef, so
pre/post scripts have little benefit (and we can use chef-solo for
easy idempotent post-extraction configuration). Using tarballs lets us
have multiple versions of an app on the machine, so
rollback/rollforward is just a symlink flip.
Our approach also decouples deployment from configuration management,
which i think is a good thing.

I completely agree that for system tools packages are the right
approach - we have internal packages for scribe, reconnoiter, etc -
but for deployments I think they're overly heavy weight.

I think we're probably in position to start opening some or all of
these tools in the near future, but I can't commit to that right now.
Hope this gives a little more insight into our approach.

-Thom