[LRUG] Continuous * (Happy New Year!)

[Matthias Berth]

Hi Sam,

great topic! Based on the Wikipedia articles you're looking for
"Continuous Deployment"

> what I'm really looking for the right term for the practise of automatically
moving code along a pipeline like the one I mentioned above: develop -->
staging --> production.

But I guess you want to solve a real problem:

> It seems like we can now create our own automatic pipeline
progression with cron jobs so that, say, on Monday at noon, the develop
code is rebased into staging, and at noon on wednesday the staging code
is rebased into production, and in each cases tests and deploys would be
kicked off.
> My main motivation to automate this is to remove the manual step which is a chore and can get put off.

My guess: you're not there yet. The manual steps are in your process
to mitigate  some risks.
You'll run into these risks when you start doing the rebases on a
schedule. Say, next Monday the develop code is rebased onto staging,
you see a
bug that you cannot fix until Wedesday noon. Now what?

Continuous Deployment is really hard (or impossible) unless you have a
few things in place:

- release really small changes, like one pull request. That implies
IMHO developing with very short-lived branches and keeping master in a
releasable state all the time.
- be able to back out of (revert) a bad release
- be able to switch off a feature, or activate it selectively to
testers / reviewers / beta customers
- be really sure that it works, before you deploy (unit tests,
functional tests etc)

Based on your post, I guess you're now releasing a few times per
month. How often is that successful, i.e. no surprises between staging
and production? How much manual testing / checking do you have to do
for a typical release?

What is your next goal? Release to production once a week? Once a day?
Continuously?

Cheers and Happy New Year!

Matthias

On Mon, Jan 7, 2019 at 10:51 AM Samuel Joseph <tansaku at gmail.com> wrote:
>
> Hi LRUG,
>
> Happy New Year!  Hope you all had a good one.
>
> Apologies in advance for what has become a bit of a long post, but I
> have a question about "Continuous *", i.e.
>
> * Continous Integration (C.I.)
> * Continous Deployment
> * Continous Delivery
>
> I think I understand Continuous Integration quite well.  I take it to
> mean that all tests are run whenever you commit code to version
> repositories in the cloud, and thus we talk about C.I. providers such as
> Travis, Semaphore, CircleCI, CodeShip etc.   I interact with Travis and
> Semaphore on a daily basis and see the results of automated tests run on
> all our pull requests and again when we merge them in.  I think
> technically the concept of C.I. originally means just having all
> developers getting their work merged in to the same place with high
> frequency, but anyway, I feel relatively comfortable with this term.
>
> The way I hear Continuous Deployment being used seems to be when the
> C.I. tests are set such that on a passing build, the code gets
> automatically deployed to a server.   We have a few pipelines where we
> have develop, staging and production servers, which are automatically
> deployed to as a result of passing builds on the develop, staging and
> master branches respectively.  These deployments are supported by hooks
> on Travis, Semaphore etc. and are very handy.  Continous Delivery I'm
> not so sure - I just found the term of google
>
> Of course I have read the wikipedia pages on all these terms:
>
> https://en.wikipedia.org/wiki/Continuous_integration
> https://en.wikipedia.org/wiki/Continuous_deployment
> https://en.wikipedia.org/wiki/Continuous_delivery
>
> and while I'd love to tune up my mental pictures of each, what I'm
> really looking for the right term for the practise of automatically
> moving code along a pipeline like the one I mentioned above: develop -->
> staging --> production.
>
> At the moment the process of running tests and deploying to each server
> for each branch is completely automatic.  However, the process of moving
> code from develop to staging, or staging to production is manual.  We've
> previously reached out to Travis, Semaphore and Heroku to ask about some
> process that would automate moves along the pipeline, but the
> conversation seems confused by the ambiguity in the technology and I'm
> left thinking they don't - but could be wrong.
>
> We've recenly moved one pipeline completely from heroku to azure/dokku
> and it seems like we can now create our own automatic pipeline
> progression with cron jobs so that, say, on Monday at noon, the develop
> code is rebased into staging, and at noon on wednesday the staging code
> is rebased into production, and in each cases tests and deploys would be
> kicked off.   My main motivation to automate this is to remove the
> manual step which is a chore and can get put off.  It's particularly
> highlighted by our use of dependabot, which is automatically putting in
> PRs based on library upgrades, so every week there is a several upgrades
> to go out along with the usual features and bug-fixes.
>
> I speculate that if we had such a setup, we'd get into the habit of
> being more careful with merging PRs (knowing they'd be automatically
> deployed to production) and regularly doing the few additional front end
> manual sanity checks when we're notified of staging and production
> deploys ...  Anyway, I'd love to know if there's a correct term to be
> using to describe the pipeline automation we want to set up, and whether
> there are any providers that make it easy to do.
>
> We hear all the time about facebook, netflix etc. deploying to
> production multiple times a day, but I'm very interested to hear about
> practices at all scales.  Sorry for the long post - here's wishing
> everyone a very prosperous 2019!
>
> Best, Sam
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> Archives: http://lists.lrug.org/pipermail/chat-lrug.org
> Manage your subscription: http://lists.lrug.org/options.cgi/chat-lrug.org
> List info: http://lists.lrug.org/listinfo.cgi/chat-lrug.org