[LRUG] Continuous * (Happy New Year!)

[Samuel Joseph]

Hi Gerhard,

On 07/01/2019 12:00, Gerhard Lazu wrote:
> Hi Sam,
>
> What determines that a build can go from your development environment 
> into staging?
Good question, the answer is:

1) that all the unit, integration and acceptance tests pass
2) that there are no merge conflicts
3) that the manual sanity checks on develop are coming back okay
> And from staging into production? If you can capture this in code, you 
> can put it into a pipeline.
>
I don't think there's any way we can remove the manual sanity checks as 
the acceptance tests are just not that reliable, and although we've 
poured 1000's of hours into them and ultimately I can't see any way of 
making them perfect.

I didn't think the presence of a manual step would prevent us using a 
pipeline, in as much as I thought of a pipeline as just being a series 
of servers with matching branches and code is them moved along them 
whether manually or automatically.  Heroku calls such things pipelines 
and seems to have no support for automatically moving code along them, 
it's purely manual from what I can see.
>
> Why do you have 3 pipelines?

I don't think we do.  As I understand it, we have one pipeline:

develop branch + develop server ---> staging branch + staging server 
---> master branch + production server

That's three paired branches/servers in one pipeline.  Here's a 
screenshot of how Heroku presents our pipeline in their pipeline 
interface.  Note the button "Promote to staging" which allows you to 
manually move the code on the develop server to the staging server, but 
doesn't actually do a rebase of the code from develop branch to staging:

> Based on the questions that you're asking, I believe that it would 
> help if you had a single pipeline.
I agree - I think we do, but perhaps I'm wrong ...
> The question that I would focus on is /What would it take to have a 
> single pipeline that has an end-goal of creating production builds/? 
> Here is a pipeline example which stops after it publishes a Docker 
> image: changelog.com, CircleCI 
> <https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f>.
Ooh, thanks for sharing! I had to log in to CircleCI to see that:

but that looks really interesting.
> If you are using something like Docker Swarm or Kubernetes, the 
> platform/ecosystem has all the necessary tools to keep deployment 
> concerns self-contained. In the changelog.com <http://changelog.com> 
> case, the Docker stack that captures the entire deployment has an 
> update component that is responsible for app updates 
> <https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31>. 
> In this specific case, if the new app version starts and is healthy 
> for 30 seconds, it gets automatically promoted to live. We have been 
> using a similar approach since October 2016 
> <http://changelog.com/podcast/254>, a Docker stack just makes it easier.
>
> I want to spur your imagination by sharing the pipeline that is 
> responsible for RabbitMQ v3.7.x 
> <https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x>. 
> This pipeline captures what is possible if imagination is set free:
Wow, that RabbitMQ pipeline looks amazing - and I've only captured part 
of it in the screenshot:

>
> * tests & builds 30+ apps...
> * on all supported major runtime version...
> * and all supported OSes
> * tests upgrades
> * tests client support
> * releases alphas, betas, RCs & GAs
> * and publishes to all supported distribution channels
>
> I hope this helps, Gerhard.

That's extremely helpful - thankyou!

But so just to be clear, there is something in these pipelines that 
you're sharing that regularly moves code from one branch to another?  
And that's something that CircleCI and RabbitMQ provide?  Or these are 
pipelines where the same code in the same branch is being moved through 
a series of servers, based on tests and checks passing at each server?

Many thanks in advance
Best, Sam

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190108/827c0f38/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nmfedjpijhmiddke.png
Type: image/png
Size: 113143 bytes
Desc: not available
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190108/827c0f38/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: geelgcfnnoegpbnp.png
Type: image/png
Size: 56951 bytes
Desc: not available
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190108/827c0f38/attachment-0007.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pdkbkpfocdjfmoki.png
Type: image/png
Size: 236620 bytes
Desc: not available
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190108/827c0f38/attachment-0008.png>