[LRUG] Continuous * (Happy New Year!)

Wed Jan 9 08:29:19 PST 2019

> so instead of thinking about "deploy staging" and "deploy production" you
instead think "deploy master to staging" and "deploy master to production".

Very well put

You probably also want a strategy for giving you back the confidence of
being able to know what is on prod/staging at any point.

It depends on your setup. Maybe it could be adding a version endpoint that
returns the git sha?
On Wed, 9 Jan 2019 at 12:05, Gareth Adams <g at rethada.ms> wrote:

> Sam,
>
> Ok, it sounds like you don't have the problem I was worried you had. If
> your environment branches don't diverge from each other then the rebase
> should be identical to the corresponding merge - git shouldn't be creating
> a merge commit in that case.
>
> Personally I don't agree with the obsession with single path for histories
> - it's a tool built to embrace branching, and by trying to avoid that
> you're removing valuable data from your history. But that's a divisive
> opinion so should probably be a whole separate thread if people have
> opinions. Don't even consider changing that approach just based on this
> conversation.
>
> In your case, if the aim is to deploy identical code everywhere, then I
> think you should try and reframe in your head, so instead of thinking about
> "deploy staging" and "deploy production" you instead think "deploy master
> to staging" and "deploy master to production". I don't think the
> environment branches are needed in your setup, and they're a legacy of a
> different workflow.
>
> On Wed, 9 Jan 2019, 09:53 Samuel Joseph <tansaku at gmail.com wrote:
>
>> Hi Gareth,
>>
>> Thanks - interesting point.
>>
>> As far as I'm able to detect the code on the staging branch is the same
>> code that we end up deploying to production, since what we deploy to
>> production in the master branch, and when we rebase the code from the
>> staging branch into master, the staging branch and the master branch end up
>> being identical, assuming there are no conflicts, and the way we operate
>> that is only once in a blue moon, and any extra changes there would be
>> pulled back to staging and develop.
>>
>> Maybe I am mis-understanding how rebasing works (entirely possible), but
>> when I rebase staging into master what I am doing is avoiding creating a
>> merge commit, so that we have a nice smooth single threaded git history.
>> Again my understanding (possibly faulty) is that when I rebase staging into
>> master I am making the code on master be identical to that on staging,
>> assuming that they have a shared history, but that staging has a few new
>> commits, e.g. currently our staging branch history looks like this:
>>
>> ```
>> * 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD -> staging,
>> origin/staging, origin/master, master) [Nick Schimek]
>> * bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to 5.6.0.349
>> (#2939) [dependabot[bot]]
>> * 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
>> [dependabot[bot]]
>> * d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919) [Sam
>> Joseph]
>> * 7f42d570 2018-12-11 | reduce events caching to one hour (#2936) [Sam
>> Joseph]
>> * aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0 (#2934)
>> [dependabot[bot]]
>> * aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
>> [dependabot[bot]]
>> * 298a8118 2018-12-11 | 2883 improve 500 error page (#2931) [ElisaRmz]
>> ```
>>
>> and our master history looks like this:
>>
>> ```
>> * 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD -> master,
>> origin/staging, origin/master, staging) [Nick Schimek]
>> * bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to 5.6.0.349
>> (#2939) [dependabot[bot]]
>> * 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
>> [dependabot[bot]]
>> * d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919) [Sam
>> Joseph]
>> * 7f42d570 2018-12-11 | reduce events caching to one hour (#2936) [Sam
>> Joseph]
>> * aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0 (#2934)
>> [dependabot[bot]]
>> * aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
>> [dependabot[bot]]
>> * 298a8118 2018-12-11 | 2883 improve 500 error page (#2931) [ElisaRmz]
>> ```
>>
>> and they are identical because when I did our last deploy (kicking off by
>> manually rebasing staging into master), the few new commits on staging were
>> added onto master.
>>
>> The environment specific branches don't have any evironment specific
>> config, and they don't contain different combinations of feature branches,
>> except in as much that at any given moment we may have a few new
>> features/bug-fixes/upgrades in develop that are not in staging that are not
>> in master, but as they move along (what I call) the pipeline (perhaps I'm
>> using the term incorrectly), all of those features/bug-fixes/upgrades will
>> end up in master/production, and if we have a quiet spell with nothing new
>> coming in, all three branches will be identical.
>>
>> Nevertheless, yes, we certainly want to deploy the same code everywhere,
>> as you suggest, and I think we are doing so - I could be wrong.  Perhaps
>> our three branches, each tied to a different server, is more complex than
>> we need, or just rather different from what everyone else is doing ...
>>
>> Thanks so much for taking the time to comment.
>>
>> Best, Sam
>>
> On 08/01/2019 10:33, Gareth Adams wrote:
>>
>> Sam, I don't have a grand answer to your whole question, but a phrase
>> leapt out at me and I wanted to flag it:
>>
>> > rebasing along the pipeline
>>
>> To me, this suggests the code on your staging branch is not the same as
>> the code you end up deploying to production (and it might not be the same
>> as the code in your master branch)
>>
>> I guess there are a few reasons that could be: you're storing some
>> environment-specific configuration on your environment branches, and can't
>> merge them all together, or maybe your environment branches contain a
>> different combination of feature branches that you're trying to keep
>> control of?
>>
>> Either way, I'd consider how (or if) you could change your workflow to
>> make sure you deploy the same code everywhere (or at least deploy the same
>> code to production that you deploy to staging). That's the basis behind
>> e.g. Heroku pipelines' "Promote" button and it's the pattern I commonly see
>> now
>>
>> On Tue, 8 Jan 2019, 10:13 Samuel Joseph <tansaku at gmail.com wrote:
>>
>>> Hi Gerhard,
>>> On 07/01/2019 12:00, Gerhard Lazu wrote:
>>>
>>> Hi Sam,
>>>
>>> What determines that a build can go from your development environment
>>> into staging?
>>>
>>> Good question, the answer is:
>>>
>>> 1) that all the unit, integration and acceptance tests pass
>>> 2) that there are no merge conflicts
>>> 3) that the manual sanity checks on develop are coming back okay
>>>
>>> And from staging into production? If you can capture this in code, you
>>> can put it into a pipeline.
>>>
>>> I don't think there's any way we can remove the manual sanity checks as
>>> the acceptance tests are just not that reliable, and although we've poured
>>> 1000's of hours into them and ultimately I can't see any way of making them
>>> perfect.
>>>
>>> I didn't think the presence of a manual step would prevent us using a
>>> pipeline, in as much as I thought of a pipeline as just being a series of
>>> servers with matching branches and code is them moved along them whether
>>> manually or automatically.  Heroku calls such things pipelines and seems to
>>> have no support for automatically moving code along them, it's purely
>>> manual from what I can see.
>>>
>>>
>>> Why do you have 3 pipelines?
>>>
>>> I don't think we do.  As I understand it, we have one pipeline:
>>>
>>> develop branch + develop server ---> staging branch + staging server
>>> ---> master branch + production server
>>>
>>> That's three paired branches/servers in one pipeline.  Here's a
>>> screenshot of how Heroku presents our pipeline in their pipeline
>>> interface.  Note the button "Promote to staging" which allows you to
>>> manually move the code on the develop server to the staging server, but
>>> doesn't actually do a rebase of the code from develop branch to staging:
>>>
>>> Based on the questions that you're asking, I believe that it would help
>>> if you had a single pipeline.
>>>
>>> I agree - I think we do, but perhaps I'm wrong ...
>>>
>>> The question that I would focus on is *What would it take to have a
>>> single pipeline that has an end-goal of creating production builds*?
>>> Here is a pipeline example which stops after it publishes a Docker image: changelog.com,
>>> CircleCI
>>> <https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f>
>>> .
>>>
>>> Ooh, thanks for sharing! I had to log in to CircleCI to see that:
>>>
>>> but that looks really interesting.
>>>
>>> If you are using something like Docker Swarm or Kubernetes, the
>>> platform/ecosystem has all the necessary tools to keep deployment concerns
>>> self-contained. In the changelog.com case, the Docker stack that
>>> captures the entire deployment has an update component that is
>>> responsible for app updates
>>> <https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31>.
>>> In this specific case, if the new app version starts and is healthy for 30
>>> seconds, it gets automatically promoted to live. We have been using a
>>> similar approach since October 2016 <http://changelog.com/podcast/254>,
>>> a Docker stack just makes it easier.
>>>
>>> I want to spur your imagination by sharing the pipeline that is
>>> responsible for RabbitMQ v3.7.x
>>> <https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x>.
>>> This pipeline captures what is possible if imagination is set free:
>>>
>>> Wow, that RabbitMQ pipeline looks amazing - and I've only captured part
>>> of it in the screenshot:
>>>
>>>
>>> * tests & builds 30+ apps...
>>> * on all supported major runtime version...
>>> * and all supported OSes
>>> * tests upgrades
>>> * tests client support
>>> * releases alphas, betas, RCs & GAs
>>> * and publishes to all supported distribution channels
>>>
>>> I hope this helps, Gerhard.
>>>
>>> That's extremely helpful - thankyou!
>>>
>>> But so just to be clear, there is something in these pipelines that
>>> you're sharing that regularly moves code from one branch to another?  And
>>> that's something that CircleCI and RabbitMQ provide?  Or these are
>>> pipelines where the same code in the same branch is being moved through a
>>> series of servers, based on tests and checks passing at each server?
>>>
>>> Many thanks in advance
>>> Best, Sam
>>> _______________________________________________
>>> Chat mailing list
>>> Chat at lists.lrug.org
>>> Archives: http://lists.lrug.org/pipermail/chat-lrug.org
>>> Manage your subscription:
>>> http://lists.lrug.org/options.cgi/chat-lrug.org
>>> List info: http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>>>
>> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org
> Archives: http://lists.lrug.org/pipermail/chat-lrug.org
> Manage your subscription: http://lists.lrug.org/options.cgi/chat-lrug.org
> List info: http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190109/6d7e14fe/attachment-0003.html>