[LRUG] Continuous * (Happy New Year!)
Samuel Joseph
tansaku at gmail.com
Wed Jan 9 01:53:46 PST 2019
Hi Gareth,
Thanks - interesting point.
As far as I'm able to detect the code on the staging branch is the same
code that we end up deploying to production, since what we deploy to
production in the master branch, and when we rebase the code from the
staging branch into master, the staging branch and the master branch end
up being identical, assuming there are no conflicts, and the way we
operate that is only once in a blue moon, and any extra changes there
would be pulled back to staging and develop.
Maybe I am mis-understanding how rebasing works (entirely possible), but
when I rebase staging into master what I am doing is avoiding creating a
merge commit, so that we have a nice smooth single threaded git
history. Again my understanding (possibly faulty) is that when I rebase
staging into master I am making the code on master be identical to that
on staging, assuming that they have a shared history, but that staging
has a few new commits, e.g. currently our staging branch history looks
like this:
```
* 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD ->
staging, origin/staging, origin/master, master) [Nick Schimek]
* bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to 5.6.0.349
(#2939) [dependabot[bot]]
* 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
[dependabot[bot]]
* d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919) [Sam
Joseph]
* 7f42d570 2018-12-11 | reduce events caching to one hour (#2936) [Sam
Joseph]
* aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0 (#2934)
[dependabot[bot]]
* aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
[dependabot[bot]]
* 298a8118 2018-12-11 | 2883 improve 500 error page (#2931) [ElisaRmz]
```
and our master history looks like this:
```
* 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD -> master,
origin/staging, origin/master, staging) [Nick Schimek]
* bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to 5.6.0.349
(#2939) [dependabot[bot]]
* 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
[dependabot[bot]]
* d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919) [Sam
Joseph]
* 7f42d570 2018-12-11 | reduce events caching to one hour (#2936) [Sam
Joseph]
* aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0 (#2934)
[dependabot[bot]]
* aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
[dependabot[bot]]
* 298a8118 2018-12-11 | 2883 improve 500 error page (#2931) [ElisaRmz]
```
and they are identical because when I did our last deploy (kicking off
by manually rebasing staging into master), the few new commits on
staging were added onto master.
The environment specific branches don't have any evironment specific
config, and they don't contain different combinations of feature
branches, except in as much that at any given moment we may have a few
new features/bug-fixes/upgrades in develop that are not in staging that
are not in master, but as they move along (what I call) the pipeline
(perhaps I'm using the term incorrectly), all of those
features/bug-fixes/upgrades will end up in master/production, and if we
have a quiet spell with nothing new coming in, all three branches will
be identical.
Nevertheless, yes, we certainly want to deploy the same code everywhere,
as you suggest, and I think we are doing so - I could be wrong. Perhaps
our three branches, each tied to a different server, is more complex
than we need, or just rather different from what everyone else is doing ...
Thanks so much for taking the time to comment.
Best, Sam
On 08/01/2019 10:33, Gareth Adams wrote:
> Sam, I don't have a grand answer to your whole question, but a phrase
> leapt out at me and I wanted to flag it:
>
> > rebasing along the pipeline
>
> To me, this suggests the code on your staging branch is not the same
> as the code you end up deploying to production (and it might not be
> the same as the code in your master branch)
>
> I guess there are a few reasons that could be: you're storing some
> environment-specific configuration on your environment branches, and
> can't merge them all together, or maybe your environment branches
> contain a different combination of feature branches that you're trying
> to keep control of?
>
> Either way, I'd consider how (or if) you could change your workflow to
> make sure you deploy the same code everywhere (or at least deploy the
> same code to production that you deploy to staging). That's the basis
> behind e.g. Heroku pipelines' "Promote" button and it's the pattern I
> commonly see now
>
> On Tue, 8 Jan 2019, 10:13 Samuel Joseph <tansaku at gmail.com
> <mailto:tansaku at gmail.com> wrote:
>
> Hi Gerhard,
>
> On 07/01/2019 12:00, Gerhard Lazu wrote:
>> Hi Sam,
>>
>> What determines that a build can go from your development
>> environment into staging?
> Good question, the answer is:
>
> 1) that all the unit, integration and acceptance tests pass
> 2) that there are no merge conflicts
> 3) that the manual sanity checks on develop are coming back okay
>> And from staging into production? If you can capture this in
>> code, you can put it into a pipeline.
>>
> I don't think there's any way we can remove the manual sanity
> checks as the acceptance tests are just not that reliable, and
> although we've poured 1000's of hours into them and ultimately I
> can't see any way of making them perfect.
>
> I didn't think the presence of a manual step would prevent us
> using a pipeline, in as much as I thought of a pipeline as just
> being a series of servers with matching branches and code is them
> moved along them whether manually or automatically. Heroku calls
> such things pipelines and seems to have no support for
> automatically moving code along them, it's purely manual from what
> I can see.
>>
>> Why do you have 3 pipelines?
>
> I don't think we do. As I understand it, we have one pipeline:
>
> develop branch + develop server ---> staging branch + staging
> server ---> master branch + production server
>
> That's three paired branches/servers in one pipeline. Here's a
> screenshot of how Heroku presents our pipeline in their pipeline
> interface. Note the button "Promote to staging" which allows you
> to manually move the code on the develop server to the staging
> server, but doesn't actually do a rebase of the code from develop
> branch to staging:
>
>> Based on the questions that you're asking, I believe that it
>> would help if you had a single pipeline.
> I agree - I think we do, but perhaps I'm wrong ...
>> The question that I would focus on is /What would it take to have
>> a single pipeline that has an end-goal of creating production
>> builds/? Here is a pipeline example which stops after it
>> publishes a Docker image: changelog.com, CircleCI
>> <https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f>.
> Ooh, thanks for sharing! I had to log in to CircleCI to see that:
>
> but that looks really interesting.
>> If you are using something like Docker Swarm or Kubernetes, the
>> platform/ecosystem has all the necessary tools to keep deployment
>> concerns self-contained. In the changelog.com
>> <http://changelog.com> case, the Docker stack that captures the
>> entire deployment has an update component that is responsible for
>> app updates
>> <https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31>.
>> In this specific case, if the new app version starts and is
>> healthy for 30 seconds, it gets automatically promoted to live.
>> We have been using a similar approach since October 2016
>> <http://changelog.com/podcast/254>, a Docker stack just makes it
>> easier.
>>
>> I want to spur your imagination by sharing the pipeline that is
>> responsible for RabbitMQ v3.7.x
>> <https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x>.
>> This pipeline captures what is possible if imagination is set free:
> Wow, that RabbitMQ pipeline looks amazing - and I've only captured
> part of it in the screenshot:
>
>>
>> * tests & builds 30+ apps...
>> * on all supported major runtime version...
>> * and all supported OSes
>> * tests upgrades
>> * tests client support
>> * releases alphas, betas, RCs & GAs
>> * and publishes to all supported distribution channels
>>
>> I hope this helps, Gerhard.
>
> That's extremely helpful - thankyou!
>
> But so just to be clear, there is something in these pipelines
> that you're sharing that regularly moves code from one branch to
> another? And that's something that CircleCI and RabbitMQ
> provide? Or these are pipelines where the same code in the same
> branch is being moved through a series of servers, based on tests
> and checks passing at each server?
>
> Many thanks in advance
> Best, Sam
>
> _______________________________________________
> Chat mailing list
> Chat at lists.lrug.org <mailto:Chat at lists.lrug.org>
> Archives: http://lists.lrug.org/pipermail/chat-lrug.org
> Manage your subscription:
> http://lists.lrug.org/options.cgi/chat-lrug.org
> List info: http://lists.lrug.org/listinfo.cgi/chat-lrug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lrug.org/pipermail/chat-lrug.org/attachments/20190109/ed295a7d/attachment-0002.html>
More information about the Chat
mailing list