> so instead of thinking about "deploy staging" and "deploy production" you instead think "deploy master to staging" and "deploy master to production". <br><br>Very well put<br><br>You probably also want a strategy for giving you back the confidence of being able to know what is on prod/staging at any point.<br><br>It depends on your setup. Maybe it could be adding a version endpoint that returns the git sha?<br><div class="gmail_quote"><div dir="ltr">On Wed, 9 Jan 2019 at 12:05, Gareth Adams <<a href="mailto:g@rethada.ms">g@rethada.ms</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto">Sam,<div dir="auto"><br></div><div dir="auto">Ok, it sounds like you don't have the problem I was worried you had. If your environment branches don't diverge from each other then the rebase should be identical to the corresponding merge - git shouldn't be creating a merge commit in that case.</div><div dir="auto"><br></div><div dir="auto">Personally I don't agree with the obsession with single path for histories - it's a tool built to embrace branching, and by trying to avoid that you're removing valuable data from your history. But that's a divisive opinion so should probably be a whole separate thread if people have opinions. Don't even consider changing that approach just based on this conversation.</div><div dir="auto"><br></div><div dir="auto">In your case, if the aim is to deploy identical code everywhere, then I think you should try and reframe in your head, so instead of thinking about "deploy staging" and "deploy production" you instead think "deploy master to staging" and "deploy master to production". I don't think the environment branches are needed in your setup, and they're a legacy of a different workflow.</div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 9 Jan 2019, 09:53 Samuel Joseph <<a href="mailto:tansaku@gmail.com" target="_blank">tansaku@gmail.com</a> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi Gareth,</p>
<p>Thanks - interesting point.</p>
<p>As far as I'm able to detect the code on the staging branch is
the same code that we end up deploying to production, since what
we deploy to production in the master branch, and when we rebase
the code from the staging branch into master, the staging branch
and the master branch end up being identical, assuming there are
no conflicts, and the way we operate that is only once in a blue
moon, and any extra changes there would be pulled back to staging
and develop.</p>
<p>Maybe I am mis-understanding how rebasing works (entirely
possible), but when I rebase staging into master what I am doing
is avoiding creating a merge commit, so that we have a nice smooth
single threaded git history. Again my understanding (possibly
faulty) is that when I rebase staging into master I am making the
code on master be identical to that on staging, assuming that they
have a shared history, but that staging has a few new commits,
e.g. currently our staging branch history looks like this:<br>
<br>
```<br>
* 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD
-> staging, origin/staging, origin/master, master) [Nick
Schimek]<br>
* bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to
5.6.0.349 (#2939) [dependabot[bot]]<br>
* 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
[dependabot[bot]]<br>
* d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919)
[Sam Joseph]<br>
* 7f42d570 2018-12-11 | reduce events caching to one hour (#2936)
[Sam Joseph]<br>
* aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0
(#2934) [dependabot[bot]]<br>
* aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
[dependabot[bot]]<br>
* 298a8118 2018-12-11 | 2883 improve 500 error page (#2931)
[ElisaRmz]<br>
```<br>
<br>
and our master history looks like this:</p>
<p>```<br>
* 2dadadbf 2018-12-13 | Fix future events query (#2922) (HEAD
-> master, origin/staging, origin/master, staging) [Nick
Schimek]<br>
* bd86c461 2018-12-13 | Bump newrelic_rpm from 5.5.0.348 to
5.6.0.349 (#2939) [dependabot[bot]]<br>
* 2d65985b 2018-12-13 | Bump grape from 1.2.1 to 1.2.2 (#2940)
[dependabot[bot]]<br>
* d1865dae 2018-12-12 | 2893 add grape api move legacy api (#2919)
[Sam Joseph]<br>
* 7f42d570 2018-12-11 | reduce events caching to one hour (#2936)
[Sam Joseph]<br>
* aa77c6ee 2018-12-11 | Bump letter_opener from 1.6.0 to 1.7.0
(#2934) [dependabot[bot]]<br>
* aa4d6bee 2018-12-11 | Bump stripe from 4.2.0 to 4.3.0 (#2933)
[dependabot[bot]]<br>
* 298a8118 2018-12-11 | 2883 improve 500 error page (#2931)
[ElisaRmz]<br>
```<br>
<br>
and they are identical because when I did our last deploy (kicking
off by manually rebasing staging into master), the few new commits
on staging were added onto master.</p>
<p>The environment specific branches don't have any evironment
specific config, and they don't contain different combinations of
feature branches, except in as much that at any given moment we
may have a few new features/bug-fixes/upgrades in develop that are
not in staging that are not in master, but as they move along
(what I call) the pipeline (perhaps I'm using the term
incorrectly), all of those features/bug-fixes/upgrades will end up
in master/production, and if we have a quiet spell with nothing
new coming in, all three branches will be identical.</p>
<p>Nevertheless, yes, we certainly want to deploy the same code
everywhere, as you suggest, and I think we are doing so - I could
be wrong. Perhaps our three branches, each tied to a different
server, is more complex than we need, or just rather different
from what everyone else is doing ... <br>
</p>
<p>Thanks so much for taking the time to comment.</p>
<p>Best, Sam<br>
</p></div></blockquote></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<div class="m_-6092159030513641403m_1679838559004216112moz-cite-prefix">On 08/01/2019 10:33, Gareth Adams
wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">Sam, I don't have a grand answer to your whole
question, but a phrase leapt out at me and I wanted to flag it:
<div dir="auto"><br>
</div>
<div dir="auto">> rebasing along the pipeline</div>
<div dir="auto"><br>
</div>
<div dir="auto">To me, this suggests the code on your staging
branch is not the same as the code you end up deploying to
production (and it might not be the same as the code in your
master branch)</div>
<div dir="auto"><br>
</div>
<div dir="auto">I guess there are a few reasons that could be:
you're storing some environment-specific configuration on your
environment branches, and can't merge them all together, or
maybe your environment branches contain a different
combination of feature branches that you're trying to keep
control of?</div>
<div dir="auto"><br>
</div>
<div dir="auto">Either way, I'd consider how (or if) you could
change your workflow to make sure you deploy the same code
everywhere (or at least deploy the same code to production
that you deploy to staging). That's the basis behind e.g.
Heroku pipelines' "Promote" button and it's the pattern I
commonly see now</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, 8 Jan 2019, 10:13 Samuel Joseph <<a href="mailto:tansaku@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">tansaku@gmail.com</a>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi Gerhard,<br>
</p>
<div class="m_-6092159030513641403m_1679838559004216112m_3868980048267634368m_-2707529618717350415m_-2020117851195068980moz-cite-prefix">On
07/01/2019 12:00, Gerhard Lazu wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Sam,
<div><br>
</div>
<div>What determines that a build can go from your
development environment into staging? </div>
</div>
</div>
</blockquote>
Good question, the answer is:<br>
<br>
1) that all the unit, integration and acceptance tests pass<br>
2) that there are no merge conflicts<br>
3) that the manual sanity checks on develop are coming back
okay<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>And from staging into production? If you can
capture this in code, you can put it into a
pipeline.<br>
<br>
</div>
</div>
</div>
</blockquote>
I don't think there's any way we can remove the manual
sanity checks as the acceptance tests are just not that
reliable, and although we've poured 1000's of hours into
them and ultimately I can't see any way of making them
perfect.<br>
<br>
I didn't think the presence of a manual step would prevent
us using a pipeline, in as much as I thought of a pipeline
as just being a series of servers with matching branches and
code is them moved along them whether manually or
automatically. Heroku calls such things pipelines and seems
to have no support for automatically moving code along them,
it's purely manual from what I can see.<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>Why do you have 3 pipelines? </div>
</div>
</div>
</blockquote>
<p>I don't think we do. As I understand it, we have one
pipeline:</p>
<p>develop branch + develop server ---> staging branch +
staging server ---> master branch + production server</p>
<p>That's three paired branches/servers in one pipeline.
Here's a screenshot of how Heroku presents our pipeline in
their pipeline interface. Note the button "Promote to
staging" which allows you to manually move the code on the
develop server to the staging server, but doesn't actually
do a rebase of the code from develop branch to staging:<br>
</p>
<p><img alt="" width="790" height="157"></p>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>Based on the questions that you're asking, I
believe that it would help if you had a single
pipeline. </div>
</div>
</div>
</blockquote>
I agree - I think we do, but perhaps I'm wrong ...<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>The question that I would focus on is <i>What
would it take to have a single pipeline that has
an end-goal of creating production builds</i>?
Here is a pipeline example which stops after it
publishes a Docker image: <a href="https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">changelog.com,
CircleCI</a>.</div>
</div>
</div>
</blockquote>
Ooh, thanks for sharing! I had to log in to CircleCI to see
that:<br>
<br>
<p><img alt="" width="470" height="208"></p>
but that looks really interesting.<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div> If you are using something like Docker Swarm or
Kubernetes, the platform/ecosystem has all the
necessary tools to keep deployment concerns
self-contained. In the <a href="http://changelog.com" rel="noreferrer
noreferrer noreferrer noreferrer" target="_blank">changelog.com</a> case, the
Docker stack that captures the entire deployment has
<a href="https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">an update
component that is responsible for app updates</a>.
In this specific case, if the new app version starts
and is healthy for 30 seconds, it gets automatically
promoted to live. <a href="http://changelog.com/podcast/254" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">We have
been using a similar approach since October 2016</a>,
a Docker stack just makes it easier.</div>
<div><br>
</div>
<div>I want to spur your imagination by sharing <a href="https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">the
pipeline that is responsible for RabbitMQ v3.7.x</a>.
This pipeline captures what is possible if
imagination is set free:</div>
</div>
</div>
</blockquote>
Wow, that RabbitMQ pipeline looks amazing - and I've only
captured part of it in the screenshot:<br>
<br>
<img alt="" width="593" height="375">
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>* tests & builds 30+ apps...</div>
<div>* on all supported major runtime version...</div>
<div>* and all supported OSes</div>
<div>* tests upgrades</div>
<div>* tests client support</div>
<div>* releases alphas, betas, RCs & GAs</div>
<div>* and publishes to all supported distribution
channels</div>
<div><br>
</div>
<div>I hope this helps, Gerhard.</div>
</div>
</div>
</blockquote>
<p>That's extremely helpful - thankyou!</p>
<p>But so just to be clear, there is something in these
pipelines that you're sharing that regularly moves code
from one branch to another? And that's something that
CircleCI and RabbitMQ provide? Or these are pipelines
where the same code in the same branch is being moved
through a series of servers, based on tests and checks
passing at each server?</p>
<p>Many thanks in advance<br>
Best, Sam<br>
</p>
</div>
_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org" rel="noreferrer
noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a><br>
Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote></div>
_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org" target="_blank">Chat@lists.lrug.org</a><br>
Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
</blockquote></div>