<div dir="ltr"><div>Hi Sam, <br></div><div><br></div><div>Why are you rebasing in the first place? <br></div><div>Can't you just make a feature branch off master, then merge it back into master? And deploy to staging and production from the master branch?</div><div><br></div><div>I also wonder why your 20 minutes sanity checks cannot be automated. Are you doing something new / creative every time you do these tests?</div><div>Great discussion!</div><div><br></div><div>Cheers</div><div><br></div><div>Matthias<br></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jan 8, 2019 at 11:48 AM Gareth Adams <<a href="mailto:g@rethada.ms">g@rethada.ms</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Sam, I don't have a grand answer to your whole question, but a phrase leapt out at me and I wanted to flag it:<div dir="auto"><br></div><div dir="auto">> rebasing along the pipeline</div><div dir="auto"><br></div><div dir="auto">To me, this suggests the code on your staging branch is not the same as the code you end up deploying to production (and it might not be the same as the code in your master branch)</div><div dir="auto"><br></div><div dir="auto">I guess there are a few reasons that could be: you're storing some environment-specific configuration on your environment branches, and can't merge them all together, or maybe your environment branches contain a different combination of feature branches that you're trying to keep control of?</div><div dir="auto"><br></div><div dir="auto">Either way, I'd consider how (or if) you could change your workflow to make sure you deploy the same code everywhere (or at least deploy the same code to production that you deploy to staging). That's the basis behind e.g. Heroku pipelines' "Promote" button and it's the pattern I commonly see now</div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, 8 Jan 2019, 10:13 Samuel Joseph <<a href="mailto:tansaku@gmail.com" rel="noreferrer noreferrer" target="_blank">tansaku@gmail.com</a> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi Gerhard,<br>
</p>
<div class="gmail-m_-4091885444582072218m_3868980048267634368m_-2707529618717350415m_-2020117851195068980moz-cite-prefix">On 07/01/2019 12:00, Gerhard Lazu
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">Hi Sam,
<div><br>
</div>
<div>What determines that a build can go from your development
environment into staging? </div>
</div>
</div>
</blockquote>
Good question, the answer is:<br>
<br>
1) that all the unit, integration and acceptance tests pass<br>
2) that there are no merge conflicts<br>
3) that the manual sanity checks on develop are coming back okay<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>And from staging into production? If you can capture this
in code, you can put it into a pipeline.<br>
<br>
</div>
</div>
</div>
</blockquote>
I don't think there's any way we can remove the manual sanity checks
as the acceptance tests are just not that reliable, and although
we've poured 1000's of hours into them and ultimately I can't see
any way of making them perfect.<br>
<br>
I didn't think the presence of a manual step would prevent us using
a pipeline, in as much as I thought of a pipeline as just being a
series of servers with matching branches and code is them moved
along them whether manually or automatically. Heroku calls such
things pipelines and seems to have no support for automatically
moving code along them, it's purely manual from what I can see.<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>Why do you have 3 pipelines? </div>
</div>
</div>
</blockquote>
<p>I don't think we do. As I understand it, we have one pipeline:</p>
<p>develop branch + develop server ---> staging branch + staging
server ---> master branch + production server</p>
<p>That's three paired branches/servers in one pipeline. Here's a
screenshot of how Heroku presents our pipeline in their pipeline
interface. Note the button "Promote to staging" which allows you
to manually move the code on the develop server to the staging
server, but doesn't actually do a rebase of the code from develop
branch to staging:<br>
</p>
<p><img alt="" width="790" height="157"></p>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>Based on the questions that you're asking, I believe that
it would help if you had a single pipeline. </div>
</div>
</div>
</blockquote>
I agree - I think we do, but perhaps I'm wrong ...<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div>The question that I would focus on is <i>What would it
take to have a single pipeline that has an end-goal of
creating production builds</i>? Here is a pipeline example
which stops after it publishes a Docker image: <a href="https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f" rel="noreferrer noreferrer noreferrer" target="_blank">changelog.com, CircleCI</a>.</div>
</div>
</div>
</blockquote>
Ooh, thanks for sharing! I had to log in to CircleCI to see that:<br>
<br>
<p><img alt="" width="470" height="208"></p>
but that looks really interesting.<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div> If you are using something like Docker Swarm or
Kubernetes, the platform/ecosystem has all the necessary
tools to keep deployment concerns self-contained. In the <a href="http://changelog.com" rel="noreferrer noreferrer noreferrer" target="_blank">changelog.com</a>
case, the Docker stack that captures the entire deployment
has <a href="https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31" rel="noreferrer noreferrer noreferrer" target="_blank">an update component that is
responsible for app updates</a>. In this specific case, if
the new app version starts and is healthy for 30 seconds, it
gets automatically promoted to live. <a href="http://changelog.com/podcast/254" rel="noreferrer noreferrer noreferrer" target="_blank">We have been using a similar
approach since October 2016</a>, a Docker stack just makes
it easier.</div>
<div><br>
</div>
<div>I want to spur your imagination by sharing <a href="https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x" rel="noreferrer noreferrer noreferrer" target="_blank">the pipeline that is responsible
for RabbitMQ v3.7.x</a>. This pipeline captures what is
possible if imagination is set free:</div>
</div>
</div>
</blockquote>
Wow, that RabbitMQ pipeline looks amazing - and I've only captured
part of it in the screenshot:<br>
<br>
<img alt="" width="593" height="375">
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div><br>
</div>
<div>* tests & builds 30+ apps...</div>
<div>* on all supported major runtime version...</div>
<div>* and all supported OSes</div>
<div>* tests upgrades</div>
<div>* tests client support</div>
<div>* releases alphas, betas, RCs & GAs</div>
<div>* and publishes to all supported distribution channels</div>
<div><br>
</div>
<div>I hope this helps, Gerhard.</div>
</div>
</div>
</blockquote>
<p>That's extremely helpful - thankyou!</p>
<p>But so just to be clear, there is something in these pipelines
that you're sharing that regularly moves code from one branch to
another? And that's something that CircleCI and RabbitMQ
provide? Or these are pipelines where the same code in the same
branch is being moved through a series of servers, based on tests
and checks passing at each server?</p>
<p>Many thanks in advance<br>
Best, Sam<br>
</p>
</div>
_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org" rel="noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a><br>
Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
</blockquote></div>
_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org" target="_blank">Chat@lists.lrug.org</a><br>
Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
</blockquote></div>