<div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr">On Wed, 9 Jan 2019, 10:08 Samuel Joseph <<a href="mailto:tansaku@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">tansaku@gmail.com</a> wrote:</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
    <p>I guess we could be deploying to both staging and production from
      the master branch, but by having the separate develop, staging and
      production branches, that each auto-deploy to the three servers,
      it means that we've got a clear place to go (locally and in
      GitHub) to see a running version of whatever is on each of those
      three branches</p></div></blockquote></div></div><div dir="auto">In the workflow I think you're describing, the "clear place to go" should be your CD tool - in this case the Heroku pipeline - <span style="font-family:sans-serif">rather than your source code repository.</span> But there are definitely nuances, and if getting that info back into your repo is important then maybe I'm simplifying too much.</div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><p>and if for some reason production has a weird
      issues and we have to hot fix (fairly rare) then we have the
      staging branch to do it on, safe in the knowledge that a hot-fix
      applied to staging can be auto-deployed to the staging server, and
      won't accidentally make thngs worse on production (which is of
      course auto-deployed off master).</p></div></blockquote></div></div><div dir="auto">I'll rephrase what I wrote in my last email to:</div><div dir="auto"><br></div><div dir="auto">…you instead think "deploy master to staging <b>and then</b> to production".<br></div><div dir="auto"><br></div><div dir="auto">In this workflow, there's no such thing as a "hotfix to staging", just a new commit to master, which follows the same "master ➡️ staging ➡️ production" process. Note this means you don't merge develop into master until you're sure there will be no more hotfixes, and you also merge hotfixes back into develop as soon as possible.</div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
    <p>But then, does
      no one do any manual tests on staging before there's a deploy to
      production?  What's the trigger that says it's okay to deploy to
      production following a staging deploy?</p></div></blockquote></div></div><div dir="auto">And now we've circled around to the original core of your question, this is a simple answer. Once your manual tests are complete, this is exactly what the "promote" button (or equivalent cli command) does. It reduces the manual part of the deploy process down to a single button click, which I think is what you wanted from the start of this conversation.</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
    <div class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-cite-prefix">On 08/01/2019 11:19, Matthias Berth
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">
        <div>Hi Sam, <br>
        </div>
        <div><br>
        </div>
        <div>Why are you rebasing in the first place? <br>
        </div>
        <div>Can't you just make a feature branch off master, then merge
          it back into master? And deploy to staging and production from
          the master branch?</div>
        <div><br>
        </div>
        <div>I also wonder why your 20 minutes sanity checks cannot be
          automated. Are you doing something new / creative every time
          you do these tests?</div>
        <div>Great discussion!</div>
        <div><br>
        </div>
        <div>Cheers</div>
        <div><br>
        </div>
        <div>Matthias<br>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr">On Tue, Jan 8, 2019 at 11:48 AM Gareth Adams <<a href="mailto:g@rethada.ms" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">g@rethada.ms</a>>
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div dir="auto">Sam, I don't have a grand answer to your whole
            question, but a phrase leapt out at me and I wanted to flag
            it:
            <div dir="auto"><br>
            </div>
            <div dir="auto">> rebasing along the pipeline</div>
            <div dir="auto"><br>
            </div>
            <div dir="auto">To me, this suggests the code on your
              staging branch is not the same as the code you end up
              deploying to production (and it might not be the same as
              the code in your master branch)</div>
            <div dir="auto"><br>
            </div>
            <div dir="auto">I guess there are a few reasons that could
              be: you're storing some environment-specific configuration
              on your environment branches, and can't merge them all
              together, or maybe your environment branches contain a
              different combination of feature branches that you're
              trying to keep control of?</div>
            <div dir="auto"><br>
            </div>
            <div dir="auto">Either way, I'd consider how (or if) you
              could change your workflow to make sure you deploy the
              same code everywhere (or at least deploy the same code to
              production that you deploy to staging). That's the basis
              behind e.g. Heroku pipelines' "Promote" button and it's
              the pattern I commonly see now</div>
          </div>
          <br>
          <div class="gmail_quote">
            <div dir="ltr">On Tue, 8 Jan 2019, 10:13 Samuel Joseph <<a href="mailto:tansaku@gmail.com" rel="noreferrer
                noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">tansaku@gmail.com</a>
              wrote:<br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF">
                <p>Hi Gerhard,<br>
                </p>
                <div class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647gmail-m_-4091885444582072218m_3868980048267634368m_-2707529618717350415m_-2020117851195068980moz-cite-prefix">On
                  07/01/2019 12:00, Gerhard Lazu wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">Hi Sam,
                      <div><br>
                      </div>
                      <div>What determines that a build can go from your
                        development environment into staging? </div>
                    </div>
                  </div>
                </blockquote>
                Good question, the answer is:<br>
                <br>
                1) that all the unit, integration and acceptance tests
                pass<br>
                2) that there are no merge conflicts<br>
                3) that the manual sanity checks on develop are coming
                back okay<br>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div>And from staging into production? If you can
                        capture this in code, you can put it into a
                        pipeline.<br>
                        <br>
                      </div>
                    </div>
                  </div>
                </blockquote>
                I don't think there's any way we can remove the manual
                sanity checks as the acceptance tests are just not that
                reliable, and although we've poured 1000's of hours into
                them and ultimately I can't see any way of making them
                perfect.<br>
                <br>
                I didn't think the presence of a manual step would
                prevent us using a pipeline, in as much as I thought of
                a pipeline as just being a series of servers with
                matching branches and code is them moved along them
                whether manually or automatically.  Heroku calls such
                things pipelines and seems to have no support for
                automatically moving code along them, it's purely manual
                from what I can see.<br>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div><br>
                      </div>
                      <div>Why do you have 3 pipelines? </div>
                    </div>
                  </div>
                </blockquote>
                <p>I don't think we do.  As I understand it, we have one
                  pipeline:</p>
                <p>develop branch + develop server ---> staging
                  branch + staging server ---> master branch +
                  production server</p>
                <p>That's three paired branches/servers in one
                  pipeline.  Here's a screenshot of how Heroku presents
                  our pipeline in their pipeline interface.  Note the
                  button "Promote to staging" which allows you to
                  manually move the code on the develop server to the
                  staging server, but doesn't actually do a rebase of
                  the code from develop branch to staging:<br>
                </p>
                <p><img alt="" width="790" height="157"></p>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div>Based on the questions that you're asking, I
                        believe that it would help if you had a single
                        pipeline. </div>
                    </div>
                  </div>
                </blockquote>
                I agree - I think we do, but perhaps I'm wrong ...<br>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div>The question that I would focus on is <i>What
                          would it take to have a single pipeline that
                          has an end-goal of creating production builds</i>?
                        Here is a pipeline example which stops after it
                        publishes a Docker image: <a href="https://circleci.com/workflow-run/065467ef-87c0-4f5e-a2ab-5e11be12403f" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">changelog.com,
                          CircleCI</a>.</div>
                    </div>
                  </div>
                </blockquote>
                Ooh, thanks for sharing! I had to log in to CircleCI to
                see that:<br>
                <br>
                <p><img alt="" width="470" height="208"></p>
                but that looks really interesting.<br>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div> If you are using something like Docker Swarm
                        or Kubernetes, the platform/ecosystem has all
                        the necessary tools to keep deployment concerns
                        self-contained. In the <a href="http://changelog.com" rel="noreferrer
                          noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">changelog.com</a> case,
                        the Docker stack that captures the entire
                        deployment has <a href="https://github.com/thechangelog/changelog.com/blob/cf2ebe0de0f35c96bf664b8bc9183bd1f3468565/docker/changelog.stack.yml#L15-L31" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">an
                          update component that is responsible for app
                          updates</a>. In this specific case, if the new
                        app version starts and is healthy for 30
                        seconds, it gets automatically promoted to live.
                        <a href="http://changelog.com/podcast/254" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">We have
                          been using a similar approach since October
                          2016</a>, a Docker stack just makes it easier.</div>
                      <div><br>
                      </div>
                      <div>I want to spur your imagination by sharing <a href="https://ci.rabbitmq.com/teams/main/pipelines/server-release:v3.7.x" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">the
                          pipeline that is responsible for RabbitMQ
                          v3.7.x</a>. This pipeline captures what is
                        possible if imagination is set free:</div>
                    </div>
                  </div>
                </blockquote>
                Wow, that RabbitMQ pipeline looks amazing - and I've
                only captured part of it in the screenshot:<br>
                <br>
                <img alt="" width="593" height="375">
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div><br>
                      </div>
                      <div>* tests & builds 30+ apps...</div>
                      <div>* on all supported major runtime version...</div>
                      <div>* and all supported OSes</div>
                      <div>* tests upgrades</div>
                      <div>* tests client support</div>
                      <div>* releases alphas, betas, RCs & GAs</div>
                      <div>* and publishes to all supported distribution
                        channels</div>
                      <div><br>
                      </div>
                      <div>I hope this helps, Gerhard.</div>
                    </div>
                  </div>
                </blockquote>
                <p>That's extremely helpful - thankyou!</p>
                <p>But so just to be clear, there is something in these
                  pipelines that you're sharing that regularly moves
                  code from one branch to another?  And that's something
                  that CircleCI and RabbitMQ provide?  Or these are
                  pipelines where the same code in the same branch is
                  being moved through a series of servers, based on
                  tests and checks passing at each server?</p>
                <p>Many thanks in advance<br>
                  Best, Sam<br>
                </p>
              </div>
              _______________________________________________<br>
              Chat mailing list<br>
              <a href="mailto:Chat@lists.lrug.org" rel="noreferrer
                noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a><br>
              Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
              Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
              List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
            </blockquote>
          </div>
          _______________________________________________<br>
          Chat mailing list<br>
          <a href="mailto:Chat@lists.lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a><br>
          Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
          Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
          List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
        </blockquote>
      </div>
      <br>
      <fieldset class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647mimeAttachmentHeader"></fieldset>
      <pre class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-quote-pre">_______________________________________________
Chat mailing list
<a class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-txt-link-abbreviated" href="mailto:Chat@lists.lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a>
Archives: <a class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-txt-link-freetext" href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a>
Manage your subscription: <a class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-txt-link-freetext" href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a>
List info: <a class="m_7990572999333527681m_8506084164337888487m_-8428862481596492984m_-6735578697607866647moz-txt-link-freetext" href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a>
</pre>
    </blockquote>
  </div>

_______________________________________________<br>
Chat mailing list<br>
<a href="mailto:Chat@lists.lrug.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">Chat@lists.lrug.org</a><br>
Archives: <a href="http://lists.lrug.org/pipermail/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/pipermail/chat-lrug.org</a><br>
Manage your subscription: <a href="http://lists.lrug.org/options.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/options.cgi/chat-lrug.org</a><br>
List info: <a href="http://lists.lrug.org/listinfo.cgi/chat-lrug.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">http://lists.lrug.org/listinfo.cgi/chat-lrug.org</a><br>
</blockquote></div></div></div>