As your organization grows and application and infrastructure
complexity increase good DevOps processes become very important to
be able to reliably deliver software. In this post we're going to
go over continuous integration and continuous delivery and show how
they should be a vital part of your DevOps processes, and how CI/CD
can enable your team to deliver software reliably and with
confidence.
What is continuous integration?
Continuous integration (CI) is the practice of integrating code
from multiple developers into a central repository (or branch),
multiple times per day. A CI build consists of an automated process
that runs tests (unit and integration), code style checks and so
on. This automated process helps the developers on your team to be
reasonably confident that the changes they pushed have not “broken
the build” and enables them to catch problems early on, before they
hit staging and well before they hit production.
Some organizations run CI on each git push (to any branch) and
some run CI only on open Pull Requests. This decision is left to
the reader to decide what strategy would best serve their
organization.
Continuous integration enables your
team to iterate freely on new features by providing them with an
easy mechanism to integrate their changes back into the main
branch. This prevents large features from being developed in silos
and integrated
only at the very end. Your team should be able to integrate
their features, even if incomplete, into the main branch, and test
that the changes have not broken anything. One common strategy for
integrating unfinished features is using feature flags, where the
given feature is not enabled to the end user but can still go
through the entire CI process to verify it doesn't impact any
existing functionality under the hood.
This allows other team members to be up to date with the coming
new features and are able to plan their work accordingly if it
somehow impacts them.
What is continuous delivery?
The job of continuous delivery is to make deploying your
application an ordinary and unremarkable event that anyone on the
team can do. We do this by automating the release and deployment
process that keeps the codebase in an always deployable state.
Continuous delivery is a natural extension of continuous
integration as it picks up right where the previous step left off.
This can mean automating the artifact creation (docker images
for instance) and automating the steps required to deploy the
application. We call this a pipeline. A pipeline consists of
multiple steps; each step has a series of tests to verify if
everything is okay and advances to the next or notifies the team
about an error.
It's important to note that this impacts each PR (or commit) so
that it becomes each developers responsibility to worry about their
code being deployable. A shared responsibility like this allows
each team member to fully own their feature, from development to CI
to production and enables them to iterate quickly without being
blocked by a dedicated team (or person) that does deployments.
Continuous deployment
It's worth noting that continuous deployment takes things a step
further by automatically deploying the artifact produced from
previous steps directly to production. This is a giant leap that
doesn't require a human gatekeeper to decide when is a good time to
trigger a deploy manually, rather each change is directly, and
automatically, deployed to production.
This requires very rigorous testing to have enough confidence
that new changes don't break anything, as well as encouraging
developing features in smaller increments. For more information
about the right level of quality assurance see here.
Benefits of CI/CD
CI/CD enables your organization to grow your team, codebase, and
infrastructure. It removes the fear and anxiety associated with
deploying changes to production. By automating the building,
testing and deployment phases of your pipeline you remove any form
of gate-keeping and make deploying to production a
non-event.
It encourages each team member to own their code throughout its
entire life-cycle (not just until it's merged into master), as well
as small iterative changes rather than huge features that have been
brewing a very long time.
Challenges when implementing CI/CD
Implementing CI/CD is not without its challenges. It requires
organizational changes and a mindset shift as well as deep
technical knowledge to automate the entire process. Depending on
your stack you might be using various technologies like Docker,
Ansible, Terraform and many others
that require a large initial investment to get going.
Infrastructure cost is a potential issue as well since your team
is either required to host their own CI server (or fleet) or use a
hosted version that suits your needs. Apart from that, you require
at least two identical environments (staging and production) that
allow you to test out your code, as well as the entire process
without disrupting production.
Best practices
- Commit early. Commit often.
As we've mentioned quite a few times already, this is a
fundamental principle to be able to implement CI/CD in your
organization. We cannot overstate this.
- Make your pipelines fast.
Making your pipeline fast is a very important requirement for
the team to be productive and to enable fast turn around. If a team
has to wait an hour long for the pipeline to finish just to be able
to deploy a one line change they will get frustrated very
quickly.
Apart from organizational changes, this is one of
the hardest parts to get right. To be successful with CI/CD your
team needs to develop a culture of writing tests for each code
change they integrate into the master branch. This includes writing
unit, integration, regression and smoke tests (to name a few).
Without this, your team cannot have the confidence that the changes
being deployed are correct and CI/CD will cause more trouble than
it's worth.
- Always run smoke tests after a deploy.
We've mentioned smoke tests in the bullet above, but we feel
they deserve a special note still. It's very useful to be able to
verify that the deployment that went through successfully is, in
fact, functioning as expected and has not broken any common user
flows.
- Provide an easy way to rollback.
Last but not least, we feel the most important part of a
successful CI/CD pipeline is an easy, “one button”, way to rollback
your changes in case something goes wrong. Usually, this entails
having idempotent deploys where doing a rollback just means
re-deploying a previous release.
Summary
In the age of distributed systems and cloud computing, as well
as ever-increasing complexity, CI/CD can provide your team, and
your entire organization, the confidence it needs to deliver
software to your customers reliably.
We showcased some key benefits that CI/CD brings as well as some
of the best practices to implement it successfully.
If you would like to share your experience with CI/CD, ask a
question, mention something that we have not covered, or just say
thanks, feel free to comment down below.
If you like this blog you may also like:
Subscribe to our blog via email
Email subscriptions come from our Atom feed and are handled by Blogtrottr. You will only receive notifications of blog posts, and can unsubscribe any time.
Do you like this blog post and need help with Next Generation Software Engineering, Platform Engineering or Blockchain & Smart Contracts? Contact us.