Continuous Integration vs. Delivery vs. Deployment

Continuous integration is the practice of running the steps that were traditionally performed during “integration” little and often throughout the development process, rather than waiting until code is complete before bringing it all together and testing it.

Assuming you have more than one person working on a software product – which is the norm for most commercial and open-source software – at some point, you need to combine the pieces each developer has worked on and check that the end result works as intended. With continuous integration that point happens at least once a day, if not more.

The rationale is simple. If you leave integration until all the code has been written, there’s a good chance that you’ll have to spend a considerable amount of time unpicking and rewriting sections of your code to get the solution to build, let alone get it working as intended. Code is complex. Even with detailed upfront design, it’s incredibly difficult to predict exactly how the logic will interact and avoid any issues. The more code there is, the greater the complexity and the more there is to unpick when something doesn’t work.

With continuous integration, your developers share their changes by committing them to source control regularly – at least once a day – and check that the solution builds and passes tests. This means that if something breaks, there are far fewer changes to go through in order to find the source of the problem. Getting feedback quickly also makes it easier to fix any issues, because you haven’t lost the context of what you were doing.

A good CI process requires a few essential ingredients:

• Use source control – A source (aka version) control system is absolutely essential. If you don’t have one, or only some of your code is in source control, the first step is getting everything in it and everyone using it.
• Commit early, commit often – Once you’re using source control, get everyone into the habit of committing early and often. Task size can have an impact here; breaking each piece of work down into smaller chunks makes it easier to complete and test a set of changes locally so that they’re ready to commit.
• Build your solution on every commit – Regularly sharing your code changes with everyone on the project is just the start. The next step is to check that the solution can still be built with the latest changes. While this could be done manually, it’s much easier and more efficient to trigger the build automatically, and this is where a CI server comes in.
• Automate tests – A successful build is a good sign, but for greater confidence, it pays to run some tests. Again, it’s much easier and more efficient to run tests automatically than manually. Although writing automated tests can seem labor intensive, they soon pay for themselves when you run them on every build to get fast feedback.
• Listen to the feedback – It’s only worth automating builds and tests if you’re going to do something with the information you get back. CI tools offer a range of feedback mechanisms, from dashboards and radiators to integration with instant messaging platforms.
• Build a DevOps culture – To really reap the benefits of continuous integration, you need to create a team culture where everyone takes responsibility for fixing broken builds or failing tests. Pinning the blame on whoever made the last commit is counterproductive as it often discourages your team from committing changes early and often despite that being in everyone’s interest.

Putting all these ingredients together gives you fast and regular feedback on your code. By checking that any change to the codebase – be it a bug fix, refactor or part of a new feature – at least results in a clean build that passes tests – teams can avoid building new work on top of bad foundations, and the inevitable re-work that follows. Implementing continuous integration on its own is one big step towards speeding up the process of delivering working software to your users.

Continuous delivery builds on the foundations of build and test automation established with continuous integration. With continuous delivery, you turn the manual steps used to release a build of your software to production into an automated process.

Whereas in the past there may have been a handoff from your developers to testers, and then from testers to release managers, with continuous delivery, your team (which may include people from a range of disciplines) owns the entire process of building, testing and releasing their product. This comes with several advantages:

• By avoiding the traditional siloes, your development team has a better understanding of the business and operational needs for delivering their product to your users.
• In turn this creates an opportunity to bring development practices to what was typically a manual, and often quite lengthy, process.
• Automating the steps involved in delivering a product to live not only speeds up the process but also reduces the risk of errors and makes it more stable and reliable.

The exact steps involved in delivering software to your users – and therefore the required stages in the delivery pipeline – vary according to business and user needs, but it’s common practice to deploy to at least one pre-production environment before they are released into the wild.

These can be testing environments for additional layers of testing, such as security, load and performance tests; sandbox environments for support and sales teams to familiarize themselves with new features; and acceptance testing environments for QA and product professionals to verify that changes work as intended.

With continuous delivery, each successful build is automatically deployed to each of the pre-production environments, with confidence in the quality increasing with every stage. This is a worthwhile goal that takes some investment to put into practice:

• Build once – Only by promoting the same build artefact through each stage of the pipeline can you have confidence that it has passed all prior stages of testing.
• Automate every deployment – the deployment to each environment should be scripted so that it can be run automatically, and therefore in exactly the same way, each time. The release to live should be scripted for the same reason, but with continuous delivery this final step is not automatic.
• Separate environmental concerns – In order to automate deployment of the same artefact to each environment, there needs to be clear separation between the application itself and environment-specific variables or parameters.
• Store configuration in source control – Keeping all these configuration files in source control helps ensure deployments are consistent and repeatable.
• Clean your environments – For a completely consistent approach, the environments themselves should be reset to the same conditions for each new deployment. Thanks to containers this is now much easier to implement, either on local infrastructure or in the cloud.
• Maintain the pipeline – As with continuous integration, it’s only worth investing in an automated delivery pipeline if you maintain it. CI/CD is as much about team culture as it is about process and tools. To be effective, your team needs to take responsibility for maintaining the pipeline and fixing any issues that arise, whether they’re due to a bug in the code itself or a problem with the automated deployments or tests.

With continuous delivery, your team takes responsibility for the delivery of software, and benefits from the timely feedback provided throughout the process. As with continuous integration, the practice can be built up and improved over time.

Continuous deployment takes the practices of continuous integration and delivery to their logical conclusion: if a build passes all previous stages in the pipeline successfully it is automatically released to production. This means that as soon as any change to your software has passed all tests it is delivered to your users. Continuous deployment shortens the feedback loop from code change to use in production, giving your team timely insight into how their changes perform in the real world without having to compromise on quality.

Although automating deployment of software to production is not suitable for every product and organization, it’s worth considering the steps required to get there as each individual element is valuable on its own:

• Have confidence in your tests – Deploying to production automatically requires a high level of confidence in your pipeline, particularly your automated tests. A great testing culture, where your team invests in test coverage and performance, and prioritizes fixing the build and the pipeline over new features, is essential.
• Choose what to release – A common objection from those new to continuous deployment is that if your developers commit early and often and let things march out of the door without some manual oversight then your users will find unfinished or half-baked features that are not ready for use. Rather than resorting to branches and missing out on the advantages of continuous integration, the solution is to use feature flags. Your developers can then control what is visible to and what’s hidden from the user when they’re writing the code.
• Control the rollout – Although continuous deployment means releasing automatically if all previous stages pass muster, that does not mean surrendering all control. There are various deployment practices that are used to minimize risk and control the roll out. Canary releases allow you to test the waters with a small group proportion of your users initially, while blue-green deployments can be used to manage the transition to a new version.
• Monitor production – Even with all the above measures in place, continuous deployment can feel like a risky practice – what happens if a bug goes undetected in testing only to emerge in production? Time, money and reputation are all potentially at stake. While the same problem could get through with a manual deployment, there’s nothing like a showstopper being released automatically by “the system” for losing the confidence of your stakeholders. This is where being proactive in looking for signs of trouble rather than waiting for bug reports to come in makes all the difference. Monitoring stats for any change from the norm, particularly just after a release, can alert you to issues before they cause a noticeable problem for your users.
• Streamline your pipeline – If something does go wrong in production you want to be able to respond quickly. In some cases, it might be possible to roll back the release to the previous version. Often, however, things aren’t that straightforward – database migrations and fixes for known issues can all prevent anything but forward travel and the only option is to put out a fix. Skipping the steps in your pipeline is a false economy as you are likely to introduce other issues that could have been caught had you run your tests as normal. Instead, it’s better to invest in streamlining your pipeline, from build speed to test performance. Not only does this mean you can deploy changes fast when you need to, but it also shortens the feedback loops throughout the process.

Continuous integration, delivery and deployment all serve the goal of delivering valuable, working software to your users frequently. Taking a little and often approach to integration and releasing makes the process more manageable and means that teams practice the steps regularly and get better at them.

Bringing automation into the mix not only speeds the process up but also makes it more reliable. Each step in the pipeline builds on the previous, and you can choose how much is right for you at any given time and return to build on it later.