Continuous integration, or CI as it’s often known, is the practice of having everyone working on the same software project share their changes to the codebase regularly and then checking that the code still works as it should after each change. Continuous integration forms a key part of the DevOps approach to building and releasing software, which promotes collaboration, automation and short feedback cycles.
Practicing continuous integration starts with committing changes to a source/version control system regularly so that everyone is building on the same foundation. Each commit triggers a build and a series of automated tests to verify the behavior and ensure the change has not broken anything. While continuous integration is beneficial by itself, it’s also the first step towards implementing a CI/CD pipeline.
We’ve already mentioned that continuous integration is an important element of DevOps. It also ties in well to agile software development. While following agile methodology is not a requirement of adopting DevOps or implementing continuous integration, they are all rooted in the same principles.
As software developers our overarching goal is, or at least should be, to solve a problem for our users. But not only is software development complex, but building the right thing is hard. To address this, agile advocates for developing and delivering in shorter cycles, making it easier to get feedback on what has been built and to adapt in response to it. DevOps provides techniques for shortening development and release cycles in order to implement those shorter feedback loops in the form of continuous integration, delivery and deployment.
Continuous integration evolved in response to the problems that development teams faced when trying to bring together the work of multiple individual developers into a single product after months – or even years – of writing code. The process of integrating separately developed modules could add long delays to a software project, as assumptions and dependencies were identified long after the code had been written, often requiring considerable rework.
DevOps takes the view that if something is painful, you should do it more often; the more you practice the easier it will get. Rather than leaving integration until all the code has been written, the idea is to integrate smaller amounts on a much more frequent basis and run automated tests each time. This gives you feedback shortly after you’ve made the changes so that any issues can be addressed early on. In turn this improves code quality, as the team are regularly checking that the software works as intended, and means less time spent debugging and more on building features.
The key ingredients of continuous integration are as follows:
For everyone to build on the same foundation you need to be working from the same repository and sharing your changes with each other frequently. A good rule of thumb is to have everyone commit their changes to master/trunk at least once a day.
After you’ve committed a change the next step is to build the solution and put it through a series of automated tests to verify the behavior. Automating this process is an integral part of continuous integration; building and/or testing manually is time-consuming and error-prone, and makes the goal of integrating changes daily impractical. The exact build tools and test frameworks you use depend on the language you’re working in.
Once the scripts and tests are in place the process needs to be maintained. That means adding automated tests as part of any new features, dealing with failures and monitoring performance of the process.
Adding a CI server to take care of monitoring your repository, triggering builds, running automated tests, and collating results helps to pull all these pieces together, saving you time on writing custom automation logic, and provides additional insights such as code coverage metrics and build history.
While these tools and processes are important in implementing continuous integration, to get the best out of continuous integration you need people to embrace the practice. As a development team, you need to adapt your processes to include committing to master regularly, adding automated tests to any new feature, and prioritizing fixing the build if something goes wrong. Working with the QA team to prioritize, design and maintain automated tests and collaborating with infrastructure staff to provision machines on which to run builds and tests will both help to break down organizational silos.
Although continuous integration offers benefits not just for developers but for the organization as a whole, that doesn’t mean it’s always greeted with open arms. For many development shops, DevOps represents a big change in the way they work and challenges existing processes. Good communication is needed to coordinate the effort between teams and instill a culture of collaboration.
If you’re already following agile methodologies, the shift is usually a little easier as the importance of listening to feedback to ensure you build the right thing, and the notion of self-organizing teams, should have already gained traction.
Where that’s not the case, recognizing that this is a significant change, engaging with people, starting small and demonstrating the benefits as you go can all help to bring your colleagues round.
Continuous integration also comes up against more practical challenges. If you’re working on a large, monolithic application, build times can be slow and if test environments are in short supply then it can be a challenge to parallelize test runs.
Having visibility of your CI process and using metrics to identify bottlenecks can help to quantify the costs and benefits of investing in architecture changes, additional infrastructure and automated test coverage.
Adopting continuous integration helps to speed up the development process while improving code quality. Automating these steps allows you to work more efficiently and focus on adding value for your users. But continuous integration is only the beginning of the CI/CD pipeline. The next stage, continuous delivery, applies the DevOps principles to the next part of the release process.