The Three Ways of DevOps

The book “The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win” that popularized DevOps and transformed complex IT management and business problems into a novel, outlined DevOps principles called “The Three Ways” of DevOps.   

The First Way: Left to Right

The fundamental principle of fast development of a product is that all departments involved have to perform as good as possible and they pass finished work from one to another. To gain this, each department (single contributor or a division) has to minimize amount of WIP (work in progress) and focus on tasks one after another. Also, each of them has to pass work down the pipeline without known defects. No local optimization should create global degradation. Each department should pass the knowledge to downstream department, so everybody involved knows what is going on in the system.

The DevOps enabled business value stream as seen in the picture is that Business identify requirements for a product, Development builds them, and Operations deliver them to a Customer as a service.

The Second Way: Right to Left

To build high quality product that meets customer needs, it is crucial to have a lot of feedback as fast as possible. In DevOps enabled business stream value from The First Way, each of department involved need to give feedback to its upstream colleagues. Development needs to provide time estimates, technology constraints and possible risks to Business. Operations need to provide Development detected issues and Customer needs to provide Operations feedback on the quality of service.

At the point of Dev to Ops handover, a lot can be automated. This is the exact spot where Continuous Integration (CI) and Continuous Delivery (CD) take place.

Continuous Integration can be provided with automatic tests that run every time a code is deployed into version control system (VCS) like Git or when it is merged into (pre)production branch. Developer can see result of a test in minutes and know if his new code breaks anything covered by tests. These tests can cover many aspects of programming. There can be code quality analysis, unit testing for parts of logic, integration testing to expose faults in integration between integrated units, automatic UI testing and other types of automated tests. They can happen before or after a software is built, depends on character of a test.

Imagine if you wanted to have several changes integrated during a single day and every time a developer sends built program to a testing department and he had to run tests all over again and then had to test the whole integrated system. This all is automated in CI and the main job of QA is to write tests and manually test systems only after they are successfully deployed to development or (pre)production environment.

Another great feature of continuous integration is automatic building. Software builds are generated automatically after push to a branch or at specified time of day. If build fails, it is detected as soon as the bug is introduced and not at the D-day of major release.

The result of complete CI pipeline is that either developer gets his feedback on things he should fix or there is a new build ready to be deployed. After that, CD pipeline comes in to process the build and deliver it to real environments to be tested and used.

There are two types of CD pipelines, Continuous Delivery and Continuous Deployment. They are very similar with just one fundamental difference. Production deployment has to be manually triggered or even manually deployed in the case of continuous delivery. In the case of Continuous Deployment, even deployment to production is fully automatized and can be prevented only by failing test.

Both CD types are meant to deliver small iterative changes into production that can be easily troubleshooted or reverted if necessary. Each of these changes has already passed a series of automated tests that ensures that there will be less trouble after they are deployed.

A great benefit of using CI/CD is that there are no more “release days” because each day is a release day. This is a great stress relief for development and operations teams and they tend to make less last-minute changes that often end up in broken releases.

The CI/CD process depends heavily on testing, integration and deployment tools. A lot of infrastructure work is even distributed to developers since virtual containers are used to simulate production environment on local computer, in development environment and in pre-production environment. This also prevents system-specific errors to happen that often.

Using these techniques, Facebook is able to make thousands of builds every single day, test them, spread them to its employees, roll out to a handful of customers and ultimately to all customers. These changes can be stopped at any moment by automated tests or by feedback from people already using them.

The Third Way

The Third Way is about the cultural shift in an organization. People don’t have to be afraid to take risks and experiment. If it doesn’t work, automated tests should stop the deployment. If it works, team tried and learnt something new. In either way, team learns something from either success or failure. Team also must have a recovery plan that can also be useful in the future in production.

Experimentation and risk-taking are the only way to push innovations forward and reach new frontiers.