Thursday, November 1, 2018

DevOps: pitfalls of manual deployments

As part of my quest of continual learning, I want to go deep in the theory and why of DevOps, not just learn the tools / buzzwords surrounding the culture. To that end I've begun reading Continuous Delivery: reliable software releases through build, test, and deployment automation by Jez Humble and David Farley. The authors open the book discussing the deployment pipeline, and it's central theme to book.

Deployment Pipeline: an automated implementation of your applications build, deploy, test, and release process; benefits include:

  1. Makes every part of the process visible to everyone involved, regardless of silo.
  2. Improves feedback process, so that problems are identified and rectified more quickly.
  3. Enables teams to deploy any version of their software, to any environment in a predictable, repeatable, that is to say automated fashion.


Humble and Farley go on to describe three common antipatterns of the deployment pipeline: deploying software manually, deploying to a production-like environment only after development is complete, and the manual configuration of production environments.

The issues raised with manual deployments struck very close to home, as many of the underlying issues for manual software deployments plague traditional operation deployments as well. In fact the first VAR / MSP I worked for, had engineers perform manual deployments exclusively for every project (ie: physical hosts, virtual server, firewall, access point, switch, ect.). As a result almost every customers environment was different, and not from a purposeful design perspective.

An example of this was a client with multiple locations; and each site was different. Routing local subnets at firewalls one place, the next routing at the switch, another on the ISP's modem, and one was routing at the wireless controller (with a default route to the firewall). Less dramatic examples include spending hours troubleshooting tracking down network loops, because each switch was either not running Spanning Tree or running a different version of it. These examples may seem ludicrous to an established enterprise IT team, but the underlying issue of non-repeatable, non-audit-able deployments, that keep talented engineers tied up with repetitive boring tasks is likely very relatable.


No comments:

Post a Comment