There are several ironies about DevOps that can sometimes cause confusion or perhaps take attention away from what really matters.
Take “Shift Security Left” for instance. Yes, it’s cheaper and safer to catch errors before they get into production and, yes, developers should take more responsibility for the quality and security of their code. No one disputes that. However, anyone who thinks that “shifting left” is all that’s necessary to be “doing DevSecOps” is overlooking two-thirds of the software delivery (and operation) lifecycle. Yes, your code has to be secure but so does the process that delivers it and the process that keeps it production. We believe that in order to be “doing DevSecOps” right, and thus doing DevOps right, you have to shift security everywhere.
The same can be said about “continuous verification,” which VMware defines as:
“A process of querying external system(s) and using information from the response to make decision(s) to improve the development and deployment process.”
It is true you should be constantly monitoring your applications in production - and be ready to instantly mitigate any issues that arise. That's the part of the DevOps infinity loop that talks about "monitor" and "feedback." Something happens in production, it gets noticed and remediated quickly, with the least amount of human interaction possible. Mean time to detect (MTTD) and mean time to repair (MTTR) are key benchmarks to judge how well you are doing in this respect.
CloudBees customers have had the ability to do this since at least 2015 so we see ‘continuous verification’ as little more than a new phrase for what has been possible for years. We’ve been able to integrate with tools like Dynatrace, AppDynamics, Jira, ServiceNow, etc. to automate the process of detecting, responding and remediating those issues. Customers can design, rehearse and automate responses to roll back the offending code or, with the help of CloudBees Feature Management, instantly disable it with a feature flag.
A New Metric for Measuring Continuous Verification
The ability to instantly and automatically deal with an issue is why we’re advocating the addition of a new DevOps metric: mean time to mitigate (MTTM). We see it as a further benchmark of DevOps success as it closes that exposure gap between problem detection (MTTD) and resolution (MTTR).
DevOps is a journey and not a destination. Initially, getting code through QA was the win, then getting it into production was the win, followed by continuously getting it into production. As teams, toolchains and processes have matured, the REAL DevOps win is getting code into production continuously, automatically mitigating issues and gracefully recovering from them - all with the least disruption to teams and customers.
If you haven’t been monitoring and providing feedback to your applications in production, you really haven’t been doing DevOps. Adding this to your processes and discipline is simply another stage of your DevOps journey.