- Carmen DeArdo, Technology Director at Nationwide Insurance
- John Allspaw, (former) CTO of Etsy
- John Esser, Senior Director of IT and Data Center Operations at AdvancedMD
- Mik Kersten, CEO of Tasktop
- Scott Nasello, Senior Manager of Platform and Systems Engineering at Columbia Sportswear
- Anders Wallgren, CTO of CloudBees
1.
Failures and breaches can happen at any moment – leaders should take an investigative approach, explains Allspaw : "How much attention did vulnerability patching get up until the day before (the breach)? If I was the leader of this organization, part of my job is to create the conditions such that the organization can bring their attention proportionally and understand what the trade-offs are."
2.
If you can't responsibly protect a certain part of your business, then you have no business trying to monetize it, per Kim : "I think the other thing in this scenario is, I would actually want business leaders to truly understand that this is an existential risk. The outside threat has never been higher, it's always escalating and it is actually left unaddressed. This is an existential threat to the business model. If we can't responsibly hold PHI, then our ability to make money from it should be put into jeopardy."
3.
Kersten is particularly peeved by the narrative around the breach: "What's so disturbing about the narrative here is that these leaders of these companies are not understanding that they have an organizational responsibility to managing their IT stack. That stack is how they're delivering value to their customers and how they're exposing their customers' data or safety."
4.
It's important for the person responsible for a system failure to step up and take responsibility, says DeArdo : "You have to accept responsibility not just because it's the right thing to do, but because it allows you to start talking in a way of not only what we can do for our customers but what we can do to our culture and our systems. If you don't take that responsibility, not only does this send the wrong message, but it doesn't let you move towards fixing things."
5.
Nasello on the finger-pointing nature of the response to the breach: " figures quite prominently in the DevOps transformations we're trying to do in our companies. It rings hollow if you're not really willing to own the consequences and outcomes of the transformation that you're trying to drive, and I think that lack of authenticity is going to hurt companies when they're trying to attract and retain and grow their organizations."
6.
The Equifax breach was an organizational-wide failure, per Esser: "What happened here was truly an organizational failure all the way up and all the way down. Any security auditor would pick up on these things in a basic audit. The question would be is how long some auditor was saying, ‘We have a problem.'"
7.
Elaborating on what he calls "normalization of deviance," Wallgren says: "We've been running with struts for so long and nothing bad has happened – maybe we'll be able to keep doing. It really is a systems failure and maybe there needs to be a NTSB-like function for these kinds of problems where you have an independent fact-finding, probable cause finding situation. We're never going to find out what the real problem was at Equifax."
8.
Lead by example – what you do as a leader will become the new standard, explains Nasello : "As leaders of organizations, the things that we tolerate become standards as well. Our organizations and the teams that we lead are very observant in terms of what we tolerate and the examples that we set. We undermine ourselves a lot by acting differently than what our words are, and so authenticity and really being clear on what the principles of the organization are and what you're really trying to achieve is primarily a leadership function."
9.
What it really comes down to is doing what is right, advises Kim : "Let us not fool ourselves, when things like this happen regulatory bodies start getting involved, investigators are getting involved and I, as the leader, would want to get ahead of that. We're not going to do something to make regulators happy, we are going to do this because we know that this is what a responsible, successful organization does. That's something I would love to see from that leader."
10.
Allspaw on the direct relationship to business success and complexity: "As you become more successful, you are proportionally becoming more complex because you are taking advantage of new opportunities. Therefore, you have to keep that ability, that capacity, to grasp new opportunities in step with investing in all of the things that you need to do to mitigate the risk that comes along with it."
11.
It's important that business leaders understand technical debt, says Kersten : "In large organizations, if they don't understand that the trade-off between investment features and technical debt or even value stream improvements – as is the case – then you need to set a value stream that can actually patch struts and an architecture that supports that, otherwise they can't lead the company adequately."
12.
The more you can reduce transaction costs around non-functional requirements, the more business buy-in you will receive, per Esser: "The spirit of the DevOps movement is how you make non-functional requirements, like security maintenance, that from a business perspective look like a liability, they look like they're costing me money. How do you reduce that transaction cost? The more you can reduce that transaction cost, the more the business is going to be amenable to you doing these functions."
13.
It's all about getting in the right mindset, per DeArdo : "You have to have a mindset beyond, ‘I'm going to patch. I'll just keep up with my patch and the problem will go away.' Yes, you should do patches, but that's not going to solve the problem. You don't have the right culture mindset to drive a stride."
14.
It's important that the technology and business organizations communicate with each other the reasoning for making certain decisions, advises Nasello : "Sometimes in the technology organization we may be constrained with vocabulary on helping our business leaders to understand why we need to continue to invest in availabilities or nonfunctional capabilities. Not understanding the broader context in the business domain of what they were using the technology organization for is a chronic conflict. I think what exists in all of our organizations is making hygiene, maintenance, everything else important along with business."
15.
It's important to explain things in terms that each stakeholder will understand, advises Kersten : "Our CFO just calls himself an accountant and so we have to bring it back to those terms. And same with some of these CEOs – it's got to go back to business terms. In the end, it's about dollars and risk. In the end business leaders should be looking at net present value of the company. They understand if you've got high velocity, but extremely high risk, and this new application has sensitive information that's exposed, then the present value will be lower."
16.
Getting security comes down to affordability, says Esser: "It's not the value of the investment, it's not the value necessarily of security. I can try to compare that value but, there's probability involved as well. In my experience you're always going to be able to do what you need to do as a technologist if you can make it affordable."
17.
Allspaw doesn't think this is actually a leadership issue: "I actually don't think that there's a leadership lesson in here. There's a leadership lesson in apologizing, a leadership lesson in setting the conditions for the organization to learn, but again in the end it all comes back to faster, better, cheaper."
Watch the full discussion below:
Learn from these DevOps experts, and many others!
DOES17 San Francisco is selling out - register today!
Register for the conference
This post originally appeared as a 2-part article on DevOps Digest.