If You're Not Experimenting in Production, You're Falling Behind

From a young age, we have a certain appreciation for presentation.  Specifically, we like the idea of quietly putting something together, working on it, polishing it, and then unveiling it.  Ta-da! It starts with things like papers and essays assigned in school.  Take it home, write it, and then double triple check it before handing it in.  Your grade and your pride both depend on it. We carry this with us through primary and secondary education and then on into adulthood.  For an iconic example, think of Steve Jobs.  "Oh, but there is one more thing..." Steve Jobs controllered the art of maintaining secrecy, followed by a grand unveil.  On a stage, he would reveal a polished, secret product, almost like a magician.  The effect, like Apple's success, was undeniable.  Grand unveils are the stuff of tech legend. But, here's the thing.  They're also the stuff of tech disasters. For every Steve Jobs unveiling an iPod, there are thousands of Steve Joneses, building some tool or offering in "stealth mode."   But, unlike the iPod, these things never see the light of day.  They may never even become complete. Maintaining secrecy and having a flair for presentation appeals to an aesthetic we all hold dear.  But they also carry enormous risk --- you can hit a home run, but you're far more likely to strike out. So I suggest that you don't go this route.

There's a good chance you've heard of The Lean Startup.  It's a book and a movement that have predictably taken the startup world by storm but have also gained much traction in the established corporate world. If not The Lean Startup, you've probably at least heard the term "minimum viable product" (MVP) tossed around.  And there's a near-certain chance you've heard people misuse that term. In casual corporate conversation, this is the definition we assume.

MVP: The result of "phase 1" of a project.

But that's not right.  It subtly but profoundly misses the point.  Let's look at the actual definition.

The cheapest/easiest thing you can build to start testing your value hypothesis and learning.

Both definitions focus on minimalism.  Both encourage you to get away from essay writing and turtleneck presentations of electronics in favor of incremental progress.  But one focuses on delivery, while the other focuses on learning. In my experience as a consultant to the enterprise, this can be sort of a fuzzy concept for those who are new to it, so let's consider an example.

This is an example that a consulting colleague of mine exposed me to, so I don't know whether it's something that actually happened or if it's more of a parable.  In either case, the lesson stands. There was a pizza shop that thought it could improve on its business by allowing customers to order online for delivery.  So they had a classic business conundrum: trying to forecast return on investment (ROI) for a potentially expensive project. A surprising number of companies will simply go ahead with the build of a website based only on this hunch.  They'll contract a "phase 1" build of the site, calling it an MVP.  It'll include dropdowns to pick toppings, delivery status, logins, and credit card processing capabilities.  That's expensive. Other companies might think to do a bit of recon.  They'll send out a survey to their customers asking if they'd pay for or be interested in an online ordering system.  But, as anyone who has done market research knows, there's a big difference between a person saying they would buy something and that person actually putting up money.  So this is cheap, but not very predictive. What's a pizza shop to do? Well, this one had a rather ingenious idea.  They put together the dead simplest imaginable website.  And, since something like 40% of their customers just wanted medium pepperoni pizzas, it just had a form for entering an address and a button that said, "order a medium pepperoni pizza and pay in cash."

Does this seem crazy or obtuse to you?  If so, that's understandable.  And it's also what makes this a great example. But remember, the purpose of the MVP isn't to build --- to create a beta version of the eventual website.  Instead, the purpose of this MVP is to see whether people will take advantage of online ordering and whether it will help sales.  In other words, the purpose of the MVP is to see whether it's a good investment to build the real thing. Once you realize that, a light bulb tends to come on and the ideas pour out.  You realize that you don't need to build a full site or even a particularly functional one.  And then you start to transfer this thinking to your own domain.

Maybe we don't need to build a full widget ordering website to see if people will buy widgets.  You know what -- maybe we don't need even need to build any actual widgets.  Maybe we can just put up a Shopify store to take pre-orders and see if anyone pre-orders.  If they do, then we'll hurry up and start building them.

The Lean Startup brings the idea of the scientific method to businesses, encouraging you to form hypotheses (people want online ordering) and to test them as cheaply as possible by running experiments (a dead simple, throwaway website).

The Lean Startup itself devotes a section to the idea of so-called "intrapreneurship" within established companies.   And a lot of companies and people in them have embraced the idea.  More and more, organizations are contriving and executing inexpensive but valuable experiments. And if you're going to do this in your organization, with your software, you're going to need to do this in production. Want to know if a change to your ordering workflow is going to increase revenue?  Roll it out and see.  Do you think you'll see less customer attrition if you make them an offer when they unsubscribe?  Try it out. The experimental methodology means that you don't operate on hunches and speculation and you don't make indirect guesses.  Instead, you constantly run inexpensive experiments, tune, and move on.

There's a reason that the Lean Startup has attained iconic status in our industry: the methodology yields results.  It has brought startups into the limelight and it has revitalized flagging enterprises besides. So it's worth pursuing. But that doesn't mean it's going to be easy. You're going to need to adjust both your tooling and process and also your culture.  The Jobsian culture of long periods of secrecy and grand unveils has both appeal and a ton of inertia, so you're going to need to work your way toward experimentation and rapid iteration. But you can get there.  And it's worth getting there because you can bet that your competitors are moving in that direction.  If they bring experiments to production first and learn more quickly than you, you will forever be playing catch-up.