I had the pleasure of attending the Continuous Deployment Breakfast with Eric Ries last week. Eric Ries, creator of the Lean Startup methodology, Co-Founded IMVU who’s known for deploying code to production as many as 50 times per day!

In Agile, there’s this notion that there’s someone “out there” (e.g., the Product Owner) who has all the answers about what the product should do. However, Lean Startup says no one has all the answers. This, of course, makes sense in a startup where where the customer is To Be Determined. But I wonder if it doesn’t hold a bit of truth even for more established products… does one person really hold all the answers?

And so – if we’re not sure what the right thing to build is, our most important task becomes learning.

What’s the fastest way to learn if the product is going to make sense for people? Get it in their hands! Of course, Agile recognizes this – which is why we work in short, 2 week iterations. But Continuous Deployment says why do we have to wait 2 weeks? Wouldn’t it be better if we could get continuous feedback on what’s working and what isn’t? Wasn’t that actually the original idea behind agile (ala eXtreme Programming) – that the customer was supposed to be sitting in the room with the development team? We don’t hear about that as much today because it’s proven so impractical in most situations. But maybe Continuous Deployment offers a better way.

50 times per day?!

I realize it sounds pretty – well, extreme. But I think that might be because the way we work today isn’t set up for it. I keep thinking back to that crazy, radical concept of “daily builds” in the late 90’s. This notion that we should integrate everyone’s code every day (zomg!). We’d work on these huge chunks of functionality that took a month or more to code. At any time within that month our code could be in any random state of working or not. It didn’t matter because we were just trying to get it working by the end of the month. But the idea of integrating that code with everyone else’s every day – well that was crazy. We weren’t working like that. In Lean-speak, we were working with month-long batch sizes.

Agile has brought our batch sizes down so we can fit several within our 2 week sprints. And, as we’ve seen, lowering our batch size has helped us work faster, be more flexible, and get more feedback so we’re better aligned to real customer needs. But still, continuous deployment is asking us to drop our batch sizes even smaller – where each developer actually arrives at “done” one or more times in a day. Is it crazy? Or is it just taking agile to the obvious next step?

But… isn’t Deploying everybody’s changes right into Production a really terrible idea?

Yes! Probably. So we have to build what Eric calls an Immune System. This starts with what we’d normally do for Continuous Integration. As soon as code is checked in, it’s integrated with the rest of the system and if the build or any of the automated tests break, the new code is rejected.

Okay, great, but the code might pass all its automated tests and still have a fundamental problem, right? Eric likes to give the example of turning the Check Out button to be white text on a white background, “effectively turning your startup from a business into a hobby.” We need more then just automated tests to make sure everything is working. We need to also be monitoring user’s behavior on the site to make sure everything is copasetic.

When we’re totally focused on learning, we want to be monitoring anyway – is anyone visiting our site? What are they doing there? Are they buying anything? Think Big Visible Charts – but about what our customers are doing rather than what the developers are doing (because, in the end, all that really matters is what the customers are doing, right?). So add some alerts if anything goes wonky (say check outs suddenly drop to 0) and now our immune system includes not only our automated tests but also evaluating user activity to make sure it’s still healthy.

Okay, but… is it going to be Usable?

Do we seriously want to throw changes out there that a human being hasn’t looked at yet?

According to Eric, 80% of the changes we make are behind the scenes, so not even visible to end users.

For the remaining 20%, yes – we do want a person to look at them. But, in the case of a brand new product, wouldn’t we prefer our early adopters be the ones to look at it? Ya know, the ones we’re trying to learn from so we can have a successful product. Or, maybe there’s a certain customer that specifically asked for this change, or provided input on it’s specs. What if you could selectively release it to just that person and get their thoughts on it. Wouldn’t that be more useful?

Release is a Marketing Term.

If there’s one thing to make the entire company crazy, it’s releases. Getting a release out on time, making sure sales doesn’t “oversell” what it can do, making sure it actually works in production and doesn’t break a bunch of other things.

At IMVU, Eric said most all of their initial deployments of features were done only to a small percentage of customers – customers who they could talk to to get feedback from – so they could learn if that feature made sense.

Then, once they’d validated/tweaked the feature into what it should be, then marketing could decide when they wanted to deploy it to everyone. This totally eliminates the stress around releases because: we know exactly what the feature will do, we know it’s ready, and – most importantly – we know it works with real users in production!

So, what do you think? Can deploying to production 50x/day be a Good idea?