I’m reading this really great book – “The Lean Startup”, buy Eric Ries. It has many great ideas on how to build a product. It is about taking the concepts from Lean Manufacturing and applying it to software development. So, the ideas are mostly geared toward software product creation, but can be applied to most types of products. I’m a bit more than halfway through and thus far he has only used consumer internet software products as examples.One of the interesting thought experiments it has engaged me in is trying to apply his ideas to enterprise software products – software that you sell to businesses. Symplified sells to businesses, not consumers.
For example, Ries talks about a process called validated learning. Which is a method by which a company can test the validity of each new product feature it puts out. So rather than being able to say “we just put in features A, B, and C and we altered our marketing strategy, and sales are up 10%”, you can be more targeted and say “feature A caused registration to increase 5% and feature B caused existing users to increase use 10%”.
One of the ways to do this is with a process called “split-testing”. When you put out a new feature, you only put it out to half your customer base. You then measure the difference between the customers with the feature and those without to see if it drives the behavior you are looking for (increased use, increased new users, etc…). If the feature doesn’t have any positive impact, you remove it from the product, or you at least do not enhance it any further.
We don’t have a formal validated learning process at Symplified. Split-tests have to be built into the product, which is a lot of effort, and you have to have a large accessible user population (which is much more prevalent when the company is selling to consumers) to make the splits test show trending data. I am currently in the middle of gathering data for a comprehensive report on how our customers are using our product and what features they are using. That could be the start of some validated learning. Issuing a customer survey would be another way to collect data. And of course, as Ries points out, talking to your customers is a great way to achieve validated learning about your features. The aforementioned report comes from metrics that our product collects and notes from many conversations with our customers.
The thought process continues and I hope to be able to get more validated learning information into the Symplified decision process as I figure out more and better ways to implement it.