Measuring Success

My previous post was a semi-theoretical statement about how to view the need to fund the continuous experimentation of consumer services.  Last week I was on a panel at Pillsbury Winthrop regarding Web 2.0.   Continuing my point about the need for continuous iteration, I made the point that it is absolutely essential to instrument your business to learn the fine grain details of what is really driving your business. 

I thought I'd share with you just a hint of what I mean.  Each day I, and every other Board member at Riya, get an email containing over 50 operating metrics of Like.com.   Some of them are obvious, like Daily Uniques, Number of Clicks, etc.  Others are not so obvious, and reflect our daily success at coaxing users to help us build the business.  Each of these metrics ties to decisions the team is making about traffic sourcing, user experience, product features, products being compared etc.

One of the cooler things in the daily stats are the SKU analyses we see.  Among all the other stats, we get to see the daily zeigeist of fashion and what's selling.  We literally get pictures with associated links for the top selling SKUs.  My last mail with the SKU report had the following products as top sellers:

About a month ago I actually bought a pair of "Von Zipper" sunglasses that came to me in my daily traffic report.  (Invest $5.5M you still pay retail!)

This fanatacism about continuous measurement and transparent communication is not something the Board imposed, nor even asked for.  I remember this was the thing that most impressed me about Munjal the first time I met him five years ago. Then he whipped out an excel spreadsheet with every operating parameter of his prior business and showed me how he and the team were moving up, one day at a time.

There are a lot of ways to grow the top line.  You can spend your way to revenue (and oblivion) - the strategy formerly known as "get big, fast."  There is only one way to grow the bottom line - make the business algebra actually work. Today Riya is using the organic traffic to see how changes improve monetization, repeat usage, etc. The business is in "fine-tune" mode.  Every change improves the lifetime value of every user and the marginal profit contribution of every dollar to be spent on marketing.  We haven't poured gasoline in the engine to accelerate the traffic.  We are tuning the user experience as measured by the economics of the business.  Of all the graphs and images I get every day, this one is the best because it shows our users are voting with their mice.  What's not to Like? 

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the state of science
- Lord Kelvin

...and these days, consumer internet is definitely a science.

Comments

[Steve Sarakas]

Once I was in a big unfamiliar city, and late leaving for the airport to go home. I was near the airport and racing to return the rental car. How hard could it be? I kept getting closer, turned here, then there. Raced along the highway and I could see the terminal. I could see all the car rental signs. I kept looking around to see how to get over there. Closer, closer. Suddenly I came to a huge chain link fence - it was a dead end, and ugh, the rental return was on the other side of the fence.

The problem with the idea that metrics can be established, so that optimizing business performance can be reduced to a panel with knobs on it, is that without original input you don't know where you COULD be. Even if you can tweak to improve, how do you know it isn't supposed to be ten times better than that?
Imagine you are at the bottom of a valley. In all directions, difficulty increases. Even a servomechanism can get into a dead-zone. It's not where it's supposed to be, but in all directions the difficulty increases, and seemingly, all other directions are wrong. Hence, paralysis.

This thinking sometimes manifests itself in the "core competency" argument. I had a bush out front of my house. One day I noticed part was dead. So, I pruned it - same as retracting to "core competency" - and it immediately looked green again. A few weeks later more brown. Hmmm. So, I pruned it again. Each time I pruned the bush, it looked great. But after several more times, it had been pruned to half of what it once was. Finally, I discovered that it had a worm in it all along. This reminds me of a company that trims and cuts, trims and cuts, each time the immediate result is good. They announce that this was another riskless return to their core competency. The rot returns, more cuts, and each time it SEEMS to bring about a measured improvement.

See? In the case of the servo, stability is reached in the wrong place and a disturbance must be introduced into the non-linear system. The same thing can happen in business - especially in new companies trying to use knowledge someone brought to the table from a different experience. Consider also Cristensen's The Innovator's Dilemma, and the nature of this when only sustaining technological improvements are pursued - anything else is unthinkable, at the time. Just check back in a few years.

Thanks for allowing me to post this. I'm not opposed to metrics, and believe me I love to design systems that work.