Audience Measurement is a Dismal Science, too

My prior post certainly stirred up a shit storm of foment.  There were really two parts of the post.  One part was the personal POV about sameness in the Web 2.0 startup community.  The second point was the consideration of some larger indicators. 

As was nearly universally pointed out, Alexa is unreliable. However, there is a difference between lack of reliability and systematic bias.  What struck me was apparently consistent phenomenon across the three sites that I chose.  What I did not do, and this is my mistake, is corroborate this measure (Alexa) with other measures.  Below comparisons for Gigaom.com, Techcrunch.com, and Technorati.com on Compete.com and Quantcast.com.  Clearly the conclusions drawn from the Alexa data are inconsistent  with these sources.  While I stand behind my original observation that the wheel of innovation is largely spinning without progressing, the data from these sources do not support the conclusion that the general population's interest in what's new in the web 2.0 community is on the wane.  My apologies to all for trusting a single source.

BTW, for some reason I could not get Quantcast to generate a graph with more than 2 days' data for Gigaom, which is why its so limited. 

Comments

[Lem Bingley]

None of the free traffic estimators are up to much: I've compared some of them with full WebTrends measurements for vnunet.com, a medium-sized site in the UK (see http://lembingley.itweek.co.uk/2007/02/are_traffic_est.html ) while VentureBeat blog has used the same technique to assess the free stats against the real data for a US consumer site and come up with the same lack of any useful accuracy (see http://tinyurl.com/2ua5xl ).

The likes of Compete and Quantcast seem to be extrapolating from rather small samples. It would be helpful to the web at large if one of the sites that provides a very widely adopted monitoring toolbar (such as Google or Yahoo) were to make some data available. For the moment, it appears that both of the big guys tend to see that audience data as proprietary and too valuable to give away even in smoothed format.

[s]

*** the wheel of innovation is largely spinning without progressing ***

Indeed, it's the consequence of a secret aversion to risk. Only after the visionary moves forward does everyone else occupy the territory behind them. The occupiers aren't tech squatters or copy cats, they serve a useful purpose in strengthening the ground for subsequent visionary leaps. The visionary edge has the most value of all.

Here's a tiny example of how value can be attributed to a vision. New auto designs had a leadtime of around five years, for a long time. Thanks to not only computing, but also organizational improvements, this leadtime has necessarily shrunk to perhaps two years. Now suppose you wanted to break into auto manufacturing cold. The cost of entry would be prohibitive, but imagine you had a vision for where the World will be in ten years. You could make a far smaller investment and simply intercept the market ten years later with a design that had been in the works for ten years. The cost of hitting targets two years down the road is far higher (just call up a fabricator and tell them you need a certain stainless enclosure in one week instead of six). The difference is the value of that vision. Absolutely no one else would have been working on the ten year track and you would appear to have emerged from nowhere.

Presumably, everyone understands how lines can be drawn between money and risk, the cost of taking risks, how a riskier investment should promise a bigger reward, etc. Vision reduces risk, hence, value can be directly attributed to it.

Anytime you see a remarkable advancement in technology seem to suddenly appear, the visionaries had been hard at work for a long, long time. During that time, they simply could not have been paid enough to NOT do what they were doing, because they could see something no else did. This helps to explain why many of the companies that are big today - you won't even be able to find them in five years let alone ten. And companies you have never even heard of today, will seem to have come from nowhere.

When the occupiers simply build upon what someone else has already pioneered, the value is intrinsically smaller. Even if the splash is big, the duration may well be shorter, or the revenue stream baffling. They're shoulder to shoulder in social networking and video sharing right now. The tiniest distinction attracts investment. Too bad money cannot buy vision. It can only buy the consequence, in a marketplace where everyone else shops, too. Vision is a product of knowledge plus emotion. Execution can still fail, but pour knowledge and emotion into the same container and vision boils over into things that have never even entered your mind.

I recommend not focusing on where we seem to be. Don't stare at the occupiers. Few have any comprehension of what lies ahead.

[Simon Brocklehurst]

I'm not claiming to be an expert on these stats sites, but I'm really not so sure the conclusions one should draw from looking at the stats are all that different. For one thing, they present their numbers in different ways, which can make comparisons tricky.

One comparison across the stats sites that I think should be reasonably valid, is the relative ranking of "popularity" of sites. If you look at things this way, you will see that the three stats sites you used: Alexa, Compete and Quantcast, ranked the three Web 2.0 sites you looked similarly.

That is Technorati ranked as much, much more popular that either Techcrunch or Gigaom, with Techcrunch being rather more popular that Gigaom.

[frank]

Peter,

Valleywag actually got it all wrong: it compared Alexa Reach (which represents the number of unique visitors per day) with Techcrunch’s pageview number. Not an apple-to-apple comparison!

If you compare pageviews from Alexa and pageviews from techcrunch's sitemeter pageviews, they match quite well.

I put together some graphs to show that on my blog:
http://lifeisaventure.wordpress.com/2007/03/27/how-to-use-alexas-traffic-stats/