Page view metrics? Bah, humbug!

    I want to come to Yahoo's defense about something. A recent spate of reports says that Yahoo has been surpassed by various companies in terms of page views. Why is that relatively bogus? Because of Yahoo's switch to AJAX for its mail. According to Alexa data, 49% of Yahoo visitors go to mail.yahoo.com. Everyone knows that I take Alexa data with a grain of salt, and that 49% fraction may be high, but Yahoo definitely gets a lot of traffic from Yahoo Mail. Yahoo's new mail system uses AJAX. And how do the metrics companies handle AJAX? Typically, not well.

At SES San Jose recently, I asked a metrics company about how they count AJAX and the metrics person got a deer-in-the-headlights look on their face. What does your traffic look like if 30-50% of your page views are suddenly converted to AJAX where a page never really reloads? Your traffic doesn't change and you may have happier and more users, but your metrics will plummet. By the way, that's probably why you saw this post recently on the Yahoo! Anecdotal blog talking about page views.

If you think about it, you'll see why AJAX breaks everything. Page views are "easy." Take all your GETs, POSTs, HEADs, or whatever and add them up. Maybe do something smart about images/CSS/JS/framesets and everyone's metrics will roughly agree. Now, what's the relative value of a Maps AJAX request vs. a Gmail AJAX request? Do you see why your head will hurt if you try to come up with good metrics for an AJAX site that isn't yours? By the way, Evan Williams pointed out months ago that Google Analytics can help you track your own AJAX applications. But that's different: if you're writing your own AJAX site, you know what events matter. But metrics companies won't know which AJAX events should be counted unless they read your code carefully. And that's assuming that the ISP they're buying data from gives them AJAX requests in the first place.

What are the takeaways from this?
1. Remember that post that said Gmail had a 2.5% market share? Shortly afterwards, you started to see a "Google succeeded in search, but hasn't done as well in other areas" meme. I wonder if we should reconsider the origin of that notion.
2. Everybody ask your favorite metrics company how they handle AJAX. Start with "exactly what level of AJAX data access do ISPs sell to you?".
3. If you're doing a start-up and want impressive page view metrics, stay the hell away from AJAX.
4. If you would even *for one second* consider staying away from AJAX for the sake of impressive metrics, you're running your start-up ass-backwards.

Okay, I'm out of steam. If I had more steam, I'd rip into the idea of an hours spent searching each month as a good metric for search engines.