comScore Misinterprets Data: MySpace is *NOT* Gray

futurelab default header

by: danah boyd

Read the comScore press release. Completely. Read the details. They have found that the unique VISITORS have gotten older. This is _not_ the same thing as USERS.

A year ago, most adults hadn't heard about MySpace. The moral panic has made it such that many US adults have now heard of it. This means that they _visit_ the site. Do they all have accounts? Probably not. Furthermore, MySpace has attracted numerous bands in the last year. If you Google most bands, their MySpace page is either first or second; you can visit these without an account. People of all ages look for bands through search.

Why is Xanga far greater in terms of young people? Most adults haven't heard of it. It's not something that comes up high in search for other things. Facebook's bimodal population pre-public launch shows that more professors/teachers are present than i thought (or maybe companies are more popular than i thought? or maybe comScore's data is somehow counting teens/college students as 35-54…).

Can someone tell me exactly how comScore measures this? Is it based on the known age of the person using a given computer? Remember that many teens are logging in through their parent's computer in the living room. Is it based on reported age? I kinda doubt it but the fact that there are more 100+ year olds on MySpace than are living should make people think about reported data. Is it based on phone interviews? How do they collect it? This isn't really parseable into English.

My problem is that all of these teen sites show a heavy usage amongst 35-54. I cannot for the life of me explain how Xanga is 36% 35-54. There's just *no* way. I don't get how the data is formulated but it seems like an odd pattern across these sites to see a drop in 25-34 and a rise in 35-54. Older folks aren't suddenly blogging on Xanga. So what gives? My hunch is that comScore's metrics are consistently counting teens as 35-54 across all sites. My hypothesis is that because comScore is measuring per computer and teens are using their parent's computer, comScore can't tell the difference between a teen user and a parent user. If so, maybe all this is telling us is that parents have definitely listened to the warnings over the last year and are now making their teens access these sites through their computer?

Finally, when we talk about data, we also need to separate Visitors from Active Users from Accounts. The number of accounts is not the same as the number of users. The number of visitors is not the same as the number of users.

All this said, there is no doubt that more older people are creating accounts. Parents are told that they should check in on their kids. Police officers, teachers, marketers… they are all logging in to look at the youth. Is that the same as meaningful users? Some yes, some no.

From my qualitative experience, the vast majority of actual users are 14-30 with a skew to the lower end. Furthermore, the majority of the accounts are presenting themselves as 14-30. To confirm the latter (which is easier), i did a random sample of 100 profiles with UIDs over 50M (to address the "last year" phenomenon). What i found was:

  • 26 are under 18
  • 45 are 18-30 (with a skew to the lower)
  • 10 are over 30 but under 70
  • 1 is over 70 (but looks less than 18)
  • 6 are bands
  • 11 are invalid or deleted
  • 1 is complete fake characters (explained in descript)

A few more things of note…

  • 18 have private profiles
  • Of those over 30, only 2 has more than 2 friends (one has 3 friends; one has 5)  

This account data hints that the general assumption that approximately 25% of users are minors is correct. Of the remaining, the bulk is under 30. Qualitatively, i'm seeing the most active use from those under 21. Given account practices, i don't think that i'm off in what i'm seeing.

I do suspect that MySpace is holding strong at being primarily for younger people but that older folks have definitely been checking it out a LOT more. Still, i'm still suspicious of the fact that 35-54 are common across all youth sites. I'd really like to see comScore's data on something that we can check. Maybe LiveJournal?

(I'd really really really love to be proven wrong on this. If anyone has data that can provide an alternate explanation to the comScore numbers, please let me know!)

Update: Fred Stutzman and i just jockeyed back and forth to find something we could agree on wrt the comScore numbers. Here are some ways of making sense of the data of VISITORS:

  • Xanga is more of a teen-flavored site than MySpace, Facebook or Friendster
  • Facebook is more of a college-flavored site than MySpace, Friendster or Xanga
  • Friendster is more of a 20/30-something flavored site than MySpace, Facebook or Xanga
  • Of users going to these four sites, MySpace does not swing to any one group; it draws people of all ages to visit the site.
  • A greater percentage of adults (most likely parents) visit MySpace than any of the other social sites

This is all fine and well and confirms most intuition. The problem is that what we CANNOT confirm via this data is that more adults visit any of these sites than minors. Again, intuitive but the comScore data seems to indicate that adults visit each of sites more than their key population. This is really visible in their "total internet" users which seems to suggest that the vast majority of visitors to all of these social sites are adults. I cannot find a single person who works for one of these companies that believes this.

I've spoked to numerous folks since i posted last nite. Most believe that comScore gets this data by running a program on people's computers. Young people are supposed to use a separate account than their parents. This data seems to indicate that comScore is wrong in assuming that people will do so. Most minors probably use their parent's account to check these social sites. So, if we assume that, Xanga is obscenely a teen site, Facebook probably has nearly as many high school users as college users and MySpace swings young but is used by a wider variety of age groups than most social sites.

Finally, it's all nice and well that Fox Interactive spokespeople confirm this data but i've watched over and over as FIM has confirmed or said things that were patently untrue in public. I don't know if this is because FIM (the parent of MySpace) doesn't know what's going on on MySpace or if it's because they don't care whether or not they are accurate publicly. I don't honestly believe that FIM has any clue about the age of its unique visitors. They know the purported age of people who have accounts and it would be patently false to say that 35-54 dominates account holders.

Frankly, i'm uber disappointed with comScore but even more disappointed with all of the press and bloggers who ran with the story that MySpace is gray without really looking at the data. This encourages inaccurate data and affects the entire tech industry as well as policy makers, advertisers, and users. I'm horrified that AP, Slashdot, Wall Street Journal, and numerous respectable bloggers are just reporting this as truth and speaking about it as though this is about users instead of visitors. C'mon now. If we're going to fetishize quantitative data, let's at least use a properly critical eye.

Original Post: http://www.zephoria.org/thoughts/archives/2006/10/10/comscore_misint.html