The Emperor Wears No Clothes: Is Bit.ly the Enron of Social Media?

Update: Chris Michaels posted a blog about these issues on 10/12 – read: “How can Modern PR Avoid its Enron?” 

A while back, a hacker group virtually destroyed the reputation for a leading Sacramento-based cybersecurity firm, HBGary, by posting thousands of confidential, highly embarrassing documents to HBGary’s website and bitter sarcasm from the CEO’s public Twitter account. Hackers also publicly shamed the firm’s alleged clients, including the Department of Defense and National Security Agency.

Could this happen to anyone? Maybe. Viruses, hacks and phishing — not to mention hacker collectives doing it “for the lulz” — are perhaps the greatest threat to social media’s relevance in American business. Companies like Facebook and Google are well aware of this threat.

The other big threat: most people are wrong about their online influence. Our social media data is terrible, non-existent, or incorrect. This accountability failure is sweeping the entire social media marketing industry, which at this point can loosely be defined as every nonprofit, business and political campaign in the world.

Could bad math be destroying social media? The industry should pay attention.

The problem is simple: Bit.ly (perhaps the worst offender), Blogger, WordPress, Google Analytics, Google’s link shortener and much of the status quo produces wildly incorrect data (it’s been documented). You can independently verify this using paid analytics systems such as Google’s Urchin software. More on that below, but Google’s own Urchin software would probably contradict Google Analytics, if anyone can run that experiment. The data is probably wrong partially as a result of efforts to counter “click fraud” (fake ad traffic) and unsafe link traffic (viruses/hacks). In effect, social media’s biggest threat is feeding its other biggest threat.

Why should we care? (feel free to skip this section if you use social media)

Money, obviously. For example: Twitter is free to use, thus, your eyeballs are the product. Many of Twitter’s roughly 300+ million users share links, all day, every day, which translate to billions of page views that are converted into tens of millions of dollars. The math is depressing: great online content makes horrifically low revenues, which is why the news business has seen a lot of trouble in spite of great journalism. According to Newsonomics, 1,000 news site page views tended to make about $10 in ad revenue in 2010 (source). That’s low compared to print and broadcast (and a big answer to why private sector journalism is suffering), but it adds up: if 20 million Twitter users click a link, it might add up to $200,000 in ad revenue. It’s safe to assume that Twitter moves millions of dollars on a nonstop basis.

"Come see where all your money went."

But that’s just from an advertising perspective: considering pure influence, Twitter rules the world (after Facebook). The top stories on CNN.com tomorrow will be there partially because Twitter boosted their popularity. Voters across America are changing their opinions of a politician because of something his campaign staff posted to Twitter, which was then published in a top-tier, multi-million-circulation household-name media outlet. Even if you’re offline, off-air and anti-Internet, your opinions are probably influenced by people who read Twitter (read: the absolute majority of news and advertising executives in the world).

The reasoning is simple: 1,000 years ago, manuscripts changed leaders’ decision-making; in 1930, politicians got news from the radio and newspapers; in 1970, that plus TV; in 2002, Google; and in 2011, CIA analysts, TV producers, Pentagon communications staff, Egyptian revolutionaries and #OccupyWallSt folks all know it’s usually a lot easier to scan people’s 140-character summaries than it is to read a never-ending supply of content on every online channel available (as in, Twitter is easier to follow than the entire summary of the Internet, which is essentially the summary of public human knowledge). Twitter is not intuitive to a non-user — searching Obama-related tweets won’t tell you much — but to anyone who’s figured it out, Twitter is perhaps the most tailored, customized, real-time answer to a big chunk of your public information needs. That’s not to say Google+ and MySpace couldn’t do that, theoretically speaking, but both are probably lacking the user design capability and definitely lacking the critical mass to custom-stream the entirety of your mainstream public information needs.

Tying everything together here, if the assumptions below are accurate, Bit.ly is probably the single greatest threat to the social media marketing industry’s relevance. It’s the Enron of the social media industry, a black box that spits out bad numbers. Reporters and staff from The Wall Street Journal, ABC, NBC, CNN, FOX and The New York Times are all in on this. Even PR Newswire uses Bit.ly. Typically, anyone who’s anyone on the internet uses Bit.ly.

If these conclusions are right, Bit.ly is also dragging down the news business. If the internet is killing journalism, these news organizations should know their enemy.

Accountability

Let’s be clear: it appears the majority of the world’s social media web analytics are flat-out wrong, unless you are a) using the minority of free, accurate sites like tiny.cc and tweetburner.com, or b) part of a major organization that has awesome, paid analytics systems (the kind that run $1,000+ monthly), and can’t be bothered with this. Here’s some evidence:

  • Bit.ly: Bit.ly is the top link shortener on social media, and thus the top analytics provider. According to Bit.ly, I’m impossibly more influential on Twitter — on a per-follower basis*** — than the New York Times, FOX News and CNN (no). In fact, according to Bit.ly, I’m more influential than Bit.ly (on Twitter, that is). Put a “+” symbol after any Bit.ly link to see its web traffic (see example from a Bit.ly tweet to 17,000 people). It’s wrong. I blogged about this in October 2010, and nothing has changed in the past year. Read what other people are saying here and here.
    • Bit.ly says The New York Times got a miserable 548 clicks for 3.8 million followers (safe assumption: $5 in ad revenue? Source for that claim): https://bitly.com/njiUYn+
    • The simplest way to prove Bit.ly is wrong: click the links yourself and watch how the numbers don’t change, even from different computers and different browsers (occasionally they do work – it’s hit or miss).
    • For the record, TechCrunch and the Huffington Post, among the rest of the top-tier, tend to reference Bit.ly as an accurate analytics provider. If everyone’s on the same page about Bit.ly, is there one single person to blame?
  • WordPress: also wrong. According to the analytics for this particular blog, I’ve only received about 700 visits to this blog since I launched it in May 2010. That verifiable undercount is very consistent with the other WordPress sites I’ve managed in the past, including official business sites hosted on WordPress. Logically, using a bit of “dumb math” (educated guessing), you can dismiss WordPress stats based on retweets alone. But you can track it. I’ve been tracking visits to this blog from my LinkedIn, and even if I disregard search engine and Twitter traffic, Tiny.cc, shows that this blog has received 1,960+ clicks from my personal LinkedIn profile in the last 8 months alone.  I’ve independently verified Tiny.cc in real-time using Google’s Urchin 6 software (Urchin is legit, unlike Google Analytics and Google’s link shortener). To summarize: see what WordPress measured in all time traffic (700+ clicks), versus Tiny.cc data (1,960+ from LinkedIn, not counting search engine and other social media traffic).
  • Blogger: You can verify this yourself. The stats are bogus. If you use Blogger (blogspot.com), you’re almost certainly getting a lot more web traffic than you might think. My evidence for this is primarily the ASU PRSSA blog, which never gave traffic numbers that matched tiny.cc statistics****. Read what other people are saying.
  • Google link shortener: Try this yourself – the links don’t register. Notice how this Tweet resulted in zero clicks, even when I repeatedly clicked it myself.

There’s no shortage of questionable data in our industry. I regularly track 30+ daily unique visits from my LinkedIn, while LinkedIn tells me my profile has been viewed by “10 people in the last 10 days.” On YouTube, I send hundreds of trackable unique visitors to videos only to see a modest increase in the video’s view count. These websites have the best talent, the best business models, and the best user bases: it’s hard to believe they take analytics seriously (the analytics we can see anyway).

Moving forward

It’s odd that this isn’t getting attention. In my conversations over the last year with advertising and public relations executives, journalists, marketers and industry colleagues from coast to coast, no one has disagreed with these conclusions when shown the data firsthand. (I’d love to hear what Bit.ly — perhaps the most significant offender — has to say.) For background, I had the opportunity to review the official Bit.ly account of a national, top-tier media outlet. The statistics were so low that they would suggest the site gets less than half of its verified web traffic.

I’ve also sat down with social media managers and shown them a lot of this data, and again, no one has disagreed with these conclusions. It’s not clear why this is such an unnoticed problem, but it would be ideal for three things to happen:

1)      Make the switch: If you manage social media for a client or even just a personal blog, consider making the switch to tiny.cc for all shortened links. Tiny.cc’s data is as bulletproof as it gets (at least according to Google’s Urchin 6, and to a lesser extent GoDaddy.com’s site analytics, which can be used to independently verify). Tweetburner.com is another great link-shortening alternative.

2)      Explain: All the sites mentioned above — WordPress included — should be inclined to better explain where their analytics are coming from and why the traffic undercount is so enormous.

3)      Manage expectations: The greater social media universe needs to recognize that it takes a lot of web traffic to make anything happen. Votes, sales, job leads, and yes, legitimate blog exposure — all these things need way more traffic than you might expect to move the needle. It appears all this undercounting has convinced the CMOs of America that low social media web traffic means big results. A big thank you to Bit.ly in particular for helping to inflate the social media bubble.

** I asked Bit.ly how one of their links posted to 17,000+ people could only receive 5 clicks. They didn’t respond and appeared to delete the specific link I was referring to. What does it look like when your tweet is exposed to 10,000+ followers? Here’s an example: A blog I wrote in January 2010 criticized Hooman Karamian, the founder of a popular gossip site. Karamian retweeted my link – this was one of the few times that Bit.ly appeared to work properly.

The tweet (at the time, this went out to 10,000+ Twitter followers):

The stats for that Bit.ly link:

You can read the original article here.

*** My tweets usually get 30-60 unique clicks, or perhaps 80+ if it mentions Justin Bieber. For a back-of-the-envelope, apples-to-oranges comparison, I can comfortably assume I get one unique visit for every 100 followers. Bit.ly says here The New York Times got 1 click for approximately every 2,000+ followers. Here, Bit.ly tell us The New York Times got 1 click for every 7,000+ followers. Unless the NYT is sharing Twitter followers with Newt Gingrich, these figures are all but impossible.

****Tiny.cc is a primary piece of evidence for why Blogger, Bit.ly, WordPress and Google Analytics give incorrect data. But you can do the math yourself if you consider how many clicks you usually get on Twitter versus what WordPress is telling you. All these sites would have you believe it’s normal to get 5-20 clicks from a tweet to 17,000 non-spam Twitter followers. Try it: add a “+” after this Bit.ly link.

Update 10/11/2011: Bit.ly probably used to be very reliable, as recently as 2009 – see here.

***

Note: Nothing here represents the views of my employer. To the best of my knowledge, this post is independent of my clients and direct client competitors. Read my philosophy on full disclosure from last October.  Please contact me with questions/comments.

Advertisements

9 Comments

Filed under Uncategorized

9 responses to “The Emperor Wears No Clothes: Is Bit.ly the Enron of Social Media?

  1. I’ve had the same thoughts on Klout and people taking it too seriously as a metrics and influence source. What are your thoughts on that outlet?

  2. Thanks for reading, Ashley! I think you have a point, and I’ve had similar concerns. This article addresses what might be one of the biggest issues with Klout (not sure if Klout has moved to fix this problem – this was posted three days ago):

    “But when a spam account such as @armandoxf on Twitter can get a Klout score of 36 with four followers and more than 2,600 tweets of spam links, as was the case last week, how can such a measurement tool be taken seriously? (The spam account scored a zero on Klout competitor PeerIndex.)”

    From: http://www.miamiherald.com/2011/09/18/2412566/measuring-the-value-of-a-tweet.html

  3. Super interesting to read about this. Being in web design and front end dev, social media 9 times out of 10 goes hand in hand with my field and there are TONS of people that assume these people/businesses/metrics are gospel. Interesting to see they are not! Thanks for reaching out and showing this to me!

  4. Thanks for the analysis. As someone with a scientific background, I appreciate the value of metrics and well-designed experiments.

  5. I’m writing a post on the ‘high school’ psch aspects of social media and came across this post. Brilliant and thank-you. I’ll be examining this post for a while, a lot of good info… now heading to find you on twitter.

    Here is an excerpt from this morn – I just slammed this down and began doing research – finding your post at #3 of the search results for ‘the emperor wears no clothes social networking’, sorry for the coffee stains here (I wrote about double this in ten minutes – I feel pretty strongly about how the web is changing persona en masse) just wanted to give you an idea of how i got here:

    Let’s just say:
    We live in a narcissistic culture where those who gather the most virtual popularity statistically as virtual entities/memes are thereby are able to reap the greatest rewards the general republic has to offer. If we look at the current virtual social structure and the effect on economy, we see this phenomenon exponentially growing before us via internet technology. To put it simply – if you were popular in high school on any level for any reason, you will likely be able to nurture that virtual meme and translate it into success in the marketplace of popularity as you grow into an adult. Think: fashion queen, computer nerd etc. and chisel it into ‘cult of personality’ game pieces.

    Not that everyone is playing a false meme game; certainly people who attract others have some kind of consistent authentic data to offer – and not that everyone can immediately be successful at the meme game either, but having a shiny meme game piece is becoming more and more of a prerequisite for success. Let’s face it, if you cannot attract people to your digital universe, you are not going to be a winner in the digital economy.

    The most interesting aspect to me, being a curious kitty on the subject of human psychology and behavior, is that everyone ignores the fact that the virtual popularity game exists; that it is shallow and self-serving in the most corrupt way. What we are all becoming involved in is a shinier newer version of The Emperor Wears No Clothes.

  6. I did exactly the same kind of testing as this author and came to similar conclusions about 3 years ago: Bit.ly is basically JUNK and always has been. The only reason the Bit.ly link shortener service grabbed the market share it has is because many social media companies (like Twitter) integrated this useless shortener into their software early on. Test using tiny.cc as the benchmark proved the same thing: bit.ly was horribly under counting. Blogger stats are almost as bad: tests comparing tiny.cc stats with Blogger stats showed that Blogger’s stats were missing about 2/3rds – 3/4ths of the page views captured by tiny.cc. Thurl provided extremely reliable stats but that link shortener was crippled when Twitter blocked it from use. I’ve found the highly touted Google Analytics about as useless as Blogger’s stats and I discovered that years ago by comparing Google Analytic stats for some of my standard websites with stats generated by software provided by professional hosting services for the websites (not blogs) they were hosting.

    So why is Bit.ly, Blogger and Google Analytics getting away with providing shoddy stats and why don’t they fix the problem? Simple, if they “fixed” the problem they would have to admit the stats they have been providing for years are useless: useless for historical data; useless for trend data. And those individuals foolish enough to provide those stats to clients would have to admit they have been providing bogus stats.

    Conclusion: if you are not using shorteners like Tiny.cc (I don’t work for the company) to verify what Bit.ly, Blogger and Google Analytics stats are telling you about clicks on your posted links (or page views on your site) then you are flying blind as a web developer.

  7. This has being such an interesting read. I actually stumbled across it when I was trying to find out how to see the stats of a shortened URL (http://tinyurl.com/tnx-flashart found here http://www.carrollfletcher.com/exhibitions/28/overview/).
    The angle you take in your piece is really good, so much of this twittersphere and blogosphere is just non-sense and lacks credibility.

  8. Chad – thanks, that is helpful to know. To the point above re: Klout…I’m not going to say Ashley called it on Klout dying off…But I think she did: http://slate.me/U420J4

    Full disclosure: I’m back to using Bit.ly as I think they’ve seriously improved their analytics features since I wrote this in October 2011. That said, many of these companies give extremely questionable web traffic data (what year is it again?). Appreciate your feedback!

  9. All – just read the other comments on this post. Appreciate the insights! I’ll take some time to digest the info and respond this week.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s