Articles on Radian6

Twitter FirehoseDrinking from the Twitter Firehose

The Twitter Firehose is a phrase that refers to the roughly 50 million daily tweets on everything from Wall Street reform to Justin Bieber’s favorite cereal. There are hundreds of tools on the market designed to monitor and measure Twitter data by allowing users to easily search and track Twitter trends. One of the big problems with most of these products is that they don’t have access to the full Twitter Firehose. Typically they might only have access to 10%-20% of all Twitter posts on any given day. There’s a couple reasons for this.

  • The cost and technical challenge of handling so much data
  • The cost – or privilege – to access the Firehose

Who’s Tapping it?

Giants like Microsoft, Yahoo, and Google have reportedly paid hundreds of millions of dollars for access to the full Twitter data stream. Surprisingly Twitter has decided to grant access to the full Firehose to a number of startups including; Twazzup, Collecta, CrowdEye, Scoopler, Kosmix, Chainn Search and Ellerdale.  No one is quite sure, at least I’m not, how much it’s costing these startups (if anything), and why they were chosen when so many others have been denied. I should also mention that there are also a small number of mid-size companies that fall somewhere between the giants and startups that now have access to the full Firehose – including Jive and Converseon.

A large number of social media monitoring companies – whose goal is to collect and consolidate all relevant social media data for its clients – are left out of the party. Which means even great products like Radian6 and Scout Labs have a significant hole in their data inventory – full Twitter Firehose access. Both companies expect to have full access at some point, but when isn’t yet clear.

What Does the Future Hold?

Some experts expect Twitter to eventually be more generous in granted access to the Firehose. However, this would be opposed to the direction they’ve taken recently in switching to their own URL shortening service (rather than using, unveiling their own iPad application, and requiring third-party Twitter applications to access user accounts through the OAuth authentication standard. The latter move caused problems for a number of high profile tools like TweetDeck and a number of widely used WordPress Twitter plugins.

The upcoming release of the Radian6 Engagement Console promises to improve the way companies monitor and engage customers, or their client’s customers, through social media.  I have looked at a variety of social media monitoring systems over the past few months – including entry level tools like Trendrr or SocialMention, as well as mid to enterprise level applications like Vocus and Nielsen’s Buzz Metrics. There are dozens more I’m anxious to take a look at – including products from Converseon and Autonomy. Take me to Your Leader has a nice list of dozens of free tools.  However, I’ve been less than impressed with most of the products I have used.

I’m currently leading a project for a large Federal Government agency to evaluate the effectiveness of social media for health communications. So these tools are of particular interest to me now.  This new Radian6 release has me excited. I’ve yet to see a product that systematically streamlines the engagement process like this. Does Autonomy offer this type of functionality? Does anybody else?

I’m also curious about what platforms Radian6 will monitor.  One capability that most systems lack is the ability to monitor and engage through non social media channels that have certain Web 2.0 components. For example, a news story on may generate dozens or hundreds of user comments. These data are just as important as anything being said on Facebook or Twitter.But many products seem to ignore these more traditional Web channels.

I’ve setup a Radian6 demo, and can post a follow up if anyone is interested.

YouTube Preview Image