If you have ever tried to use the Twitter search you have probably noticed it doesn’t feel much like a search.  It is more of a live filter or live pulse for a specific word, showing you a stream of live tweets containing that word.  Try using the Twitter search at http://search.twitter.com and search for something like “#CS5″ (Adobe Creative Suite 5) and you will see what I mean.  It doesn’t return the most interesting results, it just returns the most current results (imagine if Google did this).  Don’t get me wrong, current is cool, like in the event of an earthquake, or zombie apocalypse it is nice to “See what’s happening — right now.”  But maybe that isn’t what you are looking for, maybe you want to find the most interesting Tweet on #CS5, or in the case of Screenr the most interesting screencast on PowerPoint, not just the most recent.  So several months ago I was bored and set out to build a better search system for Screenr, and I did, and here is how I did it.

How to make a better search

This morning the New York Times posted an article on TweetUp, a company that is building a smarter search for Twitter (read the article here).  Essentially, what they are trying to do is turn the search into more than just a pulse, they want an actual search, like Google’s that ranks the search based on factors to determine if it will be the most relevant.

TweetUp measures a Tweets popularity by:

• Measuring how often readers repost the tweets (number of ReTweets)
• How many clicks each links get

They also just raised $3.5 million in funding for this concept (along with an interesting monetization idea).

Measure interestingness by the number of ReTweets

I think this is a pretty cool concept, if something is ReTweeted it usually means it is interesting, and the more ReTweets a tweet gets, the more interesting it is.  What I was happy about was this was the same conclusion I came to when building my search.  So tweets that are more heavily ReTweeted should appear higher in the search results, so this is what I did.

Other ways to measure interestingness

So I kept thinking about this and wondered if there were other ways that you could measure “interestingness.”

Here were some alternative ideas I had for measuring interestingness:

• Reach – “Reach” I defined as the total number of followers of everyone who tweeted the screencast.
• Views – Simply measuring the number of views of a particular screencast.
• Views per Reach - The number of views of a particular screencast divided by the “Reach” of the tweet.  The idea behind this though would be that the most interesting screencast would be the one that compelled people that saw the Tweet to actually view the video.

The difficulty of these approaches is that at Screenr we don’t give you access to the number of views through any type of API so that I can analyze the data, and when trying to determine the “Reach” of a Tweet it can use several hundred API calls to twitter that quickly push you over the API call limit.  So I abandoned any of these other approaches.

How I built my search

Here is a quick rundown of what I did:

• Dump all Screenr tweets from Twitter in a database every hour using the Twitter API. Twitter only gives you access to a couple of weeks worth of data using their search.  To deal with this I run an automated script every hour that dumps all tweets that contain the text “http://screenr.com” into a database.  This database is what I use for the search queries.  Obviously on screencasts that have been tweeted will show up in my search results because I am using the Twitter API.
• Search the database when someone searches, then rank results based on RTs. When you search using my search tool, I will query the database and find every screencast that uses your search term.  I then look for which screencast has been tweeted the most, then display that Tweet at the top of the results.

Pretty simple huh?

Use my Twitter based search for #Screenr

If you want to try it out you can use my Screenr search here:

Or by visiting it directly here:

http://www.mozealous.com/screenr/

This isn’t an official Screenr search

Just so you know, this is something I was just playing around with, at some point in the future Screenr will release an official Screenr search that is better than this.

How this search can be improved

There are a couple of things I don’t like about my search, and I think that it could be improved.  Here is what I think should be done:

• Only display the original tweet in the search results (no need to show all RTs)
• Create a better search results page, like maybe something like what YouTube does, or Bing video search, that shows the video, or video thumb, not just video description.  After all, you are looking for the video, not the description.
• Handle apostrophes better.  I don’t handle them well.

You should improve it

If you are interested in improving this search, and are comfortable programming in PHP I would be more than happy to give you access to source code I used to build this search.  If you are, let me know in the comments and I’ll give you the code and the current database I am using that stores the tweets.

Which are you: A zombie or a survivor?

Thursday (10/08/09) was an unusually quiet day on the Twitter front, for almost 3 hours I didn’t get a single update from anyone I follow, and I normally get updates from the people I follow once every couple of minutes, so going 3 hours with no posts I naturally began to worry, and I began to wonder…

Had a zombie apocalypse already devoured all the people I follow on Twitter?

Zombie Survival Kit

So I start to load my gun, and grab my sword when Twitter user @rhysatwork sends me a note telling me
(via Twitter search) that Twitter was actually frozen, and wasn’t sending out any updates (Twitter frozen).  Ok, good news, the immediate threat of a Zombie attack seemed to be over, but what lessons can be learned from this very frightening experience?

In the event of a zombie apocalypse Twitter can save your life.

You will get the news sooner on Twitter

Tim O’Reilly and Sarah Milstein point out in The Twitter Book that:

Twitter has also turned out to be one of the world’s best seismograph

They point to the graph below to show you how quickly the Los Angeles earthquake was mentioned on Twitter compared to traditional media outlets.

Twitter Earthquake Connection

So if you were paying attention to Twitter during the earthquake in July of 2008 you would have heard about it 4 minutes before any traditional main stream media would have mentioned it.

4 minutes could be the difference between life and death as you see that fleshy looking stranger run at you with out stretched arms mumbling or shouting incoherently about “BRAINNNNSSSSS!” as you debate whether you should pull the trigger.

Twitter will give you better local information

If you followed the Iran Election Crisis in July you found out first hand how beneficial Twitter can be in a real crisis to get information from a region where traditional media is either not allowed to go, or too afraid to go.  In fact, Twitter was so valuable during the Iran Election Crisis that the US State Department asked Twitter to delay scheduled downtime so that the coverage could continue.

Imagine how beneficial Twitter could be during a zombie uprising…traditional media wouldn’t dare go near zombie hot zones, much like traditional media was not allowed to cover the Iran crisis in Iran, but people forced into those situations will be able to share their experiences via Twitter, and give advice about locations safe from zombie activity (it is well known fact that zombies lack the mental capacity to grasp or understand how to use Twitter so it would be safe to Tweet about such locations).

One could argue that Twitter’s instability would make it useless in such a horrific event, but I am not so sure that is true.  Although Twitter has had stability issues but those have been mainly the result of malicious hackers or ill-conceived upgrades they have made to their platform, and in an actual zombie uprising hackers and Twitter engineers would be spending their time preparing their defenses, not making upgrades or attacking Twitter.  It would likely be weeks days or even weeks before Twitter went down, and would be more likely to be caused power outages or the internet going down.

How to use Twitter to save you from the Zombie Apocalypse

Embrace the #ZA Hashtag

Hashtags (#) in Twitter are a way for users to categorize their Tweets.  For example, if someone Tweet’s something with the #Articulate tag they are likely Tweeting something directly related to the company Articulate.  This allows people to easily find that information via the Twitter Search.  For example, check out the following search for the #Articulate tag:

http://search.twitter.com/search?q=%23Articulate

And look at the Search Results that it returns:

#Articulate hashtag Twitter search

All the topics that are returned are directly related to the Articulate Studio learning tools, and you will notice that absent from the search is any posts related directly to the adjective form of the word, which is good.

The #ZA hashtag

So what hashtag should we be looking for in the event of a Zombie Apocalypse?  I would like to suggest that we start to make #ZA the official hashtag of the Zombie Apocalypse.  It is nice because it is short, as it only use 3 characters which will enable Tweeple to use more of the 140 character Twitter limit to post important details about the attack.  Also it isn’t a widely adopted hashtag for anything else making it a perfect hashtag for following the Zombie Apocalypse.

Get a Twitter Client like TweetDeck

You can configure Twitter clients like Tweekdeck to watch specific search terms and notify you when someone tweets a specific term (like #ZA).  This will help you get notified faster in the event of an attack.

I created a short 4:50 screencast below that shows you how to configure search notifications in TweetDeck.

There are other Twitter clients that can also help you quickly do Twitter searches, and even some mobile options that work well on the iPhone, like Echofon that might be worth checking out.

Twitter will save your life in a Zombie uprising

In the event of a Zombie Apocalypse Twitter can help save your life…isn’t that reason enough to start using Twitter?