Jens Ode

Alice in Twitterland
- Twitter Seen Through The Lens Of Alice
Two weeks ago a friend and I discussed crowdsourcing (harnessing the power of the general public) online. During breakfast the following day, while reading an article in the morning paper Sydsvenskan, something hit me:

The public is capable of so much more than it actually knows. For example, doesn’t the online community reproduce (albeit randomly) the works of great authors every day without even knowing it? How long would it actually take for, let’s say Tweeters around the world, to (re)write a famous novel?

I emailed my friend asking: ”Wouldn’t it be interesting to link (every word of) a book to excerpts of people’s discussions and watch the book evolve ”live” on a web site?  We quickly agreed to go with the idea. The world wide release of Tim Burton’s remake of the movie Alice in Wonderland created the perfect back drop for the experiment. We decided that we would let the Twitter community write Lewis Carroll’s old master piece.

Step By Step

We got the text to Alice in Wonderland from Project Guthenberg and split it up into single words. The original idea was to move through the text word by word and search the global Twitter stream to find messages (tweets) that contained the words of the book. Through the course of the project our approach had to be tweaked a few times which created some valuable lessons.

Lesson 1: Time Is Relative

At first the idea was to move sequentially through the information stream, that is, every tweet had to be time stamped later than the previous one. The purpose was to make sure that the text was as close to real time as possible. The way to do this was simply to check the time stamp of a message and compare it to the one previously used. A later time stamp would indicate a more up to date message. Well, that is unfortunately not true. If two messages are submitted simultaneously, one in Sweden and one in New York, there will be a time stamp difference of 6 hours since the time indicated is local rather than from a global time line.

Lesson 2: Twitter API Is Limited

Twitter has applied a limit on how often you may call it. In the original approach we wanted to move through the text word by word, and thus needed to query each word one by one. However, Twitter only allows 150 calls per hour if you’re not whitelisted (which requires going through a manual application process which may take weeks). When making a search through the API you can get away with a higher amount of calls (Twitter won’t say how many), but due to this limitations we had to scrap the idea. Instead of moving word by word it had to be sentence by sentence to make due with fewer searches and instead cache the result and replay it through the web site.

Lesson 3: User Generated Content = Body Count

As soon as you try to harness user generated content, you run the risk of getting explicit content (and we’re not just talking the occasional mentioning of boobs…). This project is no exception. Already in the first test run of only ten messages there was content from the ”red light district”. Still it was decided to forge ahead without any filters, since the experiment is a snapshot in time, and these kind of messages are the reality of the online world today. However, if you plan to run a site with user generated comments be prepared to monitor it, because you will end up with inappropriate content.

Lesson 4: Leave An Opening

One of the features of this project is that there are words that probably aren’t mentioned too often (like the names of some of the characters). If we stepped through the text and only displayed the result world for word, no one would be able to foresee upcoming words and (hopefully) include them in tweets to move the process forward. To remedy this we display one page of the book at the time and show the progress by highlighting every tweeted word. This is not in line with the original idea, we believe it will make it more visually interesting.

Final Thoughts

This experiment is meant to capture a moment in time. The Alice in Wonderland-text will only be run once and when we reach the end the ”gathering of words” is over. We then have a static version of Alice in Wonderland expressed through thousands of twitter messages posted globally during the time of the experiment. It will be a snapshot of dialogues, expressions, ideas and mumbo jumbo seen through the lens of Alice.

Hopefully the experiment works, otherwise failure is always an option. Visit http://www.aliceintwitterland.com to follow (and perhaps participate in) the process.

Press release (swedish):
http://www.mynewsdesk.com/se/pressroom/wannaplay/pressrelease/view/twitter-anvandare-valrden-over-skriver-om-alice-i-underlandet-klart-paa-24-timmar-379945