At the Forge

I'm a very quick adopter of many new software technologies. I try new programming languages, browsers, databases and frameworks without hesitation. But when it comes to social networks, I'm a bit of a Luddite, waiting to see what all the fuss is about before making them a part of my life. Sure, I signed up for Facebook almost as soon as it was available, but I haven't really posted much there. I do use LinkedIn, mostly to collect and find contacts, but I don't post there very often either, unless I'm announcing a presentation that I've added to SlideShare.

Twitter is something of a different story. There are people, it seems, for whom Twitter is the ultimate in communication. I've been on Twitter for some time, but other than an occasional foray into that world, I didn't really pay it much attention. Even now, after having decided several months ago that I should try to get into Twitter more heavily, I find that while I look through my feed several times a day, I tweet only once every few weeks. Call me a dinosaur, but I still prefer to use e-mail to be in touch with friends and family, rather than 140-character messages.

Although I don't see Twitter as a great medium for interpersonal communication, I recently have begun to appreciate it for other reasons. Specifically, I have discovered (perhaps long after the rest of the world has done so) that using Twitter as a sort of public logfile can make a Web application more visible, updating the rest of the world as to the status of your work and your on-line community. Doing so not only lets people hear about what you are doing—and potentially rebroadcast it to the world, by “retweeting” your message to followers—but it also increases your application's SEO, or visibility on various search engines. Finally, you can use Twitter to bring attention to your on-line presence by following other people. (The idea is that when they receive your follow request, they may try to find out more about you, exploring your site or even following you back.)

I might sound like a social-media consultant, but I've seen the difference that Twitter can make in an application. I recently connected my PhD dissertation project (the Modeling Commons, at modelingcommons.org) to Twitter, such that each public action is sent to the Twitter feed. The combination of tweeting updates and following other people has had a remarkable and direct effect on the number of visitors who come to my site, the length of time they remain and the number of pages they view. Now, I'm not talking about millions of visitors per month. My application is still of interest mainly to a small community of people working with the NetLogo modeling environment. But the change has been obvious, and I grudgingly admit that I owe some of it to Twitter.

In this article, I explore some of the things I did to use Twitter in my application. From a technology perspective, you'll see that the implementation was fairly straightforward. But I think that what I've learned can be of interest to anyone running a Web application, particularly one that is trying to get the word out to the public. In addition, although there are plenty of good reasons to question Twitter's business practices and its relationship with developers, there is no doubt that its attention to detail with its API offers a model for all of us who want to provide APIs to our applications.

I'm going to assume that anyone reading this article already has created a Twitter account or is able to figure out how to do so at Twitter.com. And of course, via the Twitter.com Web site, you can do all the things that you might expect, such as tweeting, retweeting, following and searching.

Twitter's API allows you to do all of these things via code. That is, you don't need to go and compose tweets personally. You can write a program that will do so for you. In order for this to happen, you need to do two things: register with Twitter's API service and install a library that knows how to communicate with the Twitter API.

In order to register with the Twitter API, you need to go to the “developer” site at dev.twitter.com. Note that you need to sign in with your Twitter user name and password, even if you already are signed in to the main Twitter site. The two sites do not seem to share login sessions.

Once you're on the developer site, you need to create a new application. The application name needs to be unique, but don't worry about it too much. You need to provide not only a name, but also a description and a URL that is associated with the application. Agree to the terms, fill in the Captcha, and you'll be on your way. Note that many types of Twitter applications exist, with many applications (including mobile) that post to Twitter on behalf of a user. The model I demonstrate in this article is of an application sending updates to Twitter, which means you won't have such issues—you don't need a callback URL or any special login configuration.

Perhaps the most confusing thing (to me, at least) about setting things up with Twitter was that the default permissions for an application allows you to retrieve tweets, but not post to them. To allow your application read-write access, go to the settings tab and indicate that you want the read-write access, or even read, write and direct message. You won't be using all of these capabilities for this example, but without write permission, your application will not be able to post to Twitter.

And now for the most important part, the keys: Twitter's authentication model requires two tokens. First, there is your access token, which allows you to access Twitter via the API. The second is the “consumer key”, which describes your particular application and usage. Each of these keys has an accompanying secret, which you should treat as a password. As such, putting these secrets directly in your application probably is a bad idea. You would be better off putting them in environment variables, thus avoiding having the secrets in version control.

Readers of this column know that I love the Ruby language, so it won't come as a surprise to hear that I intend to use Ruby for my examples. However, there are Twitter API clients in virtually every modern language, making it easy to access from whatever you prefer to use in your programming.

The twitter Ruby gem, as is the case for all Ruby gems (libraries), is available for installation via the gem program, which comes with modern versions of Ruby. The gem currently is maintained by Erik Michaels-Ober, also known as “sferik” on GitHub. You can type:

gem install twitter -V

and the gem should be installed. On many systems, including those not running a Ruby version manager like rvm, you need to execute the above line while logged in as root.

Once you have installed the gem, you can use it. There are three parts to this process: bringing the gem into the program, configuring it to use your keys and secrets, and then executing a Twitter command. The first is handled with the Ruby require command, which looks at installed gems, as well as the Ruby core and standard libraries.

Configuration of the client is handled fairly straightforwardly from within a block that looks like this (filling in the values you got from Twitter's API documentation):

twitter_client = Twitter::REST::Client.new do |config|
  config.consumer_key = CONSUMER_KEY
  config.consumer_secret = CONSUMER_SECRET
  config.oauth_token = OAUTH_TOKEN
  config.oauth_token_secret = OAUTH_SECRET
end

Notice that you are not merely executing the “new” method on Twitter::REST::Client, but that you also are returning a value. Thus, in contrast to previous versions of Ruby's Twitter gem, you should accept the returned object, which is then the basis for all of the additional actions you wish to take.

Finally, you send the tweet with the “update” method:

tweet = twitter_client.update("Hello, world. Tweet tweet.")

Invoking the #update method has the effect of sending the message to Twitter. If you go to the Web page for your Twitter user, you'll find that a new message has been sent, as if you had typed it.

If you capture the return value from the invocation of twitter_client.update, you'll see that it is an instance of Twitter::Tweet, a Ruby object that represents a tweet. This object provides the functionality that you would want and expect from something associated from Twitter. For example:

tweet.user          # tells us who wrote the tweet
tweet.retweeted?    # indicates whether it was retweeted
tweet.favorited?    # indicates whether it was marked as a favorite

Now, it's also possible that you will not get a tweet object back at all, but rather that the “update” method will raise an exception. For example, Twitter forbids users from sending an identical tweet, at least within a short period of time. Thus, if you send the above “Hello, world” tweet (from the example above) a second time, you'll get an exception:

Twitter::Error::Forbidden: Status is a duplicate.

Of course, you can catch such errors with:

begin
  tweet = twitter_client.update("Hello again, 
  ↪@reuvenmlerner  Tweet tweet.")
rescue Twitter::Error::Forbidden => e
  puts "You already tweeted that."
rescue => e
  puts e.class    # Twitter::Error::Forbidden
  puts e.message  # 'Status is a duplicate.'
end

If you include a Twitter @username, hashtag or URL in your tweet, the appropriate magic will happen automatically. Thus:

tweet = twitter_client.update("Go to @reuvenmlerner's 
 ↪site at http://lerner.co.il/")

In the above tweet, the URL automatically will be shortened, using Twitter's standard t.co domain. Similarly, the @reuvenmlerner (my Twitter handle) will turn into a link. You can access both of these using methods on your tweet:

tweet.urls           # returns an array of Twitter::Entity::URI
tweet.user_mentions  # returns an array of 
                     # Twitter::Entity::UserMention

You can more generally ask Twitter for information about tweets. For example, you can get the most recent tweets a user has sent with:

twitter_client.user_timeline("reuvenmlerner")

which returns an array of tweet objects. You can apply the “text” method to the first element, thus getting the text back from the user's most recent tweet:

twitter_client.user_timeline("reuvenmlerner")[0].text

If there are URLs embedded in the tweet, you can get those back:

twitter_client.user_timeline("reuvenmlerner")[1].urls

This method returns an array of Twitter::Entity::URI objects, each of which has attributes, such as “url” and “expanded URL”.

As you can see, working with Twitter is surprisingly easy. The startup time for connecting to Twitter can take a little bit of time—up to two seconds, in my experience—but tweeting and querying Twitter take very little time. It's obvious, as a consumer of the API, that they have worked hard to make it execute as quickly as possible. This is a lesson to all of us who create APIs. We all know that Web pages should load quickly, and that slow load times can discourage people from staying on a site.

API calls typically are embedded within another application, meaning that if the API call takes time, the application itself will feel sluggish. As a result, a slow API call will lead to slow responses from the API clients—and may discourage people from using your API.

But where would you use such API calls? Why would you want to use Twitter on your site?

One simple use of the Twitter API would be to display a user's most recent tweets. For example, if your company (or you personally) use Twitter to send messages about what you are doing, you can see that it would be fairly easy to include those tweets in a Web page. Using an MVC system, such as Rails, you simply would grab the tweets (with the “user_timeline” method, as shown above), and stick the results on your home page. Now your home page provides another view to your Twitter feed, re-enforcing its importance and usage to your company.

I have been doing something slightly different. As I mentioned previously, I have begun to use Twitter to log public activity in the application I've developed for my dissertation. Every time a new user joins, new content is posted or someone adds a posting to a discussion forum, I send a new tweet on the subject. In and of itself, this doesn't do very much; Twitter is full of text and URLs. But I have certainly found by ensuring that my tweets are followed and seen by a large number of people, I have increased the number of users coming to my site.

In other words, by tweeting about activity on my site, I have given my site additional exposure to the world. Moreover, people who really want to see what my application is doing can follow the link in their Twitter feed and follow along.

By adding a #NetLogo hashtag to my tweets, I also have made it possible, and even easy, for my tweets (and thus my site) to be found and identified by people searching Twitter for mentions of our modeling environment. The fact that Google indexes tweets increases my site's visibility on-line among people who are searching for modeling-related sites.

The net effect has been rather huge. Within two weeks of starting to use Twitter to announce updates on my site, the number of people coming to visit has increased dramatically. Not coincidentally, my site's ranking in Google has improved noticeably.

Now, if this were a commercial site, rather than a free infrastructure for collaborative modeling, I would want to check a second thing, namely the “conversion rate”—that is, how many people who came to my site also became paying customers. But for my small, educational site, it has been fascinating to see what a difference tweeting made.

And what did I do? Truth be told, not much. I set up things such that a new tweet would be sent, using the “update” method demonstrated above, every time a new model version, forum posting or person was added to the system. Because of the relatively low latency on the “update” method, I even do this inline on an after_create callback within Rails, rather than queueing it in a background job.

The biggest technical challenge I have faced so far in all of this is the issue of duplicate tweets. When I first set up the Twitter feed, I defined the tweet for an additional discussion forum post to be:

Reuven Lerner has added a comment about the Foobar model!

The problem with this style of tweet is that it quickly can lead to duplicates—and thus errors from within the application. As a result, I have made sure that every tweet has a unique number in it somewhere, typically counting how many similar objects already have been created. For example:

Reuven Lerner wrote the 5th comment about the Foobar model!

The above ensures—assuming that user and model names are unique—that there cannot be duplicates, thus avoiding the problem.

Beyond the advantages for users, SEO and people interested in following my work, I also have found it to be enormously satisfying to see tweets come out even when I'm not aware of it. It's similar in some ways to seeing my children's creative output, but (obviously) less emotionally charged.

Adding automatic tweets to a Web application is easy to do and can have significant benefits. For your users, it gives them a way to follow what is happening in your application without needing to visit the site or use an RSS reader. For your site, automatic tweets will help bring in new visitors, improve SEO and generally improve your project's visibility.