The long awaited (and delayed) change to the Twitter API is now here: API 1.1 is now the only service available, the long used API 1.0 is gone.

This has an impact on people who have been collecting and analyzing data from Twitter.  Twitter has given and taken away with the new 1.1 API.  Mostly taken away.  More Tweets are sometimes available from the new API, up to 18,000 rather than the old 1,500 tweet limit.  This is a big change, but normal users often do not get much benefit from the limit increase if the topic they are interested in has fewer tweets.  The length of time tweets are retained and served is not much longer than it was.

The big change is the effective loss of the “Follows” edge.  Some users of the 1.0 API used to be able to get a significant number of queries that asked about who each user followed.  These queries generated data that allowed a network to be created based on which users followed which other users.  The “Follows” network in Twitter has been very informative, pointing to the key people and groups in social media discussions.  But now the “Follows” edge will be effectively impossible to use.

Twitter API 1.1 changes the limit on the number of queries about who follow who in Twitter to 60 per hour.  In practice, a network may have several hundred or thousand people in it, making a query for each person’s network of followers impractical. With the follows edge effectively gone, the remaining edges, “reply” and “mention” become more important.  These edges are far less common than the “Follows’ edge.  Many people follow lots of other people but mention the name or directly reply to very few. With the loss of the Followers edge, Twitter networks can become very sparse, with few connections remaining.  Dense structures give way to confetti.

Here is a map of the topic #scaladays with the Followers edges compared to the same map with no Follower edges:

With the “Follows” edges gone, the loss of insight into the nature of the network is profound, but not fatal. The reply and mention network does have some density in many discussions, allowing many kinds of network positions and structures to be observed. Edges can also be synthesized from other evidence, for example a link could be created when two people use words in common that are not commonly used by others.

The NodeXL project has released a version that connects to the new Twitter API 1.1 and we will be releasing additional edge types that will link people when they share content like hashtags, URLs, words and word pairs with other people.  These shared content edges are based on a presumption that when people use similar content that is rarely used by others they are likely to have an underlying connection.  The assumption that shared content use is a surrogate for the “follows” relationship requires additional testing (which will be difficult with out access to the data that Twitter just removed). For now, these connections do return density to networks that have been shattered by the loss of the visibility of the Follows connection and can indicate common interests among Twitter users.