This is the collection of keyword pairs that appeared in two clusters of people who Tweeted about “Paul Ryan”, the Republican Congressman from Wisconsin who delivered the GOP rebuttal to the 2011 United States State of the Union Address.  This network illustrates the ways that certain word pairs appears only or predominantly in one cluster (colored here Red and Blue) or the other. Terms that appeared in both clusters appear as purple.

Social networks are built from relationships between people.  Keyword networks are built from relationships between words and other text strings.  When two words appear in the same message, sentence, or alongside one another ties of different strengths are created.  The networks that result can illuminate the relationships among topics of importance in a collection of messages.

Markus Strohmaier from the Technical University Graz (TUG) along with Claudia Wagner gave us inspiration in a paper:

C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)

in which they defined a range of ways two words (technically these are strings, they may not really be words) can be associated with one another.  Words could be linked if they are in the same tweet, next to one another, or sequential among other ways to link terms.

NodeXL has not had any features for exploring the networks in texts.  Now with the addition of a new macro from Scott Golder, it is fairly simple to extract pairs of keywords from collection of tweets.  NodeXL’s Twitter importer can optionally include the content of the tweet that included the search term and this column of text can now be processed itself into a new network based on the ways words appear together in tweets.

This feature builds on the work of several people.  Scott Golder from Cornell started the ball rolling with a simple but effective VBA script that allowed others to build and refine the models of what counts as a tie between two words.  Vladimir Barash added several refinements including support for stop word lists to remove common terms.  Scott then picked up the code again and added a set of features for selecting the nature of the graph and making it easier to select the options needed.

The code for the Keyword Network macro is below.

The instructions to use it take a few steps to complete:

1. Create a new workbook, eg a list of tweets or an import from a Twitter search, whatever. Save it as .xlsm. The m is important. This can be an existing NodeXL workbook.

2. Go to Developer -> Macros. Make up a name; it doesn’t matter because it’ll get overwritten. Then press Create. the VBA window will open.

3. In the big text are that says “Sub whatever() End Sub”, select all that text and delete it. Paste in the contents of the text file below.

4. Go to Tools->Reference. Check the checkboxes for “Microsoft Scripting Runtime” and “Microsott VBScript Regular Expressions”. Press OK. Save the file (File->Save) then exit (“Close and return to Microsoft Excel”).

5. Now go to Developer -> Macros. Choose CreateWordNet and press the Run button.

6. It’ll ask you for a worksheet name, a column and a start-row. Then it’ll create a new worksheet with the edgelist in it.

The edge list is not directed (there isn’t really a concept of direction in “co-occurs”) but is weighted. Each pair is weighted by the number of times it appears.

This version also includes options for edge creation.

First, it is now possible to suppress edges of weight=1, which is helpful in getting rid of a lot of garbage.

Second, it is now possible to defined edges by adjacency or co-tweeting. Given a tweet of words “w1 w2 w3″ adjacency will give edges w1-w2 and w2-w3, while co-tweeting will give edges w1-w2, w1-w3, w2-w3.

For edges defined by adjacency, you may choose directed or undirected edges. So a tweet of “Marc Smith Marc” (for example) would generate the weighted directed edges Marc,Smith,1 and Smith,Marc,1 while the sole undirected edge would be Marc,Smith,2. That is, for undirected edges (where ordering doesn’t matter) the words are alphabetized.

Create Word Network VB Macro Vdb5

This map of connections among people who tweeted Teaparty starts on 11/15/2011 14:22 UTC and ends on 11/15/2011 17:23, a total of 3 hours and 1 minute of traffic.

The Teaparty data set contained 1,533 tweets, replies and mentions.
Blue edges are connections created by replies and mentions. Grey lines are follows relationships.

Top most between users:
@ronpaul
@michellemalkin
@christopherhull
@theteaparty_net
@capaction
@thedailyedge
@bill1phd
@dbargen
@gulagbound
@rightcandidates

Graph Metric: Value
Graph Type: Directed
Vertices: 659
Unique Edges: 8808
Edges With Duplicates: 1423
Total Edges: 10231
Self-Loops: 1084
Connected Components: 49
Single-Vertex Connected Components: 44
Maximum Vertices in a Connected Component: 606
Maximum Edges in a Connected Component: 10148
Maximum Geodesic Distance (Diameter): 6
Average Geodesic Distance:2.693965
Graph Density: 0.02036797
NodeXL Version: 1.0.1.193

The major clusters are composed of teaparty supporters. The center bottom cluster are teaparty critics.

This map of the connections among people who tweeted Occupywallstreet starts on 11/15/2011 23:08 and ends on 11/15/2011 23:34 UTC, a total of 26 minutes of traffic.

Occupywallstreet 1,370 tweets, replies and mentions
Blue edges are connections created by replies and mentions. Grey lines are follows relationships.

Top most between users:
@occupywallst
@mmflint
@nyclu
@allisonkilkenny
@andrewbreitbart
@operationleaks
@occupydenver
@theatlantic
@usgeneralstrike
@rt_com

Graph Metric: Value
Graph Type: Directed
Vertices: 1000
Unique Edges: 3546
Edges With Duplicates: 826
Total Edges: 4372
Self-Loops: 794
Connected Components: 241
Single-Vertex Connected Components: 230
Maximum Vertices in a Connected Component: 747
Maximum Edges in a Connected Component: 3998
Maximum Geodesic Distance (Diameter): 7
Average Geodesic Distance: 2.65438
Graph Density: 0.003246246
NodeXL Version: 1.0.1.194

Some notable contrasts:
Teaparty Graph Density: 0.002652645
Occupywallstreet Graph Density: 0.02036797 – significantly lower levels of interconnection
Teaparty: Single-Vertex Connected Components 44 of 1000
Occupywallstreet: Single-Vertex Connected Components 283 of 1000

Many more “isolates” (Single-Vertex Connected Components) in Occupywallstreet.
Many more hubs, and more retweeting activity in Occupywallstreet.

The difference in duration of these data sets illustrates the relative speed of content creation in the topics. The data sets are commensurable in that they are both the result of a single query against the Twitter search API. So both maps are the results of charting connections among the authors of the last 1500 tweets, how ever long that takes to create.

The Workshop on Information in Networks 2011 was held at NYU September 30-October 1st.

Over the course of the event a number of people tweeted using the hashtag #win2011.

These are the connections among the Twitter users who recently tweeted the word win2011 when queried on October 3, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

See: winworkshop.net/

Layout using the “Group Layout” composed of tiled bounded regions. Clusters calculated by the Clauset-Newman-Moore algorithm are also encoded by color.

A larger version of the image is here:http://www.flickr.com/photos/marc_smith/6209064926/sizes/l/in/photostream/

Betweenness Centrality is defined here: en.wikipedia.org/wiki/Centrality#Betweenness_centrality

Clauset-Newman-Moore algorithm is defined here: pre.aps.org/abstract/PRE/v70/i6/e066111

Top most between users:

@sinanaral
@winworkshop
@barrywellman
@ladamic
@seanjtaylor
@drewconway
@chrisdiehl
@dylanwalker
@marc_smith
@ariegoldshlager

Graph Metric: Value
Graph Type: Directed
Vertices: 38
Unique Edges: 227
Edges With Duplicates: 193
Total Edges: 420
Self-Loops: 68
Connected Components: 1
Single-Vertex Connected Components: 0
Maximum Vertices in a Connected Component: 38
Maximum Edges in a Connected Component: 420
Maximum Geodesic Distance (Diameter): 3
Average Geodesic Distance: 1.836565
Graph Density: 0.194167852
NodeXL Version: 1.0.1.179

These are the connections among the Twitter users who tweeted the word win2011 when queried a few days earlier on September 30, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

A larger version of the image is here: www.flickr.com/photos/marc_smith/6198785387/sizes/l/in/ph…

Top most between users:
@sinanaral
@winworkshop
@ariegoldshlager
@drewconway
@chrisdiehl
@kateaanderson
@dylanwalker
@barrywellman
@kristinalerman
@pboothe

Graph Metric: Value
Graph Type: Directed
Vertices: 16
Unique Edges: 60
Edges With Duplicates: 45
Total Edges: 105
Self-Loops: 26
Connected Components: 1
Single-Vertex Connected Components: 0
Maximum Vertices in a Connected Component: 16
Maximum Edges in a Connected Component: 105
Maximum Geodesic Distance (Diameter): 3
Average Geodesic Distance: 1.617188
Graph Density: 0.270833333
NodeXL Version: 1.0.1.179

More NodeXL network visualizations are here: www.flickr.com/photos/marc_smith/sets/72157622437066929/
NodeXL is free and open and available from www.codeplex.com/nodexl
NodeXL is developed by the Social Media Research Foundation (www.smrfoundation.org) – which is dedicated to open tools, open data, and open scholarship.

http://www.ischools.org/iConference11/2011index/

Our paper, Finding Social Roles in Wikipedia,  about the variety of  roles people perform in Wikis received the best paper award (along with 4 others) in a field of 86 papers.   The 2011 iConference accepted 86 papers, and had about 550 attendees.

The paper is authored by: Howard T. Welser at Ohio University, Austin Lin at Cornell University and Microsoft, Dan Cosley, Fedor Dokshin, Gueorgi Kossinets and Geri Gay at  Cornell University, and Marc Smith from Connected Action.

The paper pdf pre-print is available here: http://oak.cats.ohiou.edu/~welser/Welser.Cosley.plus.Wiki.Roles.pdf

The link to the ACM abstract and pdf: http://portal.acm.org/citation.cfm?id=1940778&CFID=9933318&CFTOKEN=58981138

Abstract: This paper investigates some of the social roles people play in the online community of Wikipedia. We start from qualitative comments posted on community oriented pages, wiki project memberships, and user talk pages in order to identify a sample of editors who represent four key roles: substantive experts, technical editors, vandal fighters, and social networkers. Patterns in edit histories and egocentric network visualizations suggest potential “structural signatures” that could be used as quantitative indicators of role adoption. Using simple metrics based on edit histories we compare two samples of Wikipedians: a collection of long term dedicated editors, and a cohort of editors from a one month window of new arrivals. According to these metrics, we find that the proportions of editor types in the new cohort are similar those observed in the sample of dedicated contributors. The number of new editors playing helpful roles in a single month’s cohort nearly equal the number found in the dedicated sample. This suggests that informal socialization has the potential provide sufficient role related labor despite growth and change in Wikipedia. These results are preliminary, and we describe several ways that the method can be improved, including the expansion and refinement of role signatures and identification of other important social roles.

On 11 March 2011, I will be speaking at:

Organizational and Social Media Network Analysis:
Next Practices in Organizational Networks, Analytics and Media

Along with Valdis Krebs, Chief Scientist, http://thenetworkthinkers.com/

Valdis will be speaking about Organizational Network Analysis - Introduction to Enterprise Networks, followed by a InFlow 3.1 Workshop: Social Network Analysis with InFlow 3.1

I will be speaking about Social Media Network Analysis with NodeXL.

Preservation Park
1233 Preservation Park Way
Oakland, CA 94612
(510) 874-7580
(click for map)

Tell a Friend
Add to Calendar

With the VOSON Data Provider Plugin for NodeXL, you can now access VOSON hyperlink network construction services from within NodeXL, an Excel 2007/2010 template for analyzing social media network data. So there are two ways you can use VOSON for analyzing hyperlink networks: (1) via the VOSON System (which you login to using a web browser); (2) via VOSON+NodeXL. The VOSON System is a special tool for hyperlink network construction and analysis. NodeXL provides access to various types of social media network data sources, e.g. Facebook, Twitter and, now, WWW hyperlink networks via VOSON.

Developers interested in creating their own data import “spigots” for NodeXL may be interested in this post on the NodeXL Codeplex discussion board: For Programmers: About NodeXL Plug-Ins.

To get access to the VOSON+NodeXL plugin and documentation you first need to register for a VOSON user account.

The Annual Meeting of the Israel Internet Association (http://www.isoc.org.il (English)) was held February 22-23 2010. I spoke at this year’s meeting: http://www.isoc.org.il/conf2010/agenda.php?lang=en

Part 1

Part 2

The previous year’s conference website is at: http://www.isoc.org.il/conf2009/program.php

The Israel Internet Association is the official Israeli Chapter of the Internet Society.  Their annual meeting is a central event of academics (sociologists, psychologists, business and law) as well as industry participants from sectors including mobile cellular companies and internet service suppliers.

Along with Professors Ben Shneiderman (Computer Science/Human Computer Interaction Lab) and Derek Hansen (College of Information Studies) from the University of Maryland I am writing and editing a book about analyzing the social media networks that form whenever people link or reply to one another, favorite, rate, read, or edit data about other people or their objects.  Social media networks can be analyzed using the methods of social network analysis, the mathematical application of graph and network theory to the social sciences.  Using social network analysis collections of connections can be analyzed and compared to identify key people and groups and measure changes over time and following interventions.

I am pleased to announce that we have signed with Elsevier/ Morgan Kaufmann to produce a book: Analyzing Social Media Networks with NodeXL: Insights from a Connected World for a Summer 2010 delivery!

A map of the relationships among the population of people who all tweet a particular keyword can lead to the discovery of the key hubs and influential people in the network.  A social network analysis of reply patterns in email collections displays clusters around projects and highlights key people and relationships.  Visualizing the connections among your friends in Facebook can reveal the various life stages and communities in which you have participated.  When you chart the links between videos and users in YouTube content with interesting network properties is exposed based on well connected content creators and influential commentators.  A graph of  the individual connections between flickr users illustrates the emergent formation of groups around social networks, locations, and topics.

These kinds of social media network data collection, scrubbing, analysis, and display tasks have historically required a remarkable collection of tools and skills.  A great example of the variety of tools that can be used in concert to extract, analyze and display social media networks can be found on Drew Conway’s blog.  This is a powerful set of tools for those who can master the demands of python and API interfaces.  In contrast, the approach the NodeXL project has taken is to provide an end-user GUI application environment built within the framework of Excel 2007 for performing basic social media network analysis and visualization for non-programmers.  The python path is certainly the high road for experts and those with demanding volumes or esoteric data requirements.  But for the non-coding user, NodeXL may be one of the easiest ways to both manipulate network graphs and get graphs from a variety of social media sources.

There are already some materials available to guide new users interested in learning about NodeXL, social networks, and social media.  A video tutorial for NodeXL demonstrates the extraction of the network of people in twitter who mentioned the term “digg”.  A tutorial guide to NodeXL offers a step by step guide to features in the NodeXL toolkit (with supporting data sets).  But the book will capture the theory, history, domain and process of social media network analysis in a single volume.

The volume contains a broad introduction to social media, social networks and the operation of the NodeXL application and then features a series of  chapters from leading researchers that focus on a particular social media system (email, Facebook, Twitter, YouTube, flickr, Wikis, the WWW hyperlink network) and the networks each contains (replies, friends, follows, subscribes, comments, favorites, edits, links, etc).   A final chapter outlines a programmer’s view of the NodeXL code, in contrast to the code-free approach of the remainder of the book.

Our intended audience is the mostly non-programming population that is interested in social media and the techniques of social network analysis.  The volume is largely in the form of a how-to guide that readers can follow and replicate all examples.  Using your own free and open copy of NodeXL, you will be able to use sample data sets or create similar live queries that map relationships in social media systems.

We have an ambitious production schedule so the book may be on a book store shelf or online retailer search result in summer 2010.

Table of contents…

The deck illustrates the use of NodeXL to extract several social media networks from systems like twitter and facebook to generate maps of communities and identify people and objects in key locations.

Tweet the Debates: Understanding Community Annotation of Uncollected Sources

Authors: Shamma, D.A.; Kennedy, L.; Churchill, E.F.
Source: ACM Multimedia, ACM, Beijing, China (2009)
URL: http://www.acmmm09.org/
Keywords: Twitter; Debates; TV; community; multimedia; social; centrality; network

Download: wsm01a-shamma.pdf

At the core of the network are the entities that have the most “eigenvector centrality” which turn out to be the major participants in the debate.

These projects point towards a world in which computers and mobile devices do more than connect us to the network, they will sense the world around us and reason about both our location and who is with us.  Combined with back-end data mining, new mobile sensor studded devices are coming that will alter the nature of social interaction in its last, most analog hold out: face-to-face, co-present interaction.

I want to explore this change in the nature of what the sociologist Erving Goffman referred to as the “interaction order“.  We are living through the early stages of the “electrification of the interaction order”, a time in which the ways we interact with one another is changed dramatically by the availability of mobile social information networks.  Online social networking, content sharing and discussion systems have effects that are multiplied when channeled through a device carried by every person and active in every interaction, however fleeting.  

Imagine going to a business meeting or conference and having Facebook suggest that you link to the people you spent the longest time talking to.  Mobile social computing will add more content to the torrent already generated by “desktop” experiences.  Some projects are already digging into this area: good examples include companies and products like nTag.com, SpotMe.Com, and the many trail and path tracking applications now appearing in the iPhone AppStore.  Scott Counts and I wrote about a location based social networking application that demonstrated many of these features as well as search and matching features that have yet to appear in the first wave of production systems.

A first step in this direction is to focus more on the analytic back-ends that will be needed for the management of all forms of social media repositories.  Community analysis servers that provide a dashboard of community health and activity indicators will be a critical differentiating feature for community hosts, managers, and leading participants.  Successful communities will be those that can cultivate contribution the best while managing conflict at the lowest cost.  Once desktop bound social encounters are channeled through an analytics console more real-world events sensed by mobile devices can be added to the mix.  

I am looking forward to some time to push back and reflect more about these changes while looking around for new ways to explore them.  I will take some time to get my family settled into our new home in California.  I hope to catch up with many people!  I will also be visiting Yale, University of Maryland and Berkely for talks this fall.  I plan to attend the Microsoft Research Social Computing Symposium in Redmond (it will be good to be back!) and the Conference on Information and Knowledge Management (CIKM) in Sonoma.

My old masmith@microsoft.com email address is no longer active, so please contact me at marc.smith.email at gmail.com.  

I look forward to staying in touch with my many friends and colleagues at Microsoft while finding the time now to meet with a wide range of people interested in social media.