This is the collection of keyword pairs that appeared in two clusters of people who Tweeted about “Paul Ryan”, the Republican Congressman from Wisconsin who delivered the GOP rebuttal to the 2011 United States State of the Union Address.  This network illustrates the ways that certain word pairs appears only or predominantly in one cluster (colored here Red and Blue) or the other. Terms that appeared in both clusters appear as purple.

Social networks are built from relationships between people.  Keyword networks are built from relationships between words and other text strings.  When two words appear in the same message, sentence, or alongside one another ties of different strengths are created.  The networks that result can illuminate the relationships among topics of importance in a collection of messages.

Markus Strohmaier from the Technical University Graz (TUG) along with Claudia Wagner gave us inspiration in a paper:

C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)

in which they defined a range of ways two words (technically these are strings, they may not really be words) can be associated with one another.  Words could be linked if they are in the same tweet, next to one another, or sequential among other ways to link terms.

NodeXL has not had any features for exploring the networks in texts.  Now with the addition of a new macro from Scott Golder, it is fairly simple to extract pairs of keywords from collection of tweets.  NodeXL’s Twitter importer can optionally include the content of the tweet that included the search term and this column of text can now be processed itself into a new network based on the ways words appear together in tweets.

This feature builds on the work of several people.  Scott Golder from Cornell started the ball rolling with a simple but effective VBA script that allowed others to build and refine the models of what counts as a tie between two words.  Vladimir Barash added several refinements including support for stop word lists to remove common terms.  Scott then picked up the code again and added a set of features for selecting the nature of the graph and making it easier to select the options needed.

The code for the Keyword Network macro is below.

The instructions to use it take a few steps to complete:

1. Create a new workbook, eg a list of tweets or an import from a Twitter search, whatever. Save it as .xlsm. The m is important. This can be an existing NodeXL workbook.

2. Go to Developer -> Macros. Make up a name; it doesn’t matter because it’ll get overwritten. Then press Create. the VBA window will open.

3. In the big text are that says “Sub whatever() End Sub”, select all that text and delete it. Paste in the contents of the text file below.

4. Go to Tools->Reference. Check the checkboxes for “Microsoft Scripting Runtime” and “Microsott VBScript Regular Expressions”. Press OK. Save the file (File->Save) then exit (“Close and return to Microsoft Excel”).

5. Now go to Developer -> Macros. Choose CreateWordNet and press the Run button.

6. It’ll ask you for a worksheet name, a column and a start-row. Then it’ll create a new worksheet with the edgelist in it.

The edge list is not directed (there isn’t really a concept of direction in “co-occurs”) but is weighted. Each pair is weighted by the number of times it appears.

This version also includes options for edge creation.

First, it is now possible to suppress edges of weight=1, which is helpful in getting rid of a lot of garbage.

Second, it is now possible to defined edges by adjacency or co-tweeting. Given a tweet of words “w1 w2 w3″ adjacency will give edges w1-w2 and w2-w3, while co-tweeting will give edges w1-w2, w1-w3, w2-w3.

For edges defined by adjacency, you may choose directed or undirected edges. So a tweet of “Marc Smith Marc” (for example) would generate the weighted directed edges Marc,Smith,1 and Smith,Marc,1 while the sole undirected edge would be Marc,Smith,2. That is, for undirected edges (where ordering doesn’t matter) the words are alphabetized.

Create Word Network VB Macro Vdb5

I will present a talk at Personal Digital Archiving 2012 titled “Arc-chiving: saving social links for study“.

The conference will be held on Thursday-Friday, February 23-24, 2012 at the Internet Archive in San Francisco.

News and updates on the conference will be posted at the conference web site, http://personalarchiving.com.

My talk this year will focus on collecting and analyzing connections between digital objects (like users) and the insights these tools make possible.

Abstract: While digital content is archived in various ways, the “arcs” or links among people and their digital objects are not systematically saved. Efforts to store social media often overlooks including data about collections of connections. The Social Media Research Foundation is dedicated to open tools, open data, and open scholarship related to social media. It is producing tools that can collect, analyze and upload social media data, including the arcs that link people and objects. Using the free and open NodeXL application, users can collect, analyze and visualize complex networks and then upload the data to a growing archive on the web at NodeXLGraphGallery.org. As the group of researchers grows, an archive is being assembled to provide researchers around the world with the data about social media needed to understand the ways computer mediated communication tools shape society.

My talk at the 2011 Personal Digital Archiving conference is available through the Internet Archive’s video service:

This map of connections among people who tweeted Teaparty starts on 11/15/2011 14:22 UTC and ends on 11/15/2011 17:23, a total of 3 hours and 1 minute of traffic.

The Teaparty data set contained 1,533 tweets, replies and mentions.
Blue edges are connections created by replies and mentions. Grey lines are follows relationships.

Top most between users:
@ronpaul
@michellemalkin
@christopherhull
@theteaparty_net
@capaction
@thedailyedge
@bill1phd
@dbargen
@gulagbound
@rightcandidates

Graph Metric: Value
Graph Type: Directed
Vertices: 659
Unique Edges: 8808
Edges With Duplicates: 1423
Total Edges: 10231
Self-Loops: 1084
Connected Components: 49
Single-Vertex Connected Components: 44
Maximum Vertices in a Connected Component: 606
Maximum Edges in a Connected Component: 10148
Maximum Geodesic Distance (Diameter): 6
Average Geodesic Distance:2.693965
Graph Density: 0.02036797
NodeXL Version: 1.0.1.193

The major clusters are composed of teaparty supporters. The center bottom cluster are teaparty critics.

This map of the connections among people who tweeted Occupywallstreet starts on 11/15/2011 23:08 and ends on 11/15/2011 23:34 UTC, a total of 26 minutes of traffic.

Occupywallstreet 1,370 tweets, replies and mentions
Blue edges are connections created by replies and mentions. Grey lines are follows relationships.

Top most between users:
@occupywallst
@mmflint
@nyclu
@allisonkilkenny
@andrewbreitbart
@operationleaks
@occupydenver
@theatlantic
@usgeneralstrike
@rt_com

Graph Metric: Value
Graph Type: Directed
Vertices: 1000
Unique Edges: 3546
Edges With Duplicates: 826
Total Edges: 4372
Self-Loops: 794
Connected Components: 241
Single-Vertex Connected Components: 230
Maximum Vertices in a Connected Component: 747
Maximum Edges in a Connected Component: 3998
Maximum Geodesic Distance (Diameter): 7
Average Geodesic Distance: 2.65438
Graph Density: 0.003246246
NodeXL Version: 1.0.1.194

Some notable contrasts:
Teaparty Graph Density: 0.002652645
Occupywallstreet Graph Density: 0.02036797 – significantly lower levels of interconnection
Teaparty: Single-Vertex Connected Components 44 of 1000
Occupywallstreet: Single-Vertex Connected Components 283 of 1000

Many more “isolates” (Single-Vertex Connected Components) in Occupywallstreet.
Many more hubs, and more retweeting activity in Occupywallstreet.

The difference in duration of these data sets illustrates the relative speed of content creation in the topics. The data sets are commensurable in that they are both the result of a single query against the Twitter search API. So both maps are the results of charting connections among the authors of the last 1500 tweets, how ever long that takes to create.

I presented on social media network analysis on October 20, 2011 in New York City at Predictive Analytics World.

A map of the connections among the people tweeting about the #Pawcon hashtag is below.

Network Maps for End Users: Collect, Analyze, Visualize and Communicate Network Insights with Zero Coding

Abstract: Networks are everywhere except the end user desktop.  NodeXL, the free and open network overview, discovery and exploration add-in for the popular and familiar Excel (2007/2010) spreadsheet allows users who are comfortable making pie charts to now make useful network visualizations.  Developed and released by the Social Media Research Foundation, NodeXL uses Excel as a framework, providing a GUI network browser (a “web browser”?) that novices can use quickly and experts can use to generate sophisticated results.  Data importers provide access to a range of social media network data sources like Twitter, flickr, YouTube, Facebook, email, the WWW, and more through standard file formats (CSV, GraphML, Matrix).  Simple to use tools can automatically analyze, visualize and highlight insights in complex network graphs.  Using NodeXL, researchers have been collecting a wide range of network data sets from various social media services.  These images reveal a range of common social formations in social media and point to people who occupy strategic locations in these graphs.

This is a map of the connections among the people who tweeted the term “PAWCON” on the first day of the event:

These are the connections among the Twitter users who recently tweeted the word #pawcon when queried on October 19, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

A larger version of the image is here: www.flickr.com/photos/marc_smith/6261006732/sizes/l/in/ph…

Top most between users:
@tapan_patel
@pawcon
@sasanalytics
@deloitteba
@kristinevick
@jamet123
@zementis
@kdnuggets
@tibcospotfire
@saspublishing

Graph Metric: Value
Graph Type: Directed
Vertices: 41
Unique Edges: 233
Edges With Duplicates: 120
Total Edges: 353
Self-Loops: 44
Connected Components: 2
Single-Vertex Connected Components: 1
Maximum Vertices in a Connected Component: 40
Maximum Edges in a Connected Component: 352
Maximum Geodesic Distance (Diameter): 4
Average Geodesic Distance: 1.87133
Graph Density: 0.15304878
NodeXL Version: 1.0.1.179

Here is an example map of the connections among the people who tweeted the term “pawcon” in Twitter on September 14th, a week prior to the event.

Manu Sharma, Principle Research Scientist at LinkedIn gave a great presentation on the patterns found in their data.  Big data, for example, showed that most of the people who previously worked at recently failed banks and financial institutions have updated their profiles to show that they mostly have new jobs at some of the remaining companies in the industry.

The event was held at the New York Hilton: Maps & Directions

http://ir12.aoir.org/

These are the  connections among the Twitter users who recently tweeted the word AOIR when queried on October 10, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

See: ir12.aoir.org/

Top most between users:
@ir12
@barrywellman
@mikemonello
@netcrit
@andresmh
@s_t_e_v_e_jones
@zizip
@kellybergstrom
@katypearce
@guillaumelatzko

Graph Metric: Value
Graph Type: Directed
Vertices: 98
Unique Edges: 119
Edges With Duplicates: 672
Total Edges: 791
Self-Loops: 153
Connected Components: 40
Single-Vertex Connected Components: 35
Maximum Vertices in a Connected Component: 55
Maximum Edges in a Connected Component: 710
Maximum Geodesic Distance (Diameter): 6
Average Geodesic Distance: 2.257477
Graph Density: 0.035766884
NodeXL Version: 1.0.1.179

By expanding the query to include #IR12, the conference hashtag, the network expands to include:

Connections among the Twitter users who recently tweeted the word AOIR OR #IR12 when queried on October 11, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

Top most between users:
@ir12
@barrywellman
@mikemonello
@netcrit
@andresmh
@s_t_e_v_e_jones
@zizip
@kellybergstrom
@katypearce
@guillaumelatzko

Graph Metric: Value
Graph Type: Directed
Vertices: 231
Unique Edges: 1984
Edges With Duplicates: 2118
Total Edges: 4102
Self-Loops: 586
Connected Components: 46
Single-Vertex Connected Components: 40
Maximum Vertices in a Connected Component: 181
Maximum Edges in a Connected Component: 4012
Maximum Geodesic Distance (Diameter): 5
Average Geodesic Distance: 2.213887
Graph Density: 0.046320346
NodeXL Version: 1.0.1.179

More NodeXL network visualizations are here: www.flickr.com/photos/marc_smith/sets/72157622437066929/

At the end of the conference on October 5th, the network of connections among the people who tweeted about Wikisym looked like this:

These are the connections among the Twitter users who recently tweeted the word wikisym when queried on October 5, 2011, scaled by numbers of followers (with outliers thresholded). Connections are created when users reply, mention or follow one another.

See: www.wikisym.org/

Top most between users:
@edchi
@jfelipe
@phauly
@readermeter
@hfordsa
@wikisym
@clifflampe
@staeiou
@geoplace
@wikimedia

Graph Metric: Value
Graph Type: Directed
Vertices: 170
Unique Edges: 1163
Edges With Duplicates: 2165
Total Edges: 3328
Self-Loops: 658
Connected Components: 5
Single-Vertex Connected Components: 3
Maximum Vertices in a Connected Component: 165
Maximum Edges in a Connected Component: 3319
Maximum Geodesic Distance (Diameter): 6
Average Geodesic Distance: 2.385502
Graph Density: 0.052941176
NodeXL Version: 1.0.1.179

In contrast, a few days earlier, these are the connections among the Twitter users who recently tweeted the word wikisym when queried on October 3, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

Top most between users:
@wikisym
@jfelipe
@andicat
@geoplace
@staeiou
@dirkriehle
@wikimedia
@edchi
@bkeegan
@hfordsa

Graph Metric: Value
Graph Type: Directed
Vertices: 78
Unique Edges: 459
Edges With Duplicates: 354
Total Edges: 813
Self-Loops: 132
Connected Components: 10
Single-Vertex Connected Components: 8
Maximum Vertices in a Connected Component: 68
Maximum Edges in a Connected Component: 800
Maximum Geodesic Distance (Diameter): 5
Average Geodesic Distance: 2.207506
Graph Density: 0.086913087
NodeXL Version: 1.0.1.179

More NodeXL network visualizations are here: www.flickr.com/photos/marc_smith/sets/72157622437066929/

NodeXL is free and open and available from www.codeplex.com/nodexl

NodeXL is developed by the Social Media Research Foundation (www.smrfoundation.org) – which is dedicated to open tools, open data, and open scholarship.

The book, Analyzing social media networks with NodeXL: Insights from a connected world, is available from Morgan Kaufmann and from Amazon.

Marc Smith on Twitter.

Here is a map of the connections among the people who tweeted the term “AEJMC” on August 7, 2011:

The top most between people in this network are:@aejmc, @jlab, @karenrussell, @terryflynn, @natcomm, @tmccorkindale, @derigansilver, @tkell, @aejmconlineads, and @jeremyhl:

I will present a Workshop on Social Media Network Analysis and NodeXL at the 9 August 2011 – Association for Education in Journalism and Mass Communication (http://www.aejmc.com) in St. Louis, Missouri along with my colleague Professor Hernando Rojas, from the School of Journalism & Mass Communication, University of Wisconsin – Madison.

See: http://www.aejmc.com/home/events/annual-convention/

The Association for Education in Journalism and Mass Communication (AEJMC) is a nonprofit, educational association of journalism and mass communication educators, students and media professionals. The Association’s mission is to promote the highest possible standards for journalism and mass communication education, to cultivate the widest possible range of communication research, to encourage the implementation of a multi-cultural society in the classroom and curriculum, and to defend and maintain freedom of communication in an effort to achieve better professional practice and a better informed public.

Our session is:

Using NodeXL for Social Network Analysis
Tuesday — 2 pm to 5 pm
Presented by Communication Theory and Methodology Division
This pre-conference workshop examines social network analysis. Social network analysis can be used to examine message boards, blogs, and friend networks (amongmany other phenomena). Participants will learn to use the NodeXL program to conduct a network analysis. For information, contact Michel M. Haigh, Pennsylvania State University at mmh25@psu.edu.

Starting in version .165 of NodeXL we have supported the idea of an options file that can be imported, exported and exchanged among users.

If you have set all the knobs and dials of your copy of NodeXL just right, you can export these adjustments and configurations into a single file.  Use the NodeXL>Options>Export feature to create a named file containing your settings.  You can now exchange this file with others.  If you receive an options file, you can use the NodeXL>Options>Import feature to pick it out from the file system and set your copy of NodeXL to the settings defined in that file.

If you use the related NodeXL>Options>Use Current for New feature you can set the defaults for NodeXL to the settings contained in any imported options file.

I have been fine tuning a settings file that generates network visualizations that are tailored for the Twitter networks I have been making many of.  I have saved these sample settings file for NodeXL and you can try them yourself here: NodeXL-Twitter Network-Options Settings.NodeXLOptions.

Download this file, rename it by removing the .txt extension and load it into your copy of NodeXL.

NodeXL allows users to gather vertices into named collections called “Groups”.  This is handy whenever the entities in the network are made up of different types or an algorithm has divided the network into sub-regions based on how densely some vertices connect to one another.  The Groups menu is found in the NodeXL>Analysis menu:

Since version v.132 of NodeXL it has been possible to Collapse a group of vertices (See: Expand and Collapse Groups of Vertices with NodeXL v.132).  When a group is collapsed all of the vertices within that group are removed from the network graph and replaced with a single vertex with a size proportionate to the number of vertices in the group.  A small “+” plus sign indicates that the vertex is a placeholder for a group of vertices.

If the user expands a collapsed group all of the vertices that had been hidden return to positions in the network visualization.  The Groups menu has commands for creating, collapsing, and expanding groups.

NodeXL (v.166) now has the ability to automatically collapse or expand any group of vertices conditionally based on any attribute in the workbook using the Autofill Columns feature.

The NodeXL Autofill columns feature allows users to map data elements to display elements.  At the bottom of this list (you may need to scroll down to see it) you will now find a new row: Group Collapsed?

There are several network metric attributes for each group that are created when the Find Groups and then the Graph Metrics command has been run on a network in NodeXL:

Selecting one of the data items in the drop down allows you to automatically decide if a group with those attributes will be presented in a collapsed or (default) expanded state.  The data about each group include the number of vertices within the group, the number of connections between those vertices, the number of non-unique connections, the number of unique connections among the vertices, the number of self-connections, the number of unique connected components, the number of isolated vertices, the number of vertices in the largest component, the number of edges in the largest component, the maximum and average width of the largest component, and the density of the group.

These metrics allow for the automated processing of the graph to measure each group and apply a test to decide if a group is too dense or populous to be seen in an expanded state.

I will be speaking at the Jacob T. Schwartz International School for Scientific Research week long Lipari School on Computational Social Science , July 17 – July 23, 2011, Lipari Island, Italy.

This year’s program is dedicated to Computational Social Science: Text and Decisions

Speakers:

• Claudio Cioffi-Revilla: Director of the Center for Social Complexity, Krasnow Institute for Advanced Study, George Mason University, Washington DC.
• Huan Liu: Community Detection and Mining in Social Media [abstract]
  School of Computing, Informatics, and Decision Systems Engineering, Arizona State University
• Roel Popping: Computer-assisted text analysis, and the relevance of decision making and text mining [abstract]
  Department of Sociology, University of Groningen

Tutorials

• Marc A. Smith: Charting Collections of Connections in Social Media: Maps and Measures with NodeXL [abstract]
  Chief Social Scientist, Connected Action Consulting Group
• Calogero Zarba: Introduction to matrix algebra [abstract]
  Neodata Intelligence s.r.l., Italy
• Alessandro Pluchino: Netlogo: An agent based simulation programmable environment [abstract], University of Catania, Italy

A recent paper “EventGraphs: Charting Collections of Conference Connections” by Marc Smith from Connected Action, Professor Derek Hansen (College of Information Studies) and Professor Ben Shneiderman (Computer Science/Human Computer Interaction Lab) both from the University of Maryland and has been accepted for publication at the 2011 Hawaii International Conference on System Sciences (HICSS) Conference.  This is the 44th year for the conference. Derek Hansen presented the paper on January 7, 2011.

Hansen, D., Smith, M., Shneiderman, B., EventGraphs: charting collections of conference connections. Hawaii International Conference on System Sciences. Forty-Forth Annual Hawaii International Conference on System Sciences (HICSS). January 4-7, 2011. Kauai, Hawaii.

http://www.cs.umd.edu/localphp/hcil/tech-reports-search.php?number=2010-13

Our paper is about visualizing social media and it describes the visualization of the patterns of connections formed when people tweet about events like conferences and news stories.

EventGraphs are social media network diagrams constructed from content selected by its association with time-bounded events, such as conferences. Many conferences now communicate a common “hashtag” or keyword to identify messages related to the event. EventGraphs help make sense of the collections of connections that form when people follow, reply or mention one another and a keyword. This paper defines EventGraphs, characterizes different types, and shows how the social media network analysis add-in NodeXL supports their creation and analysis. The paper also identifies the structural and conversational patterns to look for and highlight in EventGraphs and provides design ideas for their improvement.

A collection of EventGraphs are available in flickr: http://www.flickr.com/photos/marc_smith/sets/72157625618025980/

The EventGraph for HICSS this year is here: 20110109-NodeXL-Twitter-HICSS

The top most between contributors to the HICSS Twitter graph this year are: @arcticpenguin @avantgame @barrywellman @clifflampe @dalprof @chandlerism @shakmatt @marc_smith @shen045 and @thecatandthekey

Here is the data set: 20110109-NodeXL-Twitter-HICSS
Graph Metric Value
Graph Type Directed
Vertices 92
Unique Edges 243
Edges With Duplicates 71
Total Edges 314
Self-Loops 0
Connected Components 21
Single-Vertex Connected Components 18
Maximum Vertices in a Connected Component 69
Maximum Edges in a Connected Component 307
Maximum Geodesic Distance (Diameter) 8
Average Geodesic Distance 3.081693
Graph Density 0.032967033

NodeXL Version 1.0.1.159

An article about social media visualization Social Seen: Analyzing and Visualizing Data from Social Networks by Hunter Whitney, appeared in UX Magazine on December 15, 2010.  It provides a great round up of tools and approaches to mapping populations in social media spaces.

Previous HICSS papers:

Gleave, Eric, Howard T. Welser, Marc Smith, and Thomas Lento.  2009.  “A conceptual and operational definition of social role in online community.”   In Proceedings of the 42nd  Hawaii International Conference on Systems Sciences (HICSS), January 5-8. Computer Society Press.  (Best Paper, Digital Media and Communication Mini-Track)

2009 – HICSS – 42 – Best Paper – A Conceptual and Operational Definition of ‘Social Role’ in Online Community

Fisher, D., Smith, M., and Welser, H. You Are Who You Talk To, Proceedings of HICSS, January 2006. (Best Paper, Digital Media and Communication Program)

http://www.connectedaction.net/wp-content/uploads/2009/08/2006-HICSS-You-Are-Who-You-Talk-To-Detecting-Roles-in-Usenet-Newsgroups.pdf

Viégas, Fernanda B., Marc Smith. “Newsgroup Crowds and AuthorLines: Visualizing the Activity of Individuals in Conversational Cyberspaces“, Proceedings of Hawaii International Conference on Software and Systems (HICSS) 2004. (Best Paper: Persistent Conversation Minitrack)

http://www.connectedaction.net/wp-content/uploads/2009/08/2004-HICSS-Viegas-and-Smith-Newsgroup-Crowds-and-Author-Lines.pdf

Here are some sample event graphs:

Once you start creating and collecting network graphs you may find you can build a significant collection: hundreds, thousands or tens of thousands of graphs may result from a study or on-going monitoring project. In a series of features in the NodeXL project we have enabled a workflow for constructing many social media  network graphs using the Network Server component (see: How to schedule the creation of a network with NodeXL and Windows Task Scheduler and: New NodeXL Network Server (v1.0.1.126) – Frequently Asked Questions).  This can result in a collection of *many* NodeXL (and GraphML) network files.  Then we implemented features that enabled “Automation”, the application of many operations in NodeXL (metrics calculation, autofill columns, layout and more) to many files without direct human engagement (see: Automatic for the people (who use the latest NodeXL!). Release v.1.0.1.128 and: Fully automatic: NodeXL can build your network graphs hands free).

A single workbook may contain data from a single NodeXL data collection, run on a particular day and collecting data from a few hours or days back from that moment  (depending on factors like the volume of activity around the selected keyword and the depth of the twitter search catalog, which is often not more than a week or two long and much shorter for active topics).  An example of a single network slice is this recent map of the connections among people who mentioned “microsoft research” in Twitter on a single day (December 18th, 2010):

:

This is a single slice of the network, a day out of months of activity.  A still frame can tell a rich story: this is a picture of a crowd that has gathered to discuss a topic of common interest: “microsoft research“.  It illustrates a structure common to many large discussions of popular topics — a large set of isolates (the rows at the bottom) who were not observed to have a followed, mentions, or replies relationship to anyone else who tweeted the same term.  These are casual mentioners of the topic.  At the end of these rows are a small number of dyads, triads, and small components of a handful of people who link to one another but not to the largest connected component. These are pairs or small groups discussing the topic among themselves, but none are connected to a larger component.  Above these rows is the “giant component” — the blob of people who do have a connection to someone else who also tweeted a message containing the same term who in turn have a connection that leads to a large number of others.  The giant component is itself composed of several sub-components of densely connected groups.  At the center of each component are the core users, the people who often hold their cluster together. Between these clusters are the bridges, the people who link otherwise disconnected sub-groups.  At the edges are the peripheral people who have just taken the first step up from being an isolate and have formed a single reply, mention, or follows relationship to someone else who also tweeted the search keyword and can bridge them back to the core of the giant component.  This is a large and active network with hybrid qualities.  There is a “brand” or broadcast element in it: the yellow cluster is a hub and spoke structure centered on the Microsoft Research Twitter account.  These people re-tweet what this account publishes but do not connect to one another.  Just a few of these people set off second and third waves of retweets.  Elsewhere in the graph there are other network structures present, for example the green and blue clusters feature people are centered around their own discussions of the term “microsoft research“.

If you collect many still frames of slices of network activity there is great value in exploring the way the network graph changes over time.  In the most recent release NodeXL provides the first step in a series of features related to time and graph comparison.  You can now create a workbook that aggregates the overall metrics (edge counts, vertex counts, connected component counts, etc.) for a folder full of NodeXL workbooks. In NodeXL follow the menu path: NodeXL>Analysis>Graph Metrics>Aggregate Overall Metrics to get this:

The result of this feature is a workbook with a row containing the summary data from each of the workbooks in the target folder.  Any arbitrary collection of network workbooks can be aggregated but this is particularly useful when the workbooks are sequential time slices.

An example is the time series plot below tracking the rise and fall of several Twitter volume and network measures for the “microsoft research” search term over several months:

This chart tracks the number of vertices (each vertex in this case is a person  our data collector saw tweet about the search term “microsoft research“) in each (almost) daily network snapshot.  In addition the unique edges or connections between these Twitter users are plotted along with the number of people with no connections (“Single-Vertex Connected Components”).  The size of the largest component in the network (“Maximum vertices in a connected component “) is a measure of the changing size of the core community of discussion participants.  Measures like the maximum and average “geodesic” distance provide a rough measure of how long and thin (high values) or generally spherical (low values) a particular network is shaped. A “geodesic” is the longest path that can be walked through the network.  Long skinny networks may indicate the presence of loosely connected smaller groups that have a few people who act as bridges.  Low geodesic values suggest dense networks with people connected to many others with few isolates and sub-groups.

The peaks are closely associated with major events on the Microsoft Research calendar, like the 2010 Microsoft Research Faculty Summit event I attended in early July.

I find the ratios between measures of the size of the large network component and the population of isolates to be interesting.  As events go on over a period of days more people connect with others who are talking about the same topic, growing the size of the large connected component.  But often the isolate population also grows during this time as people at the periphery of the topic network catch sight of mentions of the event and tweet about it.  I could imagine one goal of social media management to be the conversion of isolates to connected component members.  Those who follow, reply or mention even a single other person also talking about a topic are more likely to return and engage than those who have zero connections.  It is not clear if more connections provide a linear increase in continued engagement, I suspect that the main effect is at the zero/one divide and drops off in effect after the first dozen or so connections.  Encouraging cohesion and network density by replying to isolates and encouraging others to do so may help keep a social media population focused and growing.

This feature follows the work done in the ManyNets project (http://www.cs.umd.edu/hcil/manynets/) at the University of Maryland by Manuel Freire, Catherine Plaisant, Ben Shneiderman, Awalin Sopan, and Miguel Rios.  ManyNets also created a framework for managing the metadata about collections of networks. ManyNets provides for  much richer interactions and linkages to the underlying networks than NodeXL can do so far.

As mobile devices become a major method for authoring and consuming social media, location data is increasingly a part of many posts, tweets, check-ins, and messages.  Many Twitter clients, for example, can add the user’s current latitude and longitude to the metadata associated with a tweet.  Other systems like Facebook Places, Google Latitude and Foursquare encourage users to declare  where they are to the world, often passing the information to other social media sites.

Using this location data in network analysis opens up a range of new opportunities.  Instead of a person – to – person social network, location data allows people to be linked to places and, by extension, places can be linked to other places based on the patterns of connection people create when located in a particular place.  A convergence of network analysis and Geographic Information Systems in underway.  A great example of this can be found in this wonderful video from the BBC which demonstrates the idea by mapping the flow of telephone calls, texts, and data around the UK and the wider world.

Link on the BBC

Even better is this video from the SensibleCity group at MIT:

Now, NodeXL (v.156) has the first of a series of features that will start to approximate the experience displayed in the video by supporting the import of location data about networks and plotting networks onto maps.

For now, we have started importing latitude and longitude data that Twitter makes available.  If you check “Add a Tweet column to the Vertices worksheet” in NodeXL, Data, Import, From Twitter Search Network or From Twitter User Network, the Twitter user’s geographical coordinates will be added to the Vertices worksheet when they are available.

Note that not every tweet has a latitude and longitude, in fact many do not (yet).  Further, note that not every latitude and longitude is accurate, many are not.

We need to implement more features for better location data support in a NodeXL workbook, but this is a start.  We are interested in exploring geospatial networks and Twitter is a great data source.  With this data in place we may look into features that emit KML files for exploration in other packages like Google Earth.  A nifty Google Earth/Spreadsheet importer can take small sets (400) of location data points in a spreadsheet and export them to a KML file, something we could implement in the future as well.  In addition we may be able to connect directly with services like Bing Maps and Google Maps to display connections between nodes with known locations.  Metrics that calculate the distance between nodes seem sensible as well.

Location coordinates are the key to a cornucopia of related data about a place.  Given a latitude and longitude it is possible to find the name of the city it is located in, look up data about that location in official records as well as resources like Wikipedia.  Income, education, property values, weather, photos, and more can be pulled together from just a simple lat/long.  All of these attributes could be used to cluster or illustrate the network visualization.

In this map nodes represent the major feature groups and functions in the NodeXL application.

This map will become the default file that will open when you run NodeXL for the first time.  You will see a dialog like this:

Select Yes to have the graph above imported into the workbook.  You can then display the graph using the Show Graph button in the NodeXL menu ribbon.

After that, it will be available via the help menus. When you import the file, all of the data is also available in the spreadsheet part of NodeXL so that you can experiment with changing values there to see the impact in the graph display after you hit the “refresh graphs” button.

Over the coming weeks we plan to release additional sample network data sets that illustrate key concepts and methods in network science.  Suggestions for sample networks are welcome!

The book Analyzing Social Media Networks with NodeXL: Insights from a connected world is now available [Amazon] from Morgan-Kaufman. Co-authored by Professor Derek Hansen (College of Information Studies) and Professor Ben Shneiderman (Computer Science/Human Computer Interaction Lab) from the University of Maryland and Marc Smith from Connected Action, the book is a introduction and guide to the application of social network analysis to social media.  The introductory chapters introduce the history and concepts of social network analysis an the varieties of social media, highlighting the presence of a common data structure, the network, in otherwise diverse social media systems including email, Twitter, Facebook, the WWW, Wikis, Blogs, flickr, an YouTube.  The central section of the book reviews a step-by-step guide to using the key features of NodeXL, the free and open social media network analysis add-in for Excel 2007 and 2010.  Readers can move from simple hand entered networks of a few nodes up to complex graphs extracted from a variety of social media services.  The remainder of the book are focused chapters dedicated to analyzing the networks found within a specific social media service.  These chapters were contributed by leading social media researchers and illustrate the insights that can be extracted from the otherwise disorganized stream of messages, tweets, posts, comments, links, likes, tags, friends, follows, mentions, replies and ratings.  A recent article about the book can be found on the Morgan-Kaufmann website.

Table of contents…

I. Getting Started with Analyzing Social Media Networks

[Chapter 1]: Introduction to Social Media and Social Networks
[Chapter 2]: Social media: New Technologies of Collaboration
[Chapter 3]: Social Network Analysis: Measuring, Mapping, and Modeling Collections of Connections

II. NodeXL Tutorial: Learning by Doing

[Chapter 4]: Getting Started with NodeXL, Layout, Visual Design, and Labeling
[Chapter 5]: Calculating and Visualizing Network Metrics
[Chapter 6]: Preparing Data and Filtering
[Chapter 7]: Clustering and Grouping

III Social Media Network Analysis Case Studies

[Chapter 8]: Email: The Lifeblood of Modern Communication
[Chapter 9]: Thread Networks: Mapping Message Boards and Email Lists

[Chapter 10]: Twitter: Conversation, Entertainment and Information, All in One Network!

By Vladimir Barash and Scott Golder

[Chapter 11]: Visualizing and Interpreting Facebook Networks

By Bernie Hogan

[Chapter 12]: WWW Hyperlink Networks

By Robert Ackland

[Chapter 13]: Flickr: Linking People, Photos, and Tags

By Eduarda Mendes Rodrigues and Natasa Milic-Frayling

[Chapter 14]: YouTube: Contrasting Patterns of Interaction and Prominence

By Dana Rotman and Jennifer Golbeck

[Chapter 15]: Wiki Networks: Networks of Creativity and Collaboration

By Howard T. Welser, Patrick Underwood, Dan Cosley, Derek Hansen, and Laura Black

I spoke at the 2010 Catalyst Conference in San Diego on July 29th. The conference hashtag is #CAT10.

The slides are here:

A few days before the conference started the #CAT10 twitter social network map looked like this:

26 July 2010 NodeXL Twitter map of the connections among people who tweet “#CAT10″ the hashtag for this year’s Catalyst conference.

This is the list of the most “between” contributors in the #CAT10 Twitter graph on July 26, 2010.

A few days later, as people began to arrive at the conference, the graph became far more dense and populous.

The network of #CAT10 mentioning users in Twitter has become much more dense, with more people and more connections among them as people reply, retweet, follow, and mention one another.

While the core people in this list are similar to the list generated a few days earlier, several people have shifted position.

Filtering the graph, we can remove all but the most between people to reveal the core members of the community.

These people are likely to play an influential role in the #CAT10 community.

There is a Stanford Media X event on July 22nd, 2010 on new tools for SNA:

Network Analysis Made Easy:  Using NodeXL To Map Social Media Networks

http://mediax.stanford.edu/WSI/marc.html

Bring a laptop (running Windows and Office 2007 or 2010) to this workshop and you can be analyzing a social media network from systems like Twitter, flickr, YouTube and your own email by the end of the day.  If you can make a pie-chart in Excel, using the free and open NodeXL (http://nodexl.codeplex.com) you can now make a rich network graph from data extracted from social media systems and other common formats.  If you have a network, bring it, if not you can bring a suggested topic that we can map during the course of the day.

Even if you leave your laptop behind or have a Mac (sorry, no version is yet available for MacOS – unless you have a virtual machine with Windows and Office) this workshop will introduce the core concepts of network science with application to social networks in general and social media networks in particular. Applied to a range of topics and services, social media network maps can illuminate a variety of “publics” – populations who share a common interest and may share connections.  Maps of topics like “oil spill”, “global warming” and other issue and event related keywords can reveal the groups and factions that cluster around different concepts and terms.  Key contributors in these maps can be identified through the application of network measurements that capture various aspects of a  person’s location in a network graph.

2010 – June – NodeXL Book Flyer.

[Chapter 10]: Twitter: Conversation, Entertainment and Information, All in One Network!

By Vladimir Barash and Scott Golder

[Chapter 11]: Visualizing and Interpreting Facebook Networks

By Bernie Hogan

[Chapter 12]: WWW Hyperlink Networks

By Robert Ackland

[Chapter 13]: Flickr: Linking People, Photos, and Tags

By Eduarda Mendes Rodrigues and Natasa Milic-Frayling

[Chapter 14]: YouTube: Contrasting Patterns of Interaction and Prominence

By Dana Rotman and Jennifer Golbeck

[Chapter 15]: Wiki Networks: Networks of Creativity and Collaboration

By Howard T Welser, Patrick Underwood, Dan Cosley, Derek Hansen, and Laura Black

This handy poster contains many details about the book contributors, chapters, and the book cover (which you can also see below):

Analyzing Social Media Networks with NodeXL: Insights from a Connected World

Fourth International AAAI Conference on Weblogs and Social Media
(ICWSM-10)
May 23-26, 2010
George Washington University, Washington, DC

Sponsored by the Association for the Advancement of Artificial Intelligence

The ICWSM 2010 conference starts Sunday.  This is a very high quality conference on the study of social media.  My colleague, Professor Derek Hansen, and I will lead a tutorial on using NodeXL to analyze social media networks.

SA2: Introduction to Social Media Network Analysis
Marc Smith (Connected Action) and
Derek Hansen (University of Maryland)

Social networks are the defining data structure of social media, created as people reply, link, click, favorite, friend, re-tweet, co-edit, mention, or tag one another. In this tutorial, we review the core concepts and methods of social network analysis and apply it to the collection, analysis, and visualization of social media networks. Using the free and open NodeXL application, learn how to extract a social media network and generate metrics and visualizations that highlight key people and positions within streams of tweets, videos, photos, or emails.

I will be speaking at the Princeton Club of Northern California on May 12, 2010.

The Stanford Network Analysis Platform (SNAP) (http://snap.stanford.edu) is a high performance library for calculating network metrics of potentially very large graphs. Working with SNAP author and Stanford Computer Science Professor Jure Leskovec, the NodeXL team is releasing a new update with expanded support for network metrics. With SNAP integrated into NodeXL we have improved the scale and speed performance significantly (*very* significantly!).  As of this release (v.1.0.1.122) the Betweenness Centrality, Closeness Centrality, and Eigenvector Centrality measures are calculated using the SNAP library. In addition we have added the Page Rank metric calculated by SNAP to the list of supported network measures.  Two additional clustering algorithms automatically group nodes together into collections. With SNAP integrated into NodeXL we have added two clustering algorithms: Girvan-Newman and Clauset-Newman-Moore.

What network metrics matter most to you?

This is a map of the connections among the population of people who recently tweeted the term “WWW2010″.  The size of the profile photos indicates the log scaled number of followers that person has.  The edges vary in weight, thicker edges mean more of the three types of connections we track, follows, replies, and mentions, appear between any two people.

This is April 29th:

This is April 28th:

A row of isolated people stand at the bottom of the graph, mentioning “www2010″ but lacking any visible connection to anyone else who tweets the same term.

The giant component is dominated by the corporate twitter accounts for information technology companies and prominent researchers in the field.

One way to rank people in a network is in terms of how “between” they are — how much they occupy a position that is like a bridge between otherwise disconnected people or groups.  There is a correlation between the number of connections and your betweenness centrality, but it is possible to be very between and have just a few (at least two!) connections.

In this layout the x axis is the log of the number of followers and the y axis is the log of the number of tweets.  Some people out perform the relationship between tweets and followers, attracting more followers for fewer tweets than others.  Above the diagonal, some people lag in terms of their tweet to follower ratio.

These reports and images were generated with NodeXL, the free and open social media network analysis add-in for Excel 2007/2010.  The dataset 2010 – April – 28 – NodeXL – Twitter www2010 is available here. A book, Analyzing social media networks with NodeXL: Insights from a connected world, is forthcoming summer 2010 from Morgan Kaufmann and from Amazon.

Travel Industry Social Media Analysis: United v Delta – How do airline twitter maps compare? from Marc Smith on Vimeo.

The two maps we discuss are: United Airlines:

and Delta Airlines:

United has a large twitter mentioning population and has a bigger main component.  The larger profile photos indicate Twitter authors with many followers.  The large population of isolates or small components at the bottom of both images are people who mention either airline but are not in a conversation or a relationship with someone else who also mentioned these brands.  They are “shouts” about a brand, not conversations.  In contrast the large component in both images are the connected collection of people who talk to people who talk about these brands.  They are committed to the topic in a way the less connected  authors are not.  They know someone who also talks about air travel.

Viewed over time, we can start to assess the ways these brand’s twitter populations are changing.  Are new people moving into central hub positions?  Are people who held those positions drifting away?

A key observation is that some people with relatively few followers occupy highly central positions in the graph.  This suggests that these authors have a location that lends their content attention from other well positioned people.  Popularity is not just about volume of connections, in a social network perspective, importance is also a function of where the person sits within the graph.

We will add axis markings and titles soon, making these charts ready to use in a variety of network reports.  These histograms will also appear in the Dynamic Filters dialog to guide users as they select segments of the distribution to include or filter out of the displayed network graph.

Other updates:

1.0.1.110 (2010-02-03)

• The Overall Metrics worksheet now includes more information about the degree, in-degree, out-degree, betweenness centrality, closeness centrality, eigenvector centrality, and clustering coefficient metrics when those metrics are computed. The additional information includes the minimum, maximum, average, and median metric values, and a histogram showing the metric value distribution.
• The “Convert Old Workbook” item on the NodeXL, Data, Import menu in the Ribbon is now called “Import from NodeXL Workbook Created on Another Computer.” This menu item can be used to work around the following problem: NodeXL workbooks created on a 64-bit Windows computer cannot be opened directly in Excel on a 32-bit Windows computer, and vice-versa. (If you attempt to do so, you will get an error message whose details include “could not find a part of the path.”)
• A Clear All Worksheet Columns Now button has been added to the Autofill Columns dialog box (NodeXL, Visual Properties, Autofill). Also, you can now clear an individual worksheet column by clicking a button in the dialog box’s Options column.
• Bug fix: On large-font machines, the buttons at the bottom of the Autofill Columns dialog box didn’t fit within the dialog box.
• Bug fix: In some circumstances, vertices were drawn below the bottom of the graph pane and were impossible to see. One such circumstance was when the selection was exported to a new workbook (NodeXL, Data, Export, Selection to New NodeXL Workbook). The graph pane in the new workbook acted as if it were taller than its real height, leading to vertices dropping off the bottom.

NodeXL has updated again (v.1.0.1.109) with new network metrics.  The application now calculates path length data for your network, reporting the Maximum Geodesic Distance and the Average Geodesic Distance.  The list of overall metrics NodeXL creates includes: Vertices (the number of nodes in the graph), Unique Edges, Edges With Duplicates, Total Edges, Self-Loops (Edges that point back at the node from which they originate), Connected Components (each set of connected nodes that are not connected to another set of nodes), Single-Vertex Connected Components (all the “singletons” of just one node in a component), Maximum Vertices in a Connected Component (the size of the “Giant” component), Maximum Edges in a Connected Component (the density of the “Giant” component), Maximum Geodesic Distance (Diameter) (the longest path that can be uniquely walked through the graph), Average Geodesic Distance (the average distance between two nodes in the graph (compare this to the “six degrees” standard), Graph Density (the density of the complete network).

More metrics and details on existing metrics are on the way!

What metrics do you need?

Penn State hosts C&T 2009

Derek Hansen (University of Maryland) and I will be running a workshop as part of the Communities and Technologies conference next week in University Park, PA. We have room for additional participants. If anyone is interested, please email me (marc.smith@telligent.com) or Derek (shakmatt@gmail.com). If there is sufficient interest by those in the DC or Bay Area we could potentially arrange for a similar workshop closer to home. We are particularly interested in reaching instructors who may be teaching social network analysis or courses on social media in the Fall of 2009 and have a tutorial we have developed to aid in that process. The tutorial is being updated, but here is the old version (which still is useful!).  We will distribute the updated version at the conference and thereafter here on the blog and the NodeXL codeplex site.

Title: NodeXL: Social Network Analysis and Visualization tools for Social Media
Date/Time: Wednesday, June 24, 8:30am through early afternoon
Place: Penn State University, University Park, PA

(http://cct2009.ist.psu.edu/index.cfm)

This session will provide a walk through the basic operation of NodeXL. Attendees are encouraged to bring an edge list of interest. Sample data sets will be provided. To download the NodeXL add-in and slides, go to:
http://www.codeplex.com/NodeXL