NodeXL allows collections of vertices in a network to be gathered together into a “Group”.  Groups have several properties:

• groups can be selected
• vertices in selected groups can be operated on as a set
• groups can be collapsed or expanded
• network metrics can be calculated for each group
• groups can be plotted within bounded regions

NodeXL supports creating clusters or groups of vertices in several ways:  by attribute or manually, by component, or algorithmically.

Group menu commands are located in the Groups Menu in the NodeXL>Analysis Menu section.

Group menu commands include:

Group by Vertex Attribute

Users can also assign vertices to groups based on any attribute in the vertex worksheet.

The NodeXL>Analysis>Groups>Group by Vertex Attribute allows groups of vertices to be defined by any attribute.

These attributes can be numeric, or categorical:

Groups can also be authored manually.  A group is created whenever a new row is populated in the Groups worksheet.  A vertex is assigned to a group when it is named with its group in the Group Vertices worksheet.

Find Connected Components:

Each component can be assigned to its own group using the NodeXL>Analysis>Groups>Find Connected Components option.

Find Clusters – Automated Group Assignment Algorithms:

NodeXL exposes three of the clustering algorithms from the Stanford Network Analysis Platform (SNAP) (http://snap.stanford.edu) library for calculating network metrics from graphs. Working with SNAP author and Stanford Computer Science Professor Jure Leskovec, the NodeXL team integrated three clustering algorithms which can be selected from Cluster>Options:

the Wakita and Tsurumi “Finding Community Structure in Mega-scale SocialNetworks“ algorithm, the Girvan-Newman or Clauset-Newman-Moore algorithm.  When these Group findings algorithms are run each vertex is assigned to one of a set of groups based on its decision rules.  In general, these algorithms try to place collectons of densely connected vertices into separate groups or clusters.

NodeXL offers a range of shapes to represent vertices, including: circle, square, triangle, and diamond in filled and unfilled forms.

Vertices can also be represented as images.

Edges can also have styles:

Once you start creating and collecting network graphs you may find you can build a significant collection: hundreds, thousands or tens of thousands of graphs may result from a study or on-going monitoring project. In a series of features in the NodeXL project we have enabled a workflow for constructing many social media  network graphs using the Network Server component (see: How to schedule the creation of a network with NodeXL and Windows Task Scheduler and: New NodeXL Network Server (v1.0.1.126) – Frequently Asked Questions).  This can result in a collection of *many* NodeXL (and GraphML) network files.  Then we implemented features that enabled “Automation”, the application of many operations in NodeXL (metrics calculation, autofill columns, layout and more) to many files without direct human engagement (see: Automatic for the people (who use the latest NodeXL!). Release v.1.0.1.128 and: Fully automatic: NodeXL can build your network graphs hands free).

A single workbook may contain data from a single NodeXL data collection, run on a particular day and collecting data from a few hours or days back from that moment  (depending on factors like the volume of activity around the selected keyword and the depth of the twitter search catalog, which is often not more than a week or two long and much shorter for active topics).  An example of a single network slice is this recent map of the connections among people who mentioned “microsoft research” in Twitter on a single day (December 18th, 2010):

:

This is a single slice of the network, a day out of months of activity.  A still frame can tell a rich story: this is a picture of a crowd that has gathered to discuss a topic of common interest: “microsoft research“.  It illustrates a structure common to many large discussions of popular topics — a large set of isolates (the rows at the bottom) who were not observed to have a followed, mentions, or replies relationship to anyone else who tweeted the same term.  These are casual mentioners of the topic.  At the end of these rows are a small number of dyads, triads, and small components of a handful of people who link to one another but not to the largest connected component. These are pairs or small groups discussing the topic among themselves, but none are connected to a larger component.  Above these rows is the “giant component” — the blob of people who do have a connection to someone else who also tweeted a message containing the same term who in turn have a connection that leads to a large number of others.  The giant component is itself composed of several sub-components of densely connected groups.  At the center of each component are the core users, the people who often hold their cluster together. Between these clusters are the bridges, the people who link otherwise disconnected sub-groups.  At the edges are the peripheral people who have just taken the first step up from being an isolate and have formed a single reply, mention, or follows relationship to someone else who also tweeted the search keyword and can bridge them back to the core of the giant component.  This is a large and active network with hybrid qualities.  There is a “brand” or broadcast element in it: the yellow cluster is a hub and spoke structure centered on the Microsoft Research Twitter account.  These people re-tweet what this account publishes but do not connect to one another.  Just a few of these people set off second and third waves of retweets.  Elsewhere in the graph there are other network structures present, for example the green and blue clusters feature people are centered around their own discussions of the term “microsoft research“.

If you collect many still frames of slices of network activity there is great value in exploring the way the network graph changes over time.  In the most recent release NodeXL provides the first step in a series of features related to time and graph comparison.  You can now create a workbook that aggregates the overall metrics (edge counts, vertex counts, connected component counts, etc.) for a folder full of NodeXL workbooks. In NodeXL follow the menu path: NodeXL>Analysis>Graph Metrics>Aggregate Overall Metrics to get this:

The result of this feature is a workbook with a row containing the summary data from each of the workbooks in the target folder.  Any arbitrary collection of network workbooks can be aggregated but this is particularly useful when the workbooks are sequential time slices.

An example is the time series plot below tracking the rise and fall of several Twitter volume and network measures for the “microsoft research” search term over several months:

This chart tracks the number of vertices (each vertex in this case is a person  our data collector saw tweet about the search term “microsoft research“) in each (almost) daily network snapshot.  In addition the unique edges or connections between these Twitter users are plotted along with the number of people with no connections (“Single-Vertex Connected Components”).  The size of the largest component in the network (“Maximum vertices in a connected component “) is a measure of the changing size of the core community of discussion participants.  Measures like the maximum and average “geodesic” distance provide a rough measure of how long and thin (high values) or generally spherical (low values) a particular network is shaped. A “geodesic” is the longest path that can be walked through the network.  Long skinny networks may indicate the presence of loosely connected smaller groups that have a few people who act as bridges.  Low geodesic values suggest dense networks with people connected to many others with few isolates and sub-groups.

The peaks are closely associated with major events on the Microsoft Research calendar, like the 2010 Microsoft Research Faculty Summit event I attended in early July.

I find the ratios between measures of the size of the large network component and the population of isolates to be interesting.  As events go on over a period of days more people connect with others who are talking about the same topic, growing the size of the large connected component.  But often the isolate population also grows during this time as people at the periphery of the topic network catch sight of mentions of the event and tweet about it.  I could imagine one goal of social media management to be the conversion of isolates to connected component members.  Those who follow, reply or mention even a single other person also talking about a topic are more likely to return and engage than those who have zero connections.  It is not clear if more connections provide a linear increase in continued engagement, I suspect that the main effect is at the zero/one divide and drops off in effect after the first dozen or so connections.  Encouraging cohesion and network density by replying to isolates and encouraging others to do so may help keep a social media population focused and growing.

This feature follows the work done in the ManyNets project (http://www.cs.umd.edu/hcil/manynets/) at the University of Maryland by Manuel Freire, Catherine Plaisant, Ben Shneiderman, Awalin Sopan, and Miguel Rios.  ManyNets also created a framework for managing the metadata about collections of networks. ManyNets provides for  much richer interactions and linkages to the underlying networks than NodeXL can do so far.

As mobile devices become a major method for authoring and consuming social media, location data is increasingly a part of many posts, tweets, check-ins, and messages.  Many Twitter clients, for example, can add the user’s current latitude and longitude to the metadata associated with a tweet.  Other systems like Facebook Places, Google Latitude and Foursquare encourage users to declare  where they are to the world, often passing the information to other social media sites.

Using this location data in network analysis opens up a range of new opportunities.  Instead of a person – to – person social network, location data allows people to be linked to places and, by extension, places can be linked to other places based on the patterns of connection people create when located in a particular place.  A convergence of network analysis and Geographic Information Systems in underway.  A great example of this can be found in this wonderful video from the BBC which demonstrates the idea by mapping the flow of telephone calls, texts, and data around the UK and the wider world.

Link on the BBC

Even better is this video from the SensibleCity group at MIT:

Now, NodeXL (v.156) has the first of a series of features that will start to approximate the experience displayed in the video by supporting the import of location data about networks and plotting networks onto maps.

For now, we have started importing latitude and longitude data that Twitter makes available.  If you check “Add a Tweet column to the Vertices worksheet” in NodeXL, Data, Import, From Twitter Search Network or From Twitter User Network, the Twitter user’s geographical coordinates will be added to the Vertices worksheet when they are available.

Note that not every tweet has a latitude and longitude, in fact many do not (yet).  Further, note that not every latitude and longitude is accurate, many are not.

We need to implement more features for better location data support in a NodeXL workbook, but this is a start.  We are interested in exploring geospatial networks and Twitter is a great data source.  With this data in place we may look into features that emit KML files for exploration in other packages like Google Earth.  A nifty Google Earth/Spreadsheet importer can take small sets (400) of location data points in a spreadsheet and export them to a KML file, something we could implement in the future as well.  In addition we may be able to connect directly with services like Bing Maps and Google Maps to display connections between nodes with known locations.  Metrics that calculate the distance between nodes seem sensible as well.

Location coordinates are the key to a cornucopia of related data about a place.  Given a latitude and longitude it is possible to find the name of the city it is located in, look up data about that location in official records as well as resources like Wikipedia.  Income, education, property values, weather, photos, and more can be pulled together from just a simple lat/long.  All of these attributes could be used to cluster or illustrate the network visualization.

In this map nodes represent the major feature groups and functions in the NodeXL application.

This map will become the default file that will open when you run NodeXL for the first time.  You will see a dialog like this:

Select Yes to have the graph above imported into the workbook.  You can then display the graph using the Show Graph button in the NodeXL menu ribbon.

After that, it will be available via the help menus. When you import the file, all of the data is also available in the spreadsheet part of NodeXL so that you can experiment with changing values there to see the impact in the graph display after you hit the “refresh graphs” button.

Over the coming weeks we plan to release additional sample network data sets that illustrate key concepts and methods in network science.  Suggestions for sample networks are welcome!

Here are some NodeXL generated maps of recent  discussions around Wikileaks, pdfleaks, and cablegate in Twitter:

These are the connections among the people who tweeted the term “wikileaks” on 11 December 2010. The “wikileaks” account is at the center of the inner circle.  It is surrounded by a collection of very lightly interconnected people (there is a low clustering coefficient) who retweet messages from the wikileaks account.  This ring is surrounded by a thinner ring of 2nd order retweets.  This is a very common pattern for a successful “brand” or broadcast account, getting lots of retweets and, even more impressive, retweets of retweets.

The most between people who tweeted the term “wikileaks” on 11 December 2010.  The top ten: @wikileaks, @melhoresfrases, @ukelectionnews, @nonradical, @nelsonbocaranda, @linuxmall, @dominiofeminino, @resermarca, @crazy_dymnd, @rodolfomoros

Here is the data file: 20101209-NodeXL-Twitter-wikileaks

These are the connections among people who recently tweeted “pdfleaks” in Twitter on 11 December 2010.  It shares the brand/broadcast structure of the wikileaks account shown above.  After the wikileaks account itself, the second most between participant is @jeffjarvis who uniquely bridged to many people who would otherwise not be connected in this graph.

The most between people who tweeted the term pdfleaks“ on 11 December 2010.

Here is the data set: 20101211-NodeXL-Twitter-pdfleaks

The most between people who tweeted the term “cablegate” on 11 December 2010.  The cablegate discussion is more multipolar, with several high betweenness hubs connecting to otherwise separate communities.  El Pais, for example, is reaching a mostly Spanish language community.  The top ten: @anonops, @el_pais, @mparent77772, @wllawyer, @wikileaks_pp, @gdncables, @milenio, @evgenymorozov, @muschelschloss, @saladeprensa.

Here is the data set:

I have a MacBookPro.  I can run Windows very nicely using various Virtual Machine products like VMWare’s Fusion or the free and open VirtualBox.  In these virtual machines I can run Windows and Office and then NodeXL.  It’s pretty neat to see Windows inside the Mac OS window.  I am always amazed that it works at all.

But I try to avoid doing this as the main way I run NodeXL at almost any cost.  My Mac session slows to a crawl, my Windows session is slow, and the overall usability of the system degrades too much.  If you focus on JUST NodeXL in the VM it works well, but context switching is too demanding.  So I mostly focus on NodeXL on a Windows machine.  When I travel I usually just have my Mac laptop and miss having a zippy version of NodeXL at hand.

Recently, I discovered that Amazon EC2 offers a remarkable way to have my cake and eat it too: I created a modestly powered .micro instance of Windows XP and Office 2010 and installed NodeXL.  I then use the Microsoft Remote Desktop Client for Mac OS X to get a window into the remote virtual instance.  This system can be a bit slow, it responds a bit like a NetBook, but it does not slow down my Mac OS X instance and can be reached from any machine with the Remote Desktop client and internet access.  For those who need a more speedy response or to handle larger data sets, Amazon is happy to sell more powerful instances of Windows for modestly more pennies per hour.  For example, the .small instance is merely $0.13/hour or $3.12/day and offers much faster responses. Of course, if you turn the instance off when you are not using it, Amazon does not charge anything.

Recently Adam Fields, my colleague at Morningside-Analytics did me the great favor of recording the process of creating a new .micro instance of an Amazon EC2 Windows system.

aws_instance_creation
Following each step will lead you to the creation of your own virtual machine into which you can install Office and NodeXL.  You can then access this image, which runs continuously until you Terminate it – even when you close computer you use to remote desktop.  This is both a bug and a feature – the system runs no matter the state of the computer you use to access it.  Just remember to really turn off the instance if you do not want to pay a recurring fee! You have to actually _Terminate_ the instance to avoid paying fees shutting it down isn’t enough.

Also note that the AMI Adam used in the video was a 64-bit one, which doesn’t have a “small” option, it jumps straight from micro to large. If you want the small option, you have to use the 32-bit AMI, which is right above the one he picked.

On November 8th, 2010 I spoke in Seoul, Korea at the 3rd International Conference on Digital Culture on the topic of “Smart Society and Civic Culture“.

Several of my colleagues also spoke at the event, including Leslie Tkach-Kawasaki from the University of Tsukuba, Japan, Han Woo Park from YeoungNam University, Korea, George Barnett from the University of California, Davis and Robert Ackland from The Australian National University, AU.

Robert Ackland from The Australian National University, AU

George Barnett from the University of California, Davis

Han Woo Park from YeoungNam University, Korea

Leslie Tkach-Kawasaki from the University of Tsukuba, Japan

Marc Smith, Connected Action Consulting Group

Han Woo Park‘s research team from YeoungNam University, Korea

I spoke at the Social Tech 2010 conference in San Jose on October 26, 2010.

The event focused on social media technologies and practices in the business to business space.

I spoke on a panel about the application of social network analysis to social media to illustrate the ways SNA can illuminate the shape and key contributors to a social media crowd inside the firewall or on the public Internet.

Other notable speakers included: Thomas Lento from Facebook, Lawrence Liu from Cisco and Michael Wu from Lithium.  The panel was moderated by Tatyana Kanzaveli.

An example of applying social network analysis and visualization is this map of the connections among the people who recently tweeted “socialtech” on the 25th of September, 2010.

The top most between users in this map can be sorted in the spreadsheet, bringing the user accounts for @mprofsevents, @socialtech, @mikewise07, @jowyang, @marketingprofs, @meganleap, @socialmediaclub, @cotweet, @zuberance, @spiral16.

CONNECTING THE DOTS
A Network Visualization Symposium

http://networks.iq.harvard.edu/

Date: October 22, 2010
The first CONNECTING THE DOTS symposium on network visualization organized by Michael Barnett, Jukka-Pekka Onnela, and Samuel Arbesman of The Christakis Lab at Harvard was held on October 22, 2010.

Keynote speakers:

Alessandro Vespignani, Professor of Informatics and Computing, Indiana University, Bloomington
Ben Fry, co-developer of Processing and data visualization expert

The NodeXL project will be represented by Ben Shneiderman who will speak at 2:00 PM on:

Analyzing Social Media Networks with NodeXL

Complete draft program for Connecting_the_Dots…

Program

10:15 AM Keynote I: Ben Fry (Fathom)

11:00 AM SocioPatterns
Wouter van Der Broeck, (ISI Foundation)

The SocioPatterns project developed an active-RFID based sociometric platform capable of sensing mutual proximity in a distributed, scalable fashion. This platform enables a large-scale, real-time assessment of the dynamics of person-to-person interaction network with a fine temporal and spatial granularity. The initial focus of this project was on empirical data collection in support of research on dynamic social interaction networks and infection dynamics. Various experiments also explored the potential of the platform for more service oriented applications, and artscience contexts. The SocioPattens platform has been deployed in more than a dozen settings, ranging from conferences and tradeshows, over primary schools, long-term public exhibitions, artscience projects, and domestic settings.

11:25 AM Visualizing Countrywide Traffic Dynamics
Wen Dong (MIT)

It is often a challenge to reveal the human mobility patterns in multi-gigabyte vehicle tracking data spanning a whole country and many years. We address this problem through a Markov model of how people drive in the road network, and animations of road-traffic conditions on top of Google Maps. We welcome researchers to use our visualization tool set.

12:50 PM Keynote II: Alessandro Vespignani (Indiana University)

1:35 PM Network Visualization Using Processing
Sergiy Nesterko (Harvard University)

Network, or relational data are becoming of increasing interest to researchers in different disciplines, partly because related problems are very hard to solve due to complex structure of underlying datasets. It is well known that a picture is worth a thousand words, and especially for networks related problems, visualization has become an invaluable tool in gaining intuition that helps tackle them. As an example, we consider network visualization using Processing as applied to problems in respondent-driven sampling, which is a variant of link tracing process to explore social networks of hard-to-reach populations. We emphasize the role of a compelling visualization in a successful research endeavor and briefly describe how Processing can be used to make it.

2:00 PM Analyzing Social Media Networks with NodeXL
Ben Shneiderman (Univ. of Maryland, College Park)

NodeXL is the free and open add-in for Excel 2007/2010 that supports network overview, discovery and exploration. Supported by Microsoft External Research for 3+ years, this effort has produced a game-changing software tool that enables students, researchers, and professional community managers to extract and download networks from email, Twitter, YouTube, Flickr, WWW, etc. Then they can compute network analytic metrics, filter out less relevant features, apply multiple layout algorithms, and produce compelling yet comprehensible displays that reveal actionable insights about complex social processes.

2:25 PM FacetAtlas: Multifaceted Visualization for Rich-context Networks
Yu-Ru Lin (Harvard University and Northeastern University)

FacetAtlas is a multifaceted visualization technique for analyzing interconnected text documents. FacetAtlas combines search technology with advanced visual analytical tools to convey both global and local patterns simultaneously. The power of FacetAtlas is demonstrated through a case study that targets patient education in the health care domain.

3:35 PM Pathline: A Tool for Exploring Biological Data in the Context of Molecular Networks

Miriah Meyer (Harvard University)

Pathline is an interactive tool that visualizes the biologically interesting relationships between multiple molecular networks, the activity levels of multiple genes, and the evolutionary relatedness of multiple species. Working in close collaboration with a group of genomics scientists we designed Pathline around a representation of a linearized network that provides appropriate topological information and supports the comparison of quantitative data. We validated the efficacy of Pathline to both confirm existing knowledge and discover new scientific insights through a series of case studies.

4:00 PM Is Network Visualization Even Useful?
John Dila (Innocentive)

Our company, InnoCentive Inc. (innocentive.com) is an Open Innovation marketplace, which connects Seeker companies (that have tractable problems they have not been able to solve) with our Global Solver Community of experts, specialists, and generalists who sign up to solve tough, meaningful problems. As part of a website redesign, we wanted to update the way we graphically display data about our global Solver network. By “eating our own dog-food” we posted a challenge on our platform asking for solutions to how we can best visualize a selected data set about our global Solver population in order to drive traffic to our site and add value to customers and the business. We received 10 solutions and they are currently under evaluation. We will present our challenge process, the results of the visualization challenge, and how we plan to use them moving forward.

I spoke at the University of Michigan, School of Information on October 19th, 2010 about “Charting Collections of Connections in Social Media: Mapping and Measuring Social Media Networks to Find Key Positions and Structures“.

I demonstrated the ways social network data sets can be extracted from social media services like Facebook, Twitter, email, YouTube, and flickr.  These network graphs can reveal information about the “shape” of the population in terms of the presence of sub-groups and communities within the larger population.  In addition, each individual participant is located or positioned within the graph, helping to identify the people who are “core” versus those who are peripheral, as well as those who occupy the position of “bridge” between two otherwise separate groups.

Yahoo! Speaker Series

The Yahoo Speaker Series at the School of Information supports distinguished guest lecturers from the fields of information and technology.

2010 Speaker: Marc Smith
VIDEO: Marc Smith

Marc Smith, chief social scientist with the Connected Action Social Group, presented “Charting Collections of Connections in Social Media: Mapping and Measuring Social Media Networks to Find Key Positions and Structures,” on Tuesday, Oct. 19, 2010.

Smith’s talk was sponsored by the Yahoo! Speaker Series, Michigan Interactive & Social Computing, and the School of Information.

In this talk, Smith described how networks are a data structure common across all social media systems — systems defined by enabling populations to author collections of connections. The Social Media Research Foundation‘s NodeXL project makes analysis of social media networks accessible to most users of the Excel spreadsheet application. Using NodeXL, networks become as easy to create and analyze as pie charts. Applying the tool to a range of social media networks has already revealed the variations present in online social spaces. A review of the tool and images of Twitter, Flickr, YouTube, and e-mail networks will be presented, illustrating different patterns created when communities, brands, and controversies are discussed.

I am coming to understand what is interesting in any particular network visualization by looking at many network visualizations.

Using NodeXL, I have made several maps of  social media networks of people talking about several topics of interest from current events to conferences I attend.  You can find a collection of them on flickr.

I look at these images and look for differences in the number of big clusters: some images have a “double yolk” – that I propose is a necessary (but not necessarily sufficient) condition of defining a topic to be “controversial”.  These two cluster networks have two well defined populations who lack much if any connection across the divide to the “other” side.

Some networks are highly populated but sparse, these are often the networks that form around brands where a central account tweets and is retweeted by many.  But these many lack much connection to one another.  So these brands form broadcast networks, not communities.

Some networks are dense single clusters with few if any “isolates”.  Isolates are people who say a term, and thus appear in the graph, but have no connections (follows, replies, ore mentions) to anyone else in the graph (at least as observed and reported by twitter at that time).  These dense clusters without isolates are topics where everyone is in-group.  Examples, like “scrm”, are technical and business terms that identify medium sized populations with high levels of density.

Have a look and see what patterns you can find.