Mapping mobile social networks with NodeXL: finding key users, groups, and locations

The video is now available:

The video reviews the creation of maps like this:

That illustrate the connections among people who tweet the term “#ecomm2010″, scaled by the number of followers.

Abstract: Social network analysis (SNA) is a powerful method for gaining insight into the massive collections of connections created when many people connect to one another through mobile devices. SNA has been widely applied to desktop social media and is moving into the mobile world. Prominent studies of the “call graph” have been produced at national scales.

Mobile providers are applying SNA to identify key subscribers who can reduce churn and help gain adoption of new services and products. Network analysis has historically had a steep learning curve, but now new tools are making SNA easier for less technical users. This talk will describe social network concepts and their application to mobile data sets. A free and open add-in for the popular Excel 2007 spreadsheet called NodeXL (http://www.codeplex.com/nodexl) can perform many complex SNA tasks like data import, scrubbing, metrics calculation, clustering, and visualization. Applying this tool to call graph and subscriber data sets can reveal key positions in the network that can attract and hold other subscribers in the system.

Examples of network analysis of social media and mobile data sets can be found on the Connected Action blog (http://www.connectedaction.net).

NodeXL has a number of data importers that can create a network of connections from social media data sources like Twitter, YouTube, flickr, email, and the WWW (along with a number of other data import formats like GraphML, UCINet, CSV, and other Excel workbooks with data).

To create a network you just select the search terms and configurations you want from the NodeXL>Data>Import menu.

If you want to create the same network every day (or at any schedule), a recent feature (since version .125) of NodeXL can help. NodeXLNetworkServer.exe is an application that ships with NodeXL along with a sample configuration file called SampleNetworkConfiguration.xml. By editing the configuration file you can set NodeXL to collect anything available in the menu through Excel.  So far we have exposed the two Twitter data collectors (more on the way) so the configuration file asks you to select a search term or a user’s name, the size of the network and the details you want reported along with the location and name of the destination file that NodeXL will create.  Answer these questions by editing the config file and save it with a useful name that includes the search term.

Step by step details after the jump:

Editing the Configuration file

NodeXL ships with a SampleConfigurationFile.xml that you should copy, rename and edit.

You may want to create a directory to hold this configuration file if you expect that there will be many files.

The configuration file is written in XML but it it is very human readable and editable.  Here are the key elements of the file that depend on you to choose and configure the file appropriately.

NetworkType Specifies the type of network to get.  Must be one of the following values: TwitterSearch or TwitterUser

After setting NetworkType, you must also edit one of the following sections:

TwitterSearchNetworkConfiguration

TwitterUserNetworkConfiguration

TwitterSearchNetworkConfiguration is used only if NetworkType is TwitterSearch.

SearchTerm - What to search for.

WhatToInclude - What to include in the network.  This must be a combination of the following values, separated by commas:

Statuses - Include each person’s status (tweet).

Statistics - Include each person’s statistics.

FollowedEdges - Include an edge for each followed relationship.

RepliesToEdges - Include an edge from person A to person B if person A’s tweet is a reply to person B.

MentionsEdges - Include an edge from person A to person B if person A’s tweet mentions person B.

<WhatToInclude>Statuses,Statistics,FollowedEdges,RepliesToEdges,MentionsEdges</WhatToInclude>

MaximumPeoplePerRequest - The maximum number of people to request for each query, or leave empty for no limit.

NetworkFileFolder - Full path to the folder where the network files should be stored.

NetworkFileFormats - Specifies the file formats to save the network to.  This must be a combination of the following values, separated by commas:

GraphML - Save the network to a GraphML file, which can be imported into a NodeXL workbook.

NodeXLWorkbook - Save the network directly to a NodeXL workbook.  To use this option, the NodeXL Excel Template must be installed on this computer.

<NetworkFileFormats>GraphML,NodeXLWorkbook</NetworkFileFormats>

AutomateNodeXLWorkbook - Specifies whether the NodeXL Excel Template’s automate feature should be run on the workbook.  Must be true or false.  This is used only if NetworkFileFormats (above) includes NodeXLWorkbook.

If true, the automate options you most recently set in the NodeXL Excel Template are used to automate the workbook.  To set the automate options, do the following:

1. Open the NodeXL Excel Template.

2. In the Excel ribbon, Go to NodeXL, Graph, Automate.

Note that the “On this workbook” and “On every NodeXL workbook in this folder” selection in the Automate dialog box is ignored when automating the workbook from the NodeXL Network Server.

<AutomateNodeXLWorkbook>true</AutomateNodeXLWorkbook>

TwitterUserNetworkConfiguration

This section is used only if NetworkType is TwitterUser.

ScreenNameToAnalyze - The screen name of the Twitter user whose network should be analyzed.

WhatToInclude - What to include in the network.  This must be a combination of the following values, separated by commas:

FollowedVertices – Include a vertex for each person followed by the user.

FollowerVertices - Include a vertex for each person following the user.

LatestStatuses - Include each person’s latest status (tweet).

FollowedFollowerEdges - Include an edge for each followed relationship if FollowedVertices is specified, and include an edge for each follower relationship if FollowerVertices is specified,

RepliesToEdges - Include an edge from person A to person B if person A’s latest tweet is a reply to person B.

MentionsEdges - Include an edge from person A to person B if person A’s latest tweet mentions person B.

<WhatToInclude>FollowedVertices,FollowerVertices,LatestStatuses,FollowedFollowerEdges,RepliesToEdges,MentionsEdges</WhatToInclude>

NetworkLevel - Network level to include.  Must be One, OnePointFive, or Two.

MaximumPeoplePerRequest - The maximum number of people to request for each query, or leave empty for no limit.

NetworkFileFolder - Full path to the folder where the network files should be stored.

NetworkFileFormats -Specifies the file formats to save the network to.  This must be a combination of the following values, separated by commas:

GraphML -Save the network to a GraphML file, which can be imported into a NodeXL workbook.

NodeXLWorkbook - Save the network directly to a NodeXL workbook.  To use this option, the NodeXL Excel Template must be installed on this computer.

AutomateNodeXLWorkbook

Specifies whether the NodeXL Excel Template’s automate feature should be run on the workbook.  Must be true or false.  This is used only if NetworkFileFormats (above) includes NodeXLWorkbook. If true, the automate options you most recently set in the NodeXL Excel Template are used to automate the workbook.  To set the automate options, do the following:

1. Open the NodeXL Excel Template.

2. In the Excel ribbon, Go to NodeXL, Graph, Automate.

Note that the “On this workbook” and “On every NodeXL workbook in this folder” selection in the Automate dialog box is ignored when automating the workbook from the NodeXL Network Server.

One key configuration is this last choice to turn on the automated processing of the resulting data set.  If you set this to true, NodeXL constructs a Twitter social network and then performs all of the steps of automation that you define.  For example, NodeXL can calculate graph metrics, find clusters, create subgraphs, map a set of autofill column mappings of data to display attributes, set graph layouts and settings and render a graph without any human intervention.

Once you have a properly edited configuration file you can simply open a command line session (go to the Start menu, type in “CMD”) and type the following:

> NodeXLNetworkServer.exe SampleNetworkConfiguration.xml

And you will get a stream of messages about what parts of the network are being actively collected:

This is a simple example and not too useful: you may as well just do this through the NodeXL Excel interface.  But things get more interesting when one more piece is added: Windows Task Scheduler.  You may not see it that often but Task Scheduler is on almost every Windows desktop and can be used to automate the collection of NodeXL data sets from Twitter and other sources of social media networks.

To get to Task Scheduler just type its name into the Start Menu search box.  It should look like this:

Using this tool you can create new Tasks that will execute on a specific time and frequency.

Create and select a new folder (call it something like “NodeXL Data Collections”) under Task Scheduler Library to hold all your NodeXL data collections separate from other scheduled tasks.

Use the Create Task menu item on the far right.

You will get a Create a task dialog box:

Enter a Name and a Description that captures the search terms of your query and then select the “Actions” tab.

Actions are where the command that you want to execute is defined.  Select “New…” to create a new Action.

This will open a dialog box in which you can define the program to be run along with any settings.

Enter the complete path to the NodeXLNetworkServer.exe application in the “Program/script” field.

Add the complete path to the configuration file in the “Add arguments” field.

To easily capture the complete path (and the quotation marks needed if there are spaces in the path) hold the SHIFT key down while right clicking the name of the configuration file you want to schedule for collection.  You should see an option “Copy as Path” which will place the needed information into the clipboard.  After selecting “Copy as Path”, return to the “New Action” dialog box and paste the path into the “Add arguments” field.

Once the path to the NodeXLNetworkServer.exe file and the path to the configuration file have been entered into the New Action dialog, shift to the Triggers tab of the Create Task dialog.  Select “New…” to create a new trigger.

Triggers are defined by many things, but we will focus on time.  When the New Trigger dialog is set to “On a schedule” you can choose the time and frequency to run the collection.  If you want to run a collection daily at 7:01:30AM each day the following settings should work:

Once the time and recurrence are set select OK and you have a new task!  You may want to create a task that starts in a minute or two to test that the event fires properly.  You can then schedule these collections to run at an hour when you will not be bothered by the interruption.  Multiple collections can be scheduled but several limits suggest that only a few can run simultaneously.  You may want to start collections only a few times a day or hour to allow one collection to complete before others begin.  When these tasks do execute you should see a collection session appear in a console window and report updates as it steps through the many stages of constructing a network dataset.

Collection is much faster if you have a rate limit lifted account, which you must request from Twitter.  With a credentialed rate-limit lifted account you can perform several queries per hour.  With a regular account with credentials (your Twitter login) you can get one or two queries per day depending on the size of the data collected.  In either case it is possible to reach the limit that Twitter will provide.  When that happens NodeXL will pause the collection and wait until the API query budget refreshes and Twitter is willing to serve more query results.  As a result, even accounts without rate-limits lifting can create large complex social media network maps, although at a much slower rate.

I spoke at the 2010 Catalyst Conference in San Diego on July 29th. The conference hashtag is #CAT10.

The slides are here:

A few days before the conference started the #CAT10 twitter social network map looked like this:

26 July 2010 NodeXL Twitter map of the connections among people who tweet “#CAT10″ the hashtag for this year’s Catalyst conference.

This is the list of the most “between” contributors in the #CAT10 Twitter graph on July 26, 2010.

A few days later, as people began to arrive at the conference, the graph became far more dense and populous.

The network of #CAT10 mentioning users in Twitter has become much more dense, with more people and more connections among them as people reply, retweet, follow, and mention one another.

While the core people in this list are similar to the list generated a few days earlier, several people have shifted position.

Filtering the graph, we can remove all but the most between people to reveal the core members of the community.

These people are likely to play an influential role in the #CAT10 community.

http://jossviz.wordpress.com/2010/06/23/friendwheel-layout-of-a-facebook-network/

The Journal of Social Structure (JoSS) is an electronic journal of the International Network for Social Network Analysis (INSNA).  Here is Bernie’s description of the graph.

This is a “pinwheel” diagram using the author’s Facebook personal network (captured July 15, 2009).

Nodes represent the author’s friends and links represent friendships among them. The author is not shown. Each ‘wing’ radiating outwards is a partition using a greedy community detection algorithm (Wakita and Tsurumi, 2007). Wings are manually labelled. Node ordering within each wing is based on degree. Node color and size is also based on degree. Nodes position is based on a polar coordinate system: each node is on an equal angle of n/360º with a radius being a log-scaled measure of betweenness. Higher values are closer to the center indicating a sort of cross-partition ‘gravity’.

This layout has several notable features:

- The angle of each wing is proportionate to its share of the network. Thus 25 percent of nodes go from 0 to 90º.

- Partitions are distinguished by their position rather than a node’s color or shape.

- The tail indicates the periphery of each partition. A wing with many tail nodes indicates many people who are only tied to other group members.

- Edges crossing the center show between-partition connections. Since nodes are sorted by degree it is easy to see if edges originate from the most highly connected nodes or the entire partition.

Bernie’s chapter on analyzing Facebook networks with NodeXL appears in the book: Analyzing Social Media Networks with NodeXL: Insights from a connected world.

There is a Stanford Media X event on July 22nd, 2010 on new tools for SNA:

Network Analysis Made Easy:  Using NodeXL To Map Social Media Networks

http://mediax.stanford.edu/WSI/marc.html

Bring a laptop (running Windows and Office 2007 or 2010) to this workshop and you can be analyzing a social media network from systems like Twitter, flickr, YouTube and your own email by the end of the day.  If you can make a pie-chart in Excel, using the free and open NodeXL (http://nodexl.codeplex.com) you can now make a rich network graph from data extracted from social media systems and other common formats.  If you have a network, bring it, if not you can bring a suggested topic that we can map during the course of the day.

Even if you leave your laptop behind or have a Mac (sorry, no version is yet available for MacOS – unless you have a virtual machine with Windows and Office) this workshop will introduce the core concepts of network science with application to social networks in general and social media networks in particular. Applied to a range of topics and services, social media network maps can illuminate a variety of “publics” – populations who share a common interest and may share connections.  Maps of topics like “oil spill”, “global warming” and other issue and event related keywords can reveal the groups and factions that cluster around different concepts and terms.  Key contributors in these maps can be identified through the application of network measurements that capture various aspects of a  person’s location in a network graph.

The 2010 Microsoft Research Faculty Summit was held July 12 and 13 in Redmond, Washington.  Among the many panels and discussions related to the state of computer science the NodeXL team had several representatives talking about the ways network science education can be expanded using an easy to use application for network analysis built on Excel.

Jimmy Lin from the University of Maryland also attended to speak about programming in the cloud.

Here is the abstract for the NodeXL talk:

NodeXL – Social Network Analysis in Excel—Natasa Milic Frayling, Microsoft Research; Ben Shneiderman, University of Maryland; Marc Smith, Connected Action

Businesses, entrepreneurs, individuals, and government agencies alike are looking to social network analysis (SNA) tools for insight into trends, connections, and fluctuations in social media. Microsoft’s NodeXL is a free, open-source SNA plug-in for use with Excel. It provides instant graphical representation of relationships of complex networked data. But it goes further than other SNA tools—NodeXL was developed by a multidisciplinary team of experts that bring together information studies, computer science, sociology, human-computer interaction, and over 20 years of visual analytic theory and information visualization into a simple tool anyone can use. This makes NodeXL of interest not only to end-users but also to researchers and students studying visual and network analytics and their application in the real world. NodeXL has the unique feature that it imports networks from Outlook email, Twitter, flickr, YouTube, WWW, and other sources, plus it offers a rich set of metrics, layouts, and clustering algorithms. This talk will describe NodeXL and our efforts to start the Social Media Research Foundation.

Some photos from the event:

Saul Greenberg

Ben Shneiderman and Andy van Dam

Ben Shneiderman, Natasa Milic-Frayling and Marc Smith

Tom McMail and Marc Smith

The NodeXL team has just released a new version (v.1.0.1.128) that contains a new “Automation” feature that allows users to define a collection of operations to perform on their network graphs and invoke the complete set in a single button click AND reuse that configuration on other workbook graphs.  In fact, the feature will apply the configuration you define to all the files you specify, allowing easy processing of large collections of network data sets.

This week the feature is partially complete.  Users can invoke the merge duplicate edges, calculate graph metrics, auto-fill columns, create sub-graph images, find clusters and show graph.  These operations can require as many as dozens of clicks when performed manually.  If you have dozens or hundreds of network data sets the result is a daunting case of repetitive strain injury and carpal tunnel syndrome.  Instead, with automation, these operations can be carried out orders of magnitude more frequently without much pain!

The next release will feature the complete package which will then include control over the layout and graph options.  As a result, automatically generated network visualizations can be produced in a pipeline: users will be able to specify a query using the NodeXL desktop network data collector and then automate the processing of  large collections of data sets.

The result should be better analysis of time series data sets that have many “slices”.  The feature points the way to additional development work for supporting the comparison between networks to evaluate their evolution.

A new paper on visualizing social media has been released on the University of Maryland, Human Computer Interaction Laboratory tech report archive.  Co-authored by Derek Hansen,  myself, and Ben Shneiderman, the paper describes and visualizes the patterns of connections formed when people tweet about events like conferences and news stories.

EventGraphs_2010_HCIL_Tech_Report

http://www.cs.umd.edu/localphp/hcil/tech-reports-search.php?number=2010-13

Hansen, D., Smith, M., Shneiderman, B.
EventGraphs: Charting Collections of Conference Connections
HCIL-2010-13

EventGraphs are social media network diagrams constructed from content selected by its association with time-bounded events, such as conferences. Many conferences now communicate a common “hashtag” or keyword to identify messages related to the event. EventGraphs help make sense of the collections of connections that form when people follow, reply or mention one another and a keyword. This paper defines EventGraphs, characterizes different types, and shows how the social media network analysis add-in NodeXL supports their creation and analysis. The paper also identifies the structural and conversational patterns to look for and highlight in EventGraphs and provides design ideas for their improvement.

The International Sunbelt Social Network Conference is the official conference of the International Network for Social Network Analysis (INSNA).

This year’s INSNA “Sunbelt” conference is at the  Riva del Garda Fierecongressi, Trento, Italy!  Here is the 2010 INSNA Sunbelt Program.

This is the NodeXL map of connections among people who tweeted the hashtag used for the conference “#sunbelt”.

Having now seen several of these maps for other topics and events (see: http://www.flickr.com/photos/marc_smith/sets/72157622437066929/) this map can be placed in context.  It is a small group, but has a high density of connections.  It lacks isolates, the people who say the term but do not connect to others who say that term.  This means that this is a very “in-group” population: if you know to use the #sunbelt hashtag, you probably connect to someone else who uses the term.  It is a single major cluster of connected people, no obvious sub-graphs or clusters are visible.  Not everyone is central in the graph, and those who are have a prominent role in the network science community.  Here is the top ten list of #sunbelt mentioning twitter users ranked by betweeness centrality.

miriamnotten
barrywellman
memeticbrand
isidromj
drewconway
gephi
kristtina
danevans87
valdiskrebs
ciro

Pierre has been a deep student of telecommunications policy regulation in the United States for many years.  He has generated a remarkable network map built from the details of filings to the FCC over more than a decade.  These filings are made by companies when they agree or disagree with a proposed policy.  When two companies file in support (or opposition) to the same policy they create a tie between them.  The collection of these connections creates a complex network of coalitions and factions.

http://www.cmu.edu/joss/content/issues/2010jossviz/5_deVries.htm

“The graph is derived from meta-data associated with documents that are filed electronically whenever an organization interacts with the FCC, in accordance with the Administrative Procedures Act. Whenever a letter, comment or other document is filed, the filer provides information on the parties involved, number of pages, relevant proceedings, date, etc.”

…

“Once the data is cleaned up, an edge list is created in Excel by running another VBA macro. A graph is created from this list with NodeXL, a social network analysis and visualization add-in for Excel 2007. NodeXL’s Fruchterman-Reingold algorithm is used to prepare a preliminary layout; nodes are then moved by hand into visually intelligible positions, respecting the clusters suggested by NodeXL’s implementation of the Wakita-Tsurumi algorithm. Nodes are colored on the basis of eigenvector centrality. The degree of investment that organizations make in lobbying is measured by the total number of filings it made in this proceeding over the period of study, and reflected in the size of the node. This information is obtained by running another VBA macro against the underlying ECFS metadata, and then matching that to the vertices in the graph.”

Read more about this industry network at JoSS.

2010 – June – NodeXL Book Flyer.

[Chapter 10]: Twitter: Conversation, Entertainment and Information, All in One Network!

By Vladimir Barash and Scott Golder

[Chapter 11]: Visualizing and Interpreting Facebook Networks

By Bernie Hogan

[Chapter 12]: WWW Hyperlink Networks

By Robert Ackland

[Chapter 13]: Flickr: Linking People, Photos, and Tags

By Eduarda Mendes Rodrigues and Natasa Milic-Frayling

[Chapter 14]: YouTube: Contrasting Patterns of Interaction and Prominence

By Dana Rotman and Jennifer Golbeck

[Chapter 15]: Wiki Networks: Networks of Creativity and Collaboration

By Howard T Welser, Patrick Underwood, Dan Cosley, Derek Hansen, and Laura Black

This handy poster contains many details about the book contributors, chapters, and the book cover (which you can also see below):

Analyzing Social Media Networks with NodeXL: Insights from a Connected World

The NodeXL team has released a new version (v.1.0.1.126) with better support for collecting data from social media network sources, starting with Twitter.  The NodeXL Network Server program now ships in every NodeXL installation.  Tony, the lead developer on the team, created the following FAQ to explain how to use the collector application.

This document describes how the NodeXL Network Server works.

• What is the NodeXL Network Server?

It’s a Windows command-line program that downloads a network from Twitter and stores the network on disk in several file formats.  It can be run directly from a command line, but is typically scheduled to run on a periodic basis via the Task Scheduler that is built into Windows.

• Where can the files be found?

The files are in NodeXL’s program folder.  To find out where the folder is, right-click the Microsoft NodeXL, Excel 2007 Template menu item in the Windows Start menu, then select Properties.  On 32-bit English computers, the folder is “C:\Program Files\Microsoft Research\Microsoft NodeXL Excel Template.”

• Who are its intended users?

The Server is meant for use by people with moderate system administration skills.  It is not difficult to use, but it is not intended for the same audience as the NodeXL Excel Template, where ease of use is of high priority.

• How do you run the Server from the Windows command line?

Like this:

NodeXLNetworkServer.exe NetworkConfiguration.xml

The program takes a single argument, which is the path to a configuration file that specifies which network should be downloaded and how the network should be saved to disk.  A particular configuration file might specify “Get the Twitter search network for people whose tweets contain ‘Sociology,’ add an edge for each ‘mentions’ relationship, limit to 100 people, include tweets, include statistics, and store the network as a GraphML file in the C:\NodeXLNetworks folder.”

The program immediately gets the requested network, saves it to disk, and exits.  On its own, it does not run on a periodic basis.

• How do I create a configuration file?

You create a configuration file by copying a provided template file and editing the copy in Notepad.  The template file is named SampleNetworkConfiguration.xml and is stored in the same folder as the program.  The file is in XML format and the XML tags are clearly named and documented.

• In what file formats can be the network be saved to disk?

You can save the network to either GraphML, which can be imported into a NodeXL workbook; directly to a NodeXL workbook; or both.

• Do you typically run the program from the command line?

No.  Instead, you typically run it as a scheduled task via a built-in Windows program called Task Scheduler

Task Scheduler is a powerful utility that lets your run any program, including NodeXL Network Server, on a periodic basis.  You can, for example, tell Task Scheduler to run NodeXL Network Server using a particular network configuration file every twelve hours starting June 1, 2010 and ending June 30, 2010; or once a week starting now and continuing forever.  The scheduling options are endless.

• Why not just include scheduling features in the NodeXL Network Server?

For two reasons.  First, Task Scheduler’s extensive scheduling options would be difficult to duplicate.  Second, if NodeXL Network Server had to download a network on a periodic basis, it would have to run as a Windows service, and Windows services are more complex to implement and to use than a simple command-line program.

• How are the network files named?

Scheduling the NodeXL Network Server to run periodically can create any number of network files in the specified directory, so a file-naming scheme is needed.  The file name format is

{NetworkConfigFileName}_{Date}_{Time}.{Extension}.

So the above example, in which NetworkConfiguration.xml specifies that networks are to be saved as GraphML, might create a set of network files that look like this:

NetworkConfiguration_2010-06-01_02-00-00.graphml
NetworkConfiguration_2010-06-01_14-00-00. graphml
NetworkConfiguration_2010-06-02_02-00-00. graphml
…

• What happens if the computer is not turned on at the scheduled time?

By default, the task won’t be performed until the next scheduled time when the computer is turned on.  However, if the computer is sleeping, you can tell Task Scheduler to wake it at the scheduled time to run the task.

• What happens if the NodeXL Network Server encounters an error?

If the error prevents the network from being downloaded, the NodeXL Network Server creates an error file instead of a network file.  The file name starts with “Error” to make it easy to spot:

Error_NetworkConfiguration_2010-06-02_14-00-00.txt

The error file contains the details of what went wrong.

If one or more errors block part of the network but other parts of the network are successfully downloaded, then the NodeXL Network Server creates the network file containing the partial network, along with a text file that explains how many errors occurred.  The text file name starts with “PartialNetworkInfo” to make it easy to spot:

NetworkConfiguration_2010-06-02_14-00-00.Graphml
PartialNetworkInfo_NetworkConfiguration_Date.txt

• What if I want to periodically download more than one network?

Simply schedule more than one task, each using a different network configuration file.  The tasks are independent of one another and can be scheduled to run at different times.

The Enterprise 2.0 conference is about to get underway in Boston. The event focuses on all the ways social media tools that are familiar on the consumer Internet are making their way behind the firewall in many enterprises and institutions. Why can’t you “friend” a colleague or “like” a spreadsheet or slide deck? Employees often come to their jobs expecting tools that resemble the social media tools with which they already spend much of their time.

Like many conferences, this one has a hashtag, actually two that I know of: #e2 and #e2conf. There is already a good deal of activity leading up to the event. Here is a map of connections among a group of people who mentioned either #e2 or #e2conf in the last few days.

In this map there are 532 Vertices and 9,395 Unique Edges, creating 13 Connected Components, 11 of which had only a Single-Vertex, the largest component had 519 vertices which were interconnected 9,393 times.  The small number of isolated components indicates that this is a cohesive community of highly connected participants.  These people know and follow, reply and mention one another.  The Graph had a Density of 0.03 and the Maximum Geodesic Distance (Diameter) was 5 steps with an Average Geodesic Distance of 2.

Within this mass of connected users is a core group of highly “between” people, those who most broadly span connections within the population. These are one possible set of “influentials” within the Enterprise 2.0 community.

Here is a two screen view of the list of the top most between #e2 OR #e2conf mentioning twitter users along with the overview graph of their internal linkages.

A closer look at the graph alone can reveal enough detail to read the names of these central participants.

This is a view of the list of authors sorted in Excel by their “Betweenness centrality” score, the measure of how much these people “bridge” across the network.

An alternative view plots these contributors in an X/Y space based on their count of followers (along the x axis) and count of tweets (along the y axis).

The top 15 are:

dhinchcliffe
jowyang
rlavigne42
enterprise20
jumpersearch
marciamarcia
itsinsider
cmswatch
philcampos
rwang0
dankeldsen
lliu
juliancaparaz
lehawes
sameerpatel

Twitter users who mentioned #e2 or #e2conf on June 13, 2010 scaled by number of followers, x = log(followers), y = log(tweets).

There is a correlation between tweets and followers, but not everyone converts tweets to followers at the same rate. Below the diagonal are those who over convert tweets to followers, those above the diagonal under convert tweets to followers.

The book, Analyzing social media networks with NodeXL: Insights from a connected world, is forthcoming summer 2010 from Morgan Kaufmann and from Amazon.

Hello!  If you would like to request a custom social media network map made with NodeXL, complete the form below.  I will generate the maps as requests come in and email you a pointer to the results which I will post to my flickr feed here: http://www.flickr.com/photos/marc_smith/sets/72157622437066929/

Here is a sample map for the term OCU2010:

http://www.cs.umd.edu/hcil/soh/

The HCIL Government Applications of Social Media Networks &
Communities Workshop, as part of the 27th Annual Human Computer Interaction Lab (HCIL) Symposium, at the University of Maryland, examined how social media can be systematically applied to increase civic participation on national priorities.

When: Friday, May 28, 2010, 9:30am-4:00pm
Where: CSIC Building, UMD, College Park
Who: Government thought leaders, system developers, and agencies; industry partners, researchers, and students

Photo by Ben Shneiderman

Front row (left to right): Brad Hesse, Betsy Rebert, Claudia Louis,
Vladimir Barash, Derek Hansen, Robin Naughton.
Middle row: Scot Golder, Rex Robison, Yan Qu, Joe Pringle, Natasa
Milic-Frayling, Amanda Shanor, Leonard Lidov, Laura Milner
Back row: Robert Altiero, Mark Edson, Keith Walker, Tim Clausner, Marc
Smith, Nick Violi, Brian Dennis, Manuel Freire, John Bertot, Derrick
Cogburn, Jennifer Preece, Francy Stilwell

Not pictured: Ben Shneiderman

The Friday, May 28th all day event focused on the use of social media data in improving the quality of government.

May 28th, 2010: Government Applications of Social Media Networks and Communities
Derek Hansen, Marc Smith, Jenny Preece, Ben Shneiderman
Federal, state, and local governments are discovering interesting and ambitious ways that social media can be used to increase civic participation in decision-making, health-care /wellness, energy sustainability, education, disaster response, community safety, scientific research, etc. This workshop will invite attendees to present current projects, design strategies, evaluation methods, and analytic tools. Issues such as universal accessibility & usability, privacy protection, and reliability will be discussed.

Attendees …
• discussed interesting and ambitious ways that federal, state, and local governments are using social media in decision-making,healthcare/wellness, energy sustainability, education, disaster response, community safety, scientific research, etc.

• explored how analytic tools like NodeXL (http://nodexl.codeplex.com/) can help systematically analyze social media initiatives and mine social media sites for useful information.

• identified the unique challenges of using social technologies in a government context and design strategies and policies that help overcome those challenges

Participants included:
• Vladimir Barash, Doctoral student, Information Science, Cornell University
• John Bertot, Professor, iSchool, UMD
• Derek Hansen – Assistant Professor, iSchool, UMD and director of Center for the Advanced Study of Communities and Information (CASCI)
• Scott Golder, Doctoral student, Sociology, Cornell University
• Bradford Hesse, Chief of the National Cancer Institute’s Health Communication and Informatics Research Branch (HCIRB)
• Natasa Milic-Frayling, Principal Researcher, Microsoft Research Cambridge
• Cynthia Parr, Director, Special Pages Group, Encyclopedia of Life, Smithsonian
• Jenny Preece – Dean, iSchool, UMD.
• Ben Shneiderman – Professor, Department of Computer Science, UMD, and founder of the Human Computer Interaction Lab (HCIL).
• Marc Smith – Chief Social Scientist, Connected Action Consulting Group, director of the Social Media Research Foundation

Slides available from:

http://www.cs.umd.edu/hcil/soh/workshop_hansen.shtml

Schedule

8:30 – 9:30am: registration and continental breakfast

9:30am: Introductions

10:00am: “Technology-Mediated Social Participation: Promoting a Research Agenda” Ben Shneiderman & Jenny Preece, University of Maryland

10:30am: “Enabling Citizen Mapping of Government Networks with NodeXL” – Marc Smith & Derek Hansen, Connected Action and University of Maryland

10:50am: Break

11:00am: “Open Government, Social Media, and Information Policy: Constraints and Barriers” – John Bertot, University of Maryland

11:20am: “New Approaches to Collecting Public Input Online” – Joe Pringle, Forum One Communications

11:40am: “Visualizing Social Data to Improve Diplomacy” – Leonard Lidov, Morningside Analytics

12:00 – 1:00pm: lunch

1:00pm: “Using New Media for Citizen Engagement: Experiences from the General Services Administration” – Leilani Martinez, Center for New Media & Citizen Engagement, U.S. General Services Administration (GSA)

1:30pm: “Supporting Lifelong Learning & Institutional Transformation with New Media” – Michael Edson, Web and New Media Strategy, Smithsonian Institution

1:50pm: “Says who? Legitimacy in Online Collective Action: Enthusiasts and Interest Groups in Wiki Government” – Scott Golder & Vladimir Barash, Cornell University

2:10pm: “From Research of Social Media to Socially Mediated Research” – Natasa Milic-Frayling, Microsoft Research, Cambridge UK

2:30pm: Break

3:00pm: “Health 2.0: Putting the Power of Open Science to the Task of Improving the Nation’s Health” – Bradford Hesse, Health Comm & Informatics Research Branch, National Cancer Institute

3:20pm: Wrap-up discussion

4:00pm: Depart

Technology-Mediated Social Participation: Promoting a Research Agenda

Jenny Preece and Ben Shneiderman, University of Maryland

With support from the National Science Foundation, we have run two research agenda-setting workshops that clarify how social media can address national priorities.  The first, in Palo Alto, CA engaged companies and the second in the Washington, DC area reached out to government agencies.  Our report will describe the national priorities that can be addressed, deep science theories, technology challenges, and scope of educational changes needed in K-12, universities, professional training, and citizen awareness.

Enabling Citizen Mapping of Government Networks with NodeXL

Marc Smith & Derek Hansen, Connected Action Consulting Group and University of Maryland

Social network analysis can serve as a powerful analysis and visualization method to understand connections between people and institutions. Until recently, only those with advanced degrees and technical know-how have been able to perform social network analysis. We will introduce NodeXL, a plugin for Excel 2007, that allows non-technical experts perform social network analysis, particularly of social media networks of interest. We will discuss applications of NodeXL to government data and social media initiatives.

Open Government, Social Media, and Information Policy: Constraints and Barriers

John Bertot, University of Maryland

Much movement and discussion surrounds the use of social media technologies by the federal government in order to create a more open, transparent, and participatory government.  Since the inauguration of the Obama Administration, government agencies have been required to engage in open dialogs, create open government plans, and otherwise use a range of social media technologies.  But these efforts belie an antiquated and contradictory information policy environment that hampers innovation and use of a range of participatory technologies by government agencies such as twitter, facebook, and YouTube.  Though agencies currently are using these technologies, there are a number of policies (e.g., laws, memos, administrative code) that prevent agencies from truly engaging in innovative uses and practices through social media technologies.  This presentation identifies selected policy conflicts and considerations that need resolution for federal government agencies to be able to successfully engage the use of participatory technologies.

New Approaches to Collecting Public Input Online

Joe Pringle, Forum One Communications

Federal government agencies are experimenting with social media, crowdsourcing, and other collaborative approaches to soliciting, collecting, and responding to public input.  This includes everything from formal rulemaking comment collection to ideascale events to online brainstorms such as USAID’s recent Global Pulse event.  Chris will highlight common online approaches to using social media and collaborative approaches for collecting input, provide some examples about who is doing what, what’s working, what’s not working, and why.

Visualizing Social Data to Improve Diplomacy

Leonard Lidov, President, Morningside Analytics

This talk will focus on how Dr. Lidov’s firm, Morningside Analytics, uses data visualization to understand public diplomacy, both in and outside government.  Morningside’s signature “dot-maps” picturing “Attentive Clusters” in the blogosphere have been featured in the media and opened business development doors in DC (http://online.wsj.com/article/SB124658422588090107.html ), but they are just one of several innovative approaches the firm uses to help clients understand how its network analyses can improve their understanding of how messages move in the online “Link Economy.

Using New Media for Citizen Engagement: Experiences from the General Services Administration

Leilani Martinez, Center for New Media & Citizen Engagement, U.S. General Services Administration (GSA)

This presentation will discuss several new media initiatives overseen by the Center for New Media & Citizen Engagement, part of the General Services Administration of the federal government. These will include discussions of developing tools, methods, and best practices related to using new media by government to engage citizens. Examples will include a citizen engagement tool used by 23 agencies to gather input for their open government plans, the Citizen Engagement Platform, and FedSpace, a secure intranet and collaborative workspace for government employees and contractors.

Supporting Lifelong Learning & Institutional Transformation with New Media

Michael Edson, Director of Web and New Media Strategy, Smithsonian Institution

The Smithsonian Institution’s Web and New Media Strategy that talks about an updated digital experience, a new learning model that helps people with their “lifelong learning journeys,” and the creation of a Smithsonian Commons-a new part of the Smithsonian’s digital presence dedicated to stimulating learning, creation, and innovation through open access to Smithsonian research, collections and communities. This talk describes the commons concept in the context of 21st century knowledge creation and the methods and drivers of organizational change.

Says who? Legitimacy in Online Collective Action: Enthusiasts and Interest Groups in Wiki Government

Scott Golder and Vladimir Barash, Cornell University

With the expansion of the World Wide Web, online communities have emerged focused on government and civic activities. These communities allow a wide user base to participate in everything from election campaigns to opinion polls to the legislative process (by making it easier for individuals to interact with members of local, state and federal government). The emergence of these communities has led to a greater citizen participation but has also opened up new venues for special interest groups to sway public opinion. For instance, politically oriented communities (dailykos, the free republic) advertise links to popular opinion polls and encourage their members to vote in such polls en masse, creating a cascade of skewed opinion. We explore the issues related to creating venues of open (often unregulated) online participation in the government process, and the role special interest groups play in these venues.

From Research of Social Media to Socially Mediated Research

Natasa Milic-Frayling, Principal Researcher and Director of Research Partnership, Microsoft Research, Cambridge UK

We are witnessing the ultimate assimilation of computing technology by the masses: millions of online users have adopted social media as part of their daily routine. At the same time, researchers are striving to understand the principles of sustaining social media and the implications the new practices have on research itself. In this presentation we discuss insights about two important aspects. First is the delicacy of social media design where seemingly small changes can cause a dramatic difference in the character of the online community. This applies to both UI features that facilitate interaction and the reward mechanism that administrators introduce to motivate specific behaviours. Second is the emergence of new methodologies that leverage the principles of social media. They are adopted by groups of individuals to achieve their common objectives. We illustrate how, for example, the issue of scale can be addressed through the use of games to mobilize a community of practice.

Health 2.0: Putting the Power of Open Science to the Task of Improving the Nation’s Health

Bradford Hesse, Chief, Health Comm & Informatics Research Branch, National Cancer Institute

Fourth International AAAI Conference on Weblogs and Social Media (ICWSM-10)

***Microblogging 2***

Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment (Tumasjan et al.)

Successful use of social media in las presidential campaign has established twitter as an integral part of political campaign toolbox

Goal: analyze on Twitter: 1. Deliberation, 2. Sentiment, 3. Prediction

Previous work:

Deliberation: Honeycutt and Herring – Twitter not only used for one-way comm, but 31% of all tweets direct a specific addressee. Kroop and Jansen – political internet discussion boards dominated by small # of heavy users

Sentiment: How accurately can Twitter inform us about the electorate’s political sentiment?

Prediction: can Twitter serve as a predictor of the election result?

Data: examined more than 100k tweets and extracted their sentiment using LIWC

Target: German federal election 2009

Results:

1. While Twitter is used as a forum for political deliberation on substantive issues, this forum is dominated by heavy users

Two widely accepted indicators of blog-based deliberation:

-The exchange of substantive issues (31% of all messages contain “@”),

-Equality of participaion: While the distribution of users across groups is almost identical with the one found on internet message boards, we find even less equality of participation for the political debate on Twitter. Additional analyses have shown users to exhibit a party-bias in the volume and sentiment of messages.

2. The online sentiment in tweets reflects nuanced offline differences between the politicians in our sample.

LIWC profiles:

-Leading candidates: Very similar profile for all leading candidates, only polarizing political characters, such as liberal leader and socialist, deviate in line with their roles as opposition leaders. Messages mentioning Steinmeir (coalition leader) are most tentative

3. Similarity of profiles is a plausible reflection of the political proximity between the parties

Key findings: high convergence of leading candidates, more divergence among politicians of governin grand coalition than among those of a potential right wing coalition

4. Activity on Twitter prior to election seems to validly reflect the election outcome (MAE 1.65%), and joint party mentions accurately reflect the political ties between parties.

From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series (Brendan O’Connor)

Measuring public opinion through social media

Old method – query via dialing, asking, etc.

New method – people write their thoughts to social media, query social media to create aggregate text sentiment measure.

Can compare results from new method to old method

Contributions:

-High correlations between very simple sentiment analysis and telephone polls

-Time series smoothing helps

Text Data: Twitter

-Large, public, ll in one place

-Sources: Archiving Twitter Streaming API (“Gardenhose”/”Sample” ~15% public tweets); Scrape earlier messages via API

-Volume ~ .7B tweets

-Poll data: consumer confidence (2008-2009) – index of consumer sentiment (Reuters/Michigan), Gallup daily. 2008 presidential elections (aggregation, pollster.com). 2009 presidential job approval (Gallup daily)

-Message selection via topic keywords

-topic frequencies change rapidly

-Sentiment analysis: word counting.

–Subjectivity Clues lexicon from OpinionFinder / U Pitt (Very simple system!)

Key: don’t need to classify individual messages correctly, just need a sentiment ratio over messages.

-Sentiment Ratio Moving Average: High day-to-day volatility. Average last k days.

-Which leads, poll or text?

–Cross-correlation analysis: between sentiment score for day t, poll for day t+L.

—Results: “jobs” text leading indicator for poll, can be turned into forecasting model

—Reminiscent of Leskovec et al. Blogpulse paper, very nice!

-Keyword message selection:

–15-day windows, no lag. “jobs” r=80%, “job” r=7%. Is stemming always good?

Presidential elections and job approval: sentiment doesn’t correlate, but pure volume does (79% for “obama” 74% for “mccain”)

Conclusions:

-Preliminary results that sentiment analysis on Twitter data can give information similar to traditional opinion polls. But, still not well-understood. Twitter bias? News vs. opinion?

-Issues: Relevant message selection, Time series smoothing

-Replacement for polls? Promising but not quite yet

Information Contagion: an Empirical Study of the Spread of News on Digg and Twitter Social Networks (Lerman et al.)

Information flow on networks

Dynamics of Social Information

-How does infromation spread on online social networks?

–How far and how fast does information flow on networks?

–What factors influence its spread?

–How does network structure affect dynamics of information flow?

–What does this tell us about quality of information?

-Study question through comparative empirical analysis of 2 social news networks – using URLs as markers

Social News: Digg, Twitter + Tweetmeme

-Tweetmeme aggregates all tweets and features most retweeted URLs on its front page

Data Scope:

-3.5K digg stories with time submitted, promoted, votes for each story (time of vote, name of voter). 140k active users who voted for at least one stroy, 71k of them following at least one user. 258k links = fan network

-398 most retweeted stories 6/11/09 – 7/3/09, extracted from tweetmeme. Retweets of each story, up to 1k most recent retweets. Follower network of users who retweeted the stories

Questions:

-Usability of social netws – do people use digg, twitter the same way? what effect do differences in user interface have?

-dynamics of social networks – how far does info spread, how fast does it spread, and what are the effects of net strucutre?

Basic terms:

-Submitter  = user who submitted link to story, or user who tweeted link to a story

-Vote = vote on Digg or retweet on Twitter

-Fan = fan on Digg or follower on Twitter

User activity: distribution of fans (Power law on Digg with up to 1e5, power law with bump ~ 10 on Twitter with up to 1e7 users)

User activity: distribution of voting: Power law on Digg and Twitter (with different slopes)

Dynamics of stories: both digg and twitter show exponential growth, but for Digg it is preceded by slow period before story is on front page, both show vote saturation

Popularity distribution of stories shows lognormal fit

Information flow on networks: information spreads on a network as fans (followers) vote for (retweet) stories their friends submit or vote for.

Dynamics of information spread on networks looks very similar to overall dynamics of information spread (evolution of fan votes qualitatively similar to evolution of all votes)

BUT distribution of popularity is different, now shows normal fit. “Inequality of popularity” no longer observed (social influence accounted for?). News spreads farther on Twitter than on Digg.

How far does information spread among submitter’s fans?

-On digg many stories get voted by submitter’s fans, opposite case on Twitter

How fast does info spread on networks?

-Two distinct phases on digg: stories spread faster through network before promotion than afterwards.

-On Twitter, info spreads at constant rate.

Network structure differences: Digg network is denser, more inter-connected than Twitter’s

Summary of results:

-Network structure and info flow

–Digg’s network is denser than Twitter’s: News spreads faster initially through Digg’s network, but it does not spread as far as on Twitter

–Twitter’s network is sparse: Fans unconnected to submitter help spread story

-User interface and information flow:

–Before promotion, Digg stories spread mainly through network (and do so faster)

–No equivalent of promotion on Twitter

Tweeting from the Town Square: measuring Geographic Local Networks (Yardi and boyd)

Two geographically bounded events: Wichita shooting and Altanta parking garage collapse

Methods: two crawls and a poll

RQ1: Do geographically local topics have more dense Twitter networks than non-local topics?

Why this is important? People living in close geo proximity may share characteristics. Connecting similar people can help them form ties, foster community

Spread of News

Spread of News Online – ongoing discussion vs. spikes of short-term high-density discussions around real-world events

Distance

Methods: searched key terms about each evenet, stored user info, crawled first degree net of users. Polled users who had tweeted twice or more about church shooting in first 24 hours after it was announced. Administered poll 3-5 days after event. Sent out 800 requests, received 164 responses.

RQ2: Are people who are central in twitter network more geographically central in physical world?

Sarita Yardi gives shout-out to NodeXL, asks for more scale!

RQ3: What sources do people go to for local news events?

Twitter maps show high level of locality to event, slow spread outward

News Sources – go to locals

News Seekers – also go to locals, then to MSM

Practical applications:

-Utilize local short paths for disseminating information. Schools have long used an “emergency phone tree” with specified # of branches and leaves

-Timely notification of unexpected events

Invited Panel: US Government and Social Media

Macon Phillips, Director of New Media for the Obama White House

Moving from Elections to Governance

Wants academics to build tools that show effect of using social media on user behavior

WH new media director Macon Phillips asks for tools that allow thousands of people to communicate with the President (thanks @sadatshami !)

Don Burke, CIA Directorate of Science and Technology, Intellipedia Project

Haym Hirsh, Director, Division of Information and Intelligent Systems

Social Media and the Federal Government

NSF

US Gov’t early crowdsourcing project – National Weather Service Cooperative Observer Program (1890)

-Experimentation:

–CIA Intellipedia

–NASA Clickworkers

–PeerToPatent

–DARPA Balloon Challenge

–EPA Greenversations

–Over 100 gov’t blogs

-Policy implications and clarifications

–70% of Airmen use YouTube

Challenges:

-Legal and Policy

–Terms of Service: Indemnification, etc.

–Advertising (e.g. alongside gov’t content)

–Procurement: Free = Gift? No competition? Charges imposed after lock-in

Additional Challenges:

-Colbert “attacks”

-Open Government Dialogue

The Open Dialogue Top 5:

1. Concerns about Obama’s Birth Certificate

2. Government spending

3. Marijuana

4. Marijuana

5. Birth Certificate

Additional Opportunities: “No matter who you are, most of the smartest people work for someone else.”

Implications:

-Foster experimentation and innovation w/in federal government

-Provide data for innovation outside the def

-align legal and policy with aspirations

-research

Question about contribution quality: do people feel their contributions are worthwhile? How do we make the value and implications of contribution clear?

What do “votes” for questions mean? Who is the right person to say that legalization of marijuana is not a big question? What questions are “big enough to matter”? The “pothole problem” – should questions about fixing potholes be crowdsourced?

Few poorly worded questions about marijuana, people will speak eloquently and argue for the issue, so it’s not just spam

Don Burke – not EVERY system has to be based on socialmedia

Questions: How do you get recognized by gov’t? Answers: open access, publishing where you’ll be noticed

Question about Intellipedia and procedures for aggregating information. Answer: without the wiki, there was no way to share tacit knowledge. But want to go beyond wiki and to the larger web

Jure Leskovec about developing APIs for gov’t data. Answer: no APIs yet, but government is collecting data in one place that’s publicly visible. Want to see scientific community analyzing datasets and finding results, government may not necessarily know what’s a “good” dataset.

***Analysis of Social Network Usage***

Governance in Social Media: A Case Study of the Wikipedia Promotion Process (Leskovec et al.)

Wikipedia promotion process

3 important features:

-deliberative process yielding a single decision

-is publicly recorder

-consequential for the community

Similarity to offline world: people evaluate other people

We study perspective of voters:

-Burke & Kraut examine candidate’s perspective

-How voters evaluate candidate?

-How do evaluations change over time?

Main findings: Relative assessment

-Voter’s evaluation of the candidate reflects different types of relative assessment

–Let voter V vote on candidate C

–we find that vote of V heavily depends on relationship and relative merit of V and C:

—past interaction

—Number of edits

—Number of “barnstars”

–Response function of vote V:

—Prob. V votes given that x other people have voted

Dataset: Wikipedia voting

-Votes are time stamped and signed by users

–2.8k elections sept ’04 – Jan ’08. 44.6% success rate: Successful: 94.7% support. Failed: 31% support votes

–114K votes (78% support). Each vote can get commented: Support votes: 7% get discussed. Oppose votes: 82% get discussed

User characteristcs

-8.3K users voted

–7.5K voters

–2.5k candidates (some go for promotion multiple times)

-Relative merit:

–How do properties of voter V and candidate C affect V’s vote?

–Two natural (but competing) hypotheses:

H1. Prob. that C receives positive vote depends primarily on characteristics of C, there are objective criteria for user to become admin

H2. Prob. that C receives positive vote depends on relationship between characteristics of C and V

Merit (level of contribution):

-Two ways to quantify merit: total #edits, total #barnstars

-Relative merit: How does prob of V voting positively depend on diff in merit of C and V?

Relative merit hypothesis: if V has higher merit than C then he is less likely to vote

Observations: V is especially unlikely to vote for candidates of the same merit (total edits or barnstars)

Direct V-C interaction: Prob of positive vote as function of prior interactions of V and C.

Observation = prior interaction increases probability of a positive vote (with diminishing returns)

Thresholds and diversity of voters:

-Aggregate response function:

–How does prob. of voting positively depend on frac. of positive votes so far?

-Aggregate response function: baseline: if voter were to flip a coin then f(x)=x

-Observation: voters more inclined to express opinion when it goes against prevailing opinion

-Personal response functions: How does prob. of voter V voting positively depend on frac. of positive votes so far?

-Enough data that we can build models of individuals

-Average is close to baseline but individual variation in shape of response function is large

-Over time voters become more conservative, response functions shift downward and to the left

Elections over time:

–Elections unfold over time: Sequence of pairs (s(t),o(t))

—Very negative elections end ealry

—Failed elections are “top-heavy” = start very positive and slowly get negative.

—Successful elections get more positive over time

—Order of early votes doesn’t matter

–False hypotheses: Candidate’s friends vote early, Herding behavior (excessive influence of first votes)

Activity Lifespan: an Analysis of User Survival Patterns in Online Knowledge Sharing Communities (Yang et al.)

-User survival analysis to show that participation patterns and performance factors can account for a considerable amount of variance in predicting user lifespan

-Compare 3 major Q&A sites: Yahoo! Answers, Baidu Knows, and Naver Knowledge-iN

-Discuss how systems might sustain users

-Characteristics of Q&A sites we studied: in Yahoo Answers, earn points at flat rate per answer / best answer, Pay flat rate in points. In Baidu and Naver, earn points at flat rate per answer + points per best answer, and asker can offer additional points

-In Yahoo, significantly more questions / answer

Method: survival analysis

Defining “death” in online communities: period of inactivity exceeding 100 days. Found model prediction not sensitive to different cutoffs (50-150 days)

General comparison: 30-70% users leave after first day, afterwards curves for all 3 sites flatten. YA users more likely to remain than users of other two sites.

Answering life on average longer than asking life across all sites.

Preference between answering and asking (A/R ratio) can account for considerable amount of variance in predicting user lifespan

Initial interaction:

Obtaining more answers to your first question, writing longer question correlated with longer lifespan on Yahoo and Baidu

Winning best answer also correlated with longer lifespan

First 30 days:

More activity, asking more questions, obtaining more answers per question positively correlated with lifespan on all 3 sites

A/R ratio negatively correlated with lifespan on Yahoo but positively correlated with lifespan on Baidu and Naivr

Winning (best answer) also positively correlated with lifespan on all three sites

Analysis: community evolution

All three sites presented a decline in survival rate from year 1 to year 2, especially for Yahoo Answers

Naivr suffered more difficulty in sustaining users in 2nd year as almost no users stayed after 250 days

Conversational vs. Informational: There is a significant and consistence difference in survival patterns between conversational categories and informational categories: more conversational categories survive for longer

*with the exception* of “computer/internet” on Baidu only (cultural difference?)

Analysis: why do YA users stay longer

Fourth International AAAI Conference on Weblogs and Social Media (ICWSM-10)

We will be liveblogging (when possible) from ICWSM 2010, going on now!

Keynote: Bob Kraut, CMU

kraut keynote
implications for community design-offline theories of socialization helpful, not definitive
-online communities can build in good socialization practice-e.g. WP welcoming committee
Two Types of Commitments to Groups-identity based groups-bond based groups
Added Identity & Bond Features to MovieLens
Introduced Subgroups into MovieLens
Identity features that focus on subgroups
Individual profiles
bond-based design:+11% loginsidentity-based design:+44% logins
interventions based on theory increased commitmen
why stronger effect of identity?-time course: social identity can form instantaneously, bonds take time
new approaches to translate theory to design-ABMs
test identity vs. bond design via abm
incorporate design into abm
what are the consequences of discussion moderation-what type of moderation should be imposed? when?
results: indiv moderation helps logins
-personalized mod improves info and social benefit
-comm level mod improves info benefit only in homogeneous communities

***Influence and Composition in Social Networks***

Measuring Influence of Neighbors in Social Networks (Cosley Huttenlocher Kleinberg, Lan Suri)

Measuring Influence in 2 types of socnet data

-Crawl

–provides state of network at specific time

–often have more than one

-Complete time seties

–often from database dump

–provides timestamped history of all events

-This talk: How do measurements of social influence in these two settings compare?

Defining Influence

p(k) = Pr(individual adopts new behavior | k neighbors have)

Analyzing Influence in Wikipedia

-Wikipedia as a social network

–registered users (500K) represented by odes

–communicate through user talk pages

—Definition: Link between u and v forms at time t if one edited the other’s talk page at time t

-Definition: A community is a set of users who edited an article

-Analyze: Pr(user joins a community | k of his friends have)

Ordinal Time Method for Measuring Influence:

-p_o(k) = # instances (over all C) where u had k neighbors in C and joined/  # instances (over all C) where u had k neighbors in C

-advantage: very easy to interpret

-disadvantage: requires complete time series

Snapshot Time Method for Measuring Influence:

-2 snapshots of the network taken at times t1 and t2

p_s(k) = # instances (over all C) where u had k neighbors in C at t_1 and joined before t2 / #instances (over all C) where u had k neighbors in C at t1 and did not join before t1

-advantages: requires only a snapshot

-disadvantage: coarse-grained, don’t know changes in friends between t1 and t2

-results: p_o(k) behaves very differently from p_s(k) on Wikipedia

How are p_o(k) and p_s(k) different?

-accumulation effect in p_o(k) where community joining events can contribute to p_o(1)…p_o(k)

Approximating Ordinal Time from Snapshots

-At each snapshot know:

–which users joined which communities

–how many friends each user had in each community

-Don’t know exact number of friends user had when joined

-Assume a constant rate of getting friends

Ethnicity in Social Networks (Facebook Data Team)

-Questions about how ethnicities engage online are prevalent

-Our goal is to better understand ethnicity on social networks

-Issues

–How do we estimate ethnic distributions on social networks?

–Many services (e.g. Facebook) do not ask their users about their ethnicity

–Our approach infers ethnicities from surnames (based on previous approaches)

Topic modeling approach

-words are names, topics are ethnicities

-Generative process: Draw ethnic bareakdown of aggregate population w/Dirichlet distribution

–For each person draw ethnicity of individiual z_n ~ Multinomial

–Draw surname of individual based on ethnicity ~ Multinomial conditioned on ethnicity

Because there is no FB ground truth, we test our method by scraping 10k users from Myspace

Results: Model does a lot better than naive guessing based on census, internet usage statistics

Also tested on FB data (no ground truth but can have qual results):

1. In 2006, AsAm and White overrepresented on FB, by 2008 ethnicities much closer to mark

2. Drew heatmaps of US, showed that Whites concentrated in the northeast, Asians on the coasts, etc.

3. Correlated with politics: AsAm more likely to be liberal, Whites more likely to be conservative/libertarian

Homophily:

-Strong racial homophily (strongest in Hispanics and Blacks, weakest in Whites)

Homophily by ordinal friendships: racial homophily is strongest for the first few friendships a person makes

Friend you communicate with the most is most likely to be homophilous friend

Who makes friends in social media and why? Rich get Richer vs Seek and Ye Shall Find (Zeynep)

Importance of friendship and social networks vs. debate about social isolation

Architecture of our Commons

-Sociality does not happen in a vacuum

-Mediated Relationships (through digital means)

–Are these relationships: Superficial, weak, fake? Supplementary to offline sociality? As real or more?

–Why does this debate refuse to die?

-Assumptions:

–Internet communication: Anonymous, text-based, fleeting

–Lack of visual cues

but today’s Internet:

–social media: non-transitory interactions, lots of visual cues

Two dominant theories: Rich get Richer vs. Seek and Ye Shall Find

Data: Sample:

-College students

-617 respondents

-multiple classrooms

-very diverse school

-diverse majors

Results: feeling about whether online friendships are possible vs. not possible – split down the middle

Qualitative Component: why do respondents feel online friendships possible / not possible

*Not Possible reasons:

-trust

-face-to-face

-body language and mannerisms

-shared emotions and experiences

*Possible:

-deeper connections – easier (“it may even be easier online as it is all dialogue and no physical characteristics involved”)

-deeper connections – judgment

-bonding is possible

-conversation as key

-experience

***Diffusion and Dynamics in Networks***

Social Dynamics of Activity in a Virtual World (Bakshy et al.)

Second Life paper

Second Life: persistent, immersive virtual world

-User driven objects, economy, society

RQs: 1. How much economic activity occurs in virtual world?

2. Role of groups?

3. How much virtual interaction?

Data:

a) 65 mln user-user transactions (virtual goods)

b) Buddy graph – 4.2 million users, 43 million relationships, focus on strong edges: reciprocal and permissive (can see each other’s online status)

c) chat between 14 million pairs of users

d) 520k groups, 23 million user/group memberships

Economic Activity in Second Life

-Same “sectors” as realworld: retail, real estate, entertainment

-29 million free transactions, 36 million paid, power-law distribution of exchange amounts

Seller in detail: shows that interaction does not necessarily correspond to exchange

Analysis hints at Seller roles? Profiles?

Social Adoption

-show how friends of friends of initial buyers go on to buy as well. Is it influence or homophily? Not sure.

Role of Social ties

-39% of free transactions (but only 7% of paid) were between friends

-40% of users that chat exchange free items, 12% of users that chat engage in paid transactions

Free transactions may be more representative of social activity than paid transasctions

Role of Groups

-Groups indicate aspect of user interest

-Co-grouped transactions

-Sellers more likely to be connected to buyers through co-group than through friendship

Long discussion of why groups are proxy for connections

Explaining Seller Success

Traditional success measures: revenue, repeat business. Regression analysis

Key predictors:

For Revenue:

-Amount made by friends (homophily? business partnerships?)

-Make connections

-Less chatting with customers

-The younger crowd (newer to 2nd Life)

For Repeat Business:

-Interaction!

–Sharing a group, chatting

-More established crowd

-More diverse buyer base

Your Brain on Facebook (Fisher and Counts)

EEG: social vs. traditional media

Why do this? Possible input/feedback to social interaction systems. Short term: Inform design through better understanding of automatic info processing

EEG very good at detecting semantic mismatch

RQs: is Myspace associated with frivolity? Is connecting with friends on Facebook connected with feelings of intimacy?

Media and concepts:

-Media: TV, Books, Social, News

-Concepts: addictive, story, interesting, frivolous, personal, useful

-Showed both front page of FB and personal FB page (asked study participants to briefly friend study on Facebook)

Measurement and Method:

1. Timed, binary decision (low conscious reasoning)

2. Lijkert scale untimed survey (high conscious reasoning)

-Computer-based task: 24 combinations x 22 trials = 528 trials

-16 participants

Results (survey): FB more addictive, but less useful than news. Tells less of a story than books or TV. More frivolous than books, news. More personal than all other forms of media.

Results (decision taks): FB very addictive, tells less of a story than books or TV, a lot more personal than other forms of media. Overall, quite similar questionnaire responses.

Viz: heatmap of head. Time chart. In time chart, look for potential drop ~400 ms after stimulus. The more similar the media to the concept, the bigger the drop.

Results (eeg): FB equal to other media in terms of addictive, interesting, useful, frivolous. Tells less of a story than other media. Is less personal than other media.

Implications, suggestions, limitations

Media parity: FB as interesting, useful, addictive and not frivolous as other media

Personalization and self-identification: Purposeful connection building; easy switching to close friends, family; identity vs. bond attachment

Form of media found most personal by eeg? Books!

Telling stories – status updates on storylines?

Making it tangible: What’s the ratty, marked-up favorite book equivalent in social media? Photo album equivalent?

Limitations: only one form of social networking / media, limited subject population, can we believe EEG results?

Opportunities: compare online media, other subject populations, corroborate with other physiology, expand into real-time capture of physiology

Social Causality and Analysis of Interpersonal Relationships in Online Blogs and Forums (Roxana Girju)

Social causality: causal reasoning used by intelligent agents in a social environment

Modeling social causality – can guide conversation strategies, facilitate modeling and understanding of social emotions, bring new insights

Our focus: social causality as capture through analysis of interpersonal relations in social media

-Pervasive set of english reciprocal textual contexts encoding interpersonal relationships

-data: 11K reciprocal relationship contexts coded

Reciprocity in Language: “The Golden Rule”

Modeling relationships

Properties of interpersonal verbs and reciprocal instances: Symmetry, Affective Value (4-state HMM), Intentionality of Actions

-Intentionality and affective values of interpersonal verbs highly correlated with blame and responsibility

Analyzing Social Interactions:

-Overall 54.34% of dataset was encoded by ambiguous symmetric patterns

-top frequent verb pairs: need-need, love-love

-followed by: hate-hate, miss-miss…

-In general we love people who love/understand/care/need us

-Gender analysis: men initiate more often than females, retaliate more often than women, are more violent and aggressive (whereas women are more forgiving), but this depends on class of verb

-Men and women generally mutually respectful, it is only when respect is broken that responses may differ (e.g. women: cheat -> hate, despise, sue. men: cheat -> dump, divorce)

-Intentionality of actions: intentionality much more often perceived as intentional in bad-bad exchanges than in good-good exchanges

-Reciprocity Chains: Dyadic (formed between 2 people of the form A v B -> B v A -> A v B). Very useful in micro-levle social interaction analysis. General (between multiple people)

In generic chains: retaliation with increased magnitude chains, good for good chains (short), good for bad chains (turn the other cheek)

How does data sampling strategy impact discovery of information diffusion in social media? (Munmun)

How can we sample social web?

Many different modes of social interaction

Scale of interest – viral marketing, ad campaigns

Is there more in social media than just scale?

Social media can have enormous power, e.g. for diffusion

However, inference of such processes is based on the quality of data

Current methods: random walk, snowball – captures structure but not content or context

RQ1: what is role of context in sampling social phenomena, RQ2: how much should we sample to capture the process

Data: Twitter. Look at diffusion via RT feature, shared URl, same hashtag

Model: Diffusion series. Has slots of individuals involved in diffusion process, links between individuals based on relationships

Sampling strategies: given N, the number of nodes to pick, topic T, social graph G

Ignoring social graph: 1. Random sampling from seeds, 2. Attribute / context sampling.

Using social graph: Forest fire (again, random or attribute)

Diffusion Saturation Metrics: user-based (volume, participation, dissemination). Topology-based: (Reach, spread, cascade instances, collection size), Time-based: rate

Our sample S distorts some metric M

Diffusion response metrics: correlate diffusion on twitter to external behavior (search and news trends)

Experimental Study

Reference Set: ~465K users, 836K edges, 30M tweets. 125 randomly chosen “trending topics” from Twitter between Oct and Nov 2009

Trending topic – theme association

Results: bias due to sampling consistent, best results come out of forest fire. For search trends, forest fire + location performs best, for news trends, forest fire + activity performs best

Larger-scale analysis: look at topic distribution by sampling strategy

Inferences about social data affected by sampling strategy. Topic + topology + seed attribute makes a difference to sampling.

Photo Tagging over Time: A Longtitudinal Study of the Role of Attention, Network Density and Motivations (Paul Russo)

Tagging over time

Overview: what factors influence users to tag?

RQs:

1. How do individual motivations affect tagging,

2. what effect does receiving attention from users affect tagging tenure

Data: Flickr – focused on established users

Individual Motivations:

-People tag for themselves (archiving, retreival) and for others (describing)

-Many ways to receive attention from others

-Huberman, Romeru, Wu demonstrated that on YuTube people whose work received more views tended to post more videos, leading to a submission cycle

-Lento et al. photo tagging behavior

New: look at network strucutre (clustering coefficient)

-Mutual Friends

H1: attention, enjoyment, commitment should increase tagging. Density should decrease tagging. ((blogger comment: what about social grooming?))

Method: 90 days of tagging on Flickr, used only “pro” users w/3 months tenure

-Combines user-reported (survey) data and system data: what people say and what they do

-Attention: comments, Density = network, motivation = survey, measured on Likert scale.

-DV = #tags/photo over 90 days

-Results: generally bear out hypotheses.

-Interesting: low density net individuals tag more than high density for same lev of attention, but higher levels of attention lead to more tagging regardless of density. On the other hand, density has positive slope w.r.t. to commitment and tagging only for low density networks, for high density networks higher levels of attention lead to less tagging for high density.

***Microblogging***

Microblogging Inside and Outside the Workplace (Ehrlich and Shami)

Method: Twitter vs. Bluetwit (internal tool)

Data: 34 users from 15 countries and 8 business units that used both Twitter and Bluetwit and 20 posts in each over 4 month period

-Twitter much more active than Bluetwit

Dataset: data collected for BlueTwit and Twitter over 4 month period. 19K posts in two tools. Extracted 4 weeks of tweets.

Manually coded 5k microblogging posts

codes: status, providing information, retweet, ask question, directed, directed q

Results:

Categories of Posts: Most frequent use – provide information, directed posts

Internal use: ask questions, directed posts, style is work-oriented

External use: provide information, style is more “social”

Public vs. Private: clear sense of what is appropriate for an internal-only audience

Reputation Management: Internal: importance of giving back

External: Publicity and Promotion

Fostering Connections: developing better awareness of professional connections in advance of a future planned or unplanned meeting

Consumption of microblogs:

-Microblogs provide early access to human selected information

Discussion:

-How is microblogging useful in a work context?

–Form of crowd-sourcing – asking questions and providing answers

–Anticipatory connections

-What differs between public and private?

–Confidentiality (very clear)

–Style of writing

–Awareness of audience knowledge and interests

–Some erosion of boundaries – despite difference lots overlap

-Why are people consuming microblogs?

–Motivation for posting – Building reputation, awareness

–Motivation for reading – early, quality news

Measuring Influence in Twitter: the Million Follower Fallacy (Cha et al.)

Goal: characterize influence in social media and study its dynamics

1. How can we measure influence of a single user?

2. Does influence of user hold across topics?

3. What behaviors make ordinary users influential?

Data: Twitter

Why Twitter? One of most popular social media, social links are primary way how information flows, traditional media soruces and word-of-mouth coexist in this environment

-54m users, 2B follow links, 1.7B links

–8.5% of profiles private

–95% users belong to the giant component

–low reciprocity (10%)

–Power law degree distribution w/extremely large hubs (500 users have more than 100K followers)

–Low tweeting activity in general (only 11% of all users posted at least 10 tweets)

Three measures of influence:

1. Indegree

2. Mentions

3. Retweets

Are these three measures related? Compared relative ranks of user across 3 measures using spearman’s rank corellation

Correlated for full population, but not for top 10% or top 1% (but retweets corr mention remains high)

top list of indegree = mix of news outlets and public figures, top retweets = celebrities, etc.

Million follower fallacy – Britney Spears has millions of followers but she doesn’t show up on top retweeted list

Measuring influence:

–Find users engage in multiple topics. Picked 3 popular topics in 2009 over 2 month period, iran election, death of  MJ, swine flu

–Focused on 13k people who talked about all 3 topics

Conclusion: just because you have a lot of followers doesn’t mean they retweet you

Along a similar idea, Tweets vs. followers in NodeXL:

Directed Closure Process in Hybrid Social-Information Networks w/Focus on Link Formation on Twitter (Romero and Kleinberg)

How do directed closed triads form?

Triadic closure vs. directed closure:

-Triadic: an edges connects 2 nodes who already have a common neighbor

-Directed: a node A links to node C to which it already has a 2-step path (through node B)

-An edge in directed graph exhibits closure if it completes a 2-step path

-Closure ratio of node C is the fraction of C’s incoming edges that exhibit closure

-could indicates how many nodes discovered C by following nodes that follow C

Data: random sample of 18 twiter micro-celebrities: Users with between 10K and 50K followers

Notation: user A is k-linked to C if A follows C and also follows k followers of C. Let s_k(C) denote set of followers k-linked to C. f(s_k(C)) = fraction of set whose edge to C exhibits closure.

Q1: Is directed closure a significant process? Randomization test

A: up to large k, f(s_k(C)) is significantly bigger than one would expect under random ordering

Observation: f(s_K(C)) increases with k, but flattens out. Why??

Properties observed: closure ratio saturates to a positive constant f, constant f is different for different micro-celebrities, constant f not closely related to total in-degree of micro-celebrity

Heuristic calculation suggests that the sum of in-degrees of incoming nodes closely predicts closure ratio

Improved Model: predict closure ratio not just by in-degree but by sum of in-degree of incoming nodes. Also sum of in-degrees of incoming nodes from same community predicts closure ratio even better!

Conclusion: definition and methodology for directed closure, evidence for directed closure on Twitter, evidence that sum of in-degrees of incoming nodes & nodes from same community predicts closure ratio on Twitter

Characterizing Microblogs with Topic Models (Ramage, Dumais and Liebling)

Do people like the posts they see?

43 users at MSR looked at 60 posts, judged as: “not really worth reading <-> maybe worth time spent reading <-> worth the time spent reading.”

the average judgment was maybe or worse, nobody on average judged posts as worth time

Fundamental problem: people followed <> Tweets worth reading

What factors go into deciding if a user is worth following?

Method: Structured interviews with heavy users, Broader survey of 56 Twitter users

Kinds of Topics for Tweets. Hobbies, professions, news, products, events = Substance. Updates about meals, travel, hygiene = status. Making plans, networking, staying in touch = Social. Humor, wit, whininess, diction, worldview = Style. Different topics liked by different users.

Shows that content is important.

Content modeling: 8.2m “Spritzer” Tweets from 2008

-Surface word features = tf idf cosine similarity, etc.

-won’t look at deeper features (e.g. parsing) – not very appropriate for Tweets

-Desparsify: Topic Models, LDA.

-Want some labels (hashtags, emoticons, questions, etc.) = Naive Bayes, SVM, etc.

-Combined surface word features, LDA, labels = labeled LDA

Content modeling with labeled LDA:

1. Discover unlabeled topics w/ k=200 latent topic dimensions (e.g. politics, sleep)

2. model common labels = 500-1000 dimensions for hashtags, emoticons, etc.

Twitter content by category: manually aggregate topics into one or more of 45 categories.

Results: 38% style, 23% social, 27% substance, 12% status

Treemap-like visualization

Filtering: Tweet stream re-ranking

Split rater’s post into train 70% and test 30%

re-rank test set by distance to positive examples

consider judgment of maybe or worth time as “positive”

Mean reciprocal Rank @ 1 Relevant: best performance = Labeled LDA + tf-idf, .75

Finding: User recommendation task

Rater’s followed user: train 6/7 followers and test 1/7. Find the test user among 8 other non-followed users. Ranking task: score by reciprocal rank of test user. Performance > .9

Next steps: better interfaces for finding and filtering and models that account for temporal dynamics

by Dru!

Can useful observations be made by studying the social media sea one bucket at a time?

NodeXL has data import “spigots” for pulling social networks out of several social media systems including Twitter, YouTube, flickr, and email.  Twitter networks of follows and followers, reply and mentions can be extracted based on either a user name or a search string “seed”.   There are additional networks inside Twitter: a tie is created whenever two people tweet the same URL, for example, or are connected by tweeting from the same general location.  For now, the NodeXL Twitter Data Importer is starting with these three initial twitter “tie-types”.

NodeXL queries are not exhaustive collections of Twitter data, we provide a more modest approach, grabbing a slice of recent content and analyzing that.  Twitter has a sea of data, NodeXL is importing something  like a study of buckets of ocean water.  A recent scientific voyage to the Great Pacific Garbage Gyre, for example, collected hundreds of samples of ocean water as they sailed to the central location of the gyre.  Each bucket revealed details about the larger state of the ocean (which does not look good).  Simlarly, NodeXL is puling buckets of social media network data from the ocean of twitter and, despite the lack of scale, can do some useful science.  In part this is a virtue imposed by necessity –  constraints imposed by Twitter (even with a rate limit lifted “whitelisted” account) impose significant limits on what can be squeezed out of the Twitter API.  For those who lack access to large data center resources, there are scale limits imposed by the capacities of a desktop/laptop device.

Access to large data sets is certainly a hallmark of the “new era of science” that generates observations not from samples but from exhaustive surveys of data terrains.  Small samples miss important phenomena it is argued.  The counter argument is that many important phenomena appear in most samples, even small ones.

Using the existing features in NodeXL, I can extract the twitter social network for a small group of user accounts.  I can provide the names or ask twitter search to deliver them.  Alternatively, a keyword can be used to collect all the users and their connections who recently tweeted containing that term.  From this selected sample, several observations can be made:

> Not every keyword is equally connected

> Not every twitter user is equally connected nor are their neighbors

> Selected data extractions can be useful in the absence of a global view

Fourth International AAAI Conference on Weblogs and Social Media
(ICWSM-10)
May 23-26, 2010
George Washington University, Washington, DC

Sponsored by the Association for the Advancement of Artificial Intelligence

The ICWSM 2010 conference starts Sunday.  This is a very high quality conference on the study of social media.  My colleague, Professor Derek Hansen, and I will lead a tutorial on using NodeXL to analyze social media networks.

SA2: Introduction to Social Media Network Analysis
Marc Smith (Connected Action) and
Derek Hansen (University of Maryland)

Social networks are the defining data structure of social media, created as people reply, link, click, favorite, friend, re-tweet, co-edit, mention, or tag one another. In this tutorial, we review the core concepts and methods of social network analysis and apply it to the collection, analysis, and visualization of social media networks. Using the free and open NodeXL application, learn how to extract a social media network and generate metrics and visualizations that highlight key people and positions within streams of tweets, videos, photos, or emails.

AuthorLine visualizations represent the weekly rates and nature of contribution from a single author over the span of a year.  Each vertical strip of bubbles represents the activity of that author in a week.  The 52 strips of bubbles in these images represent a year of contribution history.  If users reply to threads other users initiate, they get blue bubbles that  sit below a middle dividing line.  Each thread to which they contribute in that week is represented as a separate bubble.  Each additional message contributed to that thread in that week adds to the size and transparency of the bubble.  If the author initiates new threads, they get a red bubble that sits above the line.

The results is an “at-a-glance” view of the pattern of participation for a year in the life of a threaded conversation message contributor.  The two images below illustrate just how different contributors can be in their patterns of engagement in a threaded conversation environment.  This image represents a contributor of mostly reply messages who starts a small amount of thread initiation at the end of the year but who contributes significantly more to threads started by others.  They have a habit of engaging in lengthy discussions, adding dozens of messages to one or more threads almost every week of the year.

http://www.boardtracker.com/cgi-bin/display_member.pl?board=4202&member=524613

Contrast this image with the distinct pattern created by this author who heavily initiates new threads while responding to those started by others more modestly until the pattern inverts at the ned of the year where they shift to a more heavy reply rather than initiation pattern.

http://premium.boardtracker.com/cgi-bin/display_member.pl?board=4202&member=501671

The paper that described the early work on these visualizations can be found in the HICSS 2004 conference:

Viégas, Fernanda B., Marc Smith. “Newsgroup Crowds and AuthorLines: Visualizing the Activity of Individuals in Conversational Cyberspaces“, Proceedings of Hawaii International Conference on Software and Systems (HICSS) 2004. [Best Paper: Persistent Conversation Minitrack]

A related paper applies these visualizations to document the range of variation found in social media spaces built around threaded discussions.

Tammara Turner, Marc Smith, Danyel Fisher and Howard Ted Welser, Picturing Usenet: Mapping computer-mediated collective action. Journal of Computer Mediated Communication, 2005. [Local copy]

[Thanks to BoardTracker CEO Ron Kass for these images!]

In contrast, this is the map of the term “global warming” with many more isolated tweets and two major clusters in the giant component, a group of global warming deniers and a smaller group of with scientific affiliations and content.

On May 21st, I will speak at the Social Graph Symposium in Mountain View, California along with Sharad Goel – Research Scientist at Yahoo and DJ Patil – Chief Scientist at LinkedIn.

The Social Graph Symposium brings together academics, experts, influencers, executives, and other prominent figures with an interest in social graphs. The event is centered around the social graph and its implications for business, technology, and the community.

Discussions this year will center around:

• State of the Social Graph
• Latest Social Graph Research
• Activism
• Advertising
• Influencer and Word-of-Mouth Marketing
• Marketing
• Retail
• Search
• Social Gaming

Here is a map of recent connections among  twitter users who mention the term “Social Graph”:

The oil spill disaster in the Gulf of Mexico is a topic of great concern to many people in twitter.

This is the map of connections among people who tweeted the term “oil spill”.  There are lots of isolated authors, people who tweeted “oil spill” but are not connected to anyone else who said the same phrase.  The “giant component” is relatively sparse, there is no dense core of “oil spill” people yet.  This is in contrast with many topics where a large, highly interconnected cluster of people defines the “center” of the discussion.  This term remains highly diffuse.

This is the map of connections among people who tweeted the term “BP”.

The BP map also has a large population of isolates, people who are not part of  a “BP” related conversation but have said the term. A small core of highly interconnected users is forming in the “BP” hashtag space but in contrast to a term like “solar”, there is still little cohesion and density in its core.

This is the map of connections among people who tweeted the term “solar”.

This is a topic with a more dense core of highly interconnected Twitter users who share the use of the string “solar” in their tweets.  Ranks of isolates are also present in this topic space but the giant component is more internall connected and cohesive.

Are social media sites data banks, secure repositories of personal assets, or data pimps, soliciting intimate exposure for profit?

I think these services need to choose.  I notice that the setting for who can see what in various systems is in flux.  I can set something to private today and may have to reset it keep it private later.

When I upload content to a site, shouldn’t the expectation be that the deposit is governed by the terms at the time of the contribution?  Why should terms change after I upload?  At least, shouldn’t new rules apply only to new content or content explicitly that has had permissions altered.

Banks do lend out the money I provide them, but only in an anonymous way.  No one knows my dollars are in their mortgage or car loan.  Only legally authorized entities can see my banking records (or so I hope).

Data pimps seem to want to give away anything I give up.  They sell my data as quickly and for as much as possible.

Banks have now developed a reputation that does not make them a great contrast for data pimps, but they still try to represent values like security, confidentiality, and reliability.

I have personally assumed that all data I upload is public.  Only my pictures of my kids have been made “private” and I would not be surprised if those pictures ultimately become public.

Photo credit: cambodia4kidsorg

This is a map of the connections among the population of people who recently tweeted the term “WWW2010″.  The size of the profile photos indicates the log scaled number of followers that person has.  The edges vary in weight, thicker edges mean more of the three types of connections we track, follows, replies, and mentions, appear between any two people.

This is April 29th:

This is April 28th:

A row of isolated people stand at the bottom of the graph, mentioning “www2010″ but lacking any visible connection to anyone else who tweets the same term.

The giant component is dominated by the corporate twitter accounts for information technology companies and prominent researchers in the field.

One way to rank people in a network is in terms of how “between” they are — how much they occupy a position that is like a bridge between otherwise disconnected people or groups.  There is a correlation between the number of connections and your betweenness centrality, but it is possible to be very between and have just a few (at least two!) connections.

In this layout the x axis is the log of the number of followers and the y axis is the log of the number of tweets.  Some people out perform the relationship between tweets and followers, attracting more followers for fewer tweets than others.  Above the diagonal, some people lag in terms of their tweet to follower ratio.

These reports and images were generated with NodeXL, the free and open social media network analysis add-in for Excel 2007/2010.  The dataset 2010 – April – 28 – NodeXL – Twitter www2010 is available here. A book, Analyzing social media networks with NodeXL: Insights from a connected world, is forthcoming summer 2010 from Morgan Kaufmann and from Amazon.

I will present a workshop on social media network analysis at the next Network Singularity event at Fort Mason on the Bay in San Francisco on April 30, 2010.

Bay Area and Silicon Valley Network: Special Action/Research Event

Social Media Network Analysis: Next Practices in Social Network Analysis, Tools and Media

Friday 30 April 2010
8:00am -5:00pm
http://www.regonline.com/BAN10
Workshop Abstract
Why do some social media and online groups succeed when others fail? How do different collections of online media and populations of authors and users differ from one another? How do patterns of contribution vary? How do these differences illustrate the roles people play within their communities?

Patterns of contribution and connection determines social media success. Visualizing these network patterns aids implementation, adoption, security and effectiveness of social media. A range of Internet social media including discussion groups, Twitter, enterprise social media, communities-of-practice, blogs and email are presented, analyzed and visualized. Network patterns are explored to illustrate the scope of variation among social media repositories and between types of contributors.

Maps and patterns of interaction deliver a far more comprehensive view of social media. These views generate actionable findings to improve social media effectiveness, syndication and collaborative outcomes. Network analysis and intelligence can guide community cultivation, coordination and development tasks. New network capabilities provision features that improve search, ranking and consumption of user generated content.

A freely available, open source tool will be demonstrated to perform basic social media network analysis for contemporary social media that should be relevant to business models and organizations of all types. Delegates will be equipped to use social network techniques to improve overall effectiveness and impact of social media, communities-of-practice and social software applications. Participants may expect to achieve immediate improvements in key business activities such as marketing, sales, engineering, support, service, innovation and overall organizational effectiveness.

Directions

Here is an example: http://twitpic.com/9rvfq

It provides a good illustration of the ways a person’s social network is clumped into clusters built around life phases, workplaces, educational institutions, teams and locations.  As people move through more of these stages of life during the Facebook era (and often before) they accumulate these clusters.

Facebook or other contact and friend management systems might could leverage this clustering to organize the presentation of contact information streams.

Bernie recently announced on the SOCNET list that he has updated his script for downloading your Facebook network.

“Features:

1. Its faster. (Presently orders of magnitude faster than Nexus, Touchgraph or ORA).
2. It gives nice feedback during the download.
3. It has less bugs!
4. It gives you the output as a file you can right-click and save rather than copy-paste.
5. IDs are names.”

Bernie writes that phase two of his project is underway.

Bernie is planning a demo at the Sunbelt social network analysis conference in Italy in 2010.

Bernie is the author of the Facebook chapter in our forthcoming book Analyzing Social Media Networks with NodeXL: Insights from a connected world available from Morgan-Kaufmann in July 2010.

In March, 2010 I had the pleasure of attending and speaking in Segovia, Spain at the IE University Department of Communication at the symposium on Transnational connections: Challenges and opportunities for communication.

The symposium is organized with Center for Global Communication Studies at the Annenberg School for Communication, University of Pennsylvania.  I participated in a video interview with several attendees that was created to summarize the themes and topics discussed at the symposium.  The video includes my colleague John Kelly, from the Berkman Center for Internet & Society, Harvard University and Founder of Morningside Analytics, along with Adrian Monck (World Economic Forum), and Shanto Iyengar (Stanford University).

In a shorter segment several people were asked “What makes a good modern communicator”: here is my reply.

Here was John Kelly’s response as well.

More video is available from IE University’s YouTube channel.

The #ashtag network is highlighted by well known technology bloggers.

In contrast the map for the tag “#ashcloud” is dominated by the UK and Scottish travel and Foreign office twitter accounts.

This image features a minor but useful tip for building multi-line tooltips.  This can help pack a lot of information about a Node into the tooltip.  To build a multi-line tooltip use the “&CHAR(10)&” feature to glue other columns of data together along with labels.

Add a column to the Vertices worksheet and paste something like this into it:

=”Twitter user: “&Vertices[[#This Row],[Vertex]]&” – Followers:”&Vertices[[#This Row],[Followers]]&CHAR(10)&”Description: “&Vertices[[#This Row],[Description]]&CHAR(10)&”Last tweet: “&Vertices[[#This Row],[Tweet]]

Which creates a tooltip with multiple lines containing the Twitter User name, the number of their followers, a new line containing their user description followed by their last tweet on another line.  Any concatenation of columns contents can be assembled in this way.

A complete social network map, with edge thickness proportional to the number of network connections (follows, replies, mentions) that link the two participants is here:

The zoomed version highlights the key players in the network and the tighter connections that surround the account for Twitter founder “Ev”.

This is the network map of follows, replies, and mentions relationships among Twitter users who mentioned “CHI2010″ as of  April 12, 2010 scaled by number of followers.

This chart is a log/linear plot of followers (X) over tweets (Y).

Some people tweet a lot in order to get lots of followers, while others can tweet only modestly and gain a vast following.  On a log/log scale this looks like:

The log scales are mislabeled (it is a known, pending work item!), but the pattern is clear: Tweet more and get more Followers.  Some people break from the pack, of course, leading or lagging the core population.  The tooltip is the latest tweet from the most followed Twitter author in the population.

by chris.corwin

– - 

We have several data import providers (spigots) in NodeXL that query popular sources of social media for information that can be processed into a network graph.  User and search term networks from Twitter, YouTube, and flickr have been implemented for a while along with a connector to email reply networks through the Windows Search Index.  NodeXL also imports data from several popular network analysis file formats, opening up data sets and sample libraries used in many courses.

Recently, we added a 3rd party provider for web hyperlink networks through the VOSON service from Prof. Robert Ackland at the Australian National University.

We plan a number of additional data import spigots and would like to know about additional sources of network data structures that would be of interest.

The following data sources are possible targets for a new data provider.  Which matter most to you? SharePoint? Exchange? Active Directory? LinkedIn? Media Wikis?

Travel Industry Social Media Analysis: United v Delta – How do airline twitter maps compare? from Marc Smith on Vimeo.

The two maps we discuss are: United Airlines:

and Delta Airlines:

United has a large twitter mentioning population and has a bigger main component.  The larger profile photos indicate Twitter authors with many followers.  The large population of isolates or small components at the bottom of both images are people who mention either airline but are not in a conversation or a relationship with someone else who also mentioned these brands.  They are “shouts” about a brand, not conversations.  In contrast the large component in both images are the connected collection of people who talk to people who talk about these brands.  They are committed to the topic in a way the less connected  authors are not.  They know someone who also talks about air travel.

Viewed over time, we can start to assess the ways these brand’s twitter populations are changing.  Are new people moving into central hub positions?  Are people who held those positions drifting away?

A key observation is that some people with relatively few followers occupy highly central positions in the graph.  This suggests that these authors have a location that lends their content attention from other well positioned people.  Popularity is not just about volume of connections, in a social network perspective, importance is also a function of where the person sits within the graph.

Along with my colleague Eduarda Mendes-Rodriguez I will be speaking at the University of Porto next Friday, March 26th.

Hands-on Workshop – Social Media Network Analysis in NodeXL

This workshop provides an overview of Social Network Analysis (SNA) and its application to social media. The network or directed graph is a common structure in a wide range of different kinds of social media. Social Network Analysis is a set of tools, concepts, and techniques that can help measure a graph and the location and connection pattern of each component part.

Using NodeXL, workshop participants will learn how to take data from common social media sources (including enterprise discussions and online communities, Twitter, Flickr, your own email) and perform various types of analysis. Through this workshop, participants will:

· be able to understand the basics of SNA, its terminology and background.
· be able to transform communication data (e.g. Twitter, email, Flickr, message boards etc.) into network data.
· understand the different possible presentations of social networks, e.g. in a matrix or a sociogram.
· apply network metrics and visualizations to find clusters and key contributors in real world social media data.
· get familiar with the use of standard SNA tools and software in general and the NodeXL social network analysis add-in for Excel in particular.
· be able to derive practical and useful information through SNA analysis that would help design an innovative and successful online community.

The Annual Meeting of the Israel Internet Association (http://www.isoc.org.il (English)) was held February 22-23 2010. I spoke at this year’s meeting: http://www.isoc.org.il/conf2010/agenda.php?lang=en

Part 1

Part 2

The previous year’s conference website is at: http://www.isoc.org.il/conf2009/program.php

The Israel Internet Association is the official Israeli Chapter of the Internet Society.  Their annual meeting is a central event of academics (sociologists, psychologists, business and law) as well as industry participants from sectors including mobile cellular companies and internet service suppliers.

After speaking at the Israel Internet Association conference February 22, I will speak at IBM Research in Haifa at an event co-sponsored by the University of Haifa Department of Sociology and Anthropology and the M.A. program in Sociology of Technology.  My hosts are Professor Gustavo Mesch from the university and Dr. Adam Perer from IBM research.

Wednesday, February 24^th, 1 pm

IBM Building
Haifa University

Title: Visualizing collections of social media connections: using social network analysis to assess, evaluate and measure social media engagement

Abstract: Social networks are created whenever people interact.  These networks become more visible when interactions take place through social media.  Social networks form when people link, reply, comment, edit, tag, and friend one another.  Sub-populations are formed whenever people mention the same company, products, event, topic, or personality.  Using social network analysis on collections of social media connections reveals important patterns: how are people clustered and grouped, where are the gaps, who plays the roles of bridge, hub, and isolate? In this talk I will display maps of twitter, you tube, flickr, and enterprise email systems and demonstrate several tools that can be used to collect, analyze, map and monitor social media, including the free and open NodeXL (network overview, discovery and exploration) add-in Excel 2007.

Here, for example, is a map of the connections among people who recently mentioned “haifa” in twitter sized by number of followers:

Some photos taken during the trip are available after the jump:

The fellowship is designed to support research in the following areas:

• Internet Economics: auction theory and algorithmic game theory relevant to online advertising auctions.
• Cloud Computing: storage, databases, and optimization for computing in a massively distributed environment.
• Social Computing: models, algorithms and systems around social networks, social media, social search and collaborative environments.
• Data Mining and Machine Learning: learning algorithms, feature generation, and evaluation methods to produce effective online and offline models of behavioral signals.
• Systems: hardware, operating system, runtime, and language support for fast, scalable, efficient data centers.
• Information Retrieval: search algorithms, information extraction, question answering, cross-lingual retrieval and multimedia retrieval

Applications are due on February 15th. For full details, check out the Facebook Fellowship page.

Please pass this information on to any PhD students you think might be interested in applying.

The deck illustrates the use of NodeXL to extract several social media networks from systems like twitter and facebook to generate maps of communities and identify people and objects in key locations.

Using surnames from users in the United States and comparing the rates at which those surnames occur in several ethnic populations, they were able to estimate the proportion of Facebook users within different populations over time. The graphic shows a time series of the estimated percentage of U.S. Facebook users from each of 4 races/ethnicities, normalized by the expected percentage. In other words, a value of 100% means the population in question has the same proportional representation on Facebook as it does in the census data. A value over 100% means the population is over-indexed among Facebook users, and a value under 100% means the population is under-represented on Facebook. It appears that Facebook has always been fairly diverse, and apart from over-indexing for Asians/Pacific Islanders the user population is now reasonably representative of the Internet using population in the U.S.

For more details on the study, including the estimation and modeling approaches used, check out the original note on the data team page, or Cameron Marlow’s full repost.  Lars Backstrom, Jonathan Chang, Cameron Marlow and Itamar Rosenn conducted this research.