This is the collection of keyword pairs that appeared in two clusters of people who Tweeted about “Paul Ryan”, the Republican Congressman from Wisconsin who delivered the GOP rebuttal to the 2011 United States State of the Union Address.  This network illustrates the ways that certain word pairs appears only or predominantly in one cluster (colored here Red and Blue) or the other. Terms that appeared in both clusters appear as purple.

Social networks are built from relationships between people.  Keyword networks are built from relationships between words and other text strings.  When two words appear in the same message, sentence, or alongside one another ties of different strengths are created.  The networks that result can illuminate the relationships among topics of importance in a collection of messages.

Markus Strohmaier from the Technical University Graz (TUG) along with Claudia Wagner gave us inspiration in a paper:

C. Wagner, M. Strohmaier, The Wisdom in Tweetonomies: Acquiring Latent Conceptual Structures from Social Awareness Streams, Semantic Search 2010 Workshop (SemSearch2010), in conjunction with the 19th International World Wide Web Conference (WWW2010), Raleigh, NC, USA, April 26-30, ACM, 2010. (pdf)

in which they defined a range of ways two words (technically these are strings, they may not really be words) can be associated with one another.  Words could be linked if they are in the same tweet, next to one another, or sequential among other ways to link terms.

NodeXL has not had any features for exploring the networks in texts.  Now with the addition of a new macro from Scott Golder, it is fairly simple to extract pairs of keywords from collection of tweets.  NodeXL’s Twitter importer can optionally include the content of the tweet that included the search term and this column of text can now be processed itself into a new network based on the ways words appear together in tweets.

This feature builds on the work of several people.  Scott Golder from Cornell started the ball rolling with a simple but effective VBA script that allowed others to build and refine the models of what counts as a tie between two words.  Vladimir Barash added several refinements including support for stop word lists to remove common terms.  Scott then picked up the code again and added a set of features for selecting the nature of the graph and making it easier to select the options needed.

The code for the Keyword Network macro is below.

The instructions to use it take a few steps to complete:

1. Create a new workbook, eg a list of tweets or an import from a Twitter search, whatever. Save it as .xlsm. The m is important. This can be an existing NodeXL workbook.

2. Go to Developer -> Macros. Make up a name; it doesn’t matter because it’ll get overwritten. Then press Create. the VBA window will open.

3. In the big text are that says “Sub whatever() End Sub”, select all that text and delete it. Paste in the contents of the text file below.

4. Go to Tools->Reference. Check the checkboxes for “Microsoft Scripting Runtime” and “Microsott VBScript Regular Expressions”. Press OK. Save the file (File->Save) then exit (“Close and return to Microsoft Excel”).

5. Now go to Developer -> Macros. Choose CreateWordNet and press the Run button.

6. It’ll ask you for a worksheet name, a column and a start-row. Then it’ll create a new worksheet with the edgelist in it.

The edge list is not directed (there isn’t really a concept of direction in “co-occurs”) but is weighted. Each pair is weighted by the number of times it appears.

This version also includes options for edge creation.

First, it is now possible to suppress edges of weight=1, which is helpful in getting rid of a lot of garbage.

Second, it is now possible to defined edges by adjacency or co-tweeting. Given a tweet of words “w1 w2 w3″ adjacency will give edges w1-w2 and w2-w3, while co-tweeting will give edges w1-w2, w1-w3, w2-w3.

For edges defined by adjacency, you may choose directed or undirected edges. So a tweet of “Marc Smith Marc” (for example) would generate the weighted directed edges Marc,Smith,1 and Smith,Marc,1 while the sole undirected edge would be Marc,Smith,2. That is, for undirected edges (where ordering doesn’t matter) the words are alphabetized.

Create Word Network VB Macro Vdb5

The ACM CHI 2011 Conference is taking place in Vancouver, BC.

May 10, 2011

May 8, 2011

These are the connections among the Twitter users who recently tweeted the word chi2011 when queried on May 8, 2011, scaled by numbers of followers (with outliers thresholded). Connections created when users reply, mention or follow one another.

Layout using the “Group Layout” composed of tiled bounded regions. Clusters calculated by the Clauset-Newman-Moore algorithm are also encoded by color.

A larger version of the network map is here: www.flickr.com/photos/marc_smith/5701022497/sizes/o/in/ph…

Top most between users:
@chi2011
@msftresearch
@edchi
@jeffbigham
@landay
@evanmpeck
@acagamic
@clifflampe
@yardi
@eytanadar

Graph Metric: Value
Graph Type: Directed
Vertices: 421
Unique Edges: 3849
Edges With Duplicates: 324
Total Edges: 4173
Self-Loops: 0
Connected Components: 32
Single-Vertex Connected Components: 29
Maximum Vertices in a Connected Component: 384
Maximum Edges in a Connected Component: 4146
Maximum Geodesic Distance (Diameter): 6
Average Geodesic Distance: 2.553249
Graph Density: 0.022587943

NodeXL Version1.0.1.167

Data set in NodeXL format.

NodeXL is free and open and available from www.codeplex.com/nodexl

NodeXL is developed by the Social Media Research Foundation (www.smrfoundation.org) – which is dedicated to open tools, open data, and open scholarship.

This is a map of the connections among the Twitter users who recently tweeted the words #rw2011 when queried on April 28, 2011, scaled by numbers of followers (with outliers thresholded), connected by replying, mentioning and following.

Layout using the “Group Layout” composed of tiled bounded regions. Clusters calculated by the Clauset-Newman-Moore algorithm are also encoded by color.

A larger version of the network map is here: www.flickr.com/photos/marc_smith/5666637994/sizes/o/

A detailed list of top most between users is here: www.flickr.com/photos/marc_smith/5666069249/

Top most between users: @nytimes, @skynews, @peoplemag, @royalweddingcnn, @itv_news, @katewilliam2011, @julieetchitv, @abc, @cbsnews, @abcroyals

Graph Metric: Value
Graph Type: Directed
Vertices: 1000
Unique Edges: 1735
Edges With Duplicates: 531
Total Edges: 2266
Self-Loops: 0
Connected Components: 271
Single-Vertex Connected Components: 246
Maximum Vertices in a Connected Component: 701
Maximum Edges in a Connected Component: 2199
Maximum Geodesic Distance (Diameter): 11
Average Geodesic Distance: 3.892905
Graph Density: 0.001984985
NodeXL Version: 1.0.1.166

NodeXL is free and open and available from www.codeplex.com/nodexl

NodeXL is developed by the Social Media Research Foundation (www.smrfoundation.org) – which is dedicated to open tools, open data, and open scholarship.

The book, Analyzing social media networks with NodeXL: Insights from a connected world, is available from Morgan Kaufmann and from Amazon.

I will be speaking at the Oxford Internet Institute on July 13 and 14, 2011 as part of the Summer Doctoral Program.

I will speak about the use of social network analysis and visualization techniques to analyze and understand social media.

The Summer Doctoral Program is designed to

…bring together advanced doctoral students engaged in dissertation research relating to the Internet and other ICTs. By sharing their work and learning from leading academics in the field, students can enhance the quality and significance of their thesis research and create a peer network of excellent young researchers.

The 2011 Summer Doctoral Programme will draw substantially upon the OII’s research strengths and will involve participation from many of our faculty, together with colleagues from other partner institutions. It will emphasise methodological innovation and good practice in research design, and will expose students to the benefits of discussing their research in a multi-disciplinary teaching environment.”

Web Science Doctoral Summer School 2011

I will be attending and speaking at the upcoming 2011 Web Science Trust Graduate Summer School tobe held at DERI, NUI Galway Ireland from July 6th to July 13th, 2011.  I will speak about social media network analysis and run a workshop on using NodeXL.  There is a packed schedule featuring a great group of researchers who are gathering at the event, including:

* Prof. Wendy Hall is Professor of Computer Science at the University of Southampton, UK, and Dean of the Faculty of Physical and Applied Sciences.
* Prof. Nigel Shadbolt is Professor of Artificial Intelligence (AI) and Deputy Head (Research) of the School of Electronics and Computer Science at the University of Southampton.
* Prof. Stefan Decker is a professor at the National University of Ireland, Galway, and director of the Digital Enterprise Research Institute.
* Dr. Harith Alani is a senior lecturer at the Knowledge Media Institute, where he is heading a group specialising in Social Semantics.
* Dr. Bernie Hogan is a Research Fellow at the Oxford Internet Institute. He specializes in novel methods for online data capture and analysis, especially via social media.
* Prof. Scott Kirkpatrick has a background in physics (AB Princeton, PhD Harvard) and 15 years of developing new technologies at IBM’s TJ Watson Research Center before coming to the Hebrew University in 2000.
* Prof. Enric Plaza holds a Ph.D. in Computer Science by the Technical University of Catalonia (UPC) and is Research Professor of the Spanish Council for Scientific Research (CSIC).
* Prof. Steffen Staab is the director of Institute WeST – Web Science and Technologies and of the Institute for Computer Science of the University of Koblenz-Landau, Germany.
* Dr. Derek Greene is a Research Fellow at the School of Computer Science and Informatics, University College Dublin. As part of the Clique Research Cluster, his focus is on network analysis and community finding, with a particular emphasis on dynamic mobile and social media networks.
* Dr. Michael Hausenblas is a Research Fellow at DERI, NUI Galway, where he leads the Linked Data Research Centre.
* Dr. Daniele Quercia is Horizon researcher at the University of Cambridge (UK). he is interested in computational social science, web science, and social computing.
* Dr. Markus Strohmaier is an Assistant Professor at the Knowledge Management Institute, Faculty of Computer Science at Graz University of Technology.

The event is focused on the methods and theories needed to grasp the nature of the social implications of the rapid changes in the technologies of communication and computation:

“The Web is the largest technological artefact in existence, comprising a global network of information sites and services. It is a social machine, delivering information between people and communities, embedded in almost all processes of human society: education, medicine, science and technology, commerce, entertainment and social activity. It is often simply supposed that the Web is a neutral technology, a stable computing platform for the delivery of information and services. What is overlooked is that the Web is changing constantly in response to the demands of human society. Incremental innovations leads to changes in how people use the Web and in turn how Web technology responds to changed human interaction. Small technological decisions influence how individuals use the Web and ripple out to have unanticipated macro-effects. Sometimes these effects are beneficial such as the emergence of Web 2.0 technologies. Other technologies such as the rise of spam bots or ‘blackhat’ search engine optimisation techniques clutter the Web with irrelevant, distracting information. While influential corporations such as Google, Facebook and Microsoft generate huge revenues from the Web, the Web itself is owned by everyone and no-one. We need to fully understand the demands placed on the Web by human society, so that its fate does not lead to a ‘tragedy of the commons’ but to a sustainable technological resource for the future.”

http://webscience.deri.ie/index.html

Read More

Lists are a recent feature of Twitter which enable users to compile collections of users to follow in a single tweet stream.  People can add up to 500 people to follow on a single list.    People on a list may be connected to one another if one follows the other.

Some people on a list may have many connections.  Some have only a few or even no connections to others on the list.

In version 161 of NodeXL you may now create maps of the connections among a list of Twitter users.

There are two options in this feature.  One makes use of the List functions in Twitter.  If you request the map for a single twitter list, NodeXL will build a map of the connections among all the people Twitter reports as being on the list.  You can create and manage the people on a list using the Twitter list features, or select an existing Twitter list created by other users.

A second option in this feature accepts a list of up to 10,000 twitter user names pasted into the query text box.  If you have a list of users and want a map of how they are connected, and the list is not already in Twitter, just paste them here and get a map.

Either way, a connection will be created for every two users if one follows the other.

Here is an example of the network map of the Twitter list of social network analysis people maintained by Valdis Krebs: valdiskrebs\network-analysts

Today, we release a new NodeXL feature in version  .164  that we think will improve the results generated by other layout algorithms: Group Layout.  In the Layout Options menu in the NodeXL Network Graph Pane (called “Document Actions” by Excel) you will find that the dialog box has been updated:

Selecting the new “Lay out each of the graph’s groups in its own box and sort the boxes by group size” option and checking the “Show each box’s outline” can generate an image that neatly places each group in a bounded region.

Here we see the set of connections among the people who tweeted the string “msrtf11 OR techfest” – referring to the Microsoft Research TechFest 2011 event that took place in early March in Redmond, Washington.  This map clarifies the set of set of relationships among sub-groups within the graph.

Contrast this with the layout created without this feature turned on:

We still have some refinements coming that will improve this layout further (for example, the neat grid of isolated nodes takes an extra step today, we will try to make that happen automatically soon) but this is a nice step in making network graphs easier to understand.  Let us know what you think!

Here is a recipe for finding and building collections of influential Twitter users relevant to your goal and message.

A common question related to social media that I am often asked is: “how do we grow our twitter followers?”  I try to re-frame that question as “how can we get our message viewed by the right people?” and its related question: “who could best pass our message along?”

The goal is not to gain many followers, but rather to gain well positioned followers!  A well positioned follower is someone who talks about topics related to your topic and is located at the center of many relationships with other people in that topic network.  These highly central people are able to get their message to be visible to many people. If they re-tweet you, your message will go far.  Why would they re-tweet you?  First, they have to be able to hear you to re-tweet you and that means that you need to get these people to follow you.  Second, you need to say something sensible enough to be worth re-tweeting.

The task is finding the important users to focus on.  To find them, create a network map of the relevant topics.  A few people in these network maps will occupy key positions in which they have many connections or have special ties that bridge gaps between sub-groups.  There are often only a few of these people in any topic area.

Find these strategically located people and follow them and often they will follow you back. A good proxy for the strategic value of a person’s location is their “betweenness centrality” – a measure of how much a person acts a bridge between others.

If the people you select and follow do not immediately follow you back there is another opportunity to get them to follow you.  If you re-tweet particularly good content from the people you have selected to follow there is a good chance that you will be followed back.

Once you have identified, followed, and been followed back by some fraction of the influential people around your topics of interest you have a increased chance that your tweets will be re-tweeted.  For each re-tweet, your message is exposed to a potentially large and highly targeted group of people.  Over time it is possible to create a relationship with a collection of highly influential participants who may be interested in sharing your message with their followers.

On 11 March 2011, I will be speaking at:

Organizational and Social Media Network Analysis:
Next Practices in Organizational Networks, Analytics and Media

Along with Valdis Krebs, Chief Scientist, http://thenetworkthinkers.com/

Valdis will be speaking about Organizational Network Analysis - Introduction to Enterprise Networks, followed by a InFlow 3.1 Workshop: Social Network Analysis with InFlow 3.1

I will be speaking about Social Media Network Analysis with NodeXL.

Preservation Park
1233 Preservation Park Way
Oakland, CA 94612
(510) 874-7580
(click for map)

Tell a Friend
Add to Calendar

The event is a globally visible opportunity for the President of the United States to present the Administration’s vision for its agenda for the year to the people of the US and the world.

The hashtag #stateoftheunion is already very active in Twitter.

This is a NodeXL map of the connections among people who recently tweeted the string “stateoftheunion” on 21 January, 2011.

The most between user, @keitholbermann, is highlighted.  Two major clusters emerge, shown arbitrarily here as green and blue, and which are more clearly seen in the filtered graph below the list.

Summary network statistics:

Graph Type Directed
Vertices 1259
Unique Edges 8014
Edges With Duplicates 544
Total Edges 8558
Connected Components 306
Single-Vertex Connected Components 293
Maximum Vertices in a Connected Component 937
Maximum Edges in a Connected Component 8513
Maximum Geodesic Distance (Diameter) 9
Average Geodesic Distance 3.140191
Graph Density 0.00522786

The most “between” contributors in the network are:

@keitholbermann, @borowitzreport, @thefix, @katyinindy,
@america1first, @wbconservative, @westwingreport, @dailykos,
@writer2go, and @gottalaff

When contributors with low “betweenness” are filtered out of the graph, two dominant clusters become easier to discern.  To the left are supporters of the President and to the right are those critical of the Administration.  A representative tweet from this cluster is highlighted.

A recent paper “EventGraphs: Charting Collections of Conference Connections” by Marc Smith from Connected Action, Professor Derek Hansen (College of Information Studies) and Professor Ben Shneiderman (Computer Science/Human Computer Interaction Lab) both from the University of Maryland and has been accepted for publication at the 2011 Hawaii International Conference on System Sciences (HICSS) Conference.  This is the 44th year for the conference. Derek Hansen presented the paper on January 7, 2011.

Hansen, D., Smith, M., Shneiderman, B., EventGraphs: charting collections of conference connections. Hawaii International Conference on System Sciences. Forty-Forth Annual Hawaii International Conference on System Sciences (HICSS). January 4-7, 2011. Kauai, Hawaii.

http://www.cs.umd.edu/localphp/hcil/tech-reports-search.php?number=2010-13

Our paper is about visualizing social media and it describes the visualization of the patterns of connections formed when people tweet about events like conferences and news stories.

EventGraphs are social media network diagrams constructed from content selected by its association with time-bounded events, such as conferences. Many conferences now communicate a common “hashtag” or keyword to identify messages related to the event. EventGraphs help make sense of the collections of connections that form when people follow, reply or mention one another and a keyword. This paper defines EventGraphs, characterizes different types, and shows how the social media network analysis add-in NodeXL supports their creation and analysis. The paper also identifies the structural and conversational patterns to look for and highlight in EventGraphs and provides design ideas for their improvement.

A collection of EventGraphs are available in flickr: http://www.flickr.com/photos/marc_smith/sets/72157625618025980/

The EventGraph for HICSS this year is here: 20110109-NodeXL-Twitter-HICSS

The top most between contributors to the HICSS Twitter graph this year are: @arcticpenguin @avantgame @barrywellman @clifflampe @dalprof @chandlerism @shakmatt @marc_smith @shen045 and @thecatandthekey

Here is the data set: 20110109-NodeXL-Twitter-HICSS
Graph Metric Value
Graph Type Directed
Vertices 92
Unique Edges 243
Edges With Duplicates 71
Total Edges 314
Self-Loops 0
Connected Components 21
Single-Vertex Connected Components 18
Maximum Vertices in a Connected Component 69
Maximum Edges in a Connected Component 307
Maximum Geodesic Distance (Diameter) 8
Average Geodesic Distance 3.081693
Graph Density 0.032967033

NodeXL Version 1.0.1.159

An article about social media visualization Social Seen: Analyzing and Visualizing Data from Social Networks by Hunter Whitney, appeared in UX Magazine on December 15, 2010.  It provides a great round up of tools and approaches to mapping populations in social media spaces.

Previous HICSS papers:

Gleave, Eric, Howard T. Welser, Marc Smith, and Thomas Lento.  2009.  “A conceptual and operational definition of social role in online community.”   In Proceedings of the 42nd  Hawaii International Conference on Systems Sciences (HICSS), January 5-8. Computer Society Press.  (Best Paper, Digital Media and Communication Mini-Track)

2009 – HICSS – 42 – Best Paper – A Conceptual and Operational Definition of ‘Social Role’ in Online Community

Fisher, D., Smith, M., and Welser, H. You Are Who You Talk To, Proceedings of HICSS, January 2006. (Best Paper, Digital Media and Communication Program)

http://www.connectedaction.net/wp-content/uploads/2009/08/2006-HICSS-You-Are-Who-You-Talk-To-Detecting-Roles-in-Usenet-Newsgroups.pdf

Viégas, Fernanda B., Marc Smith. “Newsgroup Crowds and AuthorLines: Visualizing the Activity of Individuals in Conversational Cyberspaces“, Proceedings of Hawaii International Conference on Software and Systems (HICSS) 2004. (Best Paper: Persistent Conversation Minitrack)

http://www.connectedaction.net/wp-content/uploads/2009/08/2004-HICSS-Viegas-and-Smith-Newsgroup-Crowds-and-Author-Lines.pdf

Here are some sample event graphs:

As mobile devices become a major method for authoring and consuming social media, location data is increasingly a part of many posts, tweets, check-ins, and messages.  Many Twitter clients, for example, can add the user’s current latitude and longitude to the metadata associated with a tweet.  Other systems like Facebook Places, Google Latitude and Foursquare encourage users to declare  where they are to the world, often passing the information to other social media sites.

Using this location data in network analysis opens up a range of new opportunities.  Instead of a person – to – person social network, location data allows people to be linked to places and, by extension, places can be linked to other places based on the patterns of connection people create when located in a particular place.  A convergence of network analysis and Geographic Information Systems in underway.  A great example of this can be found in this wonderful video from the BBC which demonstrates the idea by mapping the flow of telephone calls, texts, and data around the UK and the wider world.

Link on the BBC

Even better is this video from the SensibleCity group at MIT:

Now, NodeXL (v.156) has the first of a series of features that will start to approximate the experience displayed in the video by supporting the import of location data about networks and plotting networks onto maps.

For now, we have started importing latitude and longitude data that Twitter makes available.  If you check “Add a Tweet column to the Vertices worksheet” in NodeXL, Data, Import, From Twitter Search Network or From Twitter User Network, the Twitter user’s geographical coordinates will be added to the Vertices worksheet when they are available.

Note that not every tweet has a latitude and longitude, in fact many do not (yet).  Further, note that not every latitude and longitude is accurate, many are not.

We need to implement more features for better location data support in a NodeXL workbook, but this is a start.  We are interested in exploring geospatial networks and Twitter is a great data source.  With this data in place we may look into features that emit KML files for exploration in other packages like Google Earth.  A nifty Google Earth/Spreadsheet importer can take small sets (400) of location data points in a spreadsheet and export them to a KML file, something we could implement in the future as well.  In addition we may be able to connect directly with services like Bing Maps and Google Maps to display connections between nodes with known locations.  Metrics that calculate the distance between nodes seem sensible as well.

Location coordinates are the key to a cornucopia of related data about a place.  Given a latitude and longitude it is possible to find the name of the city it is located in, look up data about that location in official records as well as resources like Wikipedia.  Income, education, property values, weather, photos, and more can be pulled together from just a simple lat/long.  All of these attributes could be used to cluster or illustrate the network visualization.

NodeXL has a number of data importers that can create a network of connections from social media data sources like Twitter, YouTube, flickr, email, and the WWW (along with a number of other data import formats like GraphML, UCINet, CSV, and other Excel workbooks with data).

To create a network you just select the search terms and configurations you want from the NodeXL>Data>Import menu.

If you want to create the same network every day (or at any schedule), a recent feature (since version .125) of NodeXL can help. NodeXLNetworkServer.exe is an application that ships with NodeXL along with a sample configuration file called SampleNetworkConfiguration.xml. By editing the configuration file you can set NodeXL to collect anything available in the menu through Excel.  So far we have exposed the two Twitter data collectors (more on the way) so the configuration file asks you to select a search term or a user’s name, the size of the network and the details you want reported along with the location and name of the destination file that NodeXL will create.  Answer these questions by editing the config file and save it with a useful name that includes the search term.

Step by step details after the jump:

Editing the Configuration file

NodeXL ships with a SampleConfigurationFile.xml that you should copy, rename and edit.

You may want to create a directory to hold this configuration file if you expect that there will be many files.

The configuration file is written in XML but it it is very human readable and editable.  Here are the key elements of the file that depend on you to choose and configure the file appropriately.

NetworkType Specifies the type of network to get.  Must be one of the following values: TwitterSearch or TwitterUser

After setting NetworkType, you must also edit one of the following sections:

TwitterSearchNetworkConfiguration

TwitterUserNetworkConfiguration

TwitterSearchNetworkConfiguration is used only if NetworkType is TwitterSearch.

SearchTerm - What to search for.

WhatToInclude - What to include in the network.  This must be a combination of the following values, separated by commas:

Statuses - Include each person’s status (tweet).

Statistics - Include each person’s statistics.

FollowedEdges - Include an edge for each followed relationship.

RepliesToEdges - Include an edge from person A to person B if person A’s tweet is a reply to person B.

MentionsEdges - Include an edge from person A to person B if person A’s tweet mentions person B.

<WhatToInclude>Statuses,Statistics,FollowedEdges,RepliesToEdges,MentionsEdges</WhatToInclude>

MaximumPeoplePerRequest - The maximum number of people to request for each query, or leave empty for no limit.

NetworkFileFolder - Full path to the folder where the network files should be stored.

NetworkFileFormats - Specifies the file formats to save the network to.  This must be a combination of the following values, separated by commas:

GraphML - Save the network to a GraphML file, which can be imported into a NodeXL workbook.

NodeXLWorkbook - Save the network directly to a NodeXL workbook.  To use this option, the NodeXL Excel Template must be installed on this computer.

<NetworkFileFormats>GraphML,NodeXLWorkbook</NetworkFileFormats>

AutomateNodeXLWorkbook - Specifies whether the NodeXL Excel Template’s automate feature should be run on the workbook.  Must be true or false.  This is used only if NetworkFileFormats (above) includes NodeXLWorkbook.

If true, the automate options you most recently set in the NodeXL Excel Template are used to automate the workbook.  To set the automate options, do the following:

1. Open the NodeXL Excel Template.

2. In the Excel ribbon, Go to NodeXL, Graph, Automate.

Note that the “On this workbook” and “On every NodeXL workbook in this folder” selection in the Automate dialog box is ignored when automating the workbook from the NodeXL Network Server.

<AutomateNodeXLWorkbook>true</AutomateNodeXLWorkbook>

TwitterUserNetworkConfiguration

This section is used only if NetworkType is TwitterUser.

ScreenNameToAnalyze - The screen name of the Twitter user whose network should be analyzed.

WhatToInclude - What to include in the network.  This must be a combination of the following values, separated by commas:

FollowedVertices – Include a vertex for each person followed by the user.

FollowerVertices - Include a vertex for each person following the user.

LatestStatuses - Include each person’s latest status (tweet).

FollowedFollowerEdges - Include an edge for each followed relationship if FollowedVertices is specified, and include an edge for each follower relationship if FollowerVertices is specified,

RepliesToEdges - Include an edge from person A to person B if person A’s latest tweet is a reply to person B.

MentionsEdges - Include an edge from person A to person B if person A’s latest tweet mentions person B.

<WhatToInclude>FollowedVertices,FollowerVertices,LatestStatuses,FollowedFollowerEdges,RepliesToEdges,MentionsEdges</WhatToInclude>

NetworkLevel - Network level to include.  Must be One, OnePointFive, or Two.

MaximumPeoplePerRequest - The maximum number of people to request for each query, or leave empty for no limit.

NetworkFileFolder - Full path to the folder where the network files should be stored.

NetworkFileFormats -Specifies the file formats to save the network to.  This must be a combination of the following values, separated by commas:

GraphML -Save the network to a GraphML file, which can be imported into a NodeXL workbook.

NodeXLWorkbook - Save the network directly to a NodeXL workbook.  To use this option, the NodeXL Excel Template must be installed on this computer.

AutomateNodeXLWorkbook

Specifies whether the NodeXL Excel Template’s automate feature should be run on the workbook.  Must be true or false.  This is used only if NetworkFileFormats (above) includes NodeXLWorkbook. If true, the automate options you most recently set in the NodeXL Excel Template are used to automate the workbook.  To set the automate options, do the following:

1. Open the NodeXL Excel Template.

2. In the Excel ribbon, Go to NodeXL, Graph, Automate.

Note that the “On this workbook” and “On every NodeXL workbook in this folder” selection in the Automate dialog box is ignored when automating the workbook from the NodeXL Network Server.

One key configuration is this last choice to turn on the automated processing of the resulting data set.  If you set this to true, NodeXL constructs a Twitter social network and then performs all of the steps of automation that you define.  For example, NodeXL can calculate graph metrics, find clusters, create subgraphs, map a set of autofill column mappings of data to display attributes, set graph layouts and settings and render a graph without any human intervention.

Once you have a properly edited configuration file you can simply open a command line session (go to the Start menu, type in “CMD”) and type the following:

> NodeXLNetworkServer.exe SampleNetworkConfiguration.xml

And you will get a stream of messages about what parts of the network are being actively collected:

This is a simple example and not too useful: you may as well just do this through the NodeXL Excel interface.  But things get more interesting when one more piece is added: Windows Task Scheduler.  You may not see it that often but Task Scheduler is on almost every Windows desktop and can be used to automate the collection of NodeXL data sets from Twitter and other sources of social media networks.

To get to Task Scheduler just type its name into the Start Menu search box.  It should look like this:

Using this tool you can create new Tasks that will execute on a specific time and frequency.

Create and select a new folder (call it something like “NodeXL Data Collections”) under Task Scheduler Library to hold all your NodeXL data collections separate from other scheduled tasks.

Use the Create Task menu item on the far right.

You will get a Create a task dialog box:

Enter a Name and a Description that captures the search terms of your query and then select the “Actions” tab.

Actions are where the command that you want to execute is defined.  Select “New…” to create a new Action.

This will open a dialog box in which you can define the program to be run along with any settings.

Enter the complete path to the NodeXLNetworkServer.exe application in the “Program/script” field.

Add the complete path to the configuration file in the “Add arguments” field.

To easily capture the complete path (and the quotation marks needed if there are spaces in the path) hold the SHIFT key down while right clicking the name of the configuration file you want to schedule for collection.  You should see an option “Copy as Path” which will place the needed information into the clipboard.  After selecting “Copy as Path”, return to the “New Action” dialog box and paste the path into the “Add arguments” field.

Once the path to the NodeXLNetworkServer.exe file and the path to the configuration file have been entered into the New Action dialog, shift to the Triggers tab of the Create Task dialog.  Select “New…” to create a new trigger.

Triggers are defined by many things, but we will focus on time.  When the New Trigger dialog is set to “On a schedule” you can choose the time and frequency to run the collection.  If you want to run a collection daily at 7:01:30AM each day the following settings should work:

Once the time and recurrence are set select OK and you have a new task!  You may want to create a task that starts in a minute or two to test that the event fires properly.  You can then schedule these collections to run at an hour when you will not be bothered by the interruption.  Multiple collections can be scheduled but several limits suggest that only a few can run simultaneously.  You may want to start collections only a few times a day or hour to allow one collection to complete before others begin.  When these tasks do execute you should see a collection session appear in a console window and report updates as it steps through the many stages of constructing a network dataset.

Collection is much faster if you have a rate limit lifted account, which you must request from Twitter.  With a credentialed rate-limit lifted account you can perform several queries per hour.  With a regular account with credentials (your Twitter login) you can get one or two queries per day depending on the size of the data collected.  In either case it is possible to reach the limit that Twitter will provide.  When that happens NodeXL will pause the collection and wait until the API query budget refreshes and Twitter is willing to serve more query results.  As a result, even accounts without rate-limits lifting can create large complex social media network maps, although at a much slower rate.

A new paper on visualizing social media has been released on the University of Maryland, Human Computer Interaction Laboratory tech report archive.  Co-authored by Derek Hansen,  myself, and Ben Shneiderman, the paper describes and visualizes the patterns of connections formed when people tweet about events like conferences and news stories.

EventGraphs_2010_HCIL_Tech_Report

http://www.cs.umd.edu/localphp/hcil/tech-reports-search.php?number=2010-13

Hansen, D., Smith, M., Shneiderman, B.
EventGraphs: Charting Collections of Conference Connections
HCIL-2010-13

EventGraphs are social media network diagrams constructed from content selected by its association with time-bounded events, such as conferences. Many conferences now communicate a common “hashtag” or keyword to identify messages related to the event. EventGraphs help make sense of the collections of connections that form when people follow, reply or mention one another and a keyword. This paper defines EventGraphs, characterizes different types, and shows how the social media network analysis add-in NodeXL supports their creation and analysis. The paper also identifies the structural and conversational patterns to look for and highlight in EventGraphs and provides design ideas for their improvement.

The International Sunbelt Social Network Conference is the official conference of the International Network for Social Network Analysis (INSNA).

This year’s INSNA “Sunbelt” conference is at the  Riva del Garda Fierecongressi, Trento, Italy!  Here is the 2010 INSNA Sunbelt Program.

This is the NodeXL map of connections among people who tweeted the hashtag used for the conference “#sunbelt”.

Having now seen several of these maps for other topics and events (see: http://www.flickr.com/photos/marc_smith/sets/72157622437066929/) this map can be placed in context.  It is a small group, but has a high density of connections.  It lacks isolates, the people who say the term but do not connect to others who say that term.  This means that this is a very “in-group” population: if you know to use the #sunbelt hashtag, you probably connect to someone else who uses the term.  It is a single major cluster of connected people, no obvious sub-graphs or clusters are visible.  Not everyone is central in the graph, and those who are have a prominent role in the network science community.  Here is the top ten list of #sunbelt mentioning twitter users ranked by betweeness centrality.

miriamnotten
barrywellman
memeticbrand
isidromj
drewconway
gephi
kristtina
danevans87
valdiskrebs
ciro

The NodeXL team has released a new version (v.1.0.1.126) with better support for collecting data from social media network sources, starting with Twitter.  The NodeXL Network Server program now ships in every NodeXL installation.  Tony, the lead developer on the team, created the following FAQ to explain how to use the collector application.

This document describes how the NodeXL Network Server works.

• What is the NodeXL Network Server?

It’s a Windows command-line program that downloads a network from Twitter and stores the network on disk in several file formats.  It can be run directly from a command line, but is typically scheduled to run on a periodic basis via the Task Scheduler that is built into Windows.

• Where can the files be found?

The files are in NodeXL’s program folder.  To find out where the folder is, right-click the Microsoft NodeXL, Excel 2007 Template menu item in the Windows Start menu, then select Properties.  On 32-bit English computers, the folder is “C:\Program Files\Microsoft Research\Microsoft NodeXL Excel Template.”

• Who are its intended users?

The Server is meant for use by people with moderate system administration skills.  It is not difficult to use, but it is not intended for the same audience as the NodeXL Excel Template, where ease of use is of high priority.

• How do you run the Server from the Windows command line?

Like this:

NodeXLNetworkServer.exe NetworkConfiguration.xml

The program takes a single argument, which is the path to a configuration file that specifies which network should be downloaded and how the network should be saved to disk.  A particular configuration file might specify “Get the Twitter search network for people whose tweets contain ‘Sociology,’ add an edge for each ‘mentions’ relationship, limit to 100 people, include tweets, include statistics, and store the network as a GraphML file in the C:\NodeXLNetworks folder.”

The program immediately gets the requested network, saves it to disk, and exits.  On its own, it does not run on a periodic basis.

• How do I create a configuration file?

You create a configuration file by copying a provided template file and editing the copy in Notepad.  The template file is named SampleNetworkConfiguration.xml and is stored in the same folder as the program.  The file is in XML format and the XML tags are clearly named and documented.

• In what file formats can be the network be saved to disk?

You can save the network to either GraphML, which can be imported into a NodeXL workbook; directly to a NodeXL workbook; or both.

• Do you typically run the program from the command line?

No.  Instead, you typically run it as a scheduled task via a built-in Windows program called Task Scheduler

Task Scheduler is a powerful utility that lets your run any program, including NodeXL Network Server, on a periodic basis.  You can, for example, tell Task Scheduler to run NodeXL Network Server using a particular network configuration file every twelve hours starting June 1, 2010 and ending June 30, 2010; or once a week starting now and continuing forever.  The scheduling options are endless.

• Why not just include scheduling features in the NodeXL Network Server?

For two reasons.  First, Task Scheduler’s extensive scheduling options would be difficult to duplicate.  Second, if NodeXL Network Server had to download a network on a periodic basis, it would have to run as a Windows service, and Windows services are more complex to implement and to use than a simple command-line program.

• How are the network files named?

Scheduling the NodeXL Network Server to run periodically can create any number of network files in the specified directory, so a file-naming scheme is needed.  The file name format is

{NetworkConfigFileName}_{Date}_{Time}.{Extension}.

So the above example, in which NetworkConfiguration.xml specifies that networks are to be saved as GraphML, might create a set of network files that look like this:

NetworkConfiguration_2010-06-01_02-00-00.graphml
NetworkConfiguration_2010-06-01_14-00-00. graphml
NetworkConfiguration_2010-06-02_02-00-00. graphml
…

• What happens if the computer is not turned on at the scheduled time?

By default, the task won’t be performed until the next scheduled time when the computer is turned on.  However, if the computer is sleeping, you can tell Task Scheduler to wake it at the scheduled time to run the task.

• What happens if the NodeXL Network Server encounters an error?

If the error prevents the network from being downloaded, the NodeXL Network Server creates an error file instead of a network file.  The file name starts with “Error” to make it easy to spot:

Error_NetworkConfiguration_2010-06-02_14-00-00.txt

The error file contains the details of what went wrong.

If one or more errors block part of the network but other parts of the network are successfully downloaded, then the NodeXL Network Server creates the network file containing the partial network, along with a text file that explains how many errors occurred.  The text file name starts with “PartialNetworkInfo” to make it easy to spot:

NetworkConfiguration_2010-06-02_14-00-00.Graphml
PartialNetworkInfo_NetworkConfiguration_Date.txt

• What if I want to periodically download more than one network?

Simply schedule more than one task, each using a different network configuration file.  The tasks are independent of one another and can be scheduled to run at different times.

The Enterprise 2.0 conference is about to get underway in Boston. The event focuses on all the ways social media tools that are familiar on the consumer Internet are making their way behind the firewall in many enterprises and institutions. Why can’t you “friend” a colleague or “like” a spreadsheet or slide deck? Employees often come to their jobs expecting tools that resemble the social media tools with which they already spend much of their time.

Like many conferences, this one has a hashtag, actually two that I know of: #e2 and #e2conf. There is already a good deal of activity leading up to the event. Here is a map of connections among a group of people who mentioned either #e2 or #e2conf in the last few days.

In this map there are 532 Vertices and 9,395 Unique Edges, creating 13 Connected Components, 11 of which had only a Single-Vertex, the largest component had 519 vertices which were interconnected 9,393 times.  The small number of isolated components indicates that this is a cohesive community of highly connected participants.  These people know and follow, reply and mention one another.  The Graph had a Density of 0.03 and the Maximum Geodesic Distance (Diameter) was 5 steps with an Average Geodesic Distance of 2.

Within this mass of connected users is a core group of highly “between” people, those who most broadly span connections within the population. These are one possible set of “influentials” within the Enterprise 2.0 community.

Here is a two screen view of the list of the top most between #e2 OR #e2conf mentioning twitter users along with the overview graph of their internal linkages.

A closer look at the graph alone can reveal enough detail to read the names of these central participants.

This is a view of the list of authors sorted in Excel by their “Betweenness centrality” score, the measure of how much these people “bridge” across the network.

An alternative view plots these contributors in an X/Y space based on their count of followers (along the x axis) and count of tweets (along the y axis).

The top 15 are:

dhinchcliffe
jowyang
rlavigne42
enterprise20
jumpersearch
marciamarcia
itsinsider
cmswatch
philcampos
rwang0
dankeldsen
lliu
juliancaparaz
lehawes
sameerpatel

Twitter users who mentioned #e2 or #e2conf on June 13, 2010 scaled by number of followers, x = log(followers), y = log(tweets).

There is a correlation between tweets and followers, but not everyone converts tweets to followers at the same rate. Below the diagonal are those who over convert tweets to followers, those above the diagonal under convert tweets to followers.

The book, Analyzing social media networks with NodeXL: Insights from a connected world, is forthcoming summer 2010 from Morgan Kaufmann and from Amazon.

Hello!  If you would like to request a custom social media network map made with NodeXL, complete the form below.  I will generate the maps as requests come in and email you a pointer to the results which I will post to my flickr feed here: http://www.flickr.com/photos/marc_smith/sets/72157622437066929/

Here is a sample map for the term OCU2010:

Fourth International AAAI Conference on Weblogs and Social Media (ICWSM-10)

***Microblogging 2***

Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment (Tumasjan et al.)

Successful use of social media in las presidential campaign has established twitter as an integral part of political campaign toolbox

Goal: analyze on Twitter: 1. Deliberation, 2. Sentiment, 3. Prediction

Previous work:

Deliberation: Honeycutt and Herring – Twitter not only used for one-way comm, but 31% of all tweets direct a specific addressee. Kroop and Jansen – political internet discussion boards dominated by small # of heavy users

Sentiment: How accurately can Twitter inform us about the electorate’s political sentiment?

Prediction: can Twitter serve as a predictor of the election result?

Data: examined more than 100k tweets and extracted their sentiment using LIWC

Target: German federal election 2009

Results:

1. While Twitter is used as a forum for political deliberation on substantive issues, this forum is dominated by heavy users

Two widely accepted indicators of blog-based deliberation:

-The exchange of substantive issues (31% of all messages contain “@”),

-Equality of participaion: While the distribution of users across groups is almost identical with the one found on internet message boards, we find even less equality of participation for the political debate on Twitter. Additional analyses have shown users to exhibit a party-bias in the volume and sentiment of messages.

2. The online sentiment in tweets reflects nuanced offline differences between the politicians in our sample.

LIWC profiles:

-Leading candidates: Very similar profile for all leading candidates, only polarizing political characters, such as liberal leader and socialist, deviate in line with their roles as opposition leaders. Messages mentioning Steinmeir (coalition leader) are most tentative

3. Similarity of profiles is a plausible reflection of the political proximity between the parties

Key findings: high convergence of leading candidates, more divergence among politicians of governin grand coalition than among those of a potential right wing coalition

4. Activity on Twitter prior to election seems to validly reflect the election outcome (MAE 1.65%), and joint party mentions accurately reflect the political ties between parties.

From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series (Brendan O’Connor)

Measuring public opinion through social media

Old method – query via dialing, asking, etc.

New method – people write their thoughts to social media, query social media to create aggregate text sentiment measure.

Can compare results from new method to old method

Contributions:

-High correlations between very simple sentiment analysis and telephone polls

-Time series smoothing helps

Text Data: Twitter

-Large, public, ll in one place

-Sources: Archiving Twitter Streaming API (“Gardenhose”/”Sample” ~15% public tweets); Scrape earlier messages via API

-Volume ~ .7B tweets

-Poll data: consumer confidence (2008-2009) – index of consumer sentiment (Reuters/Michigan), Gallup daily. 2008 presidential elections (aggregation, pollster.com). 2009 presidential job approval (Gallup daily)

-Message selection via topic keywords

-topic frequencies change rapidly

-Sentiment analysis: word counting.

–Subjectivity Clues lexicon from OpinionFinder / U Pitt (Very simple system!)

Key: don’t need to classify individual messages correctly, just need a sentiment ratio over messages.

-Sentiment Ratio Moving Average: High day-to-day volatility. Average last k days.

-Which leads, poll or text?

–Cross-correlation analysis: between sentiment score for day t, poll for day t+L.

—Results: “jobs” text leading indicator for poll, can be turned into forecasting model

—Reminiscent of Leskovec et al. Blogpulse paper, very nice!

-Keyword message selection:

–15-day windows, no lag. “jobs” r=80%, “job” r=7%. Is stemming always good?

Presidential elections and job approval: sentiment doesn’t correlate, but pure volume does (79% for “obama” 74% for “mccain”)

Conclusions:

-Preliminary results that sentiment analysis on Twitter data can give information similar to traditional opinion polls. But, still not well-understood. Twitter bias? News vs. opinion?

-Issues: Relevant message selection, Time series smoothing

-Replacement for polls? Promising but not quite yet

Information Contagion: an Empirical Study of the Spread of News on Digg and Twitter Social Networks (Lerman et al.)

Information flow on networks

Dynamics of Social Information

-How does infromation spread on online social networks?

–How far and how fast does information flow on networks?

–What factors influence its spread?

–How does network structure affect dynamics of information flow?

–What does this tell us about quality of information?

-Study question through comparative empirical analysis of 2 social news networks – using URLs as markers

Social News: Digg, Twitter + Tweetmeme

-Tweetmeme aggregates all tweets and features most retweeted URLs on its front page

Data Scope:

-3.5K digg stories with time submitted, promoted, votes for each story (time of vote, name of voter). 140k active users who voted for at least one stroy, 71k of them following at least one user. 258k links = fan network

-398 most retweeted stories 6/11/09 – 7/3/09, extracted from tweetmeme. Retweets of each story, up to 1k most recent retweets. Follower network of users who retweeted the stories

Questions:

-Usability of social netws – do people use digg, twitter the same way? what effect do differences in user interface have?

-dynamics of social networks – how far does info spread, how fast does it spread, and what are the effects of net strucutre?

Basic terms:

-Submitter  = user who submitted link to story, or user who tweeted link to a story

-Vote = vote on Digg or retweet on Twitter

-Fan = fan on Digg or follower on Twitter

User activity: distribution of fans (Power law on Digg with up to 1e5, power law with bump ~ 10 on Twitter with up to 1e7 users)

User activity: distribution of voting: Power law on Digg and Twitter (with different slopes)

Dynamics of stories: both digg and twitter show exponential growth, but for Digg it is preceded by slow period before story is on front page, both show vote saturation

Popularity distribution of stories shows lognormal fit

Information flow on networks: information spreads on a network as fans (followers) vote for (retweet) stories their friends submit or vote for.

Dynamics of information spread on networks looks very similar to overall dynamics of information spread (evolution of fan votes qualitatively similar to evolution of all votes)

BUT distribution of popularity is different, now shows normal fit. “Inequality of popularity” no longer observed (social influence accounted for?). News spreads farther on Twitter than on Digg.

How far does information spread among submitter’s fans?

-On digg many stories get voted by submitter’s fans, opposite case on Twitter

How fast does info spread on networks?

-Two distinct phases on digg: stories spread faster through network before promotion than afterwards.

-On Twitter, info spreads at constant rate.

Network structure differences: Digg network is denser, more inter-connected than Twitter’s

Summary of results:

-Network structure and info flow

–Digg’s network is denser than Twitter’s: News spreads faster initially through Digg’s network, but it does not spread as far as on Twitter

–Twitter’s network is sparse: Fans unconnected to submitter help spread story

-User interface and information flow:

–Before promotion, Digg stories spread mainly through network (and do so faster)

–No equivalent of promotion on Twitter

Tweeting from the Town Square: measuring Geographic Local Networks (Yardi and boyd)

Two geographically bounded events: Wichita shooting and Altanta parking garage collapse

Methods: two crawls and a poll

RQ1: Do geographically local topics have more dense Twitter networks than non-local topics?

Why this is important? People living in close geo proximity may share characteristics. Connecting similar people can help them form ties, foster community

Spread of News

Spread of News Online – ongoing discussion vs. spikes of short-term high-density discussions around real-world events

Distance

Methods: searched key terms about each evenet, stored user info, crawled first degree net of users. Polled users who had tweeted twice or more about church shooting in first 24 hours after it was announced. Administered poll 3-5 days after event. Sent out 800 requests, received 164 responses.

RQ2: Are people who are central in twitter network more geographically central in physical world?

Sarita Yardi gives shout-out to NodeXL, asks for more scale!

RQ3: What sources do people go to for local news events?

Twitter maps show high level of locality to event, slow spread outward

News Sources – go to locals

News Seekers – also go to locals, then to MSM

Practical applications:

-Utilize local short paths for disseminating information. Schools have long used an “emergency phone tree” with specified # of branches and leaves

-Timely notification of unexpected events

Invited Panel: US Government and Social Media

Macon Phillips, Director of New Media for the Obama White House

Moving from Elections to Governance

Wants academics to build tools that show effect of using social media on user behavior

WH new media director Macon Phillips asks for tools that allow thousands of people to communicate with the President (thanks @sadatshami !)

Don Burke, CIA Directorate of Science and Technology, Intellipedia Project

Haym Hirsh, Director, Division of Information and Intelligent Systems

Social Media and the Federal Government

NSF

US Gov’t early crowdsourcing project – National Weather Service Cooperative Observer Program (1890)

-Experimentation:

–CIA Intellipedia

–NASA Clickworkers

–PeerToPatent

–DARPA Balloon Challenge

–EPA Greenversations

–Over 100 gov’t blogs

-Policy implications and clarifications

–70% of Airmen use YouTube

Challenges:

-Legal and Policy

–Terms of Service: Indemnification, etc.

–Advertising (e.g. alongside gov’t content)

–Procurement: Free = Gift? No competition? Charges imposed after lock-in

Additional Challenges:

-Colbert “attacks”

-Open Government Dialogue

The Open Dialogue Top 5:

1. Concerns about Obama’s Birth Certificate

2. Government spending

3. Marijuana

4. Marijuana

5. Birth Certificate

Additional Opportunities: “No matter who you are, most of the smartest people work for someone else.”

Implications:

-Foster experimentation and innovation w/in federal government

-Provide data for innovation outside the def

-align legal and policy with aspirations

-research

Question about contribution quality: do people feel their contributions are worthwhile? How do we make the value and implications of contribution clear?

What do “votes” for questions mean? Who is the right person to say that legalization of marijuana is not a big question? What questions are “big enough to matter”? The “pothole problem” – should questions about fixing potholes be crowdsourced?

Few poorly worded questions about marijuana, people will speak eloquently and argue for the issue, so it’s not just spam

Don Burke – not EVERY system has to be based on socialmedia

Questions: How do you get recognized by gov’t? Answers: open access, publishing where you’ll be noticed

Question about Intellipedia and procedures for aggregating information. Answer: without the wiki, there was no way to share tacit knowledge. But want to go beyond wiki and to the larger web

Jure Leskovec about developing APIs for gov’t data. Answer: no APIs yet, but government is collecting data in one place that’s publicly visible. Want to see scientific community analyzing datasets and finding results, government may not necessarily know what’s a “good” dataset.

***Analysis of Social Network Usage***

Governance in Social Media: A Case Study of the Wikipedia Promotion Process (Leskovec et al.)

Wikipedia promotion process

3 important features:

-deliberative process yielding a single decision

-is publicly recorder

-consequential for the community

Similarity to offline world: people evaluate other people

We study perspective of voters:

-Burke & Kraut examine candidate’s perspective

-How voters evaluate candidate?

-How do evaluations change over time?

Main findings: Relative assessment

-Voter’s evaluation of the candidate reflects different types of relative assessment

–Let voter V vote on candidate C

–we find that vote of V heavily depends on relationship and relative merit of V and C:

—past interaction

—Number of edits

—Number of “barnstars”

–Response function of vote V:

—Prob. V votes given that x other people have voted

Dataset: Wikipedia voting

-Votes are time stamped and signed by users

–2.8k elections sept ’04 – Jan ’08. 44.6% success rate: Successful: 94.7% support. Failed: 31% support votes

–114K votes (78% support). Each vote can get commented: Support votes: 7% get discussed. Oppose votes: 82% get discussed

User characteristcs

-8.3K users voted

–7.5K voters

–2.5k candidates (some go for promotion multiple times)

-Relative merit:

–How do properties of voter V and candidate C affect V’s vote?

–Two natural (but competing) hypotheses:

H1. Prob. that C receives positive vote depends primarily on characteristics of C, there are objective criteria for user to become admin

H2. Prob. that C receives positive vote depends on relationship between characteristics of C and V

Merit (level of contribution):

-Two ways to quantify merit: total #edits, total #barnstars

-Relative merit: How does prob of V voting positively depend on diff in merit of C and V?

Relative merit hypothesis: if V has higher merit than C then he is less likely to vote

Observations: V is especially unlikely to vote for candidates of the same merit (total edits or barnstars)

Direct V-C interaction: Prob of positive vote as function of prior interactions of V and C.

Observation = prior interaction increases probability of a positive vote (with diminishing returns)

Thresholds and diversity of voters:

-Aggregate response function:

–How does prob. of voting positively depend on frac. of positive votes so far?

-Aggregate response function: baseline: if voter were to flip a coin then f(x)=x

-Observation: voters more inclined to express opinion when it goes against prevailing opinion

-Personal response functions: How does prob. of voter V voting positively depend on frac. of positive votes so far?

-Enough data that we can build models of individuals

-Average is close to baseline but individual variation in shape of response function is large

-Over time voters become more conservative, response functions shift downward and to the left

Elections over time:

–Elections unfold over time: Sequence of pairs (s(t),o(t))

—Very negative elections end ealry

—Failed elections are “top-heavy” = start very positive and slowly get negative.

—Successful elections get more positive over time

—Order of early votes doesn’t matter

–False hypotheses: Candidate’s friends vote early, Herding behavior (excessive influence of first votes)

Activity Lifespan: an Analysis of User Survival Patterns in Online Knowledge Sharing Communities (Yang et al.)

-User survival analysis to show that participation patterns and performance factors can account for a considerable amount of variance in predicting user lifespan

-Compare 3 major Q&A sites: Yahoo! Answers, Baidu Knows, and Naver Knowledge-iN

-Discuss how systems might sustain users

-Characteristics of Q&A sites we studied: in Yahoo Answers, earn points at flat rate per answer / best answer, Pay flat rate in points. In Baidu and Naver, earn points at flat rate per answer + points per best answer, and asker can offer additional points

-In Yahoo, significantly more questions / answer

Method: survival analysis

Defining “death” in online communities: period of inactivity exceeding 100 days. Found model prediction not sensitive to different cutoffs (50-150 days)

General comparison: 30-70% users leave after first day, afterwards curves for all 3 sites flatten. YA users more likely to remain than users of other two sites.

Answering life on average longer than asking life across all sites.

Preference between answering and asking (A/R ratio) can account for considerable amount of variance in predicting user lifespan

Initial interaction:

Obtaining more answers to your first question, writing longer question correlated with longer lifespan on Yahoo and Baidu

Winning best answer also correlated with longer lifespan

First 30 days:

More activity, asking more questions, obtaining more answers per question positively correlated with lifespan on all 3 sites

A/R ratio negatively correlated with lifespan on Yahoo but positively correlated with lifespan on Baidu and Naivr

Winning (best answer) also positively correlated with lifespan on all three sites

Analysis: community evolution

All three sites presented a decline in survival rate from year 1 to year 2, especially for Yahoo Answers

Naivr suffered more difficulty in sustaining users in 2nd year as almost no users stayed after 250 days

Conversational vs. Informational: There is a significant and consistence difference in survival patterns between conversational categories and informational categories: more conversational categories survive for longer

*with the exception* of “computer/internet” on Baidu only (cultural difference?)

Analysis: why do YA users stay longer

Fourth International AAAI Conference on Weblogs and Social Media (ICWSM-10)

We will be liveblogging (when possible) from ICWSM 2010, going on now!

Keynote: Bob Kraut, CMU

kraut keynote
implications for community design-offline theories of socialization helpful, not definitive
-online communities can build in good socialization practice-e.g. WP welcoming committee
Two Types of Commitments to Groups-identity based groups-bond based groups
Added Identity & Bond Features to MovieLens
Introduced Subgroups into MovieLens
Identity features that focus on subgroups
Individual profiles
bond-based design:+11% loginsidentity-based design:+44% logins
interventions based on theory increased commitmen
why stronger effect of identity?-time course: social identity can form instantaneously, bonds take time
new approaches to translate theory to design-ABMs
test identity vs. bond design via abm
incorporate design into abm
what are the consequences of discussion moderation-what type of moderation should be imposed? when?
results: indiv moderation helps logins
-personalized mod improves info and social benefit
-comm level mod improves info benefit only in homogeneous communities

***Influence and Composition in Social Networks***

Measuring Influence of Neighbors in Social Networks (Cosley Huttenlocher Kleinberg, Lan Suri)

Measuring Influence in 2 types of socnet data

-Crawl

–provides state of network at specific time

–often have more than one

-Complete time seties

–often from database dump

–provides timestamped history of all events

-This talk: How do measurements of social influence in these two settings compare?

Defining Influence

p(k) = Pr(individual adopts new behavior | k neighbors have)

Analyzing Influence in Wikipedia

-Wikipedia as a social network

–registered users (500K) represented by odes

–communicate through user talk pages

—Definition: Link between u and v forms at time t if one edited the other’s talk page at time t

-Definition: A community is a set of users who edited an article

-Analyze: Pr(user joins a community | k of his friends have)

Ordinal Time Method for Measuring Influence:

-p_o(k) = # instances (over all C) where u had k neighbors in C and joined/  # instances (over all C) where u had k neighbors in C

-advantage: very easy to interpret

-disadvantage: requires complete time series

Snapshot Time Method for Measuring Influence:

-2 snapshots of the network taken at times t1 and t2

p_s(k) = # instances (over all C) where u had k neighbors in C at t_1 and joined before t2 / #instances (over all C) where u had k neighbors in C at t1 and did not join before t1

-advantages: requires only a snapshot

-disadvantage: coarse-grained, don’t know changes in friends between t1 and t2

-results: p_o(k) behaves very differently from p_s(k) on Wikipedia

How are p_o(k) and p_s(k) different?

-accumulation effect in p_o(k) where community joining events can contribute to p_o(1)…p_o(k)

Approximating Ordinal Time from Snapshots

-At each snapshot know:

–which users joined which communities

–how many friends each user had in each community

-Don’t know exact number of friends user had when joined

-Assume a constant rate of getting friends

Ethnicity in Social Networks (Facebook Data Team)

-Questions about how ethnicities engage online are prevalent

-Our goal is to better understand ethnicity on social networks

-Issues

–How do we estimate ethnic distributions on social networks?

–Many services (e.g. Facebook) do not ask their users about their ethnicity

–Our approach infers ethnicities from surnames (based on previous approaches)

Topic modeling approach

-words are names, topics are ethnicities

-Generative process: Draw ethnic bareakdown of aggregate population w/Dirichlet distribution

–For each person draw ethnicity of individiual z_n ~ Multinomial

–Draw surname of individual based on ethnicity ~ Multinomial conditioned on ethnicity

Because there is no FB ground truth, we test our method by scraping 10k users from Myspace

Results: Model does a lot better than naive guessing based on census, internet usage statistics

Also tested on FB data (no ground truth but can have qual results):

1. In 2006, AsAm and White overrepresented on FB, by 2008 ethnicities much closer to mark

2. Drew heatmaps of US, showed that Whites concentrated in the northeast, Asians on the coasts, etc.

3. Correlated with politics: AsAm more likely to be liberal, Whites more likely to be conservative/libertarian

Homophily:

-Strong racial homophily (strongest in Hispanics and Blacks, weakest in Whites)

Homophily by ordinal friendships: racial homophily is strongest for the first few friendships a person makes

Friend you communicate with the most is most likely to be homophilous friend

Who makes friends in social media and why? Rich get Richer vs Seek and Ye Shall Find (Zeynep)

Importance of friendship and social networks vs. debate about social isolation

Architecture of our Commons

-Sociality does not happen in a vacuum

-Mediated Relationships (through digital means)

–Are these relationships: Superficial, weak, fake? Supplementary to offline sociality? As real or more?

–Why does this debate refuse to die?

-Assumptions:

–Internet communication: Anonymous, text-based, fleeting

–Lack of visual cues

but today’s Internet:

–social media: non-transitory interactions, lots of visual cues

Two dominant theories: Rich get Richer vs. Seek and Ye Shall Find

Data: Sample:

-College students

-617 respondents

-multiple classrooms

-very diverse school

-diverse majors

Results: feeling about whether online friendships are possible vs. not possible – split down the middle

Qualitative Component: why do respondents feel online friendships possible / not possible

*Not Possible reasons:

-trust

-face-to-face

-body language and mannerisms

-shared emotions and experiences

*Possible:

-deeper connections – easier (“it may even be easier online as it is all dialogue and no physical characteristics involved”)

-deeper connections – judgment

-bonding is possible

-conversation as key

-experience

***Diffusion and Dynamics in Networks***

Social Dynamics of Activity in a Virtual World (Bakshy et al.)

Second Life paper

Second Life: persistent, immersive virtual world

-User driven objects, economy, society

RQs: 1. How much economic activity occurs in virtual world?

2. Role of groups?

3. How much virtual interaction?

Data:

a) 65 mln user-user transactions (virtual goods)

b) Buddy graph – 4.2 million users, 43 million relationships, focus on strong edges: reciprocal and permissive (can see each other’s online status)

c) chat between 14 million pairs of users

d) 520k groups, 23 million user/group memberships

Economic Activity in Second Life

-Same “sectors” as realworld: retail, real estate, entertainment

-29 million free transactions, 36 million paid, power-law distribution of exchange amounts

Seller in detail: shows that interaction does not necessarily correspond to exchange

Analysis hints at Seller roles? Profiles?

Social Adoption

-show how friends of friends of initial buyers go on to buy as well. Is it influence or homophily? Not sure.

Role of Social ties

-39% of free transactions (but only 7% of paid) were between friends

-40% of users that chat exchange free items, 12% of users that chat engage in paid transactions

Free transactions may be more representative of social activity than paid transasctions

Role of Groups

-Groups indicate aspect of user interest

-Co-grouped transactions

-Sellers more likely to be connected to buyers through co-group than through friendship

Long discussion of why groups are proxy for connections

Explaining Seller Success

Traditional success measures: revenue, repeat business. Regression analysis

Key predictors:

For Revenue:

-Amount made by friends (homophily? business partnerships?)

-Make connections

-Less chatting with customers

-The younger crowd (newer to 2nd Life)

For Repeat Business:

-Interaction!

–Sharing a group, chatting

-More established crowd

-More diverse buyer base

Your Brain on Facebook (Fisher and Counts)

EEG: social vs. traditional media

Why do this? Possible input/feedback to social interaction systems. Short term: Inform design through better understanding of automatic info processing

EEG very good at detecting semantic mismatch

RQs: is Myspace associated with frivolity? Is connecting with friends on Facebook connected with feelings of intimacy?

Media and concepts:

-Media: TV, Books, Social, News

-Concepts: addictive, story, interesting, frivolous, personal, useful

-Showed both front page of FB and personal FB page (asked study participants to briefly friend study on Facebook)

Measurement and Method:

1. Timed, binary decision (low conscious reasoning)

2. Lijkert scale untimed survey (high conscious reasoning)

-Computer-based task: 24 combinations x 22 trials = 528 trials

-16 participants

Results (survey): FB more addictive, but less useful than news. Tells less of a story than books or TV. More frivolous than books, news. More personal than all other forms of media.

Results (decision taks): FB very addictive, tells less of a story than books or TV, a lot more personal than other forms of media. Overall, quite similar questionnaire responses.

Viz: heatmap of head. Time chart. In time chart, look for potential drop ~400 ms after stimulus. The more similar the media to the concept, the bigger the drop.

Results (eeg): FB equal to other media in terms of addictive, interesting, useful, frivolous. Tells less of a story than other media. Is less personal than other media.

Implications, suggestions, limitations

Media parity: FB as interesting, useful, addictive and not frivolous as other media

Personalization and self-identification: Purposeful connection building; easy switching to close friends, family; identity vs. bond attachment

Form of media found most personal by eeg? Books!

Telling stories – status updates on storylines?

Making it tangible: What’s the ratty, marked-up favorite book equivalent in social media? Photo album equivalent?

Limitations: only one form of social networking / media, limited subject population, can we believe EEG results?

Opportunities: compare online media, other subject populations, corroborate with other physiology, expand into real-time capture of physiology

Social Causality and Analysis of Interpersonal Relationships in Online Blogs and Forums (Roxana Girju)

Social causality: causal reasoning used by intelligent agents in a social environment

Modeling social causality – can guide conversation strategies, facilitate modeling and understanding of social emotions, bring new insights

Our focus: social causality as capture through analysis of interpersonal relations in social media

-Pervasive set of english reciprocal textual contexts encoding interpersonal relationships

-data: 11K reciprocal relationship contexts coded

Reciprocity in Language: “The Golden Rule”

Modeling relationships

Properties of interpersonal verbs and reciprocal instances: Symmetry, Affective Value (4-state HMM), Intentionality of Actions

-Intentionality and affective values of interpersonal verbs highly correlated with blame and responsibility

Analyzing Social Interactions:

-Overall 54.34% of dataset was encoded by ambiguous symmetric patterns

-top frequent verb pairs: need-need, love-love

-followed by: hate-hate, miss-miss…

-In general we love people who love/understand/care/need us

-Gender analysis: men initiate more often than females, retaliate more often than women, are more violent and aggressive (whereas women are more forgiving), but this depends on class of verb

-Men and women generally mutually respectful, it is only when respect is broken that responses may differ (e.g. women: cheat -> hate, despise, sue. men: cheat -> dump, divorce)

-Intentionality of actions: intentionality much more often perceived as intentional in bad-bad exchanges than in good-good exchanges

-Reciprocity Chains: Dyadic (formed between 2 people of the form A v B -> B v A -> A v B). Very useful in micro-levle social interaction analysis. General (between multiple people)

In generic chains: retaliation with increased magnitude chains, good for good chains (short), good for bad chains (turn the other cheek)

How does data sampling strategy impact discovery of information diffusion in social media? (Munmun)

How can we sample social web?

Many different modes of social interaction

Scale of interest – viral marketing, ad campaigns

Is there more in social media than just scale?

Social media can have enormous power, e.g. for diffusion

However, inference of such processes is based on the quality of data

Current methods: random walk, snowball – captures structure but not content or context

RQ1: what is role of context in sampling social phenomena, RQ2: how much should we sample to capture the process

Data: Twitter. Look at diffusion via RT feature, shared URl, same hashtag

Model: Diffusion series. Has slots of individuals involved in diffusion process, links between individuals based on relationships

Sampling strategies: given N, the number of nodes to pick, topic T, social graph G

Ignoring social graph: 1. Random sampling from seeds, 2. Attribute / context sampling.

Using social graph: Forest fire (again, random or attribute)

Diffusion Saturation Metrics: user-based (volume, participation, dissemination). Topology-based: (Reach, spread, cascade instances, collection size), Time-based: rate

Our sample S distorts some metric M

Diffusion response metrics: correlate diffusion on twitter to external behavior (search and news trends)

Experimental Study

Reference Set: ~465K users, 836K edges, 30M tweets. 125 randomly chosen “trending topics” from Twitter between Oct and Nov 2009

Trending topic – theme association

Results: bias due to sampling consistent, best results come out of forest fire. For search trends, forest fire + location performs best, for news trends, forest fire + activity performs best

Larger-scale analysis: look at topic distribution by sampling strategy

Inferences about social data affected by sampling strategy. Topic + topology + seed attribute makes a difference to sampling.

Photo Tagging over Time: A Longtitudinal Study of the Role of Attention, Network Density and Motivations (Paul Russo)

Tagging over time

Overview: what factors influence users to tag?

RQs:

1. How do individual motivations affect tagging,

2. what effect does receiving attention from users affect tagging tenure

Data: Flickr – focused on established users

Individual Motivations:

-People tag for themselves (archiving, retreival) and for others (describing)

-Many ways to receive attention from others

-Huberman, Romeru, Wu demonstrated that on YuTube people whose work received more views tended to post more videos, leading to a submission cycle

-Lento et al. photo tagging behavior

New: look at network strucutre (clustering coefficient)

-Mutual Friends

H1: attention, enjoyment, commitment should increase tagging. Density should decrease tagging. ((blogger comment: what about social grooming?))

Method: 90 days of tagging on Flickr, used only “pro” users w/3 months tenure

-Combines user-reported (survey) data and system data: what people say and what they do

-Attention: comments, Density = network, motivation = survey, measured on Likert scale.

-DV = #tags/photo over 90 days

-Results: generally bear out hypotheses.

-Interesting: low density net individuals tag more than high density for same lev of attention, but higher levels of attention lead to more tagging regardless of density. On the other hand, density has positive slope w.r.t. to commitment and tagging only for low density networks, for high density networks higher levels of attention lead to less tagging for high density.

***Microblogging***

Microblogging Inside and Outside the Workplace (Ehrlich and Shami)

Method: Twitter vs. Bluetwit (internal tool)

Data: 34 users from 15 countries and 8 business units that used both Twitter and Bluetwit and 20 posts in each over 4 month period

-Twitter much more active than Bluetwit

Dataset: data collected for BlueTwit and Twitter over 4 month period. 19K posts in two tools. Extracted 4 weeks of tweets.

Manually coded 5k microblogging posts

codes: status, providing information, retweet, ask question, directed, directed q

Results:

Categories of Posts: Most frequent use – provide information, directed posts

Internal use: ask questions, directed posts, style is work-oriented

External use: provide information, style is more “social”

Public vs. Private: clear sense of what is appropriate for an internal-only audience

Reputation Management: Internal: importance of giving back

External: Publicity and Promotion

Fostering Connections: developing better awareness of professional connections in advance of a future planned or unplanned meeting

Consumption of microblogs:

-Microblogs provide early access to human selected information

Discussion:

-How is microblogging useful in a work context?

–Form of crowd-sourcing – asking questions and providing answers

–Anticipatory connections

-What differs between public and private?

–Confidentiality (very clear)

–Style of writing

–Awareness of audience knowledge and interests

–Some erosion of boundaries – despite difference lots overlap

-Why are people consuming microblogs?

–Motivation for posting – Building reputation, awareness

–Motivation for reading – early, quality news

Measuring Influence in Twitter: the Million Follower Fallacy (Cha et al.)

Goal: characterize influence in social media and study its dynamics

1. How can we measure influence of a single user?

2. Does influence of user hold across topics?

3. What behaviors make ordinary users influential?

Data: Twitter

Why Twitter? One of most popular social media, social links are primary way how information flows, traditional media soruces and word-of-mouth coexist in this environment

-54m users, 2B follow links, 1.7B links

–8.5% of profiles private

–95% users belong to the giant component

–low reciprocity (10%)

–Power law degree distribution w/extremely large hubs (500 users have more than 100K followers)

–Low tweeting activity in general (only 11% of all users posted at least 10 tweets)

Three measures of influence:

1. Indegree

2. Mentions

3. Retweets

Are these three measures related? Compared relative ranks of user across 3 measures using spearman’s rank corellation

Correlated for full population, but not for top 10% or top 1% (but retweets corr mention remains high)

top list of indegree = mix of news outlets and public figures, top retweets = celebrities, etc.

Million follower fallacy – Britney Spears has millions of followers but she doesn’t show up on top retweeted list

Measuring influence:

–Find users engage in multiple topics. Picked 3 popular topics in 2009 over 2 month period, iran election, death of  MJ, swine flu

–Focused on 13k people who talked about all 3 topics

Conclusion: just because you have a lot of followers doesn’t mean they retweet you

Along a similar idea, Tweets vs. followers in NodeXL:

Directed Closure Process in Hybrid Social-Information Networks w/Focus on Link Formation on Twitter (Romero and Kleinberg)

How do directed closed triads form?

Triadic closure vs. directed closure:

-Triadic: an edges connects 2 nodes who already have a common neighbor

-Directed: a node A links to node C to which it already has a 2-step path (through node B)

-An edge in directed graph exhibits closure if it completes a 2-step path

-Closure ratio of node C is the fraction of C’s incoming edges that exhibit closure

-could indicates how many nodes discovered C by following nodes that follow C

Data: random sample of 18 twiter micro-celebrities: Users with between 10K and 50K followers

Notation: user A is k-linked to C if A follows C and also follows k followers of C. Let s_k(C) denote set of followers k-linked to C. f(s_k(C)) = fraction of set whose edge to C exhibits closure.

Q1: Is directed closure a significant process? Randomization test

A: up to large k, f(s_k(C)) is significantly bigger than one would expect under random ordering

Observation: f(s_K(C)) increases with k, but flattens out. Why??

Properties observed: closure ratio saturates to a positive constant f, constant f is different for different micro-celebrities, constant f not closely related to total in-degree of micro-celebrity

Heuristic calculation suggests that the sum of in-degrees of incoming nodes closely predicts closure ratio

Improved Model: predict closure ratio not just by in-degree but by sum of in-degree of incoming nodes. Also sum of in-degrees of incoming nodes from same community predicts closure ratio even better!

Conclusion: definition and methodology for directed closure, evidence for directed closure on Twitter, evidence that sum of in-degrees of incoming nodes & nodes from same community predicts closure ratio on Twitter

Characterizing Microblogs with Topic Models (Ramage, Dumais and Liebling)

Do people like the posts they see?

43 users at MSR looked at 60 posts, judged as: “not really worth reading <-> maybe worth time spent reading <-> worth the time spent reading.”

the average judgment was maybe or worse, nobody on average judged posts as worth time

Fundamental problem: people followed <> Tweets worth reading

What factors go into deciding if a user is worth following?

Method: Structured interviews with heavy users, Broader survey of 56 Twitter users

Kinds of Topics for Tweets. Hobbies, professions, news, products, events = Substance. Updates about meals, travel, hygiene = status. Making plans, networking, staying in touch = Social. Humor, wit, whininess, diction, worldview = Style. Different topics liked by different users.

Shows that content is important.

Content modeling: 8.2m “Spritzer” Tweets from 2008

-Surface word features = tf idf cosine similarity, etc.

-won’t look at deeper features (e.g. parsing) – not very appropriate for Tweets

-Desparsify: Topic Models, LDA.

-Want some labels (hashtags, emoticons, questions, etc.) = Naive Bayes, SVM, etc.

-Combined surface word features, LDA, labels = labeled LDA

Content modeling with labeled LDA:

1. Discover unlabeled topics w/ k=200 latent topic dimensions (e.g. politics, sleep)

2. model common labels = 500-1000 dimensions for hashtags, emoticons, etc.

Twitter content by category: manually aggregate topics into one or more of 45 categories.

Results: 38% style, 23% social, 27% substance, 12% status

Treemap-like visualization

Filtering: Tweet stream re-ranking

Split rater’s post into train 70% and test 30%

re-rank test set by distance to positive examples

consider judgment of maybe or worth time as “positive”

Mean reciprocal Rank @ 1 Relevant: best performance = Labeled LDA + tf-idf, .75

Finding: User recommendation task

Rater’s followed user: train 6/7 followers and test 1/7. Find the test user among 8 other non-followed users. Ranking task: score by reciprocal rank of test user. Performance > .9

Next steps: better interfaces for finding and filtering and models that account for temporal dynamics

by Dru!

Can useful observations be made by studying the social media sea one bucket at a time?

NodeXL has data import “spigots” for pulling social networks out of several social media systems including Twitter, YouTube, flickr, and email.  Twitter networks of follows and followers, reply and mentions can be extracted based on either a user name or a search string “seed”.   There are additional networks inside Twitter: a tie is created whenever two people tweet the same URL, for example, or are connected by tweeting from the same general location.  For now, the NodeXL Twitter Data Importer is starting with these three initial twitter “tie-types”.

NodeXL queries are not exhaustive collections of Twitter data, we provide a more modest approach, grabbing a slice of recent content and analyzing that.  Twitter has a sea of data, NodeXL is importing something  like a study of buckets of ocean water.  A recent scientific voyage to the Great Pacific Garbage Gyre, for example, collected hundreds of samples of ocean water as they sailed to the central location of the gyre.  Each bucket revealed details about the larger state of the ocean (which does not look good).  Simlarly, NodeXL is puling buckets of social media network data from the ocean of twitter and, despite the lack of scale, can do some useful science.  In part this is a virtue imposed by necessity –  constraints imposed by Twitter (even with a rate limit lifted “whitelisted” account) impose significant limits on what can be squeezed out of the Twitter API.  For those who lack access to large data center resources, there are scale limits imposed by the capacities of a desktop/laptop device.

Access to large data sets is certainly a hallmark of the “new era of science” that generates observations not from samples but from exhaustive surveys of data terrains.  Small samples miss important phenomena it is argued.  The counter argument is that many important phenomena appear in most samples, even small ones.

Using the existing features in NodeXL, I can extract the twitter social network for a small group of user accounts.  I can provide the names or ask twitter search to deliver them.  Alternatively, a keyword can be used to collect all the users and their connections who recently tweeted containing that term.  From this selected sample, several observations can be made:

> Not every keyword is equally connected

> Not every twitter user is equally connected nor are their neighbors

> Selected data extractions can be useful in the absence of a global view

The oil spill disaster in the Gulf of Mexico is a topic of great concern to many people in twitter.

This is the map of connections among people who tweeted the term “oil spill”.  There are lots of isolated authors, people who tweeted “oil spill” but are not connected to anyone else who said the same phrase.  The “giant component” is relatively sparse, there is no dense core of “oil spill” people yet.  This is in contrast with many topics where a large, highly interconnected cluster of people defines the “center” of the discussion.  This term remains highly diffuse.

This is the map of connections among people who tweeted the term “BP”.

The BP map also has a large population of isolates, people who are not part of  a “BP” related conversation but have said the term. A small core of highly interconnected users is forming in the “BP” hashtag space but in contrast to a term like “solar”, there is still little cohesion and density in its core.

This is the map of connections among people who tweeted the term “solar”.

This is a topic with a more dense core of highly interconnected Twitter users who share the use of the string “solar” in their tweets.  Ranks of isolates are also present in this topic space but the giant component is more internall connected and cohesive.

The #ashtag network is highlighted by well known technology bloggers.

In contrast the map for the tag “#ashcloud” is dominated by the UK and Scottish travel and Foreign office twitter accounts.

This image features a minor but useful tip for building multi-line tooltips.  This can help pack a lot of information about a Node into the tooltip.  To build a multi-line tooltip use the “&CHAR(10)&” feature to glue other columns of data together along with labels.

Add a column to the Vertices worksheet and paste something like this into it:

=”Twitter user: “&Vertices[[#This Row],[Vertex]]&” – Followers:”&Vertices[[#This Row],[Followers]]&CHAR(10)&”Description: “&Vertices[[#This Row],[Description]]&CHAR(10)&”Last tweet: “&Vertices[[#This Row],[Tweet]]

Which creates a tooltip with multiple lines containing the Twitter User name, the number of their followers, a new line containing their user description followed by their last tweet on another line.  Any concatenation of columns contents can be assembled in this way.

A complete social network map, with edge thickness proportional to the number of network connections (follows, replies, mentions) that link the two participants is here:

The zoomed version highlights the key players in the network and the tighter connections that surround the account for Twitter founder “Ev”.

by chris.corwin

– - 

We have several data import providers (spigots) in NodeXL that query popular sources of social media for information that can be processed into a network graph.  User and search term networks from Twitter, YouTube, and flickr have been implemented for a while along with a connector to email reply networks through the Windows Search Index.  NodeXL also imports data from several popular network analysis file formats, opening up data sets and sample libraries used in many courses.

Recently, we added a 3rd party provider for web hyperlink networks through the VOSON service from Prof. Robert Ackland at the Australian National University.

We plan a number of additional data import spigots and would like to know about additional sources of network data structures that would be of interest.

The following data sources are possible targets for a new data provider.  Which matter most to you? SharePoint? Exchange? Active Directory? LinkedIn? Media Wikis?

The deck illustrates the use of NodeXL to extract several social media networks from systems like twitter and facebook to generate maps of communities and identify people and objects in key locations.

Registration: http://smanalysistwitter.eventbrite.com/

For example, this is a map of the population of people who recently mentioned “NodeXL” in Twitter.

I see a large cluster of French speaking people who recently picked up on a tweet:

seekoeur: [Blog] Identifier des communautés et cartographier des relations sur Twitter avec NodeXL http://www.seekoeur.com/?p=854

This is a very distinct group from the second cluster around my own account and the recently created NodeXL twitter account.

In contrast, this is the map of all the people who mentioned the keyword “SharePoint” recently in Twitter.

This is clearly a much larger population and one with a dense core of highly connected individuals.  A number of peripheral groups surround the core.  Drilling in can reveal who the center of the center of the network is, a potentially highly influential person.

2009 – November – NodeXL – Demo – Mapping Twitter Social Networks “Digg” from Marc Smith on Vimeo.

This is a good demonstration of several features in NodeXL: social media network importers (in this case from Twitter), the use of a variety of layouts, auto-fill column mappings, and dynamic filters to reveal some important structures, groups and people in the graph.

Here is the source data in a NodeXL workbook: 2009 – November – NodeXL – Twitter Digg 3

Note!  Excel has issues with security and workbooks.  The easiest way to use that file is to open a blank NodeXL template and use the “Import from older workbook ” feature to pull the data into a workbook that your copy of Excel *can* trust.

What social media data most interest you?  We are considering integration with web and wiki crawlers, and support for YouTube, delicious, and enterprise data sources like Active Directory (LDAP), SharePoint, and Exchange.

This release also improves support for images, particularly those pulled from URLS, like twitter or facebook profile photos!

Here, for example, is a map of the connections among twitter accounts that tweeted the “WIN09″ tag that was used in the recent Social Networks Summit at NYU (http://winworkshop.net/) The map illustrates the way the summit brought together previously separate clusters of people from the various disciplines that have been attracted to the study of networks in general and social networks in particular.  Size of the image equals the number of tweets that person created.

A refined version adds Edge Labels and color to highlight the different tie types in the graph: “follows” relationships and “replies to” and “mentions” and now scaled by “Followers”.

In both views, the high betweenness role of one twitter account is clear.

Release details below the fold….

1.0.1.95 (2009/09/28)

• When an email network is analyzed (NodeXL, Data, Import, From Email Network), the resulting graph is now directed. This means that the relationships (John,Mary) and (Mary,John) are no longer combined into a single edge with an edge weight of 2; instead, they are considered unique edges.

1.0.1.94 (2009/09/28)

• When importing a Twitter search network (NodeXL, Data, Import, From Twitter Search Network), you can now add a Tweet column to the Vertices worksheet.
• When importing a Twitter user network (NodeXL, Data, Import, From Twitter User Network), you now have some options for how edges get added to the graph. You can add an edge for each followed/following relationship (which was always done in previous versions), each “replies-to” relationship in the people’s latest tweets, and each “mentions” relationship in the latest tweets.
• Ditto for the Twitter search network, although in this case the “replies-to” and “mentions” relationships apply to the tweets that satisfied the search criteria, which aren’t necessarily the people’s latest tweets.
• Twitter search networks and Twitter user networks now add an Image URL column to the Vertices worksheet. The images are those of the people who wrote the tweets.
• Bug fix: In the Import from Twitter Search Network dialog box, the word “latest” was removed from the text, “Search for people whose latest tweets contain…” Reason: The tweets returned by the search aren’t necessarily the people’s latest tweets.
• Bug fix: When importing a Twitter search network, a tweet posted by a person with “protected” status in Twitter would bring the search to a halt with a “There is no Twitter user with that screen name” error. Now, such people are skipped.
• Bug fix: Using any of the NodeXL, Data, Import items in the ribbon failed to clear the Images worksheet before importing the new data.

1.0.1.93 (2009/9/18)

• You can now import a Twitter network of users who have tweeted a specified term. For example, you can create a graph with a vertex for each person who has included the hashtag “#chi2010″ in a tweet, with an edge between the people who follow each other. In the Excel Ribbon, go to NodeXL, Data, Import, From Twitter Search Network.
• When including an image in a graph, you can now specify an URL to an image on the Internet.
• When an image file isn’t available, an error message is no longer displayed. Instead, a small red X is shown in the graph in place of the missing image.
• Images can now be resized using the Size column on the Vertices worksheet. There are new options in the Options dialog box for setting the default image size. (Known bug: Changing the default image size doesn’t update the graph pane until the workbook is refreshed.)
• When importing from a Twitter user’s network (NodeXL, Data, Import, From Twitter User’s Network), a Relationship column is added to the Edges worksheet. This gets set to Followed or Follower.
• The Twitter dialog boxes now provide feedback on what they’re doing as they assemble the requested network.
• Bug fix: Attempting to get a Twitter user network that included someone who “protected” her Twitter identity would cause a failure. Now, that user is skipped and the rest of the network is obtained.
• Bug fix: There was a rounding error with very small numbers (on the order of 1.0E-22) that could cause some vertices or edges to always be filtered out by Dynamic Filters, even if the filters were reset to their entire range.

1.0.1.92 (2009/9/4)

• In the Autofill Columns dialog box (NodeXL, Visual Properties, Autofill Columns), you can now specify a logarithmic mapping instead of a linear mapping when autofilling Edge Color, Edge Width, Edge Opacity, Vertex Color, Vertex Size, Vertex Opacity, Vertex Primary Label Fill Color, Vertex Layout Order, Vertex X, Vertex Y, Vertex R, or Vertex Polar Angle. Click one of the Options buttons, then check “use a logarithmic mapping.”
• You can now import a network of Flickr tags related to a specified tag. Use NodeXL, Import, From Network of Related Flickr Tags. You will need what Flickr calls an “API key.” There is a link in the Import dialog box for requesting a key from Flickr.
• The Twitter import feature has been expanded. (It’s at NodeXL, Import, From Twitter User’s Network.) You can now import the network of people followed by a user, people following a user, or both. There is a new option for selecting a 1, 1.5, or 2-level network, and you can limit the network to a specified number of people. New columns are added to the Vertices worksheet: Followed, Followers, Tweets, and (optionally) Latest Tweet. Also, you can right-click a vertex in the graph pane and select the new Open Twitter Page for This Person menu item.
• Bug fix: When using dynamic filters, a filtered edge’s label obscured what was under it even though the edge itself was hidden.
• For programmers only: The IGraphDataProvider interface used by data provider plug-ins for the Excel Template has changed. The GetGraphData() method is now called TryGetGraphData() and it now returns a Boolean. (The original design failed to accommodate failures while getting graph data.)

1.0.1.91 (2009/8/19)

• You can now label edges by filling in a new Label column on the Edges worksheet. The Label column is hidden by default. To make it visible, use NodeXL, Show/Hide, Workbook Columns, Labels in the Excel Ribbon.
• You can now import a graph from a GraphML file. GraphML is an XML-based file format used by a variety of graph applications and libraries, including Pajek, “R,” and JUNG. NodeXL will import all edge and vertex attributes in a GraphML file, including those that correspond to standard NodeXL columns such as Edge Color and Vertex Size. Use NodeXL, Data, Import, From GraphML File.
• Importing from a Twitter network (NodeXL, Data, Import, From Twitter Network) is now more reliable, thanks to automatic retries that will occur if a request to Twitter fails.
• Of possible interest to developers: NodeXL now supports custom “plug-in” .NET assemblies that will import graphs from custom data sources. For details, see the IGraphDataProvider Interface topic in the NodeXLApi.chm help file and the sample implementation in SampleGraphDataProvider.cs.
• The graph legend is now more compact. (To show the legend, use NodeXL, Show/Hide, Graph Elements, Legend.)
• Bug fix: When importing from an open edge workbook (NodeXL, Data, Import, From Open Edge Workbook), columns in the open edge workbook that have the same name as standard NodeXL columns will be copied to those NodeXL columns. Before, a column named “Color” was copied to “Color 2,” for example.
• Bug fix: Your settings, such as those entered in the Options dialog box, are now stored in a single file in your local Windows profile. Before, each named NodeXL workbook got its own settings file.

1.0.1.90 (2009/7/24)

• When you import data into the workbook (NodeXL, Data, Import), you now have the option to append the imported data to the workbook’s contents instead of clearing the workbook first. Check or uncheck the NodeXL, Data, Import, Clear NodeXL Workbook First checkbox to control this.
• The Closeness Centrality graph metric (NodeXL, Analysis, Graph Metrics) is now computed much more quickly. For example, with a graph containing about 1,000 vertices and 1,000 edges, the computation time went from 31 seconds to 3 seconds, and with a larger graph containing 5,000 vertices and 8,000 edges, the time went from 63 minutes to 2 minutes.
• The graph legend is now hidden by default.
• Your settings for showing or hiding the graph legend and axes (NodeXL, Show/Hide, Graph Elements) are now saved along with the rest of your settings.
• If you autofill the X and Y columns in the Vertices worksheet (NodeXL, Visual Properties, Autofill Columns), the Locked column is no longer automatically set to Yes. Instead, the Layout (NodeXL, Graph, Layout) is set to None, which achieves the same effect but is easier to undo. If you no longer want the autofilled X and Y values, just set the Layout to something else.
• In the options dialog boxes within the Autofill Columns dialog box, there is now a “Swap” button that will quickly swap the colors or numbers you are autofilling.
• You can now change the font used for the graph axes. In the graph pane, go to Options, Axis Font.
• The Auto Refresh checkbox that used to be in the NodeXL, Visual Properties Ribbon group is now in the Options dialog box, reachable from the graph pane.
• Bug fix: In the Import From Twitter Network feature, the screen name and password were not being correctly sent to Twitter. This caused Twitter rate limiting to kick in even if your rate limit had been lifted by Twitter.

1.0.1.89 (2009/7/9)

• There are now X and Y axes in the graph pane. To show them, check the NodeXL, Show/Hide, Graph Elements, Axes checkbox in the Excel Ribbon. If you autofill the X and Y columns in the Vertices worksheet (NodeXL, Visual Properties, Autofill Columns), the axes will show the range of autofilled values. Otherwise, the axes simply show NodeXL’s full range of coordinate values (0 to 9,999).
• You can now export the NodeXL workbook’s edges to a UCINET file. The file format is what UCINET calls “full matrix DL.” Go to NodeXL, Data, Export, To UCINET Full Matrix DL File in the Ribbon.
• You can also import a UCINET full matrix DL file. Go to NodeXL, Data, Import, From UCINET Full Matrix DL File in the Ribbon. If you have a file in a different UCINET format, you will need to use the UCINET program to convert it to full matrix DL. Click on the “What if my file is not in full matrix DL format?” link in the Import dialog box for instructions.
• You can now export the NodeXL workbook’s edges to a Pajek file. Go to NodeXL, Data, Export, To Pajek File in the Ribbon. The vertex coordinates are exported, but no other edge or vertex attributes are.

Prof. Jen Golbeck and Justin Grimes analyzed 6,000 tweets from United States Congress members.  They found some interaction but a dominant broadcast pattern of use with a focus on self-promotion.  The Washington Post headlines the results as “Tweeting Their Own Horns Study Finds Posts By Lawmakers Boastful or Boring”

Here is the video:

From the UMD News desk:
“A new study by University of Maryland researchers finds a
growing use of Twitter among members of Congress, but that
the purpose and content of their messages fall short of
improving government transparency.

Jennifer Golbeck, assistant professor in the College of Information
Studies, Maryland’s iSchool, a doctoral student and an undergraduate
assistant analyzed more than 5,000 tweets sent by 69 members of
Congress in February. They found that House and Senate members
were using the social media platform mostly to promote themselves,
rather than engage in dialogue with constituents and the public at large.

“Members of Congress were not sharing much new information on
Twitter, and there were few posts that improve transparency,”
Golbeck says.”

Tweet the Debates: Understanding Community Annotation of Uncollected Sources

Authors: Shamma, D.A.; Kennedy, L.; Churchill, E.F.
Source: ACM Multimedia, ACM, Beijing, China (2009)
URL: http://www.acmmm09.org/
Keywords: Twitter; Debates; TV; community; multimedia; social; centrality; network

Download: wsm01a-shamma.pdf

At the core of the network are the entities that have the most “eigenvector centrality” which turn out to be the major participants in the debate.

My colleague Prof. Ben Shniederman is positively impressed by these images.  He writes, “Notice how the node layout remains stable as edges are removed, so by the 4th figure the edges can all be followed easily….”.  This is one of the themes he highlights in his paper and presentations about problems and improvements in network graph drawing (see: http://www.cs.umd.edu/hcil/nvss/and in particular http://www.cs.umd.edu/hcil/pubs/presentations/NVSS-3.ppt). Prof. Shniederman’s  5th edition of Designing the User Interface is now available with two full chapters on the website with wordles to open each chapter.

A somewhat related article ran the same day in the Style section on The Celebrity Twitter Ecosystem (http://www.nytimes.com/2009/03/29/fashion/29twitter.html). This image focused on the linkages between well known people using Twitter and, by extension, revealing who they follow and who follows them in the social network.

In the first image no names are associated with the nodes, in the second the names are the major point of the diagram.

The practice of “anonymization” of network graphs may be moot in light of a recent publication mentioned on the Social Network Analysis email list (SOCNET) by Mark Round from QinetiQ of a paper:

De-anonymizing Social Networks – Arvind Narayanan & Vitaly Shmatikov
(http://33bits.org/2009/03/19/de-anonymizing-social-networks/)

which suggests that just publishing the unique pattern of links around an individual is sufficient to identify them in an otherwise anonymized data base.

Abstract:
Operators of online social networks are increasingly sharing
potentially sensitive information about users and their relationships
with advertisers, application developers, and data-mining researchers.
Privacy is typically protected by anonymization, i.e., removing names,
addresses, etc.
We present a framework for analyzing privacy and anonymity in social
networks and develop a new re-identification algorithm targeting
anonymized social-network graphs. To demonstrate its effectiveness on
real-world networks, we show that a third of the users who can be
verified to have accounts on both Twitter, a popular microblogging
service, and Flickr, an online photo-sharing site, can be re-identified
in the anonymous Twitter graph with only a 12% error rate.
Our de-anonymization algorithm is based
purely on the network topology, does not require creation of a large
number of dummy “sybil” nodes, is robust to noise and all existing
defenses, and works even when the overlap between the target network
and the adversary’s auxiliary information is small.