intel2.0

“Social networking applications” ≠ “social networks”

Guy Hagen — Thu, 19 Jun 2008 15:14:21 +0000

OK, I have to post about a growing pet peeve. I’m a very, very small lone voice against the vast and growing trend of referring to online social networking applications / platforms (MySpace, FaceBook, white label platforms, etc.) as “social networks”. While social networking applications contain, and are a type of human social network, “social networks” predate the internet by the age of homo sapiens and include different types, scales, and structures of network relationships, interactions and exchanges than are witnessed online.

The following s a pretty crude BlogPulse trend comparison, but I think it’s pretty clear that the common usage has won out over the more accurate term! (Note - the labels don’t reflect the actual boolean search terms which were rather long; the first two trends specifically exclude sites that contain “social network analysis”)

“Social networking applications” or “social networking websites” are unwieldy terms, so it’s easy to understand the abbreviated moniker. I am grateful that this web 2.0 phenomenon has brought so much attention and prestige to the field of social network analysis but at the same time it has trivialized it.

Quality Control for the Research Process

Guy Hagen — Wed, 18 Jun 2008 18:16:02 +0000

Recently, I was called to document quality control / quality assurance steps for a public relations / communications research proposal. I thought it was a useful topic, and worth summarizing some of what I’ve learned over 15 years of scientific research administration and working as a PR / marketing research consultant. In no particular order:

1. Documentation

One of the most important quality control categories involves documentation during the research process. There are different tools and strategies for managing documentation.

Project management / mapping tools. Business project management tools are used by most engineers and large businesses for very good reasons. GANTT charts, PERT and critical path charts help to document the progress and potential sources of error and difficulty in a research process, and help to delineate expectations and responsibilities. Once created however, their potential is only manifested if they are tracked and updated.
Internal WIKIs and blogs are becoming increasingly used to foster document sharing and collaborative knowledge building.
Meta notes - my term for the “field journal”, the lead researcher’s ongoing journal of events, progress, and most importantly, observiations, questions, thoughts and insights that are generated during the research process. Generally, these are private documents, but can be critical for consistently documenting, preserving, and fostering potential analytical insights which can develop throughout the research.
Research plan. An important, but sometimes understated or overlooked, component is a documented research plan that states the data collection and analytical methodologies to be used, the expected outcomes of each method, the objectives of the research, and what form the findings will take. Inevitably, a project that involves applied, “real world” research will have to be adjusted “on the fly”, but it helps to work from a documented baseline. The technical / research plan can usually be incorporated into the proposal or statement of work.

2. Communication and Collaboration

The scientific research process has a wide body of literature oriented toward improving outcomes in the collaborative research process, particularly when responsibilities are divided across organizational and stakeholder boundaries. The lessons are relevant to marketing, public relations, and organizational research as well.

Champions. It is necessary to clearly identify, and get commitments from, champions within each key organization that have a vested interest in the successful outcome of the research. It is also important to ensure that these champions have the blessing of their superiors, resources and time necessary to follow through, and an identified backup / replacement if they are removed from the project for any reason.
Regular communication schedules. Champions and representatives from key stakeholders must meet on a regularly established schedule to review progress and discuss development. The meetings should be frequent enough to not be formalities and to foster relaxed relationships between the participants, but not so frequent as to be an excessive burden or to interfere with performance.
Milestone reviews. Periodic reviews with key stakeholders, champions, and clients should be organized to review progress toward milestones, and discuss if modifications need to be made to schedules and research plans. However, this does not mean that preliminary research findings need to be shared.
Managing client expectations. The most critical part of a research process often begins prior to the research itself - a focus group with clients and client stakeholders to reveal potential obstacles, political barriers, expectations, and a careful understanding of how the research findings will be utilized.

3. Transparency

Another key quality control category speaks to transparency of the research process, the research methodologies, and the activities, framework and guidelines from which the researchers operate.

Transparent methodology. It’s important to ensure that no proprietary or undocumented methodologies are used during a research process. Some consultants utilize a “black box” or proprietary research process, and claim that the proof is in the results. However, the proof is also inevitably unverifiable in such cases. It’s always preferable to use research methods that are open to examination, replication, discussion or critique.
Estimations of confidence and error. Any research methodologies that seeks to make observations about large populations (extrapolations from census or survey research) should include estimations of statistical confidence and error intervals, and statistical significance measures where appropriate.
Review of vested interest and bias. Although academia has long discarded the mythical archetype of the truly objective researcher, the lay public still clings to the idea of the impartial, objective journalist or scientist. Every individual has a vested interest or bias in certain outcomes prior to research onset, and it is the responsibility of the researchers to examine their own motives prior to initiation and make any significant or relevant motives “up front” to stakeholders. Although researchers may resist such a step, consumers of research are often suspicious about potential bias and may dismiss credibility of findings purely upon such suspicions. Researchers may actually benefit more by being up-front about what the end-user will already suspect, thereby earning a degree of trust.

4. Multimodality

It’s almost always preferable to use multiple, complimentary research methods to research the same subject matter, where such is possible. When results from interviews, random sample surveys, and focus groups all point to and support the same general findings, the researcher can be confident about the analysis.

5. Peer Review / Second Opinions

It never hurts to have a second expert not otherwise directly involved in the process review the research plans, methodologies, and findings for overall soundness and critique.

6. Compliance and standards

Published and academically accepted standards / methodologies. One of the most solid guarantees of quality is to document a peer-reviewed or academic standard, and adhere to it during the research process. It helps to frame the research findings, and make the potential strengths and weaknesses of the method available for discussion. Where less-standard methods are utilized, it helps to document the justification, advantages and disadvantages of the alternative method.
ISO standards. There is an international ISO standard for marketing and public opinion research - ISO20252. It has broader acceptance in the EU, and is generally adopted by larger organizations. Like all ISO standards, it is at its base a documentation, certification and business process standard.
Professional standards. Most professional and academic organizations have an adopted standard of ethics and behavior to which all members are expected to embody. These standards are not intended to speak to process quality assurance, but are relevant as they speak to maintaining a standard of diligence and performance by the people who perform that research.

Reality Mining: More Tools for Automated Network Discovery

Guy Hagen — Fri, 06 Jun 2008 14:44:46 +0000

Last February, I posted about an automated sensor/logger device by researchers at MIT for the purpose of automated real-time discovery of human social networks. As interesting as that device is - and the implications for smaller, cheaper successor devices - I believe that the trend toward increasing computing power, location sensitivity, and “friend discovery” tools in mobile telecommunications devices is going to blow right past specialty research devices.

Witness the Nokia Sensor. From wikipedia:

Nokia Sensor is designed to promote spontaneous communication between users in sociable settings such as bars, nightclubs and railway platforms, business functions etc. Bluetooth wireless technology is used to detect the presence of other suitably enabled mobile phones located within a radius of 10 meters.

Also the work of Dr. Nathan Eagle (at MIT, from whom I share the term “reality mining”):

My doctoral research at the MIT Media Lab used 100 mobile phones as behavioral sensors, programmed to continually log communication (call logs), movement and location (cellular tower IDs), and other people within 5-10 meters (regular Bluetooth scans). The resultant 400,000 hours of behavioral data provided insight into individuals’ routines, relationships, and the underlying dynamics governing aggregate behavior.

Let’s take this one step further. The soon-to-be-released SDK for the iPhone is going to open up the device’s accelerometers and GPS sensors to 3rd party developers, some of which is already turning into friend-finding / social discovery applications such as Whrrl iPhone application by Pelago.

The software begins with the user’s position on the iPhone’s map …[and]… shows the positions of nearby friends who have enabled a feature that lets them be seen by others.

While the mobile social network discovery applications so far are aimed at the device user, to help them find and share with their friends, this also means the consumer marketspace is quickly becoming saturated with devices that have the ability to automatically collect and construct complex pictures about dynamic real-life social networks.

This will present a tremendous opportunity for market and behavioral research, and potential danger from mobile device viruses and privacy loss. You can read more on the subject in the Technology Review article, “The Future of Mobile Social Networking“.

More Notes on Social Media and the Military

Guy Hagen — Wed, 21 May 2008 19:49:36 +0000

Recently, Wired posted an article that demonstrates that the USDOD’s struggle with social media continues unabated - “Air Force Backtracks on Social Network Ban“.

Part of the problem is generational, and part of it is organizational. The generational part is the military is coming to a crash realization that the latest wave of recruits are digital natives, who expect to be networked and communicate 24-7 worldwide and to learn in digital, collaborative environments. To quote a military specialist at a recent focus group I conducted, “You used to be able to drop a manual in front of a soldier, and tell them to learn it. That doesn’t work any more.” Soldiers are using mobile devices to find information “on demand” and teach themselves what they need, outside of approved and expected channels.

The other part is organizational. Parting conclusions from this focus group - discussing how to integrate social / digital media innovations into the military - concluded that the barrier between small, creative digital companies and military acquisition, procurement and services opportunities may be more or less insurmountable. There’s a tremendous language and cultural discrepency between the two communities, and while there may be interest in the part of small private sector companies to help the military “come up to speed”, the only realistic path to partnership is navigating traditional USDOD procurement or attempting to partner with big defense contractors like Lockheed, Raytheon, Evans & Sutherland etc.

And to borrow another quote from the focus group - “That’s like being trapped with a bear. When the bear loses interest in its other food, you’re on the menu.”

network position + research = negotiation advantage

Guy Hagen — Thu, 08 May 2008 15:57:55 +0000

The concepts of “betweenness centrality” and “structural holes” are some of the most powerful in the social network analysis toolset (pun intended - but forgive me for lumping the two concepts together for this post). In a nutshell, high betweenness indicates that the overall network is disproportionally reliant on an individual for communication between points in the network, resulting in a “broker” advantage. There are plenty of books and articles that demonstrate the advantages gained by a high betweeness, but it seems to me that few if any actually demonstrate the process by which network position is leveraged for maximum gain. Like all network measures, betweenness is a measure of potential, and the high-betweenness individual must

be aware of the advantage of their position (that they bridge two+ disconnected alters with a common need or interest, and know what that need or interest is)
be motivated to take advantage of their position
must avoid losing their position (avoid letting their connections bypass them)
must have sufficient information to broker (the ability to extract and selectively communicate information)

Some time ago, I decided to utilize what I know about networks to help me negotiate resort peak season hotel packages for a large seminar I was organizing, including discount room rates, amenities, and conference hall rentals. It was my first experience negotiating this sort of thing, and it made me aware that information gathering (research) is a critical component for translating network position into network advantage.

Here is the process I used, that worked phenomenally well:

I requested generic package rates and deals from a dozen roughly comparable hotels.
I picked 5 hotels (based upon personal preference), and mix-and-matched the best offers from each hotel into one “ideal” package even though the hotels were not strictly comparable (some were on the beach, some weren’t). For example, one hotel would offer lower room rates and free parking / spa access, another would offer a less expensive conference hall, another would offer VIP upgrades for speakers, etc.
I individually contacted each hotel, letting them know I was receiving quotes from their competitors (and whom those competitors were), as well as the current “package” I was looking at. It helped to let them know I was strongly considering them, but I was working with a committee that might have to make a decision based upon “best cost”.
I asked the current hotel if they could improve their offering in any way to help sway the decision process. If the hotel improved their prices or offerings, I would then go to the next hotel with the “improved ideal package”, repeating this step until the hotels stopped improving their offers or response delay started becoming substantial.
I then picked the best offer on the table mixed with personal preference.

I had to repeat step 4 with all five hotels about 2 or 3 times, and each time every hotel would improve one feature of their package (price, amenities, upgrades) a little bit, thus forcing their competitors to reciprocate. It was in effect the process of intentionally creating the “prisoner’s dilemma” among them; if at any point they were willing to actually contact the competing hotels and work among themselves, they would have negated all of my negotiating advantage.

The critical lesson about extracting power and advantage from network position relies on the ability to gather and selectively share information about one’s alters.

For the network analyst, this means that betweenness measurements only really become useful when the analyst digs deeper into the context of the network to understand precisely what can be leveraged for local advantage. Perhaps there is an opportunity for natural language processing/mining to augment social network analysis in this regard.

the key obstacles to researching innovation industries

Guy Hagen — Tue, 06 May 2008 02:10:15 +0000

About eight years ago, I wrote a paper for CUED (now the International Economic Development Council) entitled “Methods for Generating Useful Databases when Industry Codes Fail.” The gist of the paper was that most industry and market data is available based upon NAICS industry classifications or some similar proxy. Forecasted and estimated employment, occupations, wages, gross national/regional product, even revenue, productivity, regional specialization and many other indicators can be derived from federal or commercial sources using NAICS definitions for the market or industry you are interested in.

However, Kurzweil’s Law of Accelerating Returns states that we are reshaping technologies, redifining and obsoleting industries, and inventing whole new markets at an exponential rate - but the NAICS system is only updated through a linear, periodic, regulatory process that guarantees that it is inevitably and increasingly inappropriate for cutting-edge, research and innovation-based, or technology- and knowledge-oriented industries.

The result is that researchers try to generate useful statistics on industries like biotech, agritech, biofuels, modeling and simulation, photonics, digital media, nanotechnology or nanomaterials using NAICS that is comparable to what they can learn about more traditional industries (I have been asked to research each of these industries recently). As long as nobody looks “under the hood”, statistics get published that are wildly inconsistent across geographies or researchers and nobody is the wiser.
The Wrong Way

The worst method I’ve seen for tackling this problem is blind aggregation. A novel “industry cluster” is defined by adding together all the NAICS that the targeted technology companies might be included in. The result is a bloated estimate that only vaguely resembles the target industry. For example, I had access to the research behind a study on digital media performed by one of the big 4 accounting firms, which simply rolled up all the public relations, advertising, newspaper and magazine, film and television, arts and entertainment related NAICS together. While these sectors are all potential users and creators of digital media, it was a big stretch (especially five years ago when the report was authored) to claim they were all digital media companies. This was validated when many of these companies were contacted for follow-on research and collaboration. This definitional methodology is probably the most innaccurate, but I suspect it may be the most prevalent because it’s the easiest solution for analysts.

The Right Way That Might Not Fit

A second category of solutions requires a bit of mathematical wizardry. My favorite is a fuzzy-logic based system developed by my colleague Dr. Ed Feser, which identifies “core” industries, “supporting” industries, and assigns various (and non-mutually-exclusive) values to how strongly each NAICS category is linked for a series of pre-identified clusters. His cluster definitions have held up in empirical, applied research projects in which I’ve implemented them, and produce comparable to superior results to many of the other strategies. Alternative solutions in this category include linear algebra and statistical solutions like factor analysis, statistical cluster analysis (hierarchical, genetic, K-means, etc.), or neural networks and pattern recognition software. Definitional strategies like these have the advantage of producing self-consistent and empirical statistics and data manipulations. However, if you can’t match up one of the clusters generated by one of these methods with your target industry (such as nanotech), then you are still completely out of luck.

The Best Way Is Difficult

The method that remains is the most difficult to implement - empirical self-identification of a cluster’s members. The precedent is called snowball sampling:

Start with a solid, pre-identified core
For each company or organization, conduct an interview or survey to confirm their membership, ask them to identify their competitors, customers, vendors and suppliers
For each competitor, customer, vendor, or supplier, repeat until the respondents no longer self-identify or associate themselves with the targeted industry or market.

It’s obvious that you could integrate the sampling process with a census or social network analysis. The converse of snowball sampling is often performed by research call centers, who will work from a pre-generated list (such as the aggregated cluster above) and call each candidate to remove those that do not match the target profile - I generally refer to this as “casting a wide net” . Unfortunately, both methods are difficult to scale up (eg nationally), and are time- and labor-intensive.

I find that often snowball or “wide net” sampling methods are the only viable methods for identifying a research population that stands up to the “common sense” and “newspaper” tests. I’m keeping my eyes out for social media tools that will allow groups of people, organizations or industries to self-organize in a way that’s useful for public relations, marketing, and economic research, but tools like discussion forum member lists, email opt-in and newsletter systems still have a long way to go. They are still ineffective for getting consistent and comprehensive participation from businesses and organizations, particularly when time constraints exist (as they always do for research projects).

If your community is promoting a cutting-edge technology or research industry, some of the best analytical investments you can make are ways to keep a comprehensive and up-to-date “opt-in” directory of members!

selling analytics to the long tail

Guy Hagen — Thu, 01 May 2008 18:51:53 +0000

Social media analytics providers are offering increasingly sophisticated tracking tools, dashboard systems, and market research report products. I’ve taken several months to familiarize myself with the types and levels of products and services that are out there, and there are more almost every day. However, as I continue to develop proposals and work with marketing / public relations clients, I continue to come across a basic problem - most potential clients that would be interested in these types of analytical products simply are too geographically focused or too small for them to be appropriate.

Companies like Biz360 and Buzzmetrics offer very sexy and sophisticated dashboards - but the list of companies that can afford a $60-$100K annual subscription is relatively limited. According to a 2006 survey by PRSA, the average budget for PR-related products and services was reported was $45,800:

$32,700 for organizations with 50 or fewer employees
$44,900 for organizations with 51 – 1,000 employees
$177,500 for those with more than 1,000 employees

This means that only companies with annual revenue of $100M or more will probably be interested in, and able to pay for, sophisticated social media dashboards. This estimated figure seemed to resonate with a few of the analytics companies I spoke to.

There are now quite a few providers that are going after the “middle market” like Andiamo and Radian6 with relatively inexpensively priced dashhboards (especially by comparison) - but even Andiamo is raising their prices this month, and it seems to me that the average small company’s needs for market intel is still not being addressed. And of course, the “small guy” makes up the “long tail” - possibly very profitable, if the right way to reach, match, and deliver to them can be found.

I find that even when potential clients are large enough to easily afford some of the social media analytics, however, there’s another problem - few companies have a truly ubiquitous brand or retail distribution profile that can take advantage of the pervasiveness of the internet. Most companies and organizations really are focused on particular markets or geographies, and really don’t much care about what is being said about their brand or their marketspace outside of their territory. Sadly, it’s still pretty tough to drill down social media analytics to a particular geography or only in context of particular market sectors. For these organizations, the best social media analytical solutions probably should take advantage of free tools (such as BuzzLogic and Google Alerts) and a lot of common sense filtering.

The “long tail” market is still unadressed by the social media analytics market. There’s probably a large opportunity for whomever can fill that space first.

Social Media Recon - Mining for Terrorists

Guy Hagen — Thu, 13 Mar 2008 20:55:51 +0000

PC Magazine published a February 27 article “The Online Hunt for Terrorists.” It focuses on the work of the Artificial Intelligence Laboratory at the University of Arizona, and in particular their Dark Web project. In my recent post “Military Social Media Intelligence“, I categorized part of the US military effort in social media as focusing on reconnaisance efforts, and the work going in this laboratory covers the length and breadth of the subject nicely.

Essentially, the Dark Web system is a collection of spidering tools for crawling and indexing content from websites, forums, blogs, and video and audio media to collect counter-terrorism intelligence. It benefits from the fact that not only do terrorists tend to discuss a certain portion of their internal activities in publicly accessible online settings, but that key communities, individuals, and their connections can be identified from social media content both central and peripheral to actual terrorism groups.

What I like is how comprehensively their system tackles social media analytics for this purpose:

Social network analysis - identifying communities and groups
Content (semantic) analysis and categorization
Citation / co-authoriation analysis (a form of network analysis)
Sentiment analysis (albeit a specialized form focusing on levels of violence)
Authorship profiling and identification (de-anonymization)
Video and audio mining

Of course, their work isn’t limited to middle eastern terrorist group activities, but domestic terrorism as well. Here’s a little terrorist network visualization that the Dark Web system generated automatically (see link for citation). While it’s not a terribly sophisticated graphic, what’s important is the resulting intelligence and how it was achieved:

The PC Magazine article is also interesting as it also captures the debate between automated analysis and human-mediated analysis similar to that seen in marketing and public relations. While manual analysis could never filter through the millions of posts that automated systems can process efficiently, many experts maintain that expert analysis provides more insightful and deep results than automation ever will.

However, nobody’s arguing against the pairing of social media analytics (such as the Dark Web system) with human research as the best of both worlds.

gen y and the future of phone surveys

Guy Hagen — Wed, 12 Mar 2008 14:17:42 +0000

I was recently visiting with Ken and Elaine Lyons, the founders of the Independent Data Collection Center about telephone surveys, particularly CATI (computer assisted telephone interviews). The IDCC specializes in providing rigorously dependable and statistically valid data from focus groups, phone surveys, and other data collection methods, allowing researchers to just focus on analyzing the data. They’ve conducted thousands of CATI surveys across the U.S. and abroad, in a dozen languages, so I value their expertise on the subject.

CATI remains a very important tool for quickly collecting representative public opinion data very quickly. I had suggested that responsiveness to CATI surveys must be significantly characterized by demographics - ethnicity, geography, income, and particularly age group. My arguments were:

Retirees would be disproportionately more likely to participate, due to accessibility, time availability, probability of owning a land-line phone, and enculturated in its use.
Cell/mobile phones are increasingly prevalant as the generation groups become younger.
Cell phones tend to be unlisted numbers, and it will be increasingly hard to generate raw call lists that contain representative proportions of cell phones.
Unsolicited survey (and other) calls to cell phones are more likely to be met with hostility or resistance; in the U.S. most tend to consider their mobile phone their “private personal line.”

Ken and Elaine admitted that it is becoming increasingly difficult to build a representative cross-generation response rate for CATI surveys. Gen-Y and Millennials are likely to utilize their mobile phone as their only phone. Consequently, it is becoming harder to obtain CATI responses that accurately include the voice of the younger generation. Given the prevalence of CATI for collecting public opinion and market research data on any number of topics from political issues to product launches, this has important implications.

I wonder if the only effective ways to quickly reach out and collect feedback from Gen-Y will be those that simply don’t have the full statistical controls that we are used to from statistical sampling, such as snowball sampling (e.g., referral networks) or through social media (facebook, social networking sites, etc.).

Ironically, we may find that Gen-Y is also less likely to see unsolicited (survey) calls to their mobile contact point as less invasive than prior generations, particularly if the calls are properly incentivized (through actual survey incentives or through personal interest in the subject issue/product).

There’s a great link at “the future place blog” that discusses these trends: “More signs of the decline of CATI“. It’s a year old and UK focused, but it cites an interesting article regarding the decline of CATI data collection trends in contrast to internet-based data collection. Following up that post’s discussion, I’ve created a graph showing results from the 2007 CASRO Trends Survey . It’s clear that we are just now seeing a very significant drop in phone surveys in favor of internet-based surveys (the CASRO Trends Survey seems to focus more on the internet as a data collection tool, in contrast to utilizing internet-based media as an analytical source).

Also prominent is the slow decline in mail surveys, and the still negligable utilization of mobile phones as a targeted data collection media. The first company that can show a successful way to systematically collect opinion data from mobile phones probably has a real opportunity on their hands!

Finally, I have watched with interest the slow innovation of products and services such as google’s grandcentral.com that centralize personal telephone communications through computer management services. Before long, we are going to see the propogation of cell phone viruses and SMS / mobile phone spam, and in parallel, smarter incoming call filtering that to some extent will imitate the email spam / spam filter war. If that happens, CATI research may find it impossible to reach its research subjects at all!

Vizualizing The Social Web

Guy Hagen — Thu, 28 Feb 2008 16:26:13 +0000

The March/April issue of MIT Tech Review has an interesting article, “Between Friends: Sites Like Facebook Are Proving The Value of the Social Graph” (nods to Nathan Gilliat and Matthew Hurst -whose work is cited- for breaking this on their blogs).

Overall, this article provides some great examples of how sociograms - social network graphs - can be used to illustrate the structure of social media networks. It shows how network analysis can be used to predict how efficient a viral marketing effort will diffuse through two different types of networks. As shown in the following image, sparse, loosely structured types of networks may be poor candidates for viral strategies (recall the recent discussion about diffusion and small world networks?)

It also touches on social network analysis for measuring internal communications, and the different considerations of what constitutes a “link” among authors/ facebookers/ bloggers/ twitter-ers for measurement purposes. Overall, the article is a nice overview, and shows some of the interesting ways that social media communities mapped to understand their underlying structures. Compare them to examples from TouchGraph (a free website connection browser)…

… and Linkfluence (a commercial analytical provider based in France, that provides custom community map generation and tracking).

For the marketing or PR practitioners, these examples may seem somewhat academic; the question for them is usually “whom do I need to influence” and demonstrating effectiveness in influencing targets probably gets lost in large-scale maps. In contrast, Buzzlogic’s dashboard technology provides the ability to map and navigate the small-scale network graphs of individual influencer networks.

The TR article shows the value of online social network analysis as a strategic measurement tool. I think there’s a lot of room for growth and exploration; the TR topic doesn’t even touch on vizualizing the semantic association networks (content) of social media communication, or the power of animation to demonstrate change in a network over time (source: Stanford.edu / McFarland; you may have to reload this post to view the animation):