The Relation between Big Data and Social Media | #bigdata #socialmedia

Summary:Today social media generates more information in a short period of time than was previously available in the entire world a few generations ago. Making sense of it and understanding what it means for your business will require all new technologies and techniques, including the emerging field of big data.

With this development — as the world continues to become more and more social — competitive advantage will come to those who understand what’s happening better than their peers and can directly connect it to their business outcomes and other useful pursuits. Social networks and enterprise social software has long been driven by two things: The connections between the people that use them and the information they share.

Just as Facebook uses the insights gleaned through its analytics on how people behave to enable personalization and better user experiences, the same phenomenon has been happening on the Enterprise 2.0 side, most recently exemplified by last month’s acquisitionof Proximal Labs by Jive Software.

While gleaning insight and contextualizing interaction in social environments is nothing new, the challenge in doing so has been pushing the boundaries of available technology for some years now. Organizations across the social business spectrum — the consumer and workplace both — are only now beginning to understand the vast intelligence that can be derived by looking at millions of conversations taking place, mostly out in the open, between those engaging in social media. While there are certainly interesting privacy, legal, and regulatory issue with doing so, even with internal social networks, it’s not likely to delay the growing adoption of such capabilities, given their potential value.

In the shorter term, the ability to analyze and mine data in scale within social networks is enabling a range of intriguing and useful applications that can plug into social media networks and make use of the knowledge inside them. Doing it well, however, has proven to be more than non-trivial, such as making making analytic sense of content types that are very large and opaque, such as high definition video, or sensing the connections between the thousands of unstructured natural language messages. Each of these require technologies that can handle the enormous scale, complexity, speed, and computation requirements in a way that remains cost effective inside a rapidly-rising exponential window. From this you can begin to see the challenge of traditional approaches to large data, which tend to break down fairly soon under large geometric growth.

Perhaps more to the point, and where the discussion of big data comes in, is that the key to social media interactions between people is that it leaves knowledge behind for others to find and reuse. This can be the original content that started the conversation or the subsequent comments, discussion, ratings, ranking, retweets, etc. These conversations will remain on the network afterwards, usually for a digital eternity, and forms an invaluable history and knowledge repository of society, culture, and business that can be discovered, brought back to life, shared, learned from, ad infinitum. Of course, some of this isn’t inherently valuable by itself (much has been made about the signal-to-noise ratio of social media). Also, finding what one is looking for in the vast sea of a million or billion human conversations is a difficult task. Thus, separating the wheat from the chaff is where big data, and the analytics it makes possible, go hand-in-hand.

But the deeper issue is not just finding the valuable nuggets in a galactic sea of social media. Rather it’s knowing what you are able to know, or even what its possible to know as social media becomes a dominant form of communication (probably now the dominant form.) Last year at Defrag, I spoke of this as the difference between being able to find a specific needle in a haystack versus having the ability to discern the shape of the haystack itself. Social analytics has arisen precisely to help us make sense in the large of the endless flow of our activity streams and social news feeds.

Social Media: A Big Data Inflection Point

Though social networks may soon contain the visible sum of humanity’s communication and interaction, the challenges of deriving what is increasingly called social business intelligence are two-fold. First, big data sets itself apart from previous approaches because it applies new ways of thinking about the capture, storage, and processing of truly vast amounts of data, precisely the kind that emanates from today’s social media ecosystems. This includes the supporting technology, often starting with emerging tech such as data mining grids or MapReduce infrastructures (see my exploration of one example, Hadoop, here) as well as software architecture that is often surprisingly non-deterministic and non-linear in design. For a quick example, see this discussion of LinkedIn’s challenges and counter intuitive solutions to data scale in social networks. In practice, this means that there is a distinct generational and technical divide between how most organizations are dealing with data today, and the very different things they’ll need to do in the future.

Rasalkhaimah, ras, al, khaimah, dubai, university, salford, manchester, @hishamsafadi, hisham, safadi, European, medical, center, business, entrepreneur, startup, economy, money, motivation, education, Leadership,  Transactional,  analysis, emotional, intelligence, organisations,  development,  innovative, technology,  care, health, investor, investment, production, shark, tank, sharktank, USA, UK, London, group, european, canada, india, china, japan, KSA, projectmanagement, datascience, bigdata, IOT, internetofthings, cloud


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s