An investment in knowledge pays the best interest. ~ Benjamin Franklin
It was a major neuron-fest (in my head) as I was attended the Ideas Economy Conference (sponsored by The Economist) on June 7/8th, 2011, in Santa Clara, California. The conference was two days of unbridled conversations on topics pertaining to “Intelligence in the Information Age”, a.k.a. Big Data and Knowledge Management. Let me give you some numerical facts regarding these 2 days: there were 39 sessions, 5 moderators, 62 speakers (including panelists), an in-room audience of ~ 250, and an online viewership of ~ 300.
The conference was publicized as “The Economist will bring together theorists, strategists, and innovators who understand how to harness data to create value and advance individual, corporate, and social good.”
It was a well thought-out and meticulously planned event with speakers and panelists ranging from some of the top leaders in the information industry, research, and medicine, to government officials and policy makers. It was not very hard to see that everyone present had one common thread of interest, which was how to manage and cope with data in this new era where information is produced faster than it can be consumed.
Below is my simplified journalist’s view of the conference…
Never before in the history of human civilization has the sharing of knowledge been so fast and effortless. For instance, today I have created 5 Facebook updates, posted a blog on IWE, IM’d with friends and family around the world, created a couple of presentations, written an article, and sent a couple dozen emails – all in a matter of few hours. With each activity, I was sharing data at a never before seen pace and reach. I am just one of an ecosystem of billions of fellow humans that creates an enormous amount of data on a daily basis – ranging from inconsequential updates to knowledge nuggets that can educate others and everything in between. Just another curve ball in this whole situation is that data is no longer just text-based; it can be video, audio, images, etc. With modern technology, any person can share data with anyone and everyone across the globe in an instant. This has made the world a bit more flat and small. But, you already knew that!
With an average of 140 million tweets/day and 30 billion pieces of content shared each month on Facebook, information has become more dynamic than ever before. Data exists in many forms such as structured and un-structured, public and private, big and small. It can come from social, corporate, private, or public forums. Data does not stay constant, it evolves through participation from multiple sources and not all of these sources are human either. Regardless of where data originates, it eventually interacts with other pieces of data from other realms to create a bigger potential for information, knowledge, intelligence, or wisdom.
Now, with the help of algorithmic inferences and other big data management technologies, large volumes of data can be crunched and analyzed very quickly. For example, petabytes of data originating from particle detectors in the Large Hadron Collider (LHC) at CERN’s Higgs Boson experiment are analyzed at 40 million inputs/second. Data can be used to gain new insights that the world has not seen before in the fields of genealogy, medicine, physics, urban planning, etc. Whether it is to create prediction models for epidemics or having Dr. Watson, from IBM’s research lab, play Jeopardy to beat the human player, the potential to derive intelligence from data is huge and not yet fully tapped.
A really good example is the Aadhaar project in India. This project was kicked off by the Indian government to tag each citizen with an Aadhaar number (much like Social Security) that will ensure proper identification of the individual. The basis for this number will be a biometric identification mechanism that gives everyone a unique identification number much like RFID. The ability to accurately map this data (for 1.21 billion people) to derive useful information will provide never before seen ways of providing social benefits, understanding census data to design welfare and education programs, and national security.
Big data has opened doors for as many possibilities as it has for bigger issues and concerns. Information in the world of big data takes on concerns beyond manageability, such as security, access, privacy, profiling, share-ability, monetize-ability, timeliness, and accuracy (just in time). In a nutshell, all of those present at the conference were as concerned about “Big Brother” knowing too much as they were about they themselves not knowing enough. Questions like “How much or how little should the Government know about you?”, “How much information is too much on social networks?”, and “Should an individual have access to their own medical history?” were just some of the topics of discussion. Conspiracy theorists imagined that this data deluge is preparing us for the “black swan effect”, where new intelligence will eventually replace humans (akin to the rise of machines in a sci-fi movie). Are we going towards Nirvana or raising a “Frankenstein”? Only time will tell…but no one disagreed that the power is in the hands of people who understand how to use data and data management technology.
Now, let’s bring this back to the corporate context: How does a company take advantage of one of the biggest untapped assets it has, i.e. data? Data does not just make a smarter company but it can also present newer ways to create value for your customers and shareholders. For all of this to happen, we need to enable foundational architectural changes to extract and flow, aggregate and interplay data to create useful insights. We need to enable intelligence architectures and technologies to augment what humans can know and do. We also need to design data governance to enable purposeful access to data and intelligence.We cannot take advantage of tomorrow’s opportunities with yesterday’s architecture. And, tomorrow’s architecture is incomplete if it does not understand how to deal with BIG DATA.
Ajay Prasad, Former Secretary, Ministry of Defense & Civil Aviation, Government of India (next to me), Gavin Newsom, Lt Governor California (center), at the Idea Economy Conference
If you are interested in this topic, here are some additional references for you to check:
The Idea Economy: http://ideas.economist.com/videos
Books: “The Shallows” by Nicolas Carr
“The Journey of Man – A Genetic Odyssey” {Discerning patterns in BIG data sets}
Wolfgram Alpha: http://www.wolframalpha.com/
Metamarkets: http://www.metamarketsgroup.com/platform/
Yandex: http://about.yandex.com/
Greenplum: http://www.greenplum.com/
Tableau: http://www.tableausoftware.com/
Urban Mapping: http://www.urbanmapping.com/
Cloudera: http://www.cloudera.com/
No comments:
Post a Comment