Welcome to the Age of Big Data. Or perhaps it’s the Age of Big Data Agnosticism. In a Newtonian twist, what started as a wave of hype for data’s transformational potential on organizations everywhere has turned into an equal and opposite backlash of big data naysaying.
It is an understandable reaction to the great over-selling of big data as a kind of enterprise cure-all. Of course, in some companies, big data pilots have produced nothing but big piles of unfulfilled expectations.
But the problem likely is not big data.
Big data remains potentially the most powerful engine for business transformation to gain currency in the 21st century. The problem is that so much of what is sold as big data isn’t. It’s typically just lots of data.
“Big data, that’s just data mining with a fancy new name.” How often have you heard that?
It’s flatly false.
The size or volume of the data does not matter in genuine big data analytics. Instead, savvy organizations already understand that big data is really about working with a mix of data types - structured and unstructured, from inside the organization and outside. It is CRM forms, but it also is Tweets, Facebook posts, TripAdvisor rants, Gmails, Outlook entries, even voicemail.
In most organizations this does not add up to petabytes of data, as I’ve written before. Terabytes is the usual quantity even though that seems small by many measures. The complexity arises in the diversity of data.
And that raises a problem. Not many databases have the flexibility to handle that many forms of data. And fewer databases have the agility to permit modifications on the fly - “Shouldn’t we add SMS data in here, too?”
The right answer is, done.
A database that cannot - with little fuss -- add a new row is too rigid for use in true big data analysis because the exciting - maybe maddening? - bit about big data today is that always there is new input that may enhance the overall result.
Then there are the other questions: why are you collecting big data in the first place? What do you want from your analysis of it and this question is key because without targeted analytics, big data is just hoarding.
As an insightful story in The Guardian recently posited, “Companies need to focus on big answers not big data. Instead of focusing upon the concept of big data, organizations should concentrate on the intelligence data can offer.”
In other words, it’s not about the data: it’s about what intelligence can be drawn from it.
The Guardian author calls himself a “big data sceptic” but, really, he isn’t. He just shares the frustration over the many mislabeled big data projects - that never were about big data - and also about the data hoarding that some companies do when they say they are committing to big data. Such projects rarely end well.
Real big data - unstructured, from multiple sources - coupled with real analytics is a game changer that gives forward-thinking organizations insight that before was merely guesswork.
One Texas city ran analyses to determine exactly what happened in parts of the city that experienced higher than anticipated growth and a resulting increase in value.
This was true big data. In the mix were police reports, zoning violations, construction permits, parking tickets, you name it. If the data existed, it was fed into the analysis and the city began to see what it did - and didn’t do - to spur growth. Where could it get out of the way? Where could it proactively spur growth?
It was real big data in action. And it’s why big data remains a big deal, despite the hype.