It is not about the data.

It is about the analytics.

An irony is that the more we dig into Big Data, the clearer it becomes that the data are not the golden nuggets, it’s the insights we can gain from them and where those insights get real power is when they are realtime.

Consider the issue of power generation. Where it gets expensive is at what power company executives call peak load and that is the extra jolt of power needed to keep a system from collapsing under strain as happens, for instance, on some very hot days in the northeast of the United States. There is no trigger event as such - no lightning strikes knocking out generators, for example. All there is are countless air conditioners straining to cool an overheated population.

Already, some large commercial customers are on plans where, for a rate break, they agree to abruptly shutdown in the event a peak is in sight. But what if this could be spread more broadly? What if every user could be notified that a peak is near?

Two things might happen. There would be fewer outages and there would be less need to build peak load plants, meaning overall cost of electricity generation might drop. The key to making this work: realtime analytics that can be extended across a wide user base, giving as many users as possible visibility into the system. But it truly has to be realtime data because, in avoiding power outages, minutes matter.

Smart Grid Norway: Making It Happen

Something similar already is happening in Norway where Smart Grid Norway has announced ambitious plans to make realtime analytics key to how the country consumes power, where and when. This potentially will help create a grid that is more reliable, less costly, and with reduced carbon footprint, making it truly win-win-win and such an outcome is only possible with realtime analytics that produce insights that can be acted on in this instant. Exactly that has become possible.

Meantime, GE has announced a realtime analytics project where the aim is to collect literally terabytes of data from machines such as jet engines and gas turbines and, then, to patch in analytical tools that allow for analysis of what is going on and what should happen now. Lower maintenance costs, increased reliability, and longer lives for the machines are the goals and, again, realtime analytics is what will unlock those payoffs.

Upping the Security Ante

Another, potentially breakthrough use of realtime analytics is securing networks. There is a great deal of gloom around username-password log-in defense based protections - newspapers have been filled with spectacular breaches of well defended networks. But where there is optimism is around harnessing data in realtime to detect anomalous and/or dangerous behaviors - and to shut them down in the instant.

In most companies, realtime analytics as the ultimate firewall frankly is still a dream but it seems certain that exactly that is becoming the best way to thwart hackers before they do significant damage.

The End of Excuses

Too much data, too little analysis. That has been the lament of many CIOs who frankly acknowledged that they have the means to collect massive amounts of data - often from multiple sources, structured and unstructured - but the honest ones acknowledged, at least off the record, that they were at wit’s end about what to do with it all.

ff there is an IT breakthrough that defines this decade, it probably will be Big Data and its associated analytical tools and as we have noted in numerous prior blogs, this came to be out of necessity. So many companies now find themselves drowning in data that cannot be cost effectively stored in relational databases. Thus was born NoSQL and that begat a generation of on the fly analytical tools, such as MongoDB.

But this is early days and, like the song says, the best is yet to come.