Dave and I were fortunate to attend a Churchill Club event on Hadoop Tuesday night. Hadoop sits at the center of the burgeoning Big Data universe, and so one might be tempted to conclude that it’s basically a finished product. Not so, said the esteemed panel, which included representatives from Cloudera, Facebook, Metamarkets, MapR, and Oracle. In fact, arguably the biggest opportunity in Hadoop isn’t Hadoop at all: it’s the cloud applications built on top of Hadoop.
Dave summarized the panel discussion on CNET, and highlights Cloudera CEO Mike Olson’s call to arms for Hadoop-based applications. It’s something Olson has said before, including here on this blog, but it was particularly poignant against the backdrop of a deep, engaging discussion about Hadoop’s pros (powerful, open source) and cons (batch-oriented, complex, somewhat inefficient).
And it’s why I think Nodeable is a sign of the times.
We’re an application that depends upon Hadoop. But we’re also a technology that improves Hadoop by front-ending it with Storm. Hadoop is powerful but limited to batch-oriented processing of data. Storm actually crunches data in real-time, in the stream. The combination of the two is potent, and something that we only discovered while building out our application to ingest systems data and extrapolate insights via in-stream data analytics.
In the near future the back-end data processing via Storm and Hadoop will be managed behind the scenes by cloud applications, as Workday co-CEO Aneel Bhusri tweeted from the Churchill Club event. For now, companies like Nodeable are helping to bridge the divide between complex infrastructure and simplified applications.