There’s one frustrating thing in common with most of the Business Intelligence and Operational Intelligence companies today: they’re very much BYOI. That is, Bring Your Own Intelligence. Tools like Tivoli or Splunk do an incredible job of indexing one’s data and making it searchable, but they don’t do much to make all of that big data small. You still need a highly competent business analyst or developer that has mastered these programs’ arcane search queries to pull insights out of the data, and in today’s incredibly fast-paced world that’s not good enough. It’s too slow.
It’s not their fault, however. Such programs were born in an era when data warehousing was all the rage; when it was enough to collect, sort, and search data. In today’s world, social is the new mantra, and “social” matters as much for machines as it does for people. APIs can make systems social, and social systems can tell you a lot more about what’s going on in a system, and what should be happening, than brute force searching of voluminous log files ever could.
Back to the BYOI concept. Splunk is the gold standard in systems management these days, and rightly so. It’s a powerful product that helps users perform root cause analysis for system failures, among other things. But to get the most from the tool, you have to be prepared to get your hands dirty. Really dirty, as TechWorld laments:
If you want to make Splunk work, you’ve got to be ready to abandon the slick GUI and dive deep into difficult technical configuration, editing configuration files, writing regular expressions, and taking the time to understand where your data are coming from and how Splunk will see them….Overall, getting data into Splunk is much more of your typical open source experience, with a confusing maze of pointers, wikis, product tech notes and documentation, but backed up by Splunk’s technical support staff. Plan on spending more than a few moments getting started.
It’s a great tool provided you know what you’re looking for, and how to structure the query to find it.
Let’s call this the old Unix world. Powerful so long as you’re geeky enough to master it. Just as in servers, however, we’re seeing an emerging “Insight as a Service” market that goes beyond providing raw tools for searching huge quantities of data, and instead serves up insight based on that data. Insight as a Service is letting hosted software take on the burden of crunching the data for you, making Big Data a game that everyone can play, and not just data scientists.
In other words, the data still needs to be parsed, but who parses it shifts from the buyer to a third-party vendor. In the cloud world, that vendor is not only going to be able to analyze your data to look for trends within your enterprise, but can also compare it to a broader database of user data, benchmarking you against competitors and peers.
Now that is real business intelligence. Social intelligence based on machine data.
And it becomes even more useful if the “insight tool” spans different systems. It’s fine to have a system that tracks my AWS instances, my Puppet deployment, or whatever, but it’s even better if these disparate views can be normalized and visualized in one place, as Redmonk analyst Stephen O’Grady posits will start to happen in 2012:
The challenges of service intregration will create commercial opportunities for aggregating services which consume individual performance streams, normalize it and present customers with a consolidated single picture of their network performance.
Even in this singular view, care needs to be taken not to overwhelm the user. In the consumer world, Facebook, more than anyone else, has been focused on this with its Highlighted Stories, moving the full news feed to the top-right corner of the screen. Users set filters on what content they want to see, and from whom, but Facebook also analyzes data on how users interact with content and each other, and surfaces stories based on the combination of the two.
Well, the same thing can be applied to systems, highlighting the “big stories” that are happening right now in your cloud systems, whether AWS or Salesforce.com or GitHub or some custom application you wrote.
That’s what we’re doing at Nodeable. Nodeable brings the best features of social networking and data analysis to the cloud by aggregating, normalizing and analyzing the data from systems and services. As an initial step, Nodeable offers real-time visibility into the cloud software development lifecycle to help identify and react to system-driven trends and patterns in real-time. Forsaking the BYOI model, Nodeable also adds intelligence to systems data, so that what you see in the data stream directly improves application development and deployment.
Nodeable is a real-time information network for cloud-based systems and services. We make systems social. And social is good.
Like this:
Like Loading...