Posted in February 2012 …

Feb 28 2012
Cloud, DevOps

IT: Increasingly a matter of dollars and cents, not ones and zeroes?

Not content to juggle spreadsheets, CFOs apparently want to manage the IT department, as well.  In a new survey of over 200 UK CFOs and finance directors, 43 percent believe the CIOs role is destined to become part of their companies’ financial functions.  And while it’s not surprising that an executive might believe she can do a better job than her peers, even outside her core skillset, it’s telling that business executives increasingly see technology as part of their job.

Blame the cloud.

After all, it’s the cloud that makes IT more a matter of “Information” and less a matter of “Technology.”  A system like Zendesk is heavy on technology, of course, but its users don’t need to worry about that.  The technology is ‘hidden’ under the covers.  The same is true of Salesforce.com and a range of other systems.  Even Nodeable, a real-time information network used to monitor geeky infrastructure like AWS instances, GitHub, and JIRA, can be and increasingly is used by business-level users.

The cloud has minimized the need to futz with installing and configuring complex infrastructure, even as IT consumerization has made hitherto obscure technology easy to use.

None of which is to suggest that CFOs are well-equipped to take on the role of CIO.  Some probably are.  Many likely aren’t.  As Ewan Leith noted to me on Twitter:

But the point is that the cloud makes the CFO’s interest in running IT a plausible possibility.  And centralizing some of the IT functions within finance actually makes sense, given that the cloud makes some formerly technical solutions a matter of dollars and cents rather than ones and zeroes.

In sum, the IT landscape is changing significantly, even dramatically.  Server huggers of the enterprise world had best beware.  The CFO may not have a valid claim to your job, but the rise of DevOps suggests that there’s a class of technical user that may not care to report up to a CIO, preferring to take this burden of managing all aspects of her application.

The cloud makes it possible.

Tagged , , ,
Feb 22 2012
3 Comments
Cloud, Monitoring, Social

Monitoring for the cloud is broken

When only 24 percent of the IT executives surveyed by InformationWeek acknowledge using a vendor-based monitoring solution for cloud infrastructure, while a full 52 percent use an in-house, custom-built solution, something is wrong with cloud monitoring.  And it’s not the IT executives.

The cloud should be a monitoring dream: programmatic interfaces can significantly expand the breadth and depth of monitoring solutions.  And yet most systems – even top-tier solutions like Splunk – don’t really take advantage of APIs.  Or if they do, they remain fixated on analyzing logs to do root-cause analysis on errors, rather than tracking the more interesting information embedded in state changes.

Which suggests one big reason cloud monitoring solutions routinely fail the needs of cloud adopters: most monitoring solutions continue to have a pre-cloud mentality.  They think about cloud problems through a data center lens.  For example, in the cloud the need for root-cause analysis diminishes.  Just ask Netflix.  Or InformationWeek, which surveyed IT executives and found that the biggest reason for embracing the cloud is speed.  If it takes seconds to pull up a new AMI, why spend hours fixing one?

The other problem is that most monitoring solutions put the burden of analysis on the user.  Even the best tools fail in this respect.  The main problem with “operations intelligence” tools like Splunk, as but one example, is that it’s a “Bring Your Own Intelligence” approach to systems management.  Splunk is basically Google for the data center, which leaves the user to figure out how to parse mountains of data with carefully constructed searches to perform root-cause analysis and other functions.

This isn’t a critique.  After all, when Splunk launched it was the state of the art.  But that was 2004, and Google was the lodestone for intelligent analysis of the web.  This approach still works in some areas, like the data center, but it’s increasingly outdated for the cloud.

For one thing, the cloud allows systems to be truly social.  The reason Nodeable’s tagline is “Twitter for machines” or “Making systems social,” depending on the day/our mood/whether @dr138 has had his iced tea yet, is that it’s no longer enough to analyze system log data in isolation, one system at a time.  Systems talk to each other and affect each other.  A modern monitoring solution needs to be able to interpret the intersections of different cloud services, and help the user understand what’s going on without resorting to arcane search queries.

In fact, a modern approach to monitoring cloud systems really needs to proactively “tweet” analyze all the disparate “tweets” that machines constantly make – their digital exhaust, if you will – and visualize for the user what is happening (trends, anomalies, etc.) in a convenient, accessible way.

Which is why Nodeable looks like a Twitter stream, except that we’re working to parse all those tweets and make sense of them for users, based on both one’s local state changes and our global pool of users.  If you’re running CPU usage at 80 percent and the industry average is 70 percent, you want to know that, even if ultimately you opt not to do anything with it.  If Amazon East is running slowly, you want to know that, too, so we can help you redeploy to Amazon West. And so forth.  You don’t necessarily need to “hear” the tweets of every single system, though we give you that.  You want your monitoring system to intelligently, conveniently visualize what’s happening in your cloud infrastructure.

This is what monitoring for the cloud should look like, and it’s the paucity of existing, vendor-based offerings that has driven so many enterprises to invest in their own custom systems (e.g., only 16 percent of IT executives surveyed built their own advanced monitoring solutions in 2011, but a full 28 percent are doing so in 2012).  We need to do better.

 

Tagged , , , ,
Feb 21 2012
3 Comments
Big Data

McKinsey misses the mark(et) on Big Data

The consultants and analysts at McKinsey & Company are laser-sharp smart.  But they’re not thinking through the implications of Big Data as much as they should.  In a recent research note, “Are You Ready for the Era of Big Data?“, McKinsey points to big potential in Big Data for a range of industries, but leaves out a few that have me scratching my head.

Take the construction industry, for example.  McKinsey gives Construction low marks for potential Big Data value and the ease of capturing data, but both seem wrong to me.

No, I don’t expect the average mid-sized construction executive to be feeding time card and materials receipts into a Watson-like computer to figure out how to optimize her business.  But can I see this same person subscribing to a service that tracks wage and materials pricing trends?  You bet.  Or how about the unemployment figures for a given region, Help Wanted postings, etc.?

All of these data can feed into smarter decisions for the construction executive, perhaps delivered via her mobile phone. Just like the finance guru on Wall Street, this manager in the construction industry needs raw data to do her job.

But just as I’ve written before, her job becomes immeasurably easier if someone takes the trouble of parsing the Big Data and serving it up “small.”

That is, it’s neither accurate to say that data are hard to come by for the construction industry, nor that the value is low.  Rather, the problem is one of interpretation of the data.  In some industries (Finance, Government, etc.), data scientists are available to make sense of oceans of data.  But most of us don’t have the luxury of having a data scientist on staff, and so we need smarter systems to do much of that work for us.

In every market that McKinsey says that Big Data is only lightly relevant, I’d argue the opposite.  The problem isn’t the value of Big Data, but rather in the form in which it’s delivered to the end user.  And I’m sure there will be no shortage of entrepreneurs willing to sign up to cull these treasure troves of data for the benefit of paying customers.

Tagged , ,
Feb 20 2012
Commentary

More signal, less noise

We live in the world of Big Data, with an increasingly computerized world spitting out massive quantities of data.  While it’s theoretically nice to have all that data, the real trick is putting data to work.  Which means, as I’ve written before, we need to figure out how to make Big Data “small.”

It turns out that this isn’t very easy.

Which is probably why über-investor Vinod Khosla points to the concept of “Data Reduction” as one of the huge, still largely untapped opportunities in technology.  By Data Reduction he means “Reducing, filtering and processing data streams to deliver the information or action that is relevant to you.”  It is increasingly clear that our problem isn’t creating abundance (of data, open-source code, or many other things), but rather parsing this abundance so that it’s relevant and useful.  This is as true for Facebook as it is for Red Hat.

Or for your average enterprise IT personnel.  Even the act of managing an enterprise’s IT is increasingly a Big Data concern.  As The 451 Group analyst Rachel Chalmers indicates:

The way we build and manage infrastructure is about to change….Machines can scale, but owing to the time-consuming and difficult-to-automate chores of raising and educating human children, the pool of people talented enough to manage them scales only by a ten- or twenty-year lag. Hence, the rise of next-generation, cloud-scale performance management; capacity planning; and intelligent workload-placement systems.

This becomes easier as systems gather a wide body of data, compare it to a particular user’s performance, and highlight the deltas and suggest ways to improve.  This is what Nodeable and other companies are starting to do, and it will make developers and operations personnel much more effective in their respective roles.

Ultimately, the company that delivers more signal and less noise will win.

Tagged ,
Feb 14 2012
Amazon, Cloud

The futility of Telcos Taking on Amazon Web Services

Another day, another mistaken assumption that Amazon’s public cloud business is ripe for disruption.  This time it’s ZDNet’s David Berlind arguing that Telcos (no, really) are on the cusp of relegating AWS to the dustbin of cloud history.  Berlind gets several things right about the raw materials Telcos have that could be used to unseat Amazon.  But he conveniently overlooks the entire of history of computing to reach his conclusion that these raw materials matter.

I’m not a fatalist, believing that once a company claims a dominant position in a market it necessarily will own that position forever.  The world changes, and in the technology world by the time government antitrust forces get around to taking action against a Microsoft or Google, the market will have changed.  Microsoft won in operating systems but the world moved to the web.  Google is winning on the web in search but we’re increasingly moving toward a social web.  And something will eventually disrupt Facebook’s hold on social.

So I don’t think Amazon’s dominant grip on cloud computing is forever.  But I also don’t think for a millisecond that Telcos will be the ones to unseat Amazon.

Berlind defends his argument by quoting analyst Ari Banerjee, noting that carriers

own the network, they own the subscribers, they are used to delivering five nines availability, they know how to provide turnkey applications and services to hot market segments like [small and midsize businesses] SMBs, and much like the way that Amazon got started in the business of IaaS-provision, they have data centers with extra capacity.

And yet they have done exactly nothing to diminish Amazon’s market power.  Nothing.

Perhaps because, as Cloudscaling CTO Randy Bias argues of enterprise clouds, their approaches are mired in the past.  Amazon is an entirely new way of thinking about IT which enterprises and the Telcos that have served them don’t seem to be able to grasp.  Yes, AWS succeeds because of its simplicity and because of its rapid feature improvements, but ultimately it’s winning because it’s not like the traditional model that Telcos have pushed.

The Telcos have been well-positioned, based on raw materials alone, to take on Apple, Google, and any number of successful companies.  But they haven’t.  They remain dumb pipes, despite buckets of money spent trying to become “un-dumb.”

Cloud isn’t simply hosted IT.  It’s a different way of thinking about IT.  Cloud changes the way we manage our systems, the way we buy/rent them, the way we secure them, and just about everything else.  Don’t look to Telcos to figure this out.  They have far too much invested in outdated ways of thinking about networks and IT.

Tagged ,
Feb 10 2012
5 Comments
Amazon, Cloud

Public cloud: it’s open source all over again

Public cloud adoption largely follows the same pattern as open source adoption, and is having to overcome the same myths that once inhibited widespread adoption of open source.  Security, control, and quality are the red herring arguments that traditional software vendors used to slow the spread of open source, and these same arguments are being resurrected to stem the flow of public cloud adoption.  But it won’t work, just as it failed to stop open source.  And as Red Hat has been cleaning up in the open source revenues sweepstakes, Amazon should win big as the public cloud continues to win converts.

By some estimates, Amazon’s share of the public cloud computing market (IaaS) is as high as 90 per cent.  This doesn’t make it invulnerable – Enstratus’ James Urquhart points to a variety of ways to unseat AWS – but it won’t be for the faint of heart, and current competitors seem mostly to be getting their messaging wrong.  Companies like Alcatel-Lucent try to categorize AWS as “coach class” while their clouds are “first class,”  but the masses seem very happy with a coach class experience in the cloud.  It’s cheap, reliable, and gets them where they want to go.

The next argument plays to the CIO’s biggest concern with the public cloud: security.  How can it possibly be safe to entrust mission-critical applications to the public cloud?  This was the same argument that kept Linux to edge-of-network sort of applications in its early days.  As the story went, Linux would never succeed in the data center.  Who could trust some community science project to mission-critical applications in the data center?

Well, today, who wouldn’t?

This is the same thing that is happening in the public cloud, and particularly with AWS, and it’s been growing on the sly for years, for reasons highlighted by R0ml Lefkowitz back at OSBC 2008 (warning: PDF).  The general adoption pattern goes like this: a developer needs to get work done, and going through traditional IT channels will either take too long or will get killed.  So she puts it up on AWS.  Perhaps she starts with dev and test instances, but soon her team becomes dependent on it and asks the question, “Why redeploy this somewhere else?  Why not just put it into production on AWS, since it works?”  And soon that enterprise is actively deploying to AWS because it’s cost effective, secure, and it works.

Yes, secure.  Amazon claims that AWS is significantly more secure than the average private data center, and there’s every reason to believe this claim.  As Jason Bloomberg argues, Amazon hires the best security people, uses the best hardware, and has experience dealing with constant security threats.  It’s not that there aren’t some private data centers that might be more secure than Amazon’s public cloud, but the odds are that they aren’t.

Once people discover this freedom of the public cloud, and its cost and security advantages, the way they manage their infrastructure also changes.  Right now, Netflix is on the bleeding edge of public cloud adoption, but its mantra of “disposable infrastructure” will soon find its way into the mainstream enterprise.  Why spend days or even hours performing root cause analysis on system failures when it takes mere seconds to spin up a new AMI to replace the failing node?

Old-school systems management, then, becomes largely irrelevant in the cloud, because it focuses too much attention on the past.  The cloud is all about watching current trends and anticipating problems, flexibly deploying configuration changes or whatever is needed to overcome problems.  This is what Nodeable and a new breed of cloud “management” tools do: focus on visibility into cloud infrastructure, rather than the tools to fix past problems.

For those of us who lived through the open source adoption curve, the public cloud adoption curve looks very familiar.  As with open source, it will change the way everyone develops, deploys, and manages software.  It may be that Amazon won’t retain its dominance forever, but its model of exceptional service at rock-bottom prices is going to be hard to beat.  Just like open source.

Tagged , ,
Feb 03 2012
1 Comment
DevOps, Social

Systems made social

There’s one frustrating thing in common with most of the Business Intelligence and Operational Intelligence companies today: they’re very much BYOI.  That is, Bring Your Own Intelligence.  Tools like Tivoli or Splunk do an incredible job of indexing one’s data and making it searchable, but they don’t do much to make all of that big data small.  You still need a highly competent business analyst or developer that has mastered these programs’ arcane search queries to pull insights out of the data, and in today’s incredibly fast-paced world that’s not good enough.  It’s too slow.

It’s not their fault, however.  Such programs were born in an era when data warehousing was all the rage; when it was enough to collect, sort, and search data.  In today’s world, social is the new mantra, and “social” matters as much for machines as it does for people.  APIs can make systems social, and social systems can tell you a lot more about what’s going on in a system, and what should be happening, than brute force searching of voluminous log files ever could.

Back to the BYOI concept.  Splunk is the gold standard in systems management these days, and rightly so.  It’s a powerful product that helps users perform root cause analysis for system failures, among other things.  But to get the most from the tool, you have to be prepared to get your hands dirty.  Really dirty, as TechWorld laments:

If you want to make Splunk work, you’ve got to be ready to abandon the slick GUI and dive deep into difficult technical configuration, editing configuration files, writing regular expressions, and taking the time to understand where your data are coming from and how Splunk will see them….Overall, getting data into Splunk is much more of your typical open source experience, with a confusing maze of pointers, wikis, product tech notes and documentation, but backed up by Splunk’s technical support staff. Plan on spending more than a few moments getting started.

It’s a great tool provided you know what you’re looking for, and how to structure the query to find it.

Let’s call this the old Unix world.  Powerful so long as you’re geeky enough to master it.  Just as in servers, however, we’re seeing an emerging “Insight as a Service” market that goes beyond providing raw tools for searching huge quantities of data, and instead serves up insight based on that data.  Insight as a Service is letting hosted software take on the burden of crunching the data for you, making Big Data a game that everyone can play, and not just data scientists.

In other words, the data still needs to be parsed, but who parses it shifts from the buyer to a third-party vendor.  In the cloud world, that vendor is not only going to be able to analyze your data to look for trends within your enterprise, but can also compare it to a broader database of user data, benchmarking you against competitors and peers.

Now that is real business intelligence.  Social intelligence based on machine data.

And it becomes even more useful if the “insight tool” spans different systems.  It’s fine to have a system that tracks my AWS instances, my Puppet deployment, or whatever, but it’s even better if these disparate views can be normalized and visualized in one place, as Redmonk analyst Stephen O’Grady posits will start to happen in 2012:

The challenges of service intregration will create commercial opportunities for aggregating services which consume individual performance streams, normalize it and present customers with a consolidated single picture of their network performance.

Even in this singular view, care needs to be taken not to overwhelm the user.  In the consumer world, Facebook, more than anyone else, has been focused on this with its Highlighted Stories, moving the full news feed to the top-right corner of the screen.  Users set filters on what content they want to see, and from whom, but Facebook also analyzes data on how users interact with content and each other, and surfaces stories based on the combination of the two.

Well, the same thing can be applied to systems, highlighting the “big stories” that are happening right now in your cloud systems, whether AWS or Salesforce.com or GitHub or some custom application you wrote.

That’s what we’re doing at Nodeable.  Nodeable brings the best features of social networking and data analysis to the cloud by aggregating, normalizing and analyzing the data from systems and services.  As an initial step, Nodeable offers real-time visibility into the cloud software development lifecycle to help identify and react to system-driven trends and patterns in real-time.  Forsaking the BYOI model, Nodeable also adds intelligence to systems data, so that what you see in the data stream directly improves application development and deployment.

Nodeable is a real-time information network for cloud-based systems and services.  We make systems social.  And social is good.

Tagged , , , ,
Follow

Get every new post delivered to your Inbox.

Join 52 other followers

%d bloggers like this: