Filed under Amazon …

May 11 2012
Amazon, Cloud, Monitoring

The cloud shifts the CIO’s role to “Chief Data Officer”

The longer I’m in tech, the more inclined I am to accept the truth of William Gibson’s quote: “The future is already here — it’s just not very evenly distributed.”  I saw this firsthand with a wealthy friend, who could afford to “see the future” by buying essentially unlimited broadband, powerful servers/computers, and more, and figuring out what the world would look like when average consumers could afford the same.

Sometimes, however, cost isn’t the gatekeeper to the future, but a willingness to risk is.  Such is the case with the cloud.

Google CIO Ben Fried thinks we’re nearing the tipping point for cloud computing, when CIOs determine that cloud computing’s cost and simplicity advantages outweigh other concerns like a lack of customizability, and jump in with both feet.  Sure, enterprises are already using cloud services: 86 percent according to Cloudability data; 81 percent by KPMG’s survey count; and 48 percent for SMBs, according to a recent survey.  But few big enterprises are using the cloud to handle the majority of their workloads.

In the future, according to Fried’s thinking, that will change.

Amazon is destined to displace big iron vendors like IBM and HP as the cloud becomes the preferred destination for enterprise computing, including mission critical workloads.  Just as Linux used to be relegated to the fringe of computing but came to dominate the heart of the data center so, too, will the cloud wreak havoc on the traditional data center business.

By taking technology out of the IT equation, Fried argues that cloud computing changes the way businesses structure themselves and do business, and may even force them to change the business they’re in altogether.  In many ways, cloud computing lets enterprises focus on the data that results from IT, rather than the IT itself.  This is a huge shift.

This isn’t to suggest that enterprises will completely forget about servers and such, but it does mean that they’ll think about compute resources differently, and will almost certainly have to think of new ways to keep tabs on them.  Companies like Boundary and Nodeable were built in the cloud for cloud resources, and focus more on surfacing actionable insights than on giving administrators or developers the tools to “spelunk” for themselves, which is inefficient and largely unnecessary in a world of semi-structured data accessed through APIs.

All of which would be a really bad idea if the cloud were just a fad.  But it’s not.  It’s how IT gets done going forward.  And it means that the Chief Information Officer is going to need to recalibrate the way she thinks about “Information.”  Namely, more a matter of “data” and less a matter of “technology.”

Tagged , , , , , ,
May 01 2012
Amazon, Big Data, Monitoring

Amazon’s essential role in delivering scale and tackling Hadoop

The cloud has many virtues, but perhaps its biggest is scale.  Scale refers to the ability to throttle resources up or down to meet inbound demand on web applications or other infrastructure.  It’s a problem that most developers, whether at startups or Fortune 500 behemoths, can only dream of having.  But for the ill-prepared, scaling an application can be a nightmare.

Which is why Amazon Web Services have become such essential infrastructure for startups and enterprises alike.  As Ryan Park, operations and infrastructure leader at Pinterest, declared earlier this month at AWS Summit:

Imagine if we were running out own data centre, and we had to go through a process of capacity planning, and ordering hardware, and racking that hardware, and so on.  It just would not have been possible scale fast enough – especially with such a small team. Until about a month ago, I was the only operations engineer at the whole company.

Think about that for a minute.  Here’s a web service with nearly 18 million visitors in February alone, which took the company just nine months to reach.  In a pre-AWS world, Pinterest would have employed scores of operations engineers to buy and manage hardware and the software to stitch it all together.  No more.

Of course, managing the infrastructure is only part of the equation.  Perhaps even more important is managing all the data that today’s businesses increasingly collect.  The lingua franca of this Big Data movement is Hadoop, which enables companies to crunch through massive piles of data to find actionable insights into how better to run one’s business.  Hadoop’s importance in our data-hungry world is perhaps best articulated by Cloudera CEO (and Nodeable board member) Mike Olson:

In the old days if you had a data problem you would write a big check for a massive piece of hardware and with any money left over you would by some very expensive but powerful software. That box with software and data became your data temple and your analysis and conclusions were done there.

There are problems with that approach. Data are now growing so fast. It is now impossible for one box to have all your data. You must have your data across multiple servers and use software that can coordinate and operate across all those servers. Hadoop is the platform designed to do this. It is designed to solve the problems of today, not the problems of yesterday.

Critically, Hadoop, too, is increasingly a matter of the cloud and, particularly, of AWS.  By some estimates Hadoop jobs comprise the majority of all AWS processing.  With petabyte-scale data clusters increasingly common, shifting that burden of storage and processing to the cloud becomes essential.

As more infrastructure and data processing moves to AWS, it becomes more and more important to analyze your AWS instances to track real-time trends (“Is my CPU running hot?”), make comparisons (“We’re running memory 25 percent lower than most companies – should we look to optimize?”), discover anomalies, and so on.  That’s where Nodeable comes in.

It used to be that the cloud is where enterprises dumped non-critical applications.  Now the inverse is true.  The cloud is the hub for mission-critical data processing.  It’s where enterprises are running applications that need serious scale.  And it just so happens to be where Nodeable shines.  Nodeable surfaces actionable insights into an easy-to-grok, Twitter-like activity stream.  Search tools like Splunk are nice, but Nodeable prefers to reveal those insights while you sip your tea or watch your daughter’s soccer game.

Why not sign up for our beta and give it a try?

Tagged , , , , ,
Apr 06 2012
Amazon, Cloud

If the cloud is all about simplicity, why is it so complex?

Software now leads hardware in terms of overall tech spending, and the biggest growth in software spending is actually not software at all, according to Forrester.  No, the biggest growth drivers in tech today are SaaS applications, general business intelligence products, and specialized analytical tools.  Sure, some of these run behind the firewall, but increasingly businesses are following consumers to the cloud.  Where businesses don’t seem content to follow consumers, however, is in the simplicity of their cloud products.

Some enterprise vendors get this.  Take Box, for example, whose cardinal rule for its content collaboration services is simplicity.  If real human beings don’t want to use the software/services, then why should enterprises waste money trying to force them to do so?

But far too many cloud vendors mire themselves and their users in complexity.  As Charles Babcock writes, “I am struck over and over again how easy it is to discuss cloud computing in high sounding terms, while those plunging into the cloud are thrust into a welter of new technology processes and complex responsibilities.”  Bingo.

Cloudscaling’s Randy Bias goes even deeper, picking apart the problems Infrastructure as a Service vendors have had in making it super-easy to run systems in the cloud.  As he notes, “[M]any engineers see understanding and developing complex systems as a rite of passage.  In reality, the true test of a great engineer is their ability to make things simpler, not more complex.”  As he goes on to say, complex systems tend to fail, but simplicity enables scalability.

Just ask Amazon, whose public cloud is by far the most widely used in large part because it’s comparatively easy for developers to use.

I see this in the systems management world, which actually tend to compound the problem of complex cloud systems with complex tooling that exacerbates the very problem it could help to solve.  Bias talks about the problems inherent in presenting users with too many choices (e.g., multi-hypervisor IaaS offerings), but the same is true with the language used to describe systems.

For example, we talk a lot about real-time, but the best description may actually be this one that I found: human real-time.  Ultimately, a user really doesn’t care how the vendor goes about processing and delivering insights into systems so long as it’s happening as fast as they need.  Nodeable uses continuous computation.  Other vendors may prefer an alternative.  But, again, I suspect very few users actually care.  They just care that the analytics serve up insights that are timely and germane to their jobs.

Cloud was supposed to make life easier for the enterprise, and I think it’s accomplishing that goal, on balance.  But we have a long ways to go toward simplifying the cloud for users, and not just in how we explain it.  Recent history suggests that the companies who do best at simplifying complex systems – think Apple, Microsoft, Amazon, and others – are those that win big in terms of revenue.  It turns out simplicity pays.

Tagged , , , ,
Feb 14 2012
Amazon, Cloud

The futility of Telcos Taking on Amazon Web Services

Another day, another mistaken assumption that Amazon’s public cloud business is ripe for disruption.  This time it’s ZDNet’s David Berlind arguing that Telcos (no, really) are on the cusp of relegating AWS to the dustbin of cloud history.  Berlind gets several things right about the raw materials Telcos have that could be used to unseat Amazon.  But he conveniently overlooks the entire of history of computing to reach his conclusion that these raw materials matter.

I’m not a fatalist, believing that once a company claims a dominant position in a market it necessarily will own that position forever.  The world changes, and in the technology world by the time government antitrust forces get around to taking action against a Microsoft or Google, the market will have changed.  Microsoft won in operating systems but the world moved to the web.  Google is winning on the web in search but we’re increasingly moving toward a social web.  And something will eventually disrupt Facebook’s hold on social.

So I don’t think Amazon’s dominant grip on cloud computing is forever.  But I also don’t think for a millisecond that Telcos will be the ones to unseat Amazon.

Berlind defends his argument by quoting analyst Ari Banerjee, noting that carriers

own the network, they own the subscribers, they are used to delivering five nines availability, they know how to provide turnkey applications and services to hot market segments like [small and midsize businesses] SMBs, and much like the way that Amazon got started in the business of IaaS-provision, they have data centers with extra capacity.

And yet they have done exactly nothing to diminish Amazon’s market power.  Nothing.

Perhaps because, as Cloudscaling CTO Randy Bias argues of enterprise clouds, their approaches are mired in the past.  Amazon is an entirely new way of thinking about IT which enterprises and the Telcos that have served them don’t seem to be able to grasp.  Yes, AWS succeeds because of its simplicity and because of its rapid feature improvements, but ultimately it’s winning because it’s not like the traditional model that Telcos have pushed.

The Telcos have been well-positioned, based on raw materials alone, to take on Apple, Google, and any number of successful companies.  But they haven’t.  They remain dumb pipes, despite buckets of money spent trying to become “un-dumb.”

Cloud isn’t simply hosted IT.  It’s a different way of thinking about IT.  Cloud changes the way we manage our systems, the way we buy/rent them, the way we secure them, and just about everything else.  Don’t look to Telcos to figure this out.  They have far too much invested in outdated ways of thinking about networks and IT.

Tagged ,
Feb 10 2012
5 Comments
Amazon, Cloud

Public cloud: it’s open source all over again

Public cloud adoption largely follows the same pattern as open source adoption, and is having to overcome the same myths that once inhibited widespread adoption of open source.  Security, control, and quality are the red herring arguments that traditional software vendors used to slow the spread of open source, and these same arguments are being resurrected to stem the flow of public cloud adoption.  But it won’t work, just as it failed to stop open source.  And as Red Hat has been cleaning up in the open source revenues sweepstakes, Amazon should win big as the public cloud continues to win converts.

By some estimates, Amazon’s share of the public cloud computing market (IaaS) is as high as 90 per cent.  This doesn’t make it invulnerable – Enstratus’ James Urquhart points to a variety of ways to unseat AWS – but it won’t be for the faint of heart, and current competitors seem mostly to be getting their messaging wrong.  Companies like Alcatel-Lucent try to categorize AWS as “coach class” while their clouds are “first class,”  but the masses seem very happy with a coach class experience in the cloud.  It’s cheap, reliable, and gets them where they want to go.

The next argument plays to the CIO’s biggest concern with the public cloud: security.  How can it possibly be safe to entrust mission-critical applications to the public cloud?  This was the same argument that kept Linux to edge-of-network sort of applications in its early days.  As the story went, Linux would never succeed in the data center.  Who could trust some community science project to mission-critical applications in the data center?

Well, today, who wouldn’t?

This is the same thing that is happening in the public cloud, and particularly with AWS, and it’s been growing on the sly for years, for reasons highlighted by R0ml Lefkowitz back at OSBC 2008 (warning: PDF).  The general adoption pattern goes like this: a developer needs to get work done, and going through traditional IT channels will either take too long or will get killed.  So she puts it up on AWS.  Perhaps she starts with dev and test instances, but soon her team becomes dependent on it and asks the question, “Why redeploy this somewhere else?  Why not just put it into production on AWS, since it works?”  And soon that enterprise is actively deploying to AWS because it’s cost effective, secure, and it works.

Yes, secure.  Amazon claims that AWS is significantly more secure than the average private data center, and there’s every reason to believe this claim.  As Jason Bloomberg argues, Amazon hires the best security people, uses the best hardware, and has experience dealing with constant security threats.  It’s not that there aren’t some private data centers that might be more secure than Amazon’s public cloud, but the odds are that they aren’t.

Once people discover this freedom of the public cloud, and its cost and security advantages, the way they manage their infrastructure also changes.  Right now, Netflix is on the bleeding edge of public cloud adoption, but its mantra of “disposable infrastructure” will soon find its way into the mainstream enterprise.  Why spend days or even hours performing root cause analysis on system failures when it takes mere seconds to spin up a new AMI to replace the failing node?

Old-school systems management, then, becomes largely irrelevant in the cloud, because it focuses too much attention on the past.  The cloud is all about watching current trends and anticipating problems, flexibly deploying configuration changes or whatever is needed to overcome problems.  This is what Nodeable and a new breed of cloud “management” tools do: focus on visibility into cloud infrastructure, rather than the tools to fix past problems.

For those of us who lived through the open source adoption curve, the public cloud adoption curve looks very familiar.  As with open source, it will change the way everyone develops, deploys, and manages software.  It may be that Amazon won’t retain its dominance forever, but its model of exceptional service at rock-bottom prices is going to be hard to beat.  Just like open source.

Tagged , ,
Follow

Get every new post delivered to your Inbox.

Join 52 other followers

%d bloggers like this: