Posted in May 2012

The problem with treating people like data: Learning from Autonomy’s mistakes

As much as we tout the importance of data in today’s fast-paced markets, Autonomy CEO Mike Lynch is a poignant reminder that people matter, too.  A lot.

HP bought Autonomy in late 2011 for $10 billion.  Autonomy was one of the UK’s brightest tech stars, but its CEO, Mike Lynch, was known to be somewhat of a difficult personality.  How difficult?  So bad that Autonomy employees gave Lynch a measly 20 percent approval rating. If the pundits think President Obama has a tough road ahead of him with a nearly 50 percent approval rating, imagine Lynch’s likelihood of getting elected.

No.  Way.

In fact, as Wired reports, the only way HP could maximize the value of its $10 billion acquisition was to fire Lynch.  This is ironic, given that Autonomy’s business is to “make sense of and process unstructured, ‘human information,’ and draw real business value from that meaning.”  The company that enables others to glean meaningful information from unstructured data was at pains to treat its employees as anything more than cogs in a machine, to be tightened and tweaked to force them to perform.

In other words, as much as we may want to boil business down to 1s and 0s, ultimately all business is about meeting human needs, not only as customers but also as employees.  Even Nodeable, which ingests machine data, processes it in real-time, and outputs useful insights is ultimately in the business of serving people, not machines.

Autonomy has built a good business based on serving customer needs. But it has started to decline as its employees struggled to enjoy apparently tyrannical working conditions.  By showing Lynch the door, HP has taken the first step toward treating both its customers and its employees with respect, which turns out to be very good business.

Tagged , ,

Survey: CIOs are confused on prioritizing IT projects, and especially on how to pay for them

CIOs are an optimistic lot these days.  According to recent survey data from InformationWeek, 61 percent of those IT executives surveyed indicate that their IT budgets will remain the same or shrink.  Yet the vast majority claim that important new projects for cloud, Big Data, security, and more will come from “new money” rather than “savings.”

How does that math work?

It’s not as if the proposed projects are useful.  As shown below, CIO’s seem to have a good handle on where money should be spent:

What they lack, of course, is a grasp on reality in terms of funding all these projects, as shown here:

And while we at Nodeable don’t have the be-all, end-all answer for how to fund these projects, we can suggest one: optimize efficiency of existing resources.  This actually fits IT priorities, generally.  After all, four of the top-five projects identified are “block-and-tackle” projects that improve existing systems rather than introducing a gee-whiz new line of business system.


The difference, of course, is that one can introduce a system like Nodeable and not only bring down costs (by tuning cloud systems based on our trending data, anomalies we flag in how resources are being used, etc.), but also drive one’s business by analyzing how resources are being used at a macro level.

I can see, for example, who is most active in handling JIRA tickets.  I can see which of my developers show up most often in the GitHub activity stream.  And while I can of course track waste in my use of AWS, I can also benchmark how my company manages its storage and compute resources against how others do.

Ultimately, what needs to be done is bring IT in better alignment with business goals.  The DevOps trend does this by reducing bureaucracy, allowing developers to get work done with a minimum of overhead.  This is the crowd Nodeable hopes to enable.

Otherwise, we end up with a mismatch of resources with goals, as InformationWeek points out:

What about hybrid clouds and cloud bursting, an activity that promises to dramatically change the face of IT spending and human resourcing as we know it? Marquee names like Zynga and DreamWorks are just two pioneers that have managed to optimize their internal infrastructure spend by balancing private and public cloud. Yet only 10% of our survey respondents identify private cloud as a top priority.

Worse, the project that came in at No.12 of 12, with a whopping 2%—launching or upgrading an enterprise social networking platform—is one that has the attention of non-IT partners….We guarantee you that if we had surveyed CMOs and their direct reports instead of CIOs and their reports, social would have been near the top.

Enterprises need to figure out how to do more with less, and that “less” means getting to more productivity with “less money,” which often will necessitate less cumbersome and costly bureaucracy.  Nodeable offers one way to accomplish this, and no doubt you can think of others.  It’s only as IT becomes more agile and joined-at-the-hip with business requirements that it’s going to be a hero in 2012.

Tagged , , , , ,

Beauty in Mobile Web Development

Nodeable mobileRecently I (Tim, front-end wizard) had the joy of creating our mobile website. It’s definitely a work in progress, but we are all proud of it. I want to start off by giving you a few insights into what makes our mobile website beautiful.

  • Total Images: 3 files for 28kb
  • Total JS Files: 15 files (11 if you don’t include all our analytics) for 140kb
  • Total HTML Files: 1 for 725b
  • Total Stylesheets: 2 for 3.67kb

Grand Total: 32 Requests for 180kb and onload hitting in 1.04s

That’s really not too bad when compared with most mainstream mobile websites. To give you an idea.*

  • Papa John’s Online Order: 10 requests for 152kb
  • Mobile Facebook: 10 requests for 197kb
  • Mobile Ebay: 22 requests for 267kb
  • Mobile Youtube: 11 requests for 290kb
  • Mobile Amazon: 8 requests for 81kb
  • Mobile Cnet: 66 requests for 489kb
  • Mobile CNN: 28 requests for 385kb
  • Mobile Google Search: 20 requests for 716kb
  • Sencha Touch Kitchen Sink: 21 requests for 818kb
  • jQuery Mobile Documentation: 11 requests for 277kb
So, as you can see our mobile website stacks up pretty well. Our mobile website was accomplished using several techniques that I recommend to anyone wanting to build a mobile web site. I’ve created quite a few mobile websites using many different frameworks and it’s amazing how quickly you learn from bad experiences.

CSS3

You should be utilizing as much CSS3 as possible. With mobile you don’t have to worry about non-standard browsers because nearly all smart phones support Webkit and thus CSS3 and HTML5. In our application, the only images used were for the logo, sprites (little icons), and a waiting spinner. The rest of it was done using border-radius, background gradients, box shadows, and CSS sprites.

The Framework

Our framework is all custom-coded and uses require.js with backbone.js, underscore.js, and jQuery. I eventually have plans to swap the jQuery with Zepto to give us even greater efficiency. There are many pre-built frameworks that exist out there, but I will never again use any of them. They all have several problems in common.

  1. You’re only going to be using about 1/3 of what the framework has to offer, so the rest of that is just bloating your mobile website.
  2. Once you deviate from their kitchen sink examples and try to customize something, the entire application is going to suffer and degrade.
  3. It’s extremely difficult to debug those frameworks. Ever try debugging Sencha Touch? Good luck!

Conclusion

Getting a great mobile website isn’t done overnight. Lots of time and care has to be put into it. On the flip-side, it also doesn’t mean that you have to break the bank on it. By using standardized tools and the latest advancements in CSS3 and HTML5, having a mobile website that looks great and runs fast is completely attainable.

*Disclaimer: this data is for initial page loading only. Subsequent requests typically contain cached resources. Some of those sites are more “image” oriented. Some of the counted requests might also be ajax requests to retrieve data.

Tagged , , , , , , ,

Nodeable May release: Introducing Insights

Over the last few months of the Nodeable beta we’ve been obsessing over how our 400+ beta customers use the service and what they want from the product. We heard loud and clear that we should increase the value of the data we present and decrease the importance of the social features that we thought were initially interesting. Hooray for user feedback!

Cruise over to beta.nodeable.com and login.

Introducing Nodeable Insights!

Nodeable Insights Nodeable Insights are the new “messages of interest” that are surfaced by our analytics engine.Nodeable analyzes your data in real-time and applies continuous computation to determine the state of your resources — including finding and revealing anomalies in addition to providing summaries and status updates.

The New Event Stream!

stream 2 We’re making big data small and doing the real-time analysis so that you know what you should be paying attention to at any moment in time.But, if you want to see every little thing that your systems are doing you can just jump over to the stream where all is revealed!


Dashboard coming soon!

There are quite a few features that we’ll be rolling over the next few weeks, including the dashboard and the Inbound Messaging Gateway that will allow you to send any type of data in Nodeable.Our rolling release cycle means you’ll automatically get the latest and greatest features while we sleep as little as possible.

Take our survey!

To make sure we’re doing the right things, we’re running a quick 10 question survey that should take you no more than 30 seconds to complete.We need your help to make Nodeable better and everyone that completes the survey is automatically eligible for an office visit from the Nodebelly himself.**
**(Note: the Nodebelly doesn’t actually exist but we will send you a t-shirt)

Microsoft’s masochistic licensing fixation: caught in The Innovator’s Dilemma

It’s like Microsoft is watching as the train barrels towards it, but refuses to get out of the way.  Even as the world moves to the cloud and open source, Microsoft’s general counsel, Brad Smith, proclaims:

No, Brad, licensing is emphatically not the path forward, for Microsoft or many other companies.  Yes, you continue to make gobs of money selling licenses to Office, Windows, and such.  But perhaps you’ve noticed where all the new money is going?  It’s not going to buy office productivity suites at all.

It’s buying services.  That run on the web.  Only a tiny fraction of which are sold by Microsoft.  Doesn’t this bother you?

Sure, you have managed to eke out some semblance of a mobile business by suing and/or scaring would-be licensees into paying you for Android (since almost no one is paying you for Windows Mobile).  But you’re missing the point.  Even Apple, proprietary in ways you never dreamed of being, doesn’t license software.  That’s what we did back in the 80′s and 90′s.  But it’s not where business is going, and your obsession with living on past business models is tripping you up from embracing the future.

I’ll do you a favor.  There’s a book that talks about this.  It’s called The Innovator’s Dilemma, and was written by Clayton Christensen  Perhaps you’ve heard of it?  I’ll give you a summary version from an excellent article in The New Yorker profiling Christensen (You’ll have to buy the magazine to get the full version because The New Yorker is part of your crowd, and paywalls its content):

Christensen didn’t blame big companies for moving upmarket.  They had to grow a certain amount every year, so selling bad products at low margins (like rebar) was never going to seem like the sensible thing for them to do.

And venturing into markets that barely existed (like teen-age radio consumers [or cloud computing or that newfangled Internet thing or tablets that aren't clunky and simply tired renditions of your desktop Windows experience]) didn’t seem sensible, either, because, without the benefit of hindsight, how could you tell the difference between a bad product poised to take over the nation and just a bad product? You couldn’t invest in every dumb thing that came along — you’d go bankrupt. The sensible thing for big companies to do was to pursue higher margins, or to wait until a new product’s market became visible enough to be analyzed and large enough to be interesting–but by then it was too late.

Meanwhile, the big companies kept doing what they were supposed to, listening to their customers and improving their products in ways that mattered to those customers, until they had improved them too much, climbed so far upmarket that they sailed right off the upper-right-hand corner of the graph, adding more features and power and degrees of perfection than anyone could possibly use, and by that time the bad, cheap, low-end product had improved to the point where it could finally appeal to the big companies’ customers, and the big companies failed.

Kind of like what happened this week with the U.S. Department of the Interior announced it would move 90,000 employees over to Google Gmail and Apps.  Microsoft has been slowly embracing online services, but when virtually all of your money comes from your old-school licensing business, what can you possibly do but continue to try to force customers down that road?  Think of how much easier it would be if your revenue were tied to an Office service rather than a one-off licensing event, with ongoing maintenance.  I bet you think of that every day.

But then you tweet about how licensing is the future, when it is so clearly the past.

Don’t believe me?  Get out of Redmond and drive down the 520 to Seattle, where you can visit Amazon, which is polishing the nails for your coffin with Amazon Web Services, which is the platform for the next 10 to 20 years of computing.  Not Windows.  AWS.  It’s already enabling the next wave of innovative startups, as Amazon CTO Werner Vogels recently pointed out, but the shift is much deeper and broader than that.  The future is not being built on Microsoft technology.  Period.

And telling the world to license your mobile technology, when it’s clearly going down the Apple or Android routes, is a losing strategy.  While Facebook goes public on a $100 billion valuation your world is threatening to layoff 25,000 people, and that’s just at one company.  Eventually those people will find their way into companies focused on the future, but not if they heed your advice and fixate on licensing.

Unfortunately, your advice isn’t even good for Microsoft, and that’s a bad thing, as we need a competitive Microsoft again.  You make great software.  You just need to start distributing software the way the world has chosen for the 21st Century: think service subscriptions, not software licenses.  We’ve met before at OSBC and at Stanford Law School, and I think you’re a wonderful person.  I just happen to think you’re woefully misguided with this licensing fixation, and it’s hurting your company and the industry.

Even developers like simplicity

How is it that we can manage to follow an average of 245 friends on Facebook and 350 people on Twitter, yet we struggle to effectively manage a handful of cloud resources?  Some will argue that it’s because social information is less important and hence requires less vigilance.  We can manage more because we actually manage less.  After all, we’re unlikely to be fired if we miss our uncle’s post about how many miles he ran today, but we could if we allow the website to crash and burn.  But the problem may also derive from the user experience.

Systems management tools get a bad rap, and for good reason.  The user experience can be less than appealing.

There’s a belief – a false one, in my experience – that technical IT folks must necessarily love complex ways in which to manage their systems.  I’m sure there are über-geeks for whom complexity is an end in itself.  But they’re the exception, not the rule.

In a conversation with an IT team at a Fortune 100 company earlier this year, one of the system administrators said that he’d buy a tool that “did whatever John would do.”  John was their Nagios expert, a system that no one else on the team could decipher.  As the sysadmin lamented, however, John sometimes goes on vacation or is unavailable while he (gasp!) spends time with family, etc.  So he wanted to receive alerts on his phone when things went awry, with one button: “Do what John would do.”

He’d click that button early and often.

Nodeable isn’t yet at the point where we learn John’s behavior in given situations and make it easily replicable by others in his absence, but as an industry I suspect we’re not terribly far off from being able to approximate this.  What we can do is simplify IT management by surfacing trending issues/anomalies/etc. so that the heavy lifting of managing cloud systems is done by Nodeable, not the developer or her operations team.  It’s not exactly “what would John do?” management, but it’s a headstart on seriously reducing complexity so that IT can focus on tailoring systems to improve business, rather than performing root cause analysis.

And, no, it’s not necessarily an easier in the cloud, even though the magic of the cloud can be the hiding of infrastructure complexity.  But the real complexity is in deciphering what’s happening in real time as apps are updated, systems are tweaked, etc.  Any changes are made on a granular level, not on a “system” level, as Enstratus James Urquhart argues,

If something goes wrong with an application, developers are on the hook to fix it, change it or kill it….However, developers and engineers can only make those changes one, or a few, components at a time. Nobody can configure the “system” to work an expected way. All you can do is constantly monitor the success and effectiveness of the technologies you deploy into the cloud, and constantly tweak them to make them as useful as they can be in that environment.

For developers to be most effective, they need to spend most of their time writing and optimizing their applications, not deciphering archaic error messages, constructing search queries in Splunk to search out root problems, or other traditional IT tasks.  A good system will surface insights into trending issues in real-time, based on continuous tracking of machine data that gives clues as to whether the changes to the system are helping or hurting.

In sum, a system is powerful not only in the various features it claims to have, but also in how well it obscures the complexity behind-the-scenes to let developers focus on writing apps.  It’s not yet “what John would do,” but it’s getting close.

Tagged , , , , ,

The cloud shifts the CIO’s role to “Chief Data Officer”

The longer I’m in tech, the more inclined I am to accept the truth of William Gibson’s quote: “The future is already here — it’s just not very evenly distributed.”  I saw this firsthand with a wealthy friend, who could afford to “see the future” by buying essentially unlimited broadband, powerful servers/computers, and more, and figuring out what the world would look like when average consumers could afford the same.

Sometimes, however, cost isn’t the gatekeeper to the future, but a willingness to risk is.  Such is the case with the cloud.

Google CIO Ben Fried thinks we’re nearing the tipping point for cloud computing, when CIOs determine that cloud computing’s cost and simplicity advantages outweigh other concerns like a lack of customizability, and jump in with both feet.  Sure, enterprises are already using cloud services: 86 percent according to Cloudability data; 81 percent by KPMG’s survey count; and 48 percent for SMBs, according to a recent survey.  But few big enterprises are using the cloud to handle the majority of their workloads.

In the future, according to Fried’s thinking, that will change.

Amazon is destined to displace big iron vendors like IBM and HP as the cloud becomes the preferred destination for enterprise computing, including mission critical workloads.  Just as Linux used to be relegated to the fringe of computing but came to dominate the heart of the data center so, too, will the cloud wreak havoc on the traditional data center business.

By taking technology out of the IT equation, Fried argues that cloud computing changes the way businesses structure themselves and do business, and may even force them to change the business they’re in altogether.  In many ways, cloud computing lets enterprises focus on the data that results from IT, rather than the IT itself.  This is a huge shift.

This isn’t to suggest that enterprises will completely forget about servers and such, but it does mean that they’ll think about compute resources differently, and will almost certainly have to think of new ways to keep tabs on them.  Companies like Boundary and Nodeable were built in the cloud for cloud resources, and focus more on surfacing actionable insights than on giving administrators or developers the tools to “spelunk” for themselves, which is inefficient and largely unnecessary in a world of semi-structured data accessed through APIs.

All of which would be a really bad idea if the cloud were just a fad.  But it’s not.  It’s how IT gets done going forward.  And it means that the Chief Information Officer is going to need to recalibrate the way she thinks about “Information.”  Namely, more a matter of “data” and less a matter of “technology.”

Tagged , , , , , ,

Hadoop as a ‘data refinery’ – good overview of how Hadoop works

Hortonworks’ Shaun Connolly is a smart man.  I say this not because he’s able to use big words and dense technical explanations to confuse me.  Lots of people can do that.  No, what makes Shaun smart is that he’s able to boil down complex systems into easily understandable ideas.  At dinner earlier this week he demonstrated this by equating Hadoop with a “data refinery,” performing a similar function for data as oil refineries manage for crude oil.

The analogy was useful to me.  I don’t work for Chevron but understanding the oil refining process is relatively simple.  You take crude oil and through distillation (i.e., boiling the oil to separate out hydrocarbons based on their vaporization temperatures) or chemical processing, you’re able to refine it into a form that is useful to power cars, lawn mowers, or laptops. (!)

Hadoop is very similar, minus the boiling and chemical processing.  It’s a data-processing framework that can ingest huge quantities of unstructured data like tweets or credit card receipts, and output reports that average humans can understand. One of the best explanations I’ve seen for how Hadoop works was written by a software engineer named Matt.  (No relation.)

The engineers at Nodeable probably don’t need any tutorials in the mechanics of Hadoop.  After all, we’ve built our data analytics service using Hadoop.

But I think it’s useful for our customers to understand the process by which their complex systems data gets turned into meaningful, actionable information.   We do a fair amount of pre-processing of inbound data to “massage” it into formats that Hadoop can more easily digest, which improves efficiency, among other things.

But ultimately we depend upon Hadoop to work its magic on the data, pulling in terabytes of data to yield insights, as shown at right (beta).

This is just one example of the “data refinery” magic that Hadoop can provide.  It’s also being used to optimize oil drilling, determine which spammy ads to send you on Facebook, enable governments to track your every move through video surveillance, and more.  In other words, our data refinery business is much more benign (and useful to you) than that of some others.  Best stick with us.

Tagged , , ,

Is curation the future of Big Data?

I hate the word “curation.”  Or, rather, I hate that it’s one of the most overused, overhyped words in Silicon Valley these days.  Or maybe it was yesterday, before “actionable insight” became the term du jour.  But curation may be about to make a comeback.

Why?  Because it turns out that it’s really hard for machines to pull “operational insights” out of big piles of data.  This is why big enterprises are scrambling to hire data scientists, and are increasingly discovering there’s far more demand than supply.

In short, we can, as Forrester does, trumpet operational insight or actionable insight as top priorities, but achieving them is easier said than done:

Which is why I found a lunchtime conversation with my neighbor and friend, Chuck Sharp, interesting.  Chuck is the founder and CEO of Right Intel, a marketing data analytics company.  This is Chuck’s second analytics company.  His first, Sharp Analytics, was acquired by iCrossing in 2007.  As Chuck told me, one thing that he learned from his first attempt was that a dashboard-based approach to analytics doesn’t work.  People don’t reliably log into a dashboard service and, even when they do, they often struggle to understand what they should do with the data being presented.

Enter curation.

Right Intel provides a platform that makes it easy to amass and amalgamate different data sources (e.g., Twitter feeds, charts and graphs found in blog posts, etc.), but that’s only half the story.  Right Intel then connects with partners who in turn service big brands (e.g., Marriott Hotels).  Those partners (or a designee within the end user/customer) then siphon through the pool of data to determine which highlights to pass on, and which actions to recommend based on the data.

It’s a pretty light-handed way to intervene to make sense of Big Data, and it might become much more common than we’d like.  As much as we’d like to assume all data can simply be crunched by machines to spit out insights, the reality is that human intelligence and intuition are going to remain relevant, and probably dispositive to getting great insight from data.

In the systems intelligence world, things are a bit easier, as a company like Nodeable, for example, can poll the APIs for AWS or GitHub and pull out somewhat structured data, and interpret that data without human intervention.  But machine data is the exception to the rule, and even here, someone needs to be looking at the output and determining the best course of action based on the data (though we do make suggestions).

Which, I suppose, is a long way of saying: it’s not too late to go back to school and get a degree in data science.  People are going to be important to Big Data for a long time.  Probably forever.

Tagged , ,

Amazon’s essential role in delivering scale and tackling Hadoop

The cloud has many virtues, but perhaps its biggest is scale.  Scale refers to the ability to throttle resources up or down to meet inbound demand on web applications or other infrastructure.  It’s a problem that most developers, whether at startups or Fortune 500 behemoths, can only dream of having.  But for the ill-prepared, scaling an application can be a nightmare.

Which is why Amazon Web Services have become such essential infrastructure for startups and enterprises alike.  As Ryan Park, operations and infrastructure leader at Pinterest, declared earlier this month at AWS Summit:

Imagine if we were running out own data centre, and we had to go through a process of capacity planning, and ordering hardware, and racking that hardware, and so on.  It just would not have been possible scale fast enough – especially with such a small team. Until about a month ago, I was the only operations engineer at the whole company.

Think about that for a minute.  Here’s a web service with nearly 18 million visitors in February alone, which took the company just nine months to reach.  In a pre-AWS world, Pinterest would have employed scores of operations engineers to buy and manage hardware and the software to stitch it all together.  No more.

Of course, managing the infrastructure is only part of the equation.  Perhaps even more important is managing all the data that today’s businesses increasingly collect.  The lingua franca of this Big Data movement is Hadoop, which enables companies to crunch through massive piles of data to find actionable insights into how better to run one’s business.  Hadoop’s importance in our data-hungry world is perhaps best articulated by Cloudera CEO (and Nodeable board member) Mike Olson:

In the old days if you had a data problem you would write a big check for a massive piece of hardware and with any money left over you would by some very expensive but powerful software. That box with software and data became your data temple and your analysis and conclusions were done there.

There are problems with that approach. Data are now growing so fast. It is now impossible for one box to have all your data. You must have your data across multiple servers and use software that can coordinate and operate across all those servers. Hadoop is the platform designed to do this. It is designed to solve the problems of today, not the problems of yesterday.

Critically, Hadoop, too, is increasingly a matter of the cloud and, particularly, of AWS.  By some estimates Hadoop jobs comprise the majority of all AWS processing.  With petabyte-scale data clusters increasingly common, shifting that burden of storage and processing to the cloud becomes essential.

As more infrastructure and data processing moves to AWS, it becomes more and more important to analyze your AWS instances to track real-time trends (“Is my CPU running hot?”), make comparisons (“We’re running memory 25 percent lower than most companies – should we look to optimize?”), discover anomalies, and so on.  That’s where Nodeable comes in.

It used to be that the cloud is where enterprises dumped non-critical applications.  Now the inverse is true.  The cloud is the hub for mission-critical data processing.  It’s where enterprises are running applications that need serious scale.  And it just so happens to be where Nodeable shines.  Nodeable surfaces actionable insights into an easy-to-grok, Twitter-like activity stream.  Search tools like Splunk are nice, but Nodeable prefers to reveal those insights while you sip your tea or watch your daughter’s soccer game.

Why not sign up for our beta and give it a try?

Tagged , , , , ,
Follow

Get every new post delivered to your Inbox.

Join 52 other followers

%d bloggers like this: