New ways emerge to head off datacenter problems while improving IT operational performance

Listen to the podcast. Or read a full transcript. Sponsor: Integrien.

Complexity in today's IT systems makes previous error prevention approaches for operators inefficient and costly. IT staffs are expensive to retain, and are increasingly hard to find. There is also insufficient information about what’s going on in the context of an entire systems setup.

Operators are using manual processes -- in reactive firefighting mode -- to maintain critical service levels. It simply takes too long to interpret and resolve IT failures and glitches. We now see 70-plus-percent of the IT operations budget spent on labor costs.

IT executives are therefore seeking more automated approaches to not only remediate problems, but also to get earlier detection. These same operators don’t want to replace their system’s management investments, they want to better use them in a cohesive manner to learn more from them, and to better extract the information that these systems emit.

To help better understand the new solutions and approaches to detection and remediation of IT operations issues, I recently chatted with Steve Henning, the Vice President of Products for Integrien, in a sponsored BriefingsDirect podcast.

Here are some excerpts:

IT operations is being told to either keep their budgets static or to reduce them. Traditionally, the way that the vice president of IT operations has been able to keep problems from occurring in these environments has been by throwing more people at it.

This is just not scalable. There is no way ... (to) possibly hire the people to support that. Even with the budget, he couldn’t find the people today.

If you look at most IT environments today, the IT people will tell you that three or four minutes before a problem occurs, they will start to understand that little pattern of events that lead to the problem.

But most of the people that I speak to tell me that’s too late. By the time they identify the pattern that repeats and leads to a particular problem -- for example, a slowdown of a particular critical transaction -- it’s too late. Either the system goes down or the slowdown is such that they are losing business.

Service oriented architecture (SOA) and virtualization increase the management problem by at least a factor of three. So you can see that this is a more complex and challenging environment to manage.

So it’s a very troubling environment these days. It’s really what’s pushing people toward looking at different approaches, of taking more of a probabilistic look, measuring variables, and looking at probable outcomes -- rather than trying to do things in a deterministic way, measuring every possible variable, looking at it as quickly as possible, and hoping that problems just don’t slip by.

If you look at the applications that are being delivered today, monitoring everything from a silo standpoint and hoping to be able to solve problems in that environment is absolutely impossible. There has to be some way for all of the data to be analyzed in a holistic fashion, understanding the normal behaviors of each of the metrics that are being collected by these monitoring systems. Once you have that normal behavior, you’re alerting only to abnormal behaviors that are the real precursors to problems.

One of the alternatives is separating the wheat from the chaff and learning the normal behavior of the system. If you look at Integrien Alive, we use sophisticated, dynamic thresholding algorithms. We have multiple algorithms looking at the data to determine that normal behavior and then alerting only to abnormal precursors of problems.

Once you've learned the normal behavior of the system, these abnormal behaviors far downstream of where the problem actually occurs are the earliest precursors to these problems. We can pick up that these problems are going to occur, sometimes an hour before the problem actually happens.

The ability to get predictive alerts ... that’s kind of the nirvana of IT operations. Once you’ve captured models of the recurring problems in the IT environment, a product like Integrien Alive can see the incoming stream of real-time data and compare that against the models in the library.

If it sees a match with a high enough probability it can let you know ahead of time, up to an hour ahead of time, that you are going to have a particular problem that has previously occurred. You can also record exactly what you did to solve the problem, and how you have diagnosed it, so that you can solve it.

We're actually enhancing the expertise of these folks. You're always going to need experts in there. You’re always going to need the folks who have the tribal knowledge of the application. What we are doing, though, is enabling them to do their job better with earlier understanding of where the problems are occurring by adding and solving this massive data correlation issue when a problem occurs.

Listen to the podcast. Or read a full transcript. Sponsor: Integrien.

Microsoft-Yahoo! combination could yield an Orwellian Web world

About 10 years ago, we used to ask Jim Barksdale, then head of Netscape, a stock question during news conferences. Did you bag any "default browser" deals lately? Inevitably Jim would demur and say they were still trying.

Those were the days when the light was swiftly fading from the Netscape's browser's beacon, and Microsoft's newcomer (and inferior) browser, Internet Explorer, was bagging the default status for PC distributors and online services. That was enough to cement Microsoft's dominance of the Web browser market globally in a few short years.

The fact is, in an online world, convenience is the killer application. For most folks starting up their PCs, whatever comes up on the screen first and easiest is what they tend to use. That's why we have craplets, it's why Netscape bit the dust, and it's why Microsoft's unsolicited bid to buy Yahoo! is Redmond's last grasp at their old worldwide Web dominion strategy. The only way for Microsoft to hold onto its PC monopoly is to gain a Web monopoly too.

And Microsoft would have a good shot at cementing those two as a monopoly with the acquisition of Yahoo!. Because for the vast majority of people who simply do no more than fire up their PCs, click to start their browsers, open a Microsoft Word document, an Outlook calendar entry, an online email or instant message -- they will entering (mostly unbeknownst to themselves) a new default Web services environment.

With both Yahoo!'s and Microsoft's directories of users integrated, the miracle of single sign-on makes them in the probable near future part of the Microsoft advertising network, the Microsoft ID management complex, the Microsoft "software plus services" environment -- all by default, all quite convenient. And once you're in as a user, and once the oxygen is cut off to the competition, the world begins to look a lot more like Windows Everywhere over time.

Indeed, the proposed Microsoft takeover of Yahoo! is really the continuation of the failed (but strategically imperative) Hailstorm initiative. You might recall how Microsoft wanted to use single sign-on to link any users of Hotmail, or Instant Messanger, or Microsoft's myriad Web portals and services (MSN) to all be onramps to the same federated ID management overlay to reach all kinds of services. It was the roach motel attempt to corner the burgeoning network -- use Internet protocols, sure, but create a separate virtual Web of, by, and for Microsoft. The initiative caused quite a donnybrook because it seemed to limited users' ability to freely navigate among other Internet services -- at least on a convenient basis (and for a price).

So Microsoft's first stab at total Web dominance worked at the level of gaining the default browser, but failed at the larger enterprise. Microsoft thought it was only a matter of time, however. And it planned prematurely to begin pulling users back from the Web into the Microsoft world of single sign-on access to Microsoft services -- from travel, to city directories, to maps, to search. Microsoft incorrectly thought that the peril of Web as a Windows-less platform had been neutralized, it's competitors' oxygen cut off. Microsoft began to leverage its own Web services and monopoly desktop status to try and keep users on its sites, using its Web server, and its Web browser and its content offerings -- making for the Microsoft Wide Web, while the real Web withered away for use by scientists (again).

But several unexpected things happened to thwart this march into a Big Brother utopia -- a place where users began and ended their digital days (as workers and consumers) within the Microsoft environment. Linux and Apache Web Server stunted the penetration of the security risk Internet Information Services (nee Server) (IIS). AOL created a bigger online home-based community. Mozilla became a fine and dandy Web browser alternative (albeit not the default choice). Java became a dominant language for distributed computing, and an accepted runtime environment standard.

Dial-up gave way to broadband for both homes and businesses. The digital gusher was provided by several sources (many of which were hostile to to Microsoft and its minions). Software as a service (Saas) became viable and Saleforce.com succeeded. And, most importantly, Google emerged as the dominant search engine and created the new economics of the Web -- search-based juxtaposed automated link ads.

Microsoft had tried to gain the Web's revenues via dominance of the platform, rather than via the compelling relationship of convenience of access to all the relevant information. Microsoft wanted Windows 2.0 instead of Web 2.0.

Social networks like MySpace LinkedIn, and Facebook replaced AOL as the communities of choice. And resurgent IBM and Apple were containing Microsoft at the edges, and even turning their hegemony back meaningfully. Mobile networks were how many of the world's newest Internet users access content and services, sans a Microsoft client.

And so only a mere three years ago, Microsoft's plans for total dominance were dashed, even though they seemingly had it all. Just like Tom Brady, they just couldn't hold on to cement the sweep, and their perfect season ended before the season itself was over. What Microsoft could not control was the Internet, thirst for unfettered knowledge, and the set of open standards -- TCP/IP & HTML -- that sidesteps Windows.

Yet at every step of the way Microsoft tried to buy, bully, create or destroy in order to control the onramps, applications, developers, content, media, and convenience of the Web - even if the genie was out of the bottle. They did their own dial-up networks, they had proxies buy up cable franchises, they tried to dominate mobile software. They created television channels, and publishing divisions, and business applications. They largely failed against an open market in everything but their original successes: PC platform, productivity apps, tools, and closed runtime.

And so the bid for Yahoo! both underscores that failure as well as demonstrates the desperate last attempt to dominate more than their desktop software monopoly. This is a make or break event for Microsoft, and has huge ramifications for the futures of several critical industries.

If Redmond succeeds with acquiring Yahoo!, imagine a world that was already once feared, back some 10 years ago. That is an Orwellian world in which a huge majority of all users of the Internet globally can -- wittingly or otherwise -- only gain their emails, their word processing, their news, their services, their spreadsheets, their data, their workflow -- all that which they do online essentially -- only by passing through the Microsoft complex and paying their tolls along the way.

All those who wish to reach that mass audience, be it on a long tail or conventional mass markets basis, must use what de facto standards Microsoft has anointed. They must buy the correct proprietary servers and infrastructure, they must develop on the proscribed frameworks. They must view the world through Microsoft Windows, at significant recurring cost. Would Microsoft's historic economic behavior translate well to such control over knowledge, experience, and personal choice?

This may sound shrill, but a dominant federated ID management function is the real killer application of convenience that is at stake today. Google knows it, and quietly and mostly responsibly linked many of its services to a single sign-on ID cloud. When you get a gmail account, it becomes your passport to many Google services, and it contains much about your online definition, as well as aids and abets the ability to power the automated advertising juggernaut that Microsoft rightly fears. But at least Google (so far) lets the content and media develop based on the open market. They don't exact a mandatory toll as much as take a portion of valued voluntary transactions, and they remain in support of open standards and choice of platforms.

We now may face a choice between a "do no evil" philosophy of seemingly much choice, or an extend-the-monopoly approach that has tended to limit choice. The Microsoft monopoly has already needed to be reigned in by global regulators who fear a blind ambition powerhouse, or who fear unmitigated control over major aspects of digital existence. Orwell didn't know how political power would be balanced or controlled in his future vision, perched as he was at the unfortunate mid-20th century.

How the power of the Internet is balanced is what now is at stake with the Microsoft-Yahoo! bid. Who can you trust with such power?

Microsoft's Yahoo bids speaks as much of failure as opportunity

Is Microsoft buying Yahoo! because it has succeeded in its own Windows Everywhere strategy and 12 years of lackluster performance on the Web?

Is Microsoft trying to buy Yahoo! because Yahoo! is seemingly at a weak point, unable to dominate in the key areas of search, advertising, and media?

Nope, Microsoft is trying to buy Yahoo! because neither Microsoft nor Yahoo! is succeeding on the Web in the ways that they should. And it's not just Google that has an edge: Consider Apple, eBay, Salesforce.com, Facebook, MySpace, Disney.

And how much sense does putting Microsoft and Yahoo! together make now? Not as much as it did two years ago when Yahoo! was stronger and Google was weaker. We should also thrown in that Apple and Amazon are also much stronger now than at any time in the past. The media conglomerates are starting to figure things out.

So once again, we have Microsoft throwing outrageous amounts of money late at what should have been an obvious merger for them a long time ago. I recall is discussion on the Gillmor Gang podcast at least two years ago that wondered when -- not if -- Microsoft would buy Yahoo! Most of those on the call, including me said it was the only outcome for Yahoo! and the only way for Microsoft to blunt Google.

But that was then, and this is now. So the burning question today is not whether a Microsoft-Yahoo! mashup makes sense -- it has made sense for years. The question is whether it makes sense now, at this outlandish price, and if this in fact marks the point where Microsoft makes a desperate and devastating mistake.

Is the Yahoo! cloud built on Windows? Nope. So the model of Windows Everywhere is junked. Accessing Yahoo! services only requires a browser -- so much for the "software plus services." Will the burgeoning Microsoft cloud and the aging Yahoo clouds work well together? Will one be able to absorb the other. I say no to both. These will be separate and ill-fitting infrastructures. Will the Redmond and Silicon Valley cultures work well, or will huge layoffs in California portend even more gridlock in the eastern Seattle suburbs?

What might be even worse -- Microsoft make try and require all the Yahoo! users to get better service via their clients. Would they be deluded enough to try and tie Microsoft client-side software to Yahoo! web services? Watch the flood to Google, if they do. Watch for Google to scream about monopoly abuses if they do. [Good thing the new mega mother of all hairballs will be under anti-trust review for a bit longer, eh?]

Does this mean what Microsoft was wrong about open source too? Because Yahoo! has built its infrastructure on a lot of open source code, including its cloud infrastructure keystone Hadoop. So Microsoft will own one of the world's most massive open source distributed datacenters. As an enterprise, should you choose a Windows platform -- or Microsoft's new choice to win on the web -- open source?

Right, so for the need to win in search, media and adverting, Microsoft is now selling its Windows Everywhere soul. They have been handing you an expensive line of proprietary crap for years, and by buying Yahoo! and its totally different approach to Web infrastructure -- they admit it.

What's more, will the world like getting their news from Microsoft? As a user, which search engine will I get when I log in to Yahoo! or MS Live? Which email will I get when I log in? Can he Yahoo! directory merge with the Live, nee Hotmail, directory? Which company will be the one I think of as the "brand"?

This spells a significant period of confusion. And that's for consumers, IT buyers, enterprise CIOs, and advertisers as well.

And for the enterprises that have invested their fates in Microsoft infrastructure, how will they get their Web services? Will it be Yahoo! for the consumers, and Microsoft Live or the business folk? Or vice versa? Both, a mish-mash? Yikes!

What's more, the Microsoft-Yahoo! amalgamation will become the enemy of the media companies worldwide. There was a certain détente between Microsoft alone and Yahoo! alone and the media world. No more. And Google could position itself as the happy medium (pun intended).

This proposed deal smacks of desperation, not multiplication of growth opportunities. But the price premium probably makes it inevitable. The only way to make this work is for Microsoft to spread itself more thickly as a media, advertising, technology, services, platform, tool -- everything to everybody. The risk is to be less and less of anything to anybody.

Microsoft is perhaps perceiving itself as pouncing on Yahoo!, given its current disarray. There's the weird board action, and the layoffs, and the performance issues. But this is weakness buying weakness, with a large period of confusion, dilution of value and brands, and risky alignments of cultures and technology.

And this from Microsoft - the hithertofore conservative acquirer that doesn't go for the big, blow-out acquisitions. Well, this is the big blow-out media merger of the year. Seems that going in the other direction, of splitting Microsoft up into logical sections that can operate and compete on their own, is out. For some time, no doubt.

The biggest risk is that if this ends up the mess it appears, that it just may end up just driving more consumers, advertisers, and businesses into the waiting arms of the singularly understood and focused Google, Apple, and IBM. It could well backfire.

And I for one will miss both Yahoo! and Microsoft because whatever they cobble together from the two won't be able to do the same that either did separately. It will be hard to define just what it is ... I think I'll call it Amalgamated Digital. It certainly isn't "micro," and it's not "soft." Any yahoo can see that.

WSO2 targets 'Social Enterprise' with combined JavaScript/Web services Mashup Server

WSO2, an open-source SOA provider, has combined JavaScript programming and Web services with the launch of its Mashup Server 1.0.

This open-source offering, which can be downloaded without subscription fees, will allow enterprises to consume, aggregate, and publish information in a variety of forms and from a variety of sources.

At the same time, WSO2, based in Colombo, Sri Lanka, and Mountain View, Calif., has announced the beta release of Mooshup.com, a hosted online version of the Mashup Server, which provides a community site for developing, running, and sharing mashups. [Disclosure: WSO2 has been a sponsor of BriefingsDirect podcasts.]

Each new service in the mashup comes with metadata that is designed to simplify consumption by other mashups and Web services clients, as well as artifacts that simplify construction of user interfaces (UIs) in browsers, rich applications, and other environments. Because it supports the separation of content and presentation, Mashup Server enables recursive mashups, meaning one mashup can be consumer by another. It also broadens the user interface beyond HTML to RSS and Atom feeds, email, and instant messaging.

The use of JavaScript leverages the broad base of developers who use the broad-based language, and mashups can be authored directly within the administrative UI, with a simple text editor, or with any popular integrated development environment (IDE).

The beta version of Mashup Server has already gotten good notices. Ohloh.net estimates that it would have cost an enterprise $571,736 to write this project from scratch, figuring nearly 45,000 lines of code and 10 person-years.

Ganesh Prasad, who blogs at The Wisdom of Ganesh, has a lot of good things to say, based on the beta release:

So is the WSO2 Mashup Server the one that will bring balance to the Force? A powerful programming language. Laughably easy XML manipulation. Simple access to SOAP services and REST resources. Transparent publication of itself as a service or resource in turn. Isn't this the holy grail of service composition?

WSO2 Mashup Server seems to be the industry's best-kept secret for now.

The Mashup Server is built on the WSO2 Web Services Application Server, based on Apache /Axis2, and WSO2’s built-in registry. Key features include:

• The ability to author and deploy mashups using notepad and a Mashup Server virtual directory.
• Auto-generation of Web service and UI artifacts, such as WSDL, REST URLs, JavaScript stubs.
• Try-It feature to help developers invoke and debug mashups or start developing their own rich HTML clients.
• Web 2.0-style console, powered by the WSO2 Registry, which natively supports different users, and allows tags, comments, and ratings and a powerful search capability.

The Mashup Server is available for download. Mooshup.com membership is free, contingent on email verification.

Progress Software adds cross-process visibility with Actional 7.1

Progress Software has beefed up its Actional SOA management offerings with the release today of Progress Actional 7.1, which provides unified visibility into business processes, and connects those business processes to the underlying SOA infrastructure.

Key features of the latest release include an automatic discovery feature that keeps information accurate, allowing users to compare how processes change from day to day. User can also set thresholds for alters about behavior and performance, and policy enforcement will automatically adjust when services or processes change.

Progress, Bedford, Mass., added the Actional product line to its SOA arsenal just a little over two years ago with the acquisition of Actional Corporation in a $32-million deal.

Progress said that Actional 7.1 will integrate with Lombardi TeamWorks, and the company plans to provide native support for other business-process management (BPM) solutions, including offerings from Software AG and Fujitsu. Actional also includes a software development kit (SDK) that allows third parties to add support for other BPM and SOA infrastructure products.

The new version also includes support for non-XML payload data, which is designed to allow users to inspect and analyze message content in such existing services as Remote Method Invocation (RMI) and Enterprise JavaBeanT (EJB).

Last July, I had a lengthy podcast discussion about Software as a Service (SaaS) with Colleen Smith, managing director of Saas for Progress. You can listen to the podcast here.

For more information on the latest offering, see the Actional Web site.