I recently had the pleasure of participating in a live discussion (now a podcast) at the Harvard Club of Boston with Jason Bloomberg, managing partner at analyst firm ZapThink, on the current state and future outlook for governance and management in SOA environments.
Moderating the discussion was Martin Milani, chief technology officer at Tidal Software, which sponsored the luncheon event. We had dozens of enterprise IT executives from the Boston area in attendance, and they joined us in a questions and answer session after the presentations.
During the hour-long "debate" Jason and I explored how IT management will evolve in the world of service-based applications. The discussion delved into issues of new standards, how SOA demands that performance management and change management should augment and elevate the role of systems management, and on how the integrity of services delivery requires a deep and wide approach to "management in total" across a service's lifecycle.
A recent survey supports some of our conclusions: That more needs to be done to provide governance and management, and that the very definition of "management" requires re-evaluation in the era of SOA.
Here are some excerpts from the event:
The key thing to keep in mind about SOA in this context is that services are an abstraction. That is, they help to provide flexibility to the business, but they don’t actually simplify the underlying technology. Many architects are a bit surprised by this, that SOA doesn't make their jobs easier or make the job of IT any easier. If anything, it’s more complex.Listen to the podcast. Read a full transcript of the discussion. Sponsor: Tidal Software.
There's more of a challenge for IT to meet the business requirements for flexible, agile, composable, and loosely coupled services. As a result, you have this need for the IT organization to rise to the challenge of services. This is especially true in the management area because the services essentially have to behave as advertised.
The implications for those dealing with applications is that you are going to service-enable those applications, decouple, and decompose them into essential core services, and then repurpose them by cross-compositing processes. What is that going to do to you, if you think you are going to go to firefighting mode when you have performance issues? It’s simply not going to work.
You need to rethink management and support, and you need to try to get proactive in how systems will be supported to head off performance issues and create insurance policies against blackouts, brownouts or other snafus. SOA is really a catalyst toward a different approach to the management and support of the services.
[SOA management] needs to be looked at not just as management of discrete parts, not just trees within the forest that each stand on their own -- but the forest itself. I'd like to see that get to the point where it becomes something that can be assimilated further than just the systems -- with the business objectives as well.
... You are going to want to tune how your applications and services are delivered, perhaps to live up to service level agreements, or perhaps so that you can give priority to certain data, application, or services streams over others. ... That’s going to require a different level of management. It’s really about leveraging the old, finding ways to assimilate and then put a more operator- or policy-driven -- perhaps even automated -- approach on top of it.
With SOA you need to gather information about your systems both deeply and broadly: deep and wide. You can already get a fire-hose of data from your systems, log files, and agent- and agentless-based approaches already on the market. You get a ton of data. It’s working with that data in the context of a horizontal business process that’s the hard part. ... If one aspect of a process goes down, that’s the weak link in that chain, the whole chain could be at disadvantage.
In the past, you might have one application down, but people could go off and do another task, because that mainframe would be back up at two o’clock. If your entire supply chain is disabled for a period of time, that’s a higher price to be paid. So, we're looking at a different level, and I don’t think we’ve seen the solution yet.
The value of IT here can be much greater. It can be an enabler, not a cost center. It can be the way in which not only is information relayed about what’s going on, but can determine what we want to happen. We want to change that supply chain. We want to change that distribution, recall these products, get a list of every single product and every serial number, and we want to relay that to our sales force.
That sounds straightforward, but if you try to do that with a lot of IT systems today, you’re going to find yourself up there with the equivalent of mimeograph and crayons, doing it by hand -- and that’s just not acceptable. So in the future, a company’s very existence could be at stake if they don’t have agility in these processes.
SOA is really not optional. Companies that don’t get this right will suffer the consequences. They will suffer lawsuits and suffer a competitive disadvantage. They are going to go out of business. This is an important thing to keep in mind. IT is not playing around here. You can't say, "Maybe we will do SOA, if we can figure it out, or maybe we won’t. We’ll just do things the old way, where we are siloed and we keep on going."
Instead of just running around and not being able to use technology, we want to have a governance plan in place, saying “This is how we’re dealing with problems. Here is how the technology will rise to these challenges." And, we make it a matter of policy. So now, instead of just having to wing it when you have some sort of issue, there is an infrastructure in place for helping you deal with issues as they come about.
A key to this is the management challenge. As management technology improves, it is less and less about just monitoring stuff, and more and more about being able to deal with issues as a matter of policy, where your policy is in place for dealing with problems that you can’t predict -- and those are the most challenging ones. That’s what we see happening over time.
What’s happening in the management standards world is a pissing match between the big vendors. You have the Java guys wanting to this and then Microsoft guys wanting to do that, and nobody listens to them, because they can't agree with each other. So, they'll realize, "Hey, all of our customers are ignoring us. We'd better get our act together." It’s become this big political thing that’s just slowing whole thing down.
From the enterprise perspective, you don’t have to wait around for the vendors to grow up. You can get stuff done today. This isn’t going to stop you from being successful with SOA initiatives today. It might mean that two products you buy off the shelf might not interoperate as well you like. That has to be part of your plan. It might mean you have to come up with your own internal standards for the time being.
As companies move closer to SOA, it forces them to grow up. It forces them to think across boundaries. In the past, complexity has forced companies to divvy up issues into small compartments, put a box around them, and assign people to that item of complexity.
But that has stifled the ability of interoperability and of addressing things holistically, of being fleet and agile. It’s made them brittle, has made them slower, and has made things expensive. SOA forces companies to start binding what happens in pre-production to post-production, what happens in an application with what happens in an infrastructure, and what happens on a service level from an outside provider to what happens as a shared service internally.
There are great risks if you try to do SOA without growing up, but there are super opportunities if it’s done properly. It can elevate IT as a function within the organization from being an inhibitor to the absolute enablement for new business and growth and opportunity.
