Tuesday, September 25, 2007

Integration infrastructure approaches adjust to new world of SaaS and shared services

Read a full transcript of the discussion. Listen to the podcast. Sponsor: Cape Clear Software.

Change is afoot for the role and requirements of integration for modern software-as-a-service (SaaS) providers and enterprises adopting shared services models. Reuse is becoming an important issue, as are patterns of automation.

The notion of reuse of integration -- with added emphasis on integration as a service -- has prompted a different approach to integration infrastructure. The new demand is driven by ecologies of services, some from the Web "cloud," as well as the need to efficiently scale the delivery of services and applications composed of many disparate component services.

Integrations require reusable patterns, high performance, as well as many different means of access from clients. As a result Cape Clear Software has this week unveiled a new major version of its enterprise service bus (ESB), Cape Clear 7.5, with an emphasis on:
  • A new graphical editor, the SOA Assembly Editor, an Eclipse-based tool to graphically clip together elements of integrations.
  • Multi-tenanting additions to the ESB that allow segmentation of integrations, data, and reporting, as well as segmenting use and reuse of integrations on reporting and management of integrations based on the identities of inbound customers, clients, or businesses.
  • A Business Process Execution Language (BPEL) management system with tools to monitor transactions, and repair transactions when they fail, to allow for rebuilding previous business information and ensure transactional integrity in running and maintaining large enterprise-class BPEL deployments.
To help better understand the new landscape for integration models, I recently moderated a sponsored podcast discussion with Phil Wainewright, an independent consultant, director of Procullux Ventures, and fellow ZDNet SaaS blogger, as well as Annrai O’Toole, CEO of Cape Clear Software.

Here are some excerpts:
... We're getting more sophisticated about SaaS, because it's being taken on board in a whole range of areas within the enterprise, and people want to do integration.

There are two forms of integration coming to the fore. The first is where data needs to be exchanged with legacy applications within the enterprise. The second form of integration that we see -- not at the moment, but it’s increasingly going to be an issue -- is where people want to integrate between different services coming in from the cloud. It’s a topic that’s familiar when we talk about mashups, fairly simple integrations of services that are done at the browser level. In the enterprise space, people tend to talk about composite applications, and it seems to be more difficult when you are dealing with a range of data sources that have to be combined.

People have realized that if you're doing integration to each separate service that's out there, then you're creating the same point-to-point spaghetti that people were trying to get away from by moving to this new IT paradigm. People are starting to think that there's a better way of doing this. If there's a better way of delivering the software, then there ought to be a better way of integrating it together as well.

Therefore, they realize that if we share the integration, rather than building it from scratch each time, we can bring into the integration field some of the benefits that we see with the shared-services architecture or SaaS. ... The new generation of SaaS providers, are really talking about a shared infrastructure, where the application is configured and tailored to the needs of individual customers. In a way, they’re segmented off from the way the infrastructure works underneath.

When you build an integration, you always end up having to customize it in some way for different customers. Customers will have different data formats. They’ll want to access it slightly differently. Some people will want to talk to it over SOAP. Some won't, and they’ll want to use something like REST. Or they might be going backwards and are only able to send it FTP drops, or something like that.

Multi-tenanting is one solution to the problem. The other is what we call multi-channel, which is the ability to have an integration, and make it available with different security policies, different transports, and different transformations going in and out.

A combination of multi-tenanting and multi-channeling allows you to build integrations once, make them accessible to different users, and make them accessible in different ways for each of those different customers. It gives you the scalability and reuse you need to make this model viable.

One point worth bearing in mind here is that this problem is going to get solved, because the economic reality of it suggests that we must solve this. One, the payoff for getting it right is huge. Second, the whole model of SaaS won’t be successful, unless we skin the integration problem. We don’t want the world to be limited to just having Salesforce.com with its siloed application.

We want SaaS to be the generic solution for everybody. That’s the way the industry is going, and that can only happen by solving this problem. So, we’re having a good stab at it, and I'll just briefly address some of the things that I think enable us to do it now, as opposed to in the past. First, there is a standardization that’s taken place. A set of standards has been created around SOA, giving us the interoperability platform that makes it possible in a way that was never possible before. Second is an acceptance of this shared-services, hosted model.

Years ago, people would have laughed at you and said, "I’m going to trust all my customer data to a provider in the cloud?" But, they’re doing it happily because of the economics of it. The whole trend toward trusting people with outsourced offerings means that the people will be more likely to trust integrations out there, because a lot of the technology to do this has been around for quite some time.

In enterprises you’re seeing this big move to virtualization and shared services. They’re saying, "Why are we having development teams build integration in all these branch offices at all these locations around the world? It’s extremely wasteful. It's a lot of skill that we've got to push out, and there are a lot of things that go wrong with these. Can't we consolidate all of those into a centralized data center? We’ll host those integrations for those individual business units or those at departments, but we'll do it here. We’ve got all the expertise in one place."

Those guys are delighted, because at the individual local level they don’t maintain all the costs and all the complexity of dealing with all the issues. It’s hosted out in their internal cloud. We haven't seen enough data points on that, but this hosted integration model can work. We’ve got it working for pure entities in SaaS companies like Workday, and we’ve got it working for a number of large enterprises. There is enough evidence for us to believe that this is really going to be the way forward for everybody in the industry.
Read a full transcript of the discussion. Listen to the podcast. Sponsor: Cape Clear Software.