Talend delivers converged platform for better data services integration across multiple applications

Seeking to make it easier to synchronize data between applications and across enterprise boundaries, Talend on Monday delivered a unified platform for both data services and applications integration requirements.

The Talend ESB Standard Edition, built on an open source enterprise service bus (ESB), frees data services and data management processes from specific applications. By abstracting them as standards-based services that can be reused by other applications, the new Talend platform offers a common environment for users to manage an entire lifecycle of a data service, regardless of its origins.

Talend also announced the 4.2 version of its Data Integration, Data Quality and Master Data Management (MDM) solutions, which now work in combination with the new Talend ESB. [Disclosure: Talend is a sponsor of BriefingsDirect podcasts.]

Thanks to such trends as cloud, hybrid computing and massive data sets, the role and impact of integration has shifted. A more comprehensive and managed approach to integration is required -- one that spans data and applications services. Moreover, the tools that support enterprise integration need to useable by more types of workers, those that are involved at the business process and data analysis levels.

By seeking to reduce middleware complexity, Talend's combined offerings unify a platform with a common development, deployment and monitoring environment that spans both data management and application integration tasks and operations, said Pat Walsh, Vice President of Marketing for the Application Integration Division at Talend.

Many touch points

"There’s now the mandate that you can no longer isolate data from application, because the touch points are just so many. You now need to look at solutions that, from the get-go, consider both aspects of the integration problem -- the data aspect and the system and application integration aspect," said Walsh.

Talend ESB Standard Edition uses the Apache CXF services framework, Apache Camel integration framework and Apache ActiveMQ enterprise messaging capabilities. Talend's ESB Standard Edition also features Service Locator capabilities for automatic failover and load balancing through the Apache Zookeeper extension for dynamic endpoint registration and lookup. The Security Token Service (STS) framework supports SAML 2.0 (Security Assertion Markup Language 2.0), and Service Activity Monitoring fosters analysis of service activity.

"We've gone to great lengths to include security mechanisms into the solution," said Walsh, "so that we can have approaches whereby there are certain permissions for just individuals. Or, IT management can look at certain aspects while opening it up maybe to a broader audience, when it comes to development and use of the interfaces that are going to be developed on the data in application side."

Talend ESB Standard Edition source code is licensed under the Apache 2.0 License. A commercial edition is also available through the company.

You may also be interested in:

• Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing
• Apache CXF: Where it's Been and What the Future Holds for Web Services Frameworks
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function
• Eclipse Swordfish OSGi ESB enters fray for SOA market acceptance, Sopera to add support
  

Open Group cloud survey shows a lack of IT preparedness, especially on measuring cloud's true costs and benefits

While more than 90 percent of companies are using -- or plan to use -- cloud computing, the cloud model is raising some concerns, as well as some paradoxes.

Unsurprisingly, security is the leading concern, while integration and governance issues are close behind. The paradox arises because industry executives know that cloud computing will impact their business, but are not yet prepared to handle that impact and, in many cases, don't yet know how to measure it.

These are some of the results from a recent survey by The Open Group, which polled 307 IT professionals who had purchasing or decision-making influence over cloud computing. The study earlier this year found that cloud computing required C-level approval at 55 percent of the companies, while 22 percent left it to IT directors and 8 percent to enterprise architects. [Disclosure: The Open Group is a sponsor of BriefingsDirect podcasts.]

Among the take-aways from the results: "You need to take the metrics and find out what your IT finances are to make a true business decision about cloud," said Dr. Chris Harding, Forum Director for The Open Group Cloud Computing Work Group. "And you need to track that as your move forward."

Among other findings from the survey:

• An impressive 92 percent of respondents said they are either currently using some cloud (49 percent) or had it on their IT road map (43 percent). The remaining 8 percent said they had no cloud plans.
  
  
• Only 17 percent of those polled said they use or would use public cloud, while the remainder were divided between private (29 percent) and hybrid (45 percent). Another 9 percent were unsure what they would use.

  The paradox arises because industry executives know that cloud computing will impact their business, but are not yet prepared to handle that impact and, in many cases, don't know how to measure it.

  
  
  
• The main drivers behind cloud adoption were cost (21 percent), timeliness and agility (19 percent), and resource optimization (17 percent).
  
  
• Data security is the biggest concern for companies (18 percent), followed by integration issues (15 percent), and governance (14 percent).
  
  
• A majority of participants (55 percent) said that cloud return on investment (ROI) would be easy to justify. However, only 35 percent of respondents said they had mechanisms in place to effectively measure ROI.
  
  
• An overwhelming 82 percent said they expected cloud to significantly impact one or more business processes, but only 28 percent are actually prepared for these changes.
  
  
• When asked if they were satisfied with cloud education and training available, 51 percent percent said they were satisfied or very satisfied, 34 percent said they were somewhat satisfied, and 15 percent said they were dissatisfied.

This survey builds on the ongoing work being orchestrated by The Open Group Cloud Computing Work Group, and the results will help guide its future development on the financial and business impact of cloud computing.

The results indicate that these enterprise architects and planners are primarily focused on private and hybrid computing, and not so much on so-called public cloud options, said Harding. The respondents expect that cloud will impact their business processes, but are not sure how.

Enterprises should start right away with measurement of shared services use and costs, said Harding. That will allow for any move to cloud with assurances that risks and rewards can be managed. Only by learning the internal costs can the right decision be made as to how external services affect the balance, he said.

If you are interested in getting involved with The Open Group Cloud Computing Work Group, visit http://www.opengroup.org/cloudcomputing/. For Cloud Computing resources published by The Open Group, visit: http://www.opengroup.org/cloud/whitepapers/.

Due out in mid-2011, The Open Group currently writing a book tentatively titled Cloud Computing for Business: The Open Group Guide, which will demonstrate a model for measuring costs, and for how to begin managing governance around costs and service-level agreements, said Harding.

You may also be interested in:

• Open Group cloud panel forecasts cloud as spurring useful transition phase for enterprise architecture
• Good insights from The Open Group Conference -- and its unconference brethren
• Cyber security top of mind for enterprise architects at Open Group conference
• Cloud brokering: Building a cloud of clouds

Rapidly evolving IT trends make open source, agile application integration platforms more important than ever

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Register for CamelOne. Sponsor: FuseSource.

Enterprise integration requirements are rapidly shifting to accommodate such trends as cloud computing, mobile devices' explosion, and increased demand for extended enterprise business processes.

Application-to-application integration inside an enterprise's four walls is well understood, but very quickly the demands placed on integration are spanning multiple enterprises, multiple types of applications, and varieties of service providers. As a result, software as a service (SaaS) and cloud computing are joining with legacy systems to form new and varied hybrid models that require whole new sets of integration needs and challenges.

Once these newer breeds of integrations are set up, can the old, brittle management and upkeep of them suffice -- or will agility and rapid upgrades and innovations require new tools to make integration a lifecycle function with ongoing management and more automated governance?

In the latest BriefingsDirect enterprise IT discussion, the panel examines how open-source integration projects like Apache Camel and lightweight integration implementations and graphical tools are making developers and architects more agile. At the same time, these open-source approaches are proving less vulnerable to the complexity, fragility, and cost that often plague aging commercial middleware integration products. [Learn more about the CamelOne conference May 24 in Washington, DC.]

Dana Gardner, Principal Analyst at Interarbor Solutions, recently sat down with Rob Davies, Chief Technology Officer at FuseSource, and Debbie Moynihan, Vice President of Marketing at FuseSource, to examine the need for innovative, new, open and agile integration capabilities.

Here are some excerpts:

Gardner: The need for integration is increasing. The things that need to be integrated are increasing, rapidly. Open source is well established. When you put these factors together this perhaps spells an historic shift. Has the ability to integrate openly become an essential ingredient of businesses?

Davies: Sometimes, it's difficult to see things happening like that, if you’re actually right inside in the middle of it. We probably are at that shift right now.

We’ve talked about cloud environment. Also there’s social networking, SaaS, and mobile devices, and you need to link all those together. It's coming to the point where organizations won’t have a choice other than to use open source as a way to try to keep up with a pace of change.

We're probably at a point now where we’re going to see that the traditional model of providing software is going to dwindle over time, probably pretty rapidly, as organizations realize that they need the flexibility and the ability to change what they’re doing very quickly.

Future-proofing applications

You have to start thinking about how you're going to future-proof your applications right from the beginning to adapt to changes in their environments. You have to architect in how you’re going to integrate and future-proof your applications, because it does get more costly if you do it as an afterthought.

Gardner: Many of the SaaS providers are doing multitenancy and providing applications as services on demand at a very attractive and aggressive price point. They're leveraging open source on the back end, I have to imagine. Do you have any insight into what the service providers themselves are building with?

Davies: Most applications now -- in particular in the cloud -- are using open source at the back end. We can't give you any specific details of vendors that are doing that, but I know they're using open-source projects, and not just the SaaS vendors, but some of the other existing product vendors use open-source as well to enable their products.

We certainly see open-source as definitely mainstream now, and we’ve seen it has been the first choice that people use for building any kind of application or service they’re providing. It's more a case of people asking the questions now of not should we be using open source, but why shouldn’t we use open source? It's starting to become a first choice for people to go to.

Gardner: Debbie, why do people need to rethink integration?

Moynihan: The business models are changing and people are being asked to do more with less. Teams and applications are more distributed than ever.

There are a lot of new technologies coming out that people are struggling to learn, and figuring out how to incorporate them into their infrastructure: cloud, mobile, the explosion of the huge amounts of data that enterprises are trying to understand and make sense out of. Not to mention the social media technologies that people are being asked about and wondering how to incorporate into their enterprise infrastructure.

There are a lot of different skills that people are looking to have that they've never been asked to have before. More and more people are being asked to perform IT tasks. It isn’t just highly skilled developers, but also business analysts and people who have never done integration before are being asked to do integration activities.

They're not sure how to keep up with all of these changes. Costs are a problem because essentially everyone has the same or smaller budget going forward and a lot of people have fewer people to do what they've been doing before.

At FuseSource, we've seen a lot people looking more and more to open source to solve some of these problems ... . There's a lot of flexibility. When the environment changes and new technologies come out, you need to integrate new things into your environment.

The community people, when they see a problem or new technology, just make it happen. They can add, expand, and modify what's involved in the various open-source integration projects without the overhead and bureaucracy of some of the traditional software development environments.

Gardner: In the past, when we had a shift in computing, we'd bring in a new set of applications, we'd update our platforms, and then think about integrating them. It was a sequential process and it could take three to five years to go through something like that.

We don’t really have that luxury anymore. Now things are happening in a simultaneous fashion. So integration really can't be an afterthought, but needs to be part-and-parcel with how you go about designing and implementing your applications.

Doesn’t open source, in a sense, allow for a compression of the time that we’ve traditionally taken with commercial products?

Moynihan: Absolutely. Open-source is a componentized, lightweight approach. As people develop their applications, they develop them in such a way that they can be broken apart in new and different ways down the road, and it's very transparent. It makes it easier over time to further integrate what you’ve built and to make changes as you need.

Gardner: One of the other aspects of this that I'm seeing in the market is that more people need to take part in integration. It can't just go through a bottleneck of "beard-and-sneaker guys" in the back room who can do coding. Integration needs to be part of process innovation. That means we need to elevate it out to a wider group of individuals, maybe as many as possible that are on the front lines of process innovation and analysis.

The addition of tooling is going to help broaden how many people can do integration, and we're real excited.

What's being done about the integration that we've been describing to make it more, well ... applicable?

Moynihan: On April 11, we announced the general availability of a new graphical tooling for Apache Camel. [Users can download a trial version of the plug-in, which includes some of the functionality of the fully paid version found on the subscription-based Fuse Mediation Router.]

The addition of graphical tooling makes it easier for more people to do integration development. They don't have to write code. They can use a drag-and-drop environment to select the integration patterns that they want to implement, and the software will implement them. They can test them and deploy them into production as well.

The addition of tooling is going to help broaden how many people can do integration, and we're real excited. We've been doing a beta program since the end of January with over 500 participants. Rob mentioned the breadth of all the components and how hot Apache Camel has been. We're not surprised that more and more people want to use it. So, the idea of having tooling on top of it is really attractive to users.

Gardner: So, what's the name and where do you go to find out more about them?

Moynihan: The Fuse IDE for Camel is the name. It plugs into an Eclipse environment and you can get it at fusesource.com.

Gardner: You know it strikes me that when we begin to talk about integration that I’d mentioned service-oriented architecture (SOA), but that was sort of yesterday’s buzzword. We're now into cloud, hybrid, and mobile. But, from an architectural perspective, you can't really scale and leverage these open components without that proper underpinning, typically an enterprise-service-bus (ESB) architecture.

Rob, help me understand why doing this correctly from an architecture -- not just an open-source -- perspective is really important as well.

Davies: You hit the core things about the SOA and the ESB architectures. We see where people are using, in particular, Apache Camel and some of our other open-source projects. They want flexibility there. So, they want to leverage a service bus, put things on, expose them as service, and expose them over the service bus, which uses different transports to enable that bus, be that messaging, HTTP, or whatever other means you want to use.

Application integration

At the same time, you also want to have the flexibility now to do it in application integration. You want to have that flexibility for some services and you very much need that enterprise service bus in place. But for other cases, you want to be able to do that more locally, where the integration points are.

The approach that we have is that we enable you to do both, because you can embed Apache Camel inside an application server, if you want it inside your application itself. If you want to use it in a more traditional sense, you can deploy it into ServiceMix. You can define your apps easily, deploy them into ServiceMix, and use it to manage the container.

Having that flexibility as well means that you can have the right architecture for your particular solution. If you look at how people would do the integration before, they’d have to get an ESB, and that would force the whole architecture of how they do things. When you’ve got more flexibility, it means that you can make the right architecture choices that you need, and you're not constrained to one particular style of integration.

Gardner: I'm facing a lot of questions more recently about how to architecturally cross the domains that we've mentioned -- SaaS, cloud, on-premises, traditional architecture, and private cloud architecture.

Does the service-bus approach and the open-source approach also give us some sort of a path or vision for how to go about this?

You can only really get that speed of innovation to keep up with the way the environment is changing by choosing open source.

Davies: Having open source enables you to have the insight into how the integration application works.

If you just look back just a couple of years, when people were starting to use the cloud, they weren’t even thinking about having hybrid clouds. Now, we're seeing more and more people, more of our customers, looking to hybrid clouds and have a private cloud for applications.

When they need the capacity, obviously they can get that capacity in a public cloud. But, to have all those PCs working together seamlessly, they need the agility that you get from an integration solution that can be deployed on a public cloud, locally, or a combination of both. That’s something that you can only get from software that has evolved at the same pace as the demands of the environment.

You can only really get that speed of innovation to keep up with the way the environment is changing by choosing open source, because the open-source community itself is driving the projects to keep up with the demands.

So, you have to try to move outside of a traditional release cycle that you would get from a traditional product company. You don’t really have any other alternatives, if you want to keep up, than to look at open-source projects, the Apache ones in particular. [Learn more about the CamelOne conference May 24 in Washington, DC.]

Apache projects certainly hit the right notes in that you've got both very business-friendly license from the Apache license and very active communities, and you’ve got diversity in that community. You know these projects are going to live beyond the lifetime of particular individuals on the projects.

Support and consultancy

You also have the benefit of having companies like FuseSource, which created the projects in the first place, and who are there and able to provide support and consultancy if you need it. You get the best of having a dynamic community, a dynamic project, and you also get the security of having professional company to back it up.

Gardner: How rapidly are the iterations within the Apache project, within Camel in particular, happening? How rapidly is innovation taking place?

Very fast pace

Davies: It’s happening at a very fast pace. When we do release these out of Apache, it's typically every three months, but in that three-month period there could be other components that have gone into the Apache Camel Framework. Because it's open source, people can actually look about, release their own components into an open-source environment, or develop them separately without necessarily releasing to Apache, just to get the functionality out.

That pace of change is very fast and it’s near real-time. When the need comes up, within a few days or a week, you would probably find someone who has already written that integration component that you need and it’s available. ... If you’ve got an open-source framework, you can actually have an insight into how the project works.

After we launched Apache Camel at the Apache Software Foundation, we provided a number of default integration components for Camel. But, as soon as they got out there and the community started to use them and saw the benefits of using them, we saw no end of contributions. People contributed adapters to weird and wonderful systems, and contributed them right back into the Apache project. [Learn more about the CamelOne conference May 24 in Washington, DC.]

We know from our customers that they’ve got specific needs. They’ve got legacy applications. Because we've gone to the effort of making sure that it's very easy to add a new component into Apache Camel, it's very straightforward for someone to add in extra functionality.

For example, if you want to write a component for legacy mainframe application, you could very easily do in a matter of hours. The old approach would take you weeks, months, maybe even years, especially if you don’t have access to the source code. So, you’ve got that added flexibility.

The fact that it's an open-source project at Apache means you can get feedback instantly, if you’ve got issues and problems. Of course, if you want professional help, there’s FuseSource as well. We have our own community at fusesource.com. So, all these things combined means that you have more flexibility and a much more agile way of doing integration.

Gardner: What's happening now in the community? I understand you have a conference that’s coming up May 24, a first of its kind. Why is this a good time to be pulling together the Camel Community?

The nice thing about Camel is that it provides a basic foundation and a terminology of well-defined patterns.

Moynihan: We’re really excited. We have an event coming up in May called CamelOne, and the reason why we focused on Camel with the name of the event is that it’s actually for open-source integration and messaging overall. It’s because Camel is a really great way for people to get started, and it brings together the entire community.

Camel is a great foundation and CamelOne is an event to bring together users of Camel and other open-source integration and messaging technologies to learn more about Camel, open-source messaging like ActiveMQ, and ESBs like Apache ServiceMix.

Camel provides a basic foundation and a terminology of well-defined patterns. The integration patterns themselves are very well-defined, but what's happening is all the different ways in which you connect and what you are connecting to have been changing and evolving over time.

Other people are going to be doing more in-depth management of many integration patterns and they may need to know all the nuances of an ESB platform. The focus of CamelOne is to bring people together to understand, learn about, and meet each other and to grow this community of open-source integration users.

Gardner: So, this is CamelOne, May 24, in the Washington D.C. area. Why Washington D.C.? Is there a lot of this going on in the public sector?

Central location

Moynihan: Actually, we do have a lot of users in the Washington D.C. area. We also thought that was a central location, where people could come from not only anywhere in the US but also from other regions of the world as well. There are a lot of direct flights to that location. But, we do have a lot of users in the area. For example, the Federal Aviation Administration (FAA) is going to be speaking and they have selected open-source integration for the next generation of their services infrastructure.

Since they connect with a lot of other agencies, there is a lot of interest in learning more specifically about that program and about the technologies that it's built upon, because a lot of other agencies need to connect.

Gardner: And how about more information on CamelOne? It’s simple, I suppose search on CamelOne will get you there.

Moynihan: Yes, camelone.com is the website as well.

Gardner: Now, you guys have been involved with a series of books and you have something new coming out in that series. Tell me about that.

Camel in Action

Moynihan: There are a couple of books that recently have come out. One is Camel in Action, which is fantastic for people who want to get going with Camel and learn how to use and deploy it. Rob is coauthor of the ActiveMQ in Action book, which has come out in print recently from Manning Publications.

Davies: ActiveMQ in Action is really a scripted book, which goes through all the different use cases of using ActiveMQ, right from getting started and what messaging is about. It walks you through different deployment options, all the way up through using clusters of ActiveMQ brokers, to using ActiveMQ as a wide area network, so you can connect geographically dispersed locations.

It shows you how to tune the performance of ActiveMQ and get the best out of it. So it's very comprehensive book about how to use ActiveMQ. It's somewhat complementary to Camel in Action as well, which goes through all the different patterns you can use.

It doesn't talk about using Camel. It talks about integration patterns as well and then describes how you can use those using Apache Camel, and you can use Apache Camel with ActiveMQ. ActiveMQ also can embed Apache Camel. So, you have routes running inside the broker from Camel. The two of them are very complementary.

On our website fusesource.com, we also have a lot of webinars, which are happening live on a regular basis. We have a lot of archived webinars, which actually walk you through technical tutorials on how to get started with these various open-source projects.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Register for CamelOne. Sponsor: FuseSource.

You may also be interested in:

• FuseSource Gains New Autonomy to Focus on OSS Infrastructure Model, Apache Community Innovation, Cloud Opportunities
• Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing
• Apache CXF: Where it's Been and What the Future Holds for Web Services Frameworks
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function

Case Study: How Fairchild Semiconductor leverages the Workday Integration Cloud

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: Workday.

The latest BriefingsDirect podcast provides a case study on how new forms of cloud-based integration are helping a major high-tech company build new relationships among and between extended enterprise business processes.

We'll examine how Fairchild Semiconductor has been an early adopter of integration platform as a service (iPaaS). The venerable Silicon Valley company has been using graphical tools to build integrations among and between far-flung applications and services but with those integration platforms housed in a newly unveiled Workday Integration Cloud. [Disclosure: Workday is a sponsor of BriefingsDirect podcasts.]

We’ll learn here from the chief technology officer at Workday on what the integration cloud approach can do and how it points to a future in which broad integration capabilities are increasingly built into software-as-a-service (SaaS) applications.

This cloud-based integration model will prove far less vulnerable to the complexity, fragility and cost that plagues traditional on-premises middleware integration methods. It should also spur the evolution of services ecosystems among multiple business service providers and application providers.

Joining the conversation to dig into what makes integration as a service (IaaS) tick and what it means for the future is Paul Lones, Senior Vice President for Information Technology at Fairchild Semiconductor, and Stan Swete, the Chief Technology Officer at Workday. The discussion is moderated by BriefingsDirect's Dana Gardner, Principal Analyst at Interarbor Solutions.

Here are some excerpts:

Gardner: What's the problem now with integration? Why is this different than a few years ago? Why is it that we need to adopt a different take on integration?

Lones: We've just recently gone live with Workday, and several of their partners, and have completely transformed our human capital management landscape.

Fairchild Semiconductor has roughly 10,000 employees worldwide. We're a semiconductor manufacturing company. We have manufacturing facilities in the United States and throughout Asia. Our customer base is global, our employee base is global. Over 70 percent of our business is in Asia and 70 percent of our employees are in Asia. Having the capability to provide a core HR platform like this to that broad a set of colleagues around the world is really exciting for us, and to be able to support our internal customers and the HR group.

... Integrations are a new challenge, a broader challenge than they have been traditionally. ... In the HR arena, there has been no such thing as a standard integration. Every benefit provider, every payroll service provider that you want to work with requires a custom integration. That’s always been true, and having the set of tools that we now have at our disposal makes that a lot easier.

Companies like Fairchild are really trying to take advantage of some of the new capabilities that SaaS providers are offering. ... It's a critical enabler.

We look for two things. One, we want to find a supplier that thinks of this in a more holistic ecosystem-like way, and that has a series of application-level partners, that we can add to our overall architecture and overall application capability.

In addition to that, we look for good integration tools, because even beyond those partnerships, we still have to do a lot of integration work.

... For the Workday partners, those integrations are handled between Workday and their partners, which reduced our integration burden. We don't have to maintain those, as both of those applications continue to improve. In addition to that, we've built 28 application integrations ourselves, largely to benefit service providers and payroll service providers around the world.

We were fortunate enough that we were able to get some early access to the toolset that Workday is now making available to their broader customer and partner base.

I had a small team of IT staff that was completely unfamiliar with Workday when they first started, and we put them to work on these integrations. We were able to complete these 28 integrations in less than 120 days, which I think was pretty good performance. ... We do know that from an overall project implementation perspective that an on-premises application typically will take 2-3 times as long to execute, and I'd expect that the integration piece would have a similar scaling.

Gardner: Stan, what needed to change and when Workday looked at this issue of your online ecosystem and how it tied things together?

Swete: We still look at it as having the same requirements for enterprise integration. Especially for hub systems like human capital management, there are ton of other systems that you have to integrate with. So the requirements are daunting and are still there. It's been the same for a while at enterprise software.

What we see as being a cloud vendor, a SaaS vendor, is just new opportunities to leverage the SaaS model to do integration a little bit differently, have the application vendor take on more of the ownership of the integration issue, and use the fact that we've got all of our customers running on a single version of the product to tie some integration logic to that and bring more control and stability to that integration for our customers and our partners.

Gardner: Why is it then that the traditional systems, platforms, and middleware that are in place are not up to this task?

Swete: There's just a split today between the technologies and the platforms that are used to execute integration and convey data and then the application’s endpoints that are involved with and tied up in the logic of that integration.

It's not that no one is up to it, but it's just that that gap splits responsibilities where maybe they don’t have to be split. What we’re trying to do is marry it, use what we know about our applications to create integration logic, and then embed technology that hasn’t been embedded with applications before to help with the delivery of that.

That hasn't replaced every single kind of middleware technology that you need. You still need a middleware technology behind your firewall. You still need specialized middleware technology in the cloud to do things that it does best. But, for the application-centric part of integration, application vendors can do more.

... The Workday Integration Cloud is an extension of Workday's cloud that we use to host and process our on-demand applications and it has several really important components. One is a platform component. The tools that Paul mentioned that they used to build integrations, up until today, have been there for Workday developers. The announcement makes these tools fully available to Workday customers and to Workday partners.

In addition to the tools, there is a rich enterprise service bus (ESB) execution environment that runs the results of these developmental tools. We offer not only the tools to build integration systems but the execution environment for the integration systems. And then we've a set of scheduling and monitoring tools that our customers can use to directly schedule and monitor the execution of their integrations.

So those three things taken together form the platform, that's part of the integration cloud. The resulting integration systems we also consider a part of the cloud. Workday for some time has been building what we call Packaged Integrations and Connectors. We have a library of those that we can make available to our customers.

Fairchild has used some of these. These integrations are built with our tooling by us and for our customers. Packaged integrations really just look like another Workday product, but they handle both ends of the integration challenge.

We also have connectors that handle our end of it but build logic out. The main example is a payroll interface product that lets our customers, gives our customers a starting point for hooking up Workday human capital management to the variety of international payrolls many of our larger customers have.

This is very solid ESB technology, well thought of by the engineering talent that we now own.

Packaged integrations from Workday is another component of the Integration Cloud and the final one is just the body of integrations that our customers and partners create.

These are the intellectual property of our customers and our partners. Workday does facilitate sharing of those definitions if the customer and partners are interested, but there is that growing body of application as well. Those things taken together are the Workday Integration Cloud.

Gardner: And just to be clear, this is designed for your customers. This isn't just a general purpose integration service that you are opening up writ large. This is about your ecosystem and your customers, is that right?

Swete: The beauty of it is that it's based on middleware from a company formerly called Cape Clear that Workday acquired three years ago. I think that's very important to mention that. So it's not like we, an apps vendor, just did our take on an ESB. This is very solid ESB technology, well thought of by the engineering talent that we now own.

Built-in integration

We're taking this technology and integrating it into our applications, building integration into our applications as the way we refer to it, and then making the combined product available to both our customers and our partners. The partners are the equally important point. Systems integration partners from Workday can get access to these tools and this platform.

Gardner: And how about the pricing?

Swete: The Workday Integration Cloud platform is being made available at no additional cost to Workday customers and Workday partners. We make our money selling our application services.

Gardner: I'm intrigued by this notion of making integration part of the application. I think the history of this, Paul, has been that over the years, new applications and platforms, and even models of computing would come along. You would get great productivity from the application, you would buy and install and master the platform, and then you would be faced with an integration problem.

This is happened over and over again. We've seen it with mainframes to client-server and then into multi-tier and distributing computing and then ultimately with web and now cloud computing.

Companies like ours and many companies working on this are moving from a monolithic internal application orientation to one that's more of a hybrid model.

Given that integration has been a bolt-on, something that's been delivered after they shift in an application model, why now change? Why is integration and the application coming together now?

Lones: Part of it is that our approach to overall enterprise architecture is changing. Companies like ours and many companies working on this are moving from a monolithic internal application orientation to one that's more of a hybrid model, where we want to really take advantage of the new capabilities and the quicker pace of development and deployment of improvements that SaaS providers offer.

Therefore, integrations naturally become a critical part of that, because the number of applications that we use in our business increases somewhat with this sort of approach.

Swete: The challenge here is that the requirements in the large problem of integration haven't changed, and there have been a lot of tools developed to address the issue. Some results have been achieved, but I don't think anyone is satisfied with how maintainable enterprise integration is. And, we happened to think the answer is to build more robust integration where the integration definitions themselves are more informed by what exists and what's changing in the application.

Hub system

That's the opportunity that we were seeing. We came on to it by just being the provider of an application that is going to be the hub system and be hooked up to a lot of different systems.

We knew that integration was going to be front and center for us as a brand new SaaS vendor six years ago. One of the differences we wanted to make was to do more about the problem. So, we started with an investment of technology.

Where that has led us is really tying what can get done with integration technology to what applications know about, everything from their security model to, in our case, we leverage a lot the fact that we know about people and how they are organized. So, we're able to have integration definitions that can get routed around for the appropriate approvals before certain steps happen.

That’s unique, but it's breaking down the separation between integration that would be built by one side of the company and tying it back to who it's really serving, the other side of the company.

For payroll integration, the payroll admin can be hooked into the fact that a major feed of HR data is going out to a payroll system and they can get a check on that before it happens. That’s something we’ve built in and we’ll continue to look for those opportunities. I still think it's actually early days for what our integration tools can leverage inside the application.

You still have to have experts on integration middleware and we have that, but the real benefit we think comes from blurring the distinction and marrying these things together.

Gardner: So, the system of record for HR and the governance and policies about employees and their roles in the organization can now be applied pretty seamlessly to who gets to do integrations and/or how integrations as part of a business process would work. Am I reading that right?

Swete: Yeah, how they get executed, how they get approved is all built in to the same sort of system that you use to schedule a report or any other thing you’d do in your application. For us, it's just an extension of the application, rather than a hard line and then some integration technology that no one on the app side understands.

There still are differences. You still have to have experts on integration middleware and we have that, but the real benefit we think comes from blurring the distinction and marrying these things together.

... We’ve taken the approach of splitting the development tools into a framework that is more geared for developing simple integrations, as we call them. This is one-way data in or one-way data out of Workday to third-party systems, and we have a tool called the Enterprise Interface Builder (EIB) that is a non-programmer could use. You still need to know that you are sending something to a secure FTP location, but you don’t have to be a developer.

Sets of choices

We give you a graphical user interface, we give you a selected set of choices for how you can source data, a selected set of choices for how you can transform it, and a select set of choices for how you can deliver it. You can save that, and then you have a definition that you can then schedule on a recurring basis. That’s built for non-developers.

The other tool that we have has a completely different personality. It's what we call Workday Studio. This is the developer tool that we have used to build our integrations, and it is now available for our customers. But, on this one, you want to be a developer. You're not doing programming, but you are working in an Eclipse-based framework with detailed control over integration components and orchestration of how data flows. So, this is a technical development tool.

The thing it creates is the same thing that the EIB creates, an integration system that can then be executed in Workday, but the creation of it is much more technical.

Gardner: So it's interesting, Stan. You have a user like Fairchild, using these tools, building these integrations, moving more toward a multiparty ecosystem process-oriented benefit, but the responsibility on those integrations is with you.

It seems as if you're really giving an awful lot here. How can you do that with a strong sense of confidence? Isn't there a risk that if these integrations start breaking that you are in the catbird seat?

Levels of the game

Swete: Yeah, well, there are levels of the game for how you can leverage the support you get out of the core application that we keep moving forward. One level of the game is for us that's very important in the integrations we build and sell are ones that can just share the application definitions. So, we support those across all the updates and verify that the logic of those is going to work.

For the integrations tools, we can put smarts into the tools that share how the applications are constructed in that. It gives our customers a leg-up that they can start with these components. Then they can create integrations that are a little bit more impervious to being broken by changes in the applications, because they're sharing metadata back into the applications.

Lots of integrations are built on our application programming interface (API) and so we've got to be rigorous about versioning the API and having a contract to support back versions that gives us a certain amount of insurance. It's not like that with some of these opened in the tools that there couldn't be logic and coding errors that are put in and those are the ones that we would have to encounter together with our customers and we're not going to debug every single one of those.

So, for different levels of the game, more packaged, complete support, on up to the more open-ended integrations, you do what you can to try to make it so the integrations are a little bit more robust than what would have been built with a separate tool set.

... We also encourage people to want to share these integrations. We didn’t need to do more to automatically support that because our partners are going to be generating these things, as our customers, and in the SaaS community, there is just this great notion about sharing the things you do. So, we see supporting that and we can ultimately see that even leading to selling some of the things you do. All of those are potential features for this space.

Gardner: Paul, it sounds less like a buyer-seller relationship than a partnership. Do you view it that way?

Our experience to date is that companies like ours have more of a voice in the feature improvements of the application.

Lones: We do. Our experience to date, working with providers like Workday and some of the other SaaS providers that we are fortunate enough to do business with, is that companies like ours have more of a voice in the feature improvements of the application.

There tends to be, and certainly it's the case with Workday, a much more active community of clients, users, that are sharing information about everything from somewhat technical to very business process-oriented experiences that all of us have had. That's a very different experience.

In some ways, it's sort of ironic to me that we view it quite a bit more as a partnership. A lot of people perhaps think that it's a SaaS application and, if things don't work out, then when your contract is up, you just go find another SaaS provider.

It is true that there might be a little bit more flexibility, but what we’re finding so far in our experience, and it is early, is that the receptivity and the sense of making improvements together, I think it will actually stick longer than maybe some of the traditional software applications.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Learn more. Sponsor: Workday.

You may also be interested in:

• Delivering Data Analytics Through Workday SaaS ERP Applications Empowers Business Managers at Actual Decision Points
• Workday Integration Cloud debut raises bar on integration as service, deeply ingrains integration as apps function
  
• Executive Interview: Workday's Aneel Bhusri on Advancement of SaaS and Cloud Models for Improved ERP
• Architecture is Destiny: Why the Revolution in Business Interactions Can't Work on Conventional Databases
• HCM SaaS Provider Workday's Advanced Architecture Brings New Business Agility Benefits to Enterprises

Learning the right lessons from the Amazon cloud outage


This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

By Jason Bloomberg

Have you noticed that ZapThink’s crystal ball has been working overtime? We sounded the warnings about cyberwarfare mere days before the Stuxnet worm hit. Then we predicted the fall of enterprise architecture frameworks right before the Zachman organization imploded. Next, we heralded a secondary market for IP addresses as the IPv4 space ran out of them. Sure enough, that secondary market is now here. And last week, we warned against putting all your eggs in any one cloud provider’s basket. Sure enough, Amazon’s public cloud went belly up immediately afterward. All I can say is that if we make a prediction that will impact your business, you’d better take heed!

In all seriousness, there’s no supernatural clairvoyance at work here. What you’re seeing is the power of the ZapThink 2020 vision for Enterprise IT, which delineates the interrelationships among the numerous trends in the IT marketplace. Just as the best psychics are in reality masters at picking up subtle clues in human behavior, we’re tuning into the complex subtleties that the multiple forces of change in our midst present to us.

One of the primary insights of the ZapThink 2020 vision is that individual trends, let alone single events, should never be taken in isolation. This insight is particularly useful when a crisis like the Amazon crash presents itself.

At this point in time, we’re experiencing a backlash from this crash. People are reconsidering the wisdom of moving to the cloud, and in particular, public clouds. Perhaps the large infrastructure vendors who were warning their customers about the security and reliability issues with public clouds in order to sell more gear to build private clouds were right after all?

Not so fast. If we place the Amazon crash into its proper context, we are in a better position to learn the right lessons from this crisis, rather than reacting out of fear to an event taken out of that context. Here, then, are some essential lessons we should take away from the crash:

• There is no such thing as 100 percent reliability. In fact, there’s nothing 100 percent about any of IT—no code is 100 percent bug free, no system is 100 percent crashproof, and no security is 100 percent impenetrable. Just because Amazon came up snake eyes on this throw of the dice doesn’t mean that public clouds are any less reliable than they were before the crisis. Whether investing in the stock market or building a high availability IT infrastructure, the best way to lower risk is to diversify. You got eggs? The more baskets the better.
• This particular crisis is unlikely to happen ever again. We can safely assume that Amazon has some wicked smart cloud experts, and that they had already built a cloud architecture that could withstand most challenges. Suffice it to say, therefore, that the latest crisis had an unusual and complex set of causes. It also goes without saying that those experts are working feverishly to root out those causes, so that this particular set of circumstances won’t happen again.


  Just because Amazon came up snake eyes on this throw of the dice doesn’t mean that public clouds are any less reliable than they were before the crisis.

• The unknown unknowns are by definition inherently unpredictable. Even though the particular sequence of events that led to the current crisis is unlikely to happen again, the chance that other entirely unpredictable issues will arise in the future is relatively likely. But such issues might very well apply to private, hybrid, or community clouds just as much as they might impact the public cloud again. In other words, bailing on public clouds to take refuge in the supposedly safer private cloud arena is an exercise in futility.

• The most important lesson for Amazon to learn is more about visibility than reliability. The weakest part of Amazon’s cloud offerings is the lack of visibility they provide their customers. This “never mind the man behind the curtain” attitude is part of how Amazon supports the cloud abstraction I discussed in the previous ZapFlash. But now it’s working against them and their customers. For Amazon to build on its success, it must open the kimono a bit and provide its customers a level of management visibility into its internal infrastructure that it’s been uncomfortable delivering to this point.


The ZapThink take

Abstractions hide complexity from consumers of technology, but if you do too good a job hiding the underlying complexity, then the abstraction can backfire. But that doesn’t mean that abstractions are bad; rather, you need different abstractions for different audiences.

The latest crisis impacted a wide swath of small cloud-based vendors, from Foursquare to DigitalChalk to EDU 2.0. These firms’ customers simply wanted their tools to work, and were disappointed and inconvenienced when they stopped working. But the end-user customer may not have even been aware that Amazon’s cloud was behind their tool of choice. Clearly, those customers wouldn’t find better visibility into the cloud particularly useful.

No, it’s the technology departments at the small vendors that require better visibility. They are the people who require management tools that enable them to gain a greater level of control over the cloud environments they leverage in their own products. Once Amazon supports such management tools, then Amazon’s customers will be better able to provide the seamless abstraction to the cloud end user, who simply wants stuff to work properly. And there’s nothing supernatural about that!


This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

---

SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

---

You may also be interested in:

• Amazon apologizes for last week's cloud crash
• Cloud brokering: Building a cloud of clouds
• Open Group cloud panel forecasts cloud as spurring useful transition for enterprise architecture
• Cloud computing drives need for open standards to define and describe a new enterprise environment