Monday, February 1, 2010

Business event processing and SOA: Joined at the hip

This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

By Jason Bloomberg, edited by Ronald Schmelzer

At the dawn of the computer age, forecasters predicted all manner of changes in day-to-day life, including fully automated kitchens, cars that drove themselves, paperless offices, and more. In all of these now-quaint views of the future, the prognostication focused on applying the technologies of the future to the problems of the present. Where an office had a filing cabinet full of paper, the future will put that paper in a computer, and voila: The paperless office!

The reality of the last 50 years, of course, is quite different. As technology evolved, so too did our appetite for information. Where the size of the filing cabinet in a 1950s office constrained the quantity of information a business could manage, today we have no such limitations. But rather than stopping at a simplification of existing business, we continue to push the limits of the technology at our disposal. If we have terabytes of storage, then to be sure we’ll soon have terabytes of information to fill it. If we have networks running at gigabit speeds, then you can rest assured the quantity of information we’ll attempt to push through such pipes will soon consume whatever capacity we have.

In many ways, in fact, it is the quantity and variety of information on the move, rather than simply at rest, that defines the modern business world. Today’s businesses operate in a complex ecosystem of connected, interrelated events, where each event creates information that flies around our networks. From fluctuating interest rates to customer transactions to manufacturing processes, business events drive the business while simultaneously causing the creation and movement of information.

In particular, it is the tight interrelationship between business events—occurrences in the real world that are relevant to the business, and software events, which are messages that such events generate, that creates both a problem as well as an opportunity for businesses. The sheer quantity and variety of events promises to swamp any organization unprepared for the onslaught of network traffic that today’s business generates. But on the other hand, business events are also the lifeblood of the organization, as everything the business does appears in real time in the event traffic on their network. Sometimes the patterns such events exhibit are easy to identify, but more often they are hard to detect and correlate. Organizations need to process and leverage business events to provide insight into the workings of their organization in order to run the business and empower the people within it.

What is business event processing?

The key to leveraging business events is to apply software that processes software events—such as messages on the network—in such a way as to gain insight into, and control over the business events that generate them. Such software is known as Business Event Processing (BEP). BEP software helps businesses detect, analyze, and respond to complex events to take advantage of emerging opportunities, handle unexpected exceptions, and redirect resources as necessary—essentially, dealing with business events on the business level, independent of the technology context for those events. BEP software often forms part of a Business Process Management (BPM) solution, which combines event pattern detection with dynamic process execution.

The goal of BEP is to detect and interpret business situations, resulting in effective business decision making. BEP enables organizations to extract events from multiple sources, detect business situations based on patterns of events, and then derive new events through aggregation of events as well as by adding new information. BEP software helps companies identify patterns and establish connections between events, and then initiates a new event, or a trigger, when an important trend emerges.

BEP is becoming increasingly important across the business environment because it enables a wide variety of organizations to proactively analyze and respond to small market changes that can have significant business impact. BEP also has a variety of other uses, for example:
  • Retailers’ BEP solutions proactively alert them about the success or failure of a product as goods move off the shelf, allowing them to make real time changes to pricing, inventory, and marketing campaigns

  • E-commerce vendors leverage BEP to help identify fraud and reduce abandoned shopping carts

  • Trading markets use BEP to uncover and compare minute changes throughout global markets to support buy/sell decisions as well as to ensure the timely execution of bids

  • The massive multi-player online game industry uses BEP to uncover unauthorized activities among tens of thousands of actions per second

  • Fleet management companies leverage BEP to help them make instantaneous decisions on how to deal with products that are lost in transit or delayed due to unforeseen circumstances.
Business event processing in an SOA context

BEP describes a wide range of ways that enterprises approach events, from simple to complex. Opening an account, making a withdrawal, buying an item, changes in sensor or meter readings, or sending an invoice are all examples of common business events. Regardless of the potential complexity of such events, organizations must both recognize new events and understand the importance of business critical events in a noisy environment. Only by recognizing important events in real time will such companies be able to leverage their IT systems and business processes to speed response and reduce the need for manual processing.

And for every type of application, there is potentially a new type of event.



Events, however, do not exist in a vacuum—they depend upon various applications and systems across the IT infrastructure to create and consume them. And for every type of application, there is potentially a new type of event. The BEP challenge, therefore, is dealing with environments of broad heterogeneity, separating what’s important to the business from the underlying complexity of the technology.

In other words, BEP does not stand alone in the IT organization. It requires a flexible architecture that can abstract the underlying heterogeneity the IT environment presents. Today’s enterprises are implementing Service-Oriented Architecture (SOA) for this purpose. SOA is a set of best practices for organizing IT resources in a flexible way to support agile business processes by representing IT capabilities and information as Services, which abstract the complexity of the underlying technology from the business. Businesses then define events and their responses through the IT perspective of interacting with Services.

Applying SOA to business events: Heterogeneity and flexibility

The story of how to apply SOA to business events take place simultaneously on two levels: above and below the Service abstraction. Above this abstraction is the business environment, where the business is able to leverage BEP to glean real time information about the business independent of the underlying technology. In contrast, below the Service abstraction, events are messages moving from one Service endpoint to another, typically (but not necessarily) in XML format.

It is below the Service abstraction, in fact, that applying SOA to business events provides much of its value to the organization. Service interfaces, by their nature, send and/or receive messages, so the broader the SOA implementation, the more the message traffic between Service endpoints represents the operations of the business. From the BEP perspective, however, such messages are events, and provide visibility into the business in an ad hoc, real time manner.

. . . If managers have visibility into business events, then they can then take more effective, proactive steps to optimize production and reduce costly slowdowns.



While the SOA-enabled BEP story offers business value beneath the Service abstraction, the benefits to the organization above the abstraction are every bit as important. After all, as the pace of business accelerates, there are business benefits from optimizing how the organization handles business events. Improved customer responsiveness, more optimal usage of physical assets and better management of complex value chains all benefit from improvements in event processing. Furthermore, if managers have visibility into business events, then they can then take more effective, proactive steps to optimize production and reduce costly slowdowns.

Similarly, event processing can improve customer service and increase customer satisfaction. Because event processing can identify important events and deliver the right information to the right place at the right time, managers can mitigate or avoid a wide range of problems. Such benefits accrue not only in individual instances, but also across business processes, as well. Visibility into events helps line of business managers deal with changes in business process, thus making the business more reactive.

Combined with SOA and BPM, therefore, BEP extends the value of each as well as the synergies between them. Following SOA best practices can leverage the value of both BPM and BEP, as SOA hides the complexity of the IT environment from the business aspects of the solutions. The bottom line is that BPM, SOA and BEP combine to meet the needs of the business more effectively than any one or two of the approaches can separately.

The ZapThink take

The exponential growth of information in the business world continues unabated, and there’s no reason to expect it to slack off in the future. This growth is driving the need for event processing, as well as the enabling technologies of Web 2.0 and the underlying architecture of SOA. The combination of these three approaches provides a foundation for flexibility, composability, integration, and scalability. At the heart of this synergy are open standards, which facilitate all the various interactions among systems that goes into Business Event Processing. Furthermore, existing security, governance, and BPM technologies round out the set of enabling technologies that feed this confluence of approaches.

The bottom line, however, is the business story. BEP, combined with SOA, further bridges the gap between business and IT.



The bottom line, however, is the business story. BEP, combined with SOA, further bridges the gap between business and IT. Only the business knows the relevance of business events, SOA abstracts the underlying technology, and Web 2.0 provides an empowering interface to increasingly powerful, real time capabilities and information.

The challenge with discussing this synergy among BEP, SOA, and Web 2.0 is that no one term does it justice. SOA is a critical part of this story, but is only a part. SOA delivers a set of principles for organizing an organization’s resources to provide a business-centric abstraction, because the business doesn’t care what server, network, or data center the implementation underlying a Service runs on. All they care is that the Service works as advertised.

This guest BriefingsDirect post comes courtesy of Jason Bloomberg, managing partner at ZapThink.


SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink's Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink's SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.