email article printer friendly

Trends in the Evolution of the ESB
An enterprise service bus does much more than get messages from point A to point B.
by Dave Chappell

October 23, 2006

Over the past four years we have seen a great deal of favorable adoption of the enterprise service bus (ESB) as a product category in the industry. Progress Software may have started the ESB technology category with its introduction of Sonic ESB in 2002, but the ESB phenomenon has taken on a life of its own since then. Application platform suite (APS) vendors such as IBM and BEA, enterprise application integration (EAI) vendors such as TIBCO and webMethods, and even Web services toolkit vendors have all adopted the ESB moniker.

Even British Telecom has embedded an ESB in a hardware box with their BT Integrate offering. We have also seen some significant changes in the way technologists in IT departments view the role of an ESB as an important part of their IT landscape. As is the case with most new technology categories, it has taken some time for the hype to subside. Now that the dust has settled we can see the practical uses of ESBs as they get deployed in the real world.

Analyst views from firms such as Gartner and Forrester have also shifted from hailing ESB as the all-you'll-ever-need for service-oriented architecture (SOA) to more of an implied part of the infrastructure in support of an SOA. Reports from these analysts and from thought-leading vendors have provided greater clarity on what makes up the definition of an ESB. The constant that still remains is that an ESB is used to connect, mediate, and control the interactions between a diverse set of applications that are exposed through the bus using service-level interfaces.

As I spend most of my time traveling around the world talking with people about SOA and ESB, I have noticed a shift in their thinking and witnessed a trend of enlightenment over the past two years. The questions on most people's minds has shifted from "what is SOA?" and "what is an ESB, and why would I need one?" to "I know I need an SOA, I know I need an ESB, but what is an ESB best suited for and what other technology infrastructure do I need to implement my SOA?"

Process-Oriented, Event-Driven Architectures
Point-to-point integrations typically have been addressed with simple request-reply, synchronous style interactions. In this type of environment, a Web service intermediary acting as a proxy for data transformation and routing can work well. However, the real sweet spot where an ESB has shown its power and flexibility is in process-oriented, event-driven architectures (EDA). When doing broad scale integrations across many disparate applications, the key to success is to have an architecture that allows for each application to be decoupled from the rest of the SOA by using the ESB as a form of mediation.

Each application should be made to operate independently from the other by being capable of receiving individual tasks or units of work that are processed as asynchronous events. If there is another action to be taken as a result of receiving and processing the asynchronous event, then so be it. That action doesn't necessarily represent a response to a request that the original initiator of the action is waiting for. It's more likely that the next action to occur is itself an asynchronous event that is placed back onto the bus to be forwarded along to the next application. The surrounding process manager that is part of the ESB, which is controlling the interactions between the individual invocations, is what is concerned with getting that event sent along to the next application.

The same is also true even if an application makes a request and expects a response to continue its processing. The request can be placed on the bus and sent across a distributed path of many applications, data sources, routers, and transformers in accordance with the process that is defined for it. That series of actions may all be processed as independent events and the response may come later as an independent event.

The event-driven interaction style is a major advantage of keeping applications decoupled form one another. When plugged into an ESB, each application doesn't need to understand the intimate details of how all the other applications want to be interacted with. Protocols, data formats, and different interaction styles are all worked with by the ESB.

EDA can only work effectively if certain conditions are in place. First, the ESB must have reliable, asynchronous messaging and high availability capabilities. In a synchronous point-to-point integration scenario, if an application fails to receive a response from a request, it can code in some error handing and retry. In the asynchronous scenario, the application simply fires off an event to the ESB and forgets about it until something else occurs to trigger further processing. There will be many points where the whole business transaction will be nowhere else but in transit through the bus. The ESB must therefore be capable of surviving failure and also sustaining the essence of the business transaction complex topologies that may involve failures and recoveries along the way.

Another condition that has to be met is the applications themselves need to be written or adapted to this event-driven style of interaction. In the case where ordering of things is important, then each application may need to check for or compensate for things occurring out of order, or the ESB itself may need to be capable of ensuring order of events across complex deployment topologies and across failures and recoveries.