Data Considerations for SOAs
Shared data services can enable SOA success.
by Chris Keene

December 1, 2004

While most experts agree on the functional benefits of service-oriented architecture (SOA) implementation, their visions of how to achieve it differ widely. Some suggest leveraging existing applications by wrapping them with a common application program interface (API). Others advocate building a completely new architecture with functional modules that expose services to end-user applications. Still others espouse a hybrid approach.

Regardless of the vision, data access logic consumes a high percentage of development resources and plays a significant role in the success or failure of an SOA initiative. Ultimately, data access logic can even determine whether the resulting systems meet performance and scalability requirements.

Our experience with SOAs in demanding enterprise environments has produced useful insights regarding the data access challenges common to SOAs. These challenges include the need to:

• Increase developer productivity by allowing developers to focus more on business-critical logic.
• Maintain data integrity when migrating existing data and application silos to enterprise services.
• Ensure the performance and scalability of the deployed system.

This article discusses the advantages of implementing shared data services to deliver on the true promise of an SOA—rapid application development through reusable components without sacrificing fast, accurate enterprise data access. By designing a shared data layer into an SOA, companies can avoid integrity, performance, scalability, and availability issues that might otherwise occur.

I include a case study of a financial services company that adopted a shared data services approach for a realtime trading platform. It was able to double developer productivity and achieve high performance and scalability while maintaining data integrity.

Defining an SOA
An SOA provides a flexible, reusable framework for building and integrating multiple applications and application components. An SOA allows an enterprise to share application services to quickly construct new applications and provide greater value to the enterprise.

The commonly cited benefits of an SOA are:

• Flexibility to enable developers to quickly build new applications.
• Reusability to leverage investments across multiple applications.
• Loose coupling to avoid the brittleness of monolithic applications.

Web services are often mentioned in discussions of SOAs. Web services generally wrap existing applications and related data and expose a consistent API for business-level services, such as credit card authorization. Web services are frequently used to integrate packaged applications. In this respect, Web services offer some of the benefits of an SOA without requiring significant rewriting or re-architecting of existing applications.

The weakness in the "pure" Web services approach appears when the resulting services must be loosely coupled and stateless. Every request goes to the Web service, the application, and then the database (see Figure 1). Statelessness combined with increased demand from exposing the new services to more users increases latency and degrades response times. To realize all of the benefits of an SOA, an organization must carefully design and implement a new set of services and applications.

The real power of an SOA is unleashed when it provides infrastructure as well as business services. Infrastructure services are building blocks from which users can create and integrate multiple applications. To achieve the greatest degree of reuse, it is important to consider key services that all applications require. Such infrastructure services for persistent data access include object-relational mapping, caching, cache synchronization, and client notification.

Shared data services reduce the number of database connections required and support a stateful architecture (see Figure 2). By caching frequently requested data, more requests can be satisfied without querying the database, which improves performance and increases scalability and reliability. In addition, the reusability and flexibility of data services allows you to develop new services and roll them out more quickly.

Removing Data Silos
Traditional applications often have their own database "silo" that contains a copy of business reference data such as customer information, product information, and inventory levels. Typically, each database is synchronized only once a day, so each application operates with slightly different data. When applications are redistributed as enterprise services without integrating the data silos, these data inconsistencies can create unanticipated business errors.

Inconsistencies can arise when silo applications are exposed as services, each with different inventory data (see Figure 3). In this example, the show_status service thinks the inventory level is 27, while the check_avail service thinks the inventory level is 0.

Given the business risks associated with inconsistent data across multiple silos, the first step for an IT department implementing an SOA is to ensure data consistency. Developers can't build custom, service-oriented applications without first implementing a shared data layer that delivers consistent data across all services.

Shared Data Services Enable SOA Success
Handling data access and updates accounts for a high percentage of enterprise application development efforts. Most IT groups today use ad hoc data access solutions such as JDBC or ADO.NET that work well within a silo architecture but are unable to support data consistency enterprise-wide.

A CIO of a Fortune 500 company recently described how his IT group had hundreds of applications, each doing the same thing, with very slight differences. Each development team within his large IT group was implementing a unique data access component—through ODBC, JDBC, or some other data access standard. He recognized that even though it was standards-based, this low-level coding was tedious, error prone, and unnecessarily redundant.

An increasing number of enterprises are recognizing the need for a shared data service that offers domain-specific data classes used by multiple applications. Each application uses a subset of the data classes managed by the data service. The data service manages relationships between the data classes and serves data changes to each application, regardless of the source of change.

Using the SOA paradigm, it is preferable to implement a credit card authorization, for example, as a single service that many applications can reuse. Similarly, it is preferable to implement a single customer data service to retrieve current customer information for a set of related applications. However, if you don't pay attention to the data access infrastructure, multiple applications vying for the same services soon run into a data bottleneck.

To be successful, an SOA initiative requires data access infrastructure software specifically designed to provide consistent performance and highly available data across distributed computing environments. Ideally, system architects should seek cross-platform data access products capable of meeting requirements across the project lifecycle—from development through tuning and deployment.

Case Study: SOA in Financial Services
A leading financial services firm has implemented more than 40 equity trading applications using an SOA that includes a shared data services layer. This business has rigorous requirements for reliability, performance, and scalability—processing $7 billion per day in trades and thousands of transactions per second at peak volume.

In equity trading, a single data consistency error can have enormous financial consequences. To ensure data integrity, this firm invested heavily in a shared data services layer. Its objectives were threefold: first, deliver immediate response to end users; second, have the ability to scale to meet the growing needs of their businesses; and finally, ensure that the system is available at all times.

In its architecture, the data service layer provides caching, optimized updates, distributed cache synchronization, load balancing, failover, and client notification. These capabilities are much more robust than the homegrown data persistence layer used in the previous generation of applications.

In the structure of this SOA deployment (see Figure 4), the data services layer provides data management for relational data and realtime market data feeds. Because the applications are related and share a common data model and common data, data services deliver up-to-date business information to each server and application.

The economic benefits of an SOA deployment in this particular environment include:

• Twice the developer productivity: Shared functional and data services account for more than 50 percent of new application functionality.
• Three times the maintenance productivity: Systems deployed using SOA can be maintained with 75 percent fewer resources.
• Dramatically higher availability: Fault tolerance within the data services layer eliminates application failures due to intermittent database or network failures.
• Significant infrastructure and operational savings: Distributed application deployment with centralized data storage—the virtual data center—can achieve 40 percent capital cost savings and 30 percent annual operating cost savings versus traditional data centers.

For applications with relatively simple requirements, the benefits of a data service include flexibility, reusability, and loose coupling. However, many enterprises also have high demands for performance, scalability, and reliability. Deploying a data service that is shared across multiple applications enables organizations to optimize this part of the infrastructure to satisfy these requirements. In contrast, when each application is implemented with a unique data access layer, each requires its own tuning and maintenance, greatly increasing complexity and total cost of ownership.

When correctly designed and rolled out, SOAs represent a great leap forward in system design. To be successful, architects need to meet the challenges of providing scalable, high-performing data access without compromising data consistency. Data access infrastructure software specifically designed to provide consistent performance and highly available data across distributed computing environments such as service-oriented architectures plays a vital role in ensuring success.

About the Author
Chris Keene is CEO of Persistence Software.