10 Ingredients for Successful SOA
Proper SOA development mandates re-engineering of the ways we approach software development
by Anant Kadiyala

August 20, 2004

The concept of service-oriented architecture (SOA) has been largely a technology buzzword. Although technology does play an important role, the right SOA approach mandates re-engineering (or refactoring) of the ways in which we approach software development. We are accustomed to taking a component- and project-based approach to application development, but SOA requires a top-down approach. That means we have to look at application design and project management in a new light.

Let's take a look at 10 of the key ingredients that are paramount to the success of SOA implementation.

1. The Enterprise Architecture Team
SOA is an enterprise-wide, long-term strategy. As such, most companies approach it by assembling a central team (the enterprise architect, or EA, team) that owns and drives this initiative forward. In most cases, this team is a small, tight group with diverse but complementary skills, which owns the overall architecture of the enterprise. The EA team is responsible for developing internal standards, blueprints, reference architectures, design patterns, templates, some of the shared and horizontal services, and so on. It works closely with line-of-business experts or has domain experts as part of the team. This central, over-arching team is essential to alleviate the risk of individual groups or divisions reinventing the wheel and formulating their own methodologies.

The EA team is the most critical factor in getting the SOA implementation right. Without a good team that understands the how-to of doing and handling SOA, the effort is likely to fail.

2. Implementation Roadmap
Once the EA team is assembled, the next major task is to work with the business and IT teams and build a roadmap for implementation. Like any good project, the more diligent the plan, the better the chance for success.

One commonly used strategy is to start with creating current state and future state documents, which make it easier to see the holistic picture and understand system interactions. This exercise also helps to identify systems that are pain points for the company.

The next step is to identify feasible milestones (in most cases, six months, 12 months, 24 months, or 36 months).

Some legacy mission-critical systems grow over a period and show the scars of time, including band-aid solutions stitched and glued together. Systems like this are ticking time bombs. Sometimes these are good candidates for initial "on ramps" to SOA. Usually the initial candidate projects are selected based on how dysfunctional a system is, how feasible is it to fix it, and the return on investment (ROI).

The roadmap has to be closely aligned with the strategic interests of the company. Factors such as projects in progress, funding, staffing, business dynamics, and industry competition could affect the progress of implementation. Since several factors could pull SOA efforts off track, tracking the progress regularly is prudent. Bear in mind that the first few wins are important to gain support for SOA and the refactoring it brings to the organization. Therefore, from a political standpoint, selecting the right candidate projects in the initial phases is critical.

An SOA roadmap typically has multiple phases. The first phase happens when the company tests the water and tries to understand the technical challenges. Simple horizontal services such as authentication, authorization, validation, and data transformations are implemented in this stage. In the second phase, more business-oriented services are developed. These services often bubble up to the portals, thereby offering easy access to information trapped in back-end systems. The third phase involves aggregating services, developing workflows, and integrating disparate systems.

3. Architecture
The main advantages of SOA (loose coupling, reuse, and abstraction of service from technology) can be achieved only with a sound architectural foundation. The EA team is the central institution that drives architecture across the enterprise. From an SOA standpoint, the following aspects of architecture and design are significant.

Platform. One of the most fundamental decisions is which technology to use. Support from the underlying platform is critical, because it helps us avoid writing our own homegrown solutions. You also need to consider the team's comfort factor with a particular platform. A good platform greatly reduces the overall cost of development and maintenance. Some companies use a combination of platforms, as well. The advantage with SOA is that after the services mature to a certain state, the interface becomes more significant than the underlying technology. Technology becomes pluggable based on ROI.

Standard. For SOA, Web services are the most commonly chosen technology in the industry. In this space, standards are evolving and multiple competing standards address a given problem. A good grasp of these fundamentals and the direction of momentum ensure that money is not spent on the wrong ones. A platform that adheres to standards protects the investment and makes integration efforts with external partners and vendors easier.

Third parties. Companies invariably have to communicate with external partners. Although we do not have control over the third parties' architecture, we should try for their SOA compliance, so that it becomes easier and cost effective for everybody in the long run. The same also applies to systems integrators and vendors, except in these cases, we have a much stronger say. Minimizing the number of strategic partnerships greatly helps in keeping complexity under check.

Services. An organization could have several kinds of services. Some are more vertical and business centric, while others are broader. Services that fall into the latter category—such as security, data transformation, translation, and event services—are typically good initial SOA candidates. The EA team is responsible for providing design patterns and guidelines in leveraging these cross-cutting services.

The EA team (in conjunction with the vertical teams) is also responsible for nurturing what Gartner calls the "productivity layer," which fosters reuse and aggregation of services. Garner states, "Without it, investments in Web services and other aspects of service orientation will reap minimal to zero rewards." Some of these services fall into this category.

Technologies like Web services enable the service to be independent of the context. This independence makes services easily usable across horizontal projects. Messaging could make applications loosely coupled, but for optimal reuse, even the context should be abstracted out.

It is highly recommended that the services go through a detailed schema and architecture review. This would provide a consistent and similar schema style and namespacing, among other options, for various vertical development teams to work with. Adopting industry or domain schemas along with architected schemas should be the preferred strategy.

Managing services. Once the services are developed and tested, they need to be deployed into production. Just like in the Web development world, it is one thing to develop an app, but it is an entirely different ball game to manage them in a production environment. Service level objectives (SLO), quality of service (QoS), security, access control, failover, monitoring, disaster recovery, auditing, and notification are just a few in the long list of tasks to be managed. Industry-specific governance measures like HIPAA and Sarbanes-Oxley (SOX) might necessitate additional infrastructure for controlling changes, auditing, and monitoring key systems. Other issues that also need attention are cyclical load handling, versioning, and life cycle management of services, and scaling.

You should have a plan for the organic growth and evolution of the services. Hence, a good change-management policy should be in place. This could be mandatory if you have legal compliances such as HIPAA and SOX.

4. Skills
A good assortment of skills—including a firm understanding of the domain, vertical technology and processes, industry and technology standards, emerging technologies and trends, compiling requirements, expertise of development platform, patterns, best practices, project management, and testing—is necessary for the success of any software project.

To be able to provide strong guidance and leadership, the EA team should be a tight group adept at these core skills. In the vertical teams, the members should have a strong understanding of the business processes and should be able to assist the EA team in identifying and designing good services.

5. Delivery Model
The delivery model consists of three major aspects: team composition and size, project duration, and development methodology. Each organization's delivery model is different, based on the nature of the project, skills set, comfort factor with technology, and domain knowledge. This model should be consistent across groups and projects, so that when developers move across projects, they have a similar experience and the learning curve is minimal.

Examples of delivery models include 6 x 12 (6 developers for 12 weeks) for moderate projects or 4 x 6 (4 developers for 6 weeks) for small projects. The teams typically consist of additional support members such as project managers, quality assurance (QA) folks, or business analysts. It is important that the support team members understand the overall SOA vision. They need to work closely with the EA team and help ensure that the efforts are strategic. Companies have also been experimenting with variations of popular methodologies like Agile Modeling and Extreme Programming for SOA.

6. Governance
Having a solid governance model in place ensures structured, nurtured, organic growth in services. Factors like dynamics in the organization, culture, skills, and trust greatly influence the model. Governance is important to ensure no duplicate efforts and projects are not bypassing SOA compliance.

Who should be in charge of this policing? The answer is not so straightforward. SOA is really a civic duty. The individuals and vertical groups themselves should make sure that they adhere to service orientation at all times. You should not yield to the temptation of the path of least resistance, unless business pressures dictate so. The EA team can initiate the policing and can oversee to an extent. However, even in a mid-sized organization, the EA team could not police every project.

The EA team and the project management office (PMO) are responsible for prioritizing and sequencing the projects. All the business requirements (that are shared or enterprise-wide) funnel through the EA team. Working closely with the line of business experts, PMO, and the vertical teams, the projects are prioritized. Once implemented, the EA team makes certain that the applications are SOA compliant. As services evolve into the next generation, the EA team provides technical guidance for vertical groups.

Working with third-party vendors and external partners entails special vigilance for SOA compliance. Sometimes, if the level of SOA compliance is unsatisfactory, the EA members work with the vendor and make sure it meets with the compliance. Measures should be established to ensure that the vendor continues SOA compliance for subsequent releases of the product.

The governing body determines who owns (and therefore is accountable for) a given service. Tying up accountability with ownership minimizes ambiguity and clarifies who should be on a response team when things go wrong. This is especially important for shared or enterprise services (such as logging and security). In the case of composite services, the overall quality of service depends on each of the constituent services. Proper accountability and ownership aid in the smooth running of the system.

7. Strategic Alignment
The excitement of new technology can sometimes obscure fundamental business factors. The IT industry is littered with lots of empirical evidence of this phenomenon. With SOA, this can happen much more easily.

One of the important tasks of the EA team, therefore, is to take constant reality checks. The EA team and business experts need to ensure that the SOA strides are always in step with the strategic business objectives. If short-term business goals conflict with the long-term goals, the team should work with the senior management and find the optimal path.

This is easier said than done. Since exceptions will occur, having a process for correcting these issues later—depending on their urgency—is important. Otherwise, these exceptions can become orphaned from the business processes that prioritize the IT projects.

No matter how good the technical solution, losing sight of business value of the solution renders all the effort futile.

8. Communication
SOA is an enterprise-wide effort. In most companies, the EA team creates reusable services that other vertical development groups could leverage. Even in a mid-sized IT team, coordinating this effort is a big task. How do other groups know what services are available to leverage? Which components can be deprecated? What is the right way to use a service?

Providing a directory list of services is one possibility, but in most practical cases, this is not enough for human comprehension. There should be procedures and best practices to publish and consume information about available services. Some companies use portals for this purpose, while others use informal media like blogs and wikis. Whatever medium you prefer, disseminating the right information to the right people at right time is critical.

SOA involves significant reengineering, and everyone is resistant to change. In any organization—especially in the initial phases of implementation—expect some resistance. Skepticism is both good and bad for the organization. It is good because the skeptics act as counterweight to the enthusiasts. They help ensure that the SOA proponents are not being overly ambitious. But skepticism is also bad because it takes extra effort to convince the skeptics and push forward the rightful changes.

One of the pitfalls with Web services technology is that without good communication channels, the architecture could become worse. Take XML schemas. When teams work in isolation, they often create custom schemas that are not necessarily standards based. Last-minute pressures and tight deadlines act as catalysts for this phenomenon. This approach works out in the short term, but the "temporary" schemas quickly proliferate and lead to systems that are overly interdependent and brittle.

Constant communication is important. Companies have found success in conducting "all-hands" meetings at regular intervals. These meetings can also be used to communicate how the on-ramped projects are bringing good ROI and how other groups can leverage existing services to make their processes more efficient. This would help quicker adoption.

9. Support from Senior Management
Support from senior management is not only important for handling the resistance, but also for aspects like funding. SOA involves up-front investment. Unless IT has the top management's blessing, driving it forward becomes difficult. If we look at major SOA deployments in the industry, one common pattern is that the CEO is a strong believer in the value proposition of this paradigm and technology.

Senior management also should make sure that the SOA implementation is in line with the business needs and that it provides the promised ROI. Management guidance is of utmost importance in making sure that there is no disparity between long-term business goals and the direction of IT.

10. Constant Re-engineering
SOA is not a fire-and-forget model. It involves constant evolution and continuous re-engineering. In the initial few phases, it involves mostly either building new services or ramping up legacy applications on to SOA (using adapters). Horizontal services (or shared services) are also usually part of the initial phases. Once the basic services are in place, the next generation of services typically involves abstracting and refining business processes. All along the path we go through several iterations. For each iteration, the feedback is looped back into the service and refined further.

The reason for running the SOA gamut is to facilitate agility and adaptability for businesses in changing market conditions. As business keeps evolving, so will the services that support it.

Your SOA Investment
Going the SOA route is like saving for retirement—it's a long-term investment. You will probably experience some short-term pains, but it pays off in the long run. Doses of diligence, perseverance, discipline, and commitment are prerequisites. It takes a lot of honest introspection, resolve, and perseverance to discard bad habits in favor of better ones. SOA is not a panacea, but it definitely helps integrate your business-critical software. SOA is part technology and part business process refactoring. Thorough understanding of both of these will help ensure long-term success.

Acknowledgements: I would like to thank Mike McCoy, Vasavi Nemani, Madhusudan Nori, and Srinivasa Nemani for their feedback and suggestions.

About the Author
Anant Kadiyala specializes in Web services, SOA, and security. He has been helping companies strategize and implement SOA from the ground up. Anant also teaches Web Services Management and Web Services Security courses at UCSD Extension, San Diego. He can be reached at anant@SOAgurus.com.