All successful SOA implementations have one thing in common – Governance! Five years ago I shared my thoughts on my blog discussing the three major work streams required for successful SOA implementation. Establishing SOA Governance is one of them. The Open Group SOA Working Group have developed and published an SOA Governance Framework so that organizations adopting SOA can leverage best practice SOA Governance. Organizations will save time and money by adopting those governance best practices provided in the framework – i.e. do not have to re-invent the wheel.

You can download the framework from the Open Group site and also listen to the webinar recording on the topic – click on the link below.

Listen to the webinar recording here

The webinar provides an overview of the SOA Governance Framework and describes the value of adopting a standard method using the SOA Governance reference model and vitality model. You will gain an understanding of the framework processes, guiding principles, roles & responsibilities and benefits. You will also learn how to apply the framework to different organizational circumstances to enhance the usage and reuse of resources (people, technology, processes) and optimize business results.

Let me know what you think of the framework by commenting on this post.

Bye for now -:) Awel Dico

I came across a blog by Richard Veryard on architectural principles. He argues against the importance of SOA/EA principles by stating that “they may have some limited use, but they are not strong enough to bear the weight that is commonly put on them”. This is only true if the architect misunderstand what the architectural principles are or the architect is unable to link the principles with the reality and measure them. How can you measure the applicability or usefulness of architectural principles? I will share with you my 2 cents thought on this.
Enterprise Architecture, SOA or any other architecture, for that matter, needs to be guided by certain architectural principles. Where do these principles come from? Well, in the first place, the architectural principles must support the Enterprise architecture objectives/goals (which in turn must be driven by business objectives or strategies). That means the principles you put together or use to guide your architecture is not just from no where. They have linkages to the business objectives. For example, the business objective may be to be the best in customer service/satisfaction – customer centric. So, the Enterprise Architecture objectives must support that business objective. The principles must be developed to guide to achieving that business objective (customer first). Your architecture will then be guided by those specific principles.
There is a challnge though. Let’s say you have an architecture that is in compliance with the priniples. But, how do you know the architecture get implemented without losing the core priniples? How do you measure the effectiviness of your priniples? Unless you make sure that they are implemented and you are able to measure the effectiveness, those priniples may be disconnected from the reality (i.e. deployed business solution); in which case Richard’s argument may hold. Here is how you may measure:

1. Develop specific guidelines based on those principles. These guidelines are NOT abstract – rather they are concrete guidelines that the solution architects, developers, etc use. The developers may not understand the “abstract” principles. You need to translate the principles into concrete guidelines that they understand. To make sure that a single principle is properly followed, you may have to develop many clear and concrete guidelines that the developers use. By doing so, you are saying that “if those guidelines are followed and implemented accordingly, the principles are followed”.
2. Put together the metrics based on the compliance with those guidelines. Here is where you make sure that your principles are followed through compliance with those concrete guidelines. Here you need a governance process to assess such compliance. How do you measure the effectiveness of the principles? Well, your metrics need to include business related metrics in addition to low level metrics. If the business is happy – you know -:). Your principles are not effective if there are many “exceptions or problems” in complying with your guidelines. In which case you may have to review your principles (or the guidelines that represent them).

If the right principles are developed based on EA objectives (which support business objectives) and the implementation of those principles are followed down to the solution deployment through concrete guidelines (which are meausrable), then priniples are very important for any architecture initiative.

Awel Dico

It seems there is still some confusion out there as to what a “Service” means in the context of Service Oriented Architecture. Just recently I came across a discussion where “service” is being communicated as just a simple web services interface. That is why I thought of sharing here the definition I used in a presentation about SOA over a year ago. I used some concepts from OASIS SOA Reference Model and added other aspects of service that I think are important. The diagram below shows the anatomy of SOA service – what it really means when we say “Service” in the context of SOA. The usual common picture people use to communicate SOA based service is by using the triangle that shows service provider, service consumer and service broker or registry. What is more important and most of the time not clearly explained is the other view of the same triangle I overlay in the diagram below. That is, service has a description, service has implementation and service is governed. 

Service Anatomy - What Service means in SOA

In short, the following aspects of “service” are implied or need to be considered when one speaks of service in SOA:

• The usual high level view
  • Service has provider
  • Service has consumer
  • Service has a registry
• The other view which is more important is:
  • Service has description. This includes tasks that the provider has to worry about to meet the consumer need and how service can be used by the consumer (for the definition of these terms please refer to OASIS model linked above):
    • Contract
    • Interface
    • Reachability
    • Metrics
    • Functionality
  • Service has implementation. This includes tasks that the provider has to worry about to meet the consumer expectation of the capability of the service :
    • Design (you know how this is important too -:))
    • Code (you know … this is where the meat is -:))
    • Test (yeah.. you think this is not important? In most of the cases this is where the money is spent – cost. You must deal with on the strategy on how to test SOA services)
    • Package (yeah… you think this is not important? Think again… this is related to service versioning, migration, change management etc.. issues. You better have a good strategy on how to package the services implementation before they get deployed)
    • Deploy (considerations of performance, scalability, availability, and other operational needs)
    • Secured (this is missing from my previous diagram – but i think it is important aspect that must be considered)
    • Monitor (why? you must capture some metrics to evaluate if the service is of value to the business as expected. This metrics comes from the top level business service design because that is where they identified as important factors to capture for business use (mostly referred to as KPI). There are also operational metrics that are collected to meet operational requirements – e.g. capacity planning, trouble-shooting, etc…)
  • Service is governed. This includes tasks that the Enterprise Architecture has to worry about:
    • Standards (i.e. service may have to be compliant with some industry standards. Those standards are best specified by enterprise architecture)
    • Guidelines (i.e. enterprise architecture should have SOA principles from which some measurable guidelines can be derived. Services need to be compliant with these guidelines, otherwise the principle is considered not applied)
    • Compliance assessments (i.e. services should be assessed against the specified standards, guidelines or principles for compliance by the governance team)
    • Exceptions management (i.e. it is a reality that a given standrd or guideline may not be followed in under some circumstances. For example the cost of implementing the guideline or standard may not be justified at the moment; and as a result a non-standard approach may be used to implement the same capability required by the business. This is about weighing the pros and cons of such a decision and if allowed then the governance team need to manage that exception. The exception is normally temporary. However there are cases where the exceptions reveal a defect in the principles or architecture itself.  This is mostly the case if too many exception is granted as a result of violation of a given standard or guideline. In such situations, the architecture or principle need to be reviewed and the broken guideline need to be updated accordingly.)

 In the above discussion, I did not mention where the service we talked about came from – I mean its granularity, usefulness from business needs perspective, etc. I assumed that a “service” we talk about here is based on top level service architecture as a part of enterprise architecture. See my discussion on Service Oriented Enterprise Architecture Framework which puts the “SOA service” discussed here in a bigger context.

From end-to-end perspective, I think when we say “service” in the context of SOA, we need to understand the above aspects of a SOA service – its complete anatomy.

Awel Dico

 I discussed earlier that SOA and EA should not be seen as separate and competing initiatives. In this post I would like to share my thoughts on the characteristics of a successful SOA initiative. I will first discuss a very useful concept introduced in TOGAF 9. Previously I only discussed an overview of TOGAF 9 here without much detail. TOGAF 9 introduced a way of dealing with EA through architecture partitioning. There are three architecture types defined, namely, Strategic Architecture, Segment (or Domain) Architecture and Capability Architecture.  The relationship between these three architecture types is depicted on the following diagram (taken from TOGAF 9 document).

Source: The Open Group (TOGAF 9 - Architecture levels)

In one of my earlier posts I promised to share some of my thoughts on SOEAF – a service oriented enterprise architecture framework. I am putting together some details on this framework which I think has the potential to simplify delivery of SOA based solutions. Before I discuss the SOEAF, I would like to describe what it takes to do end-to-end SOA work. You may have your own thoughts on what is involved in an end-to-end SOA work. For some end to end SOA is just services and its implementation; and for others it is just the architecture part of it; and so on…. For me? End-to-end SOA contains five key elements. These are shown on the diagram below.

I will provide description of this here …

If you have been in discussions about Business Architecture and SOA, you may already know what I am writing about. During my discussions with many architects at the Open Group Architecture Practitioners conference in San Francisco, I noticed that the term Business Service has different meaning for Business Architect and someone designing SOA services. SOA service architects have their own definition for it. But is that what the business architect understands?  For the Business Architect, the business service is what the enterprise provides to its customers, such as Negotiate Sales, handle Customer Order services. The “negotiate sales service” is a business service that may include many tasks to be performed.

The goal of SOA architect is to take this business service produced by business architect and create a service architecture that fulfills the business service. For example, a “Customer Order Handling” business service may be a business process that is fulfilled (or realized) by orchestrating many SOA services. For example, Customer, Customer Interaction, Product, Product Offering and other related services are required to be orchestrated to implement the Customer Order Handling business service. These services are viewed by SOA service architect or designer as business services. The point I am trying to make here is that the term “Business service” is used by both Business Architect and SOA service Architect and that they mean different things. As a result, it is important to pay attention to the audience (Business Architect or SOA Architect) we are addressing when we use the term “business service” in an SOA projects.

Awel Dico, PhD

I have been chatting with colleagues from other organizations working on SOA or planning to do so. During those discussions I observed that many of the organizations approach SOA work from pure technology perspective. Did you lately get a chance to talk to Enterprise Service Bus (ESB) product vendors? For example, many vendors and consultants advise you to buy ESB if you need SOA. But is that really SOA? The answer is No. ESB and other technologies are useful and of value if and only if you know what to do with them. Yes, such technologies make your SOA implementation easier and support many of the SOA principles. However, I think any SOA initiative must address business needs. Business needs must drive business services identification; which in turn drives the need for technology services. I think SOA will fail, if SOA is viewed purely from technology perspective without much attention to business needs. Why would one invest in technology products (such as ESB) if the need for it is not justified from business services perspective?

Fortunately, this is no more a point of argument as it used to be (remember those discussions on top-down versus bottom-up approaches?). Today, many enterprises have already realized that the right approach to SOA is from business perspective. Such a realization is a good step; but not enough to guarantee SOA success. Actually many organizations seem to be struggling with how to approach the SOA work from business perspective. I observed two camps of enterprises. The first camp is one that has some activity with Enterprise Architecture framework and at the same time adopting SOA. The second camp is one that has no EA activity but still want to adopt SOA. The enterprise in the first camp has difficulty bringing together their EA work and SOA work. The worst case is for some of those enterprises EA and SOA are treated differently and the teams are separate. The goal of EA is to close/minimize the gap between business and technology by providing enterprise wide support for both business and technology groups through standards, guidelines, governance, etc. If such an EA activity doesn’t include SOA as one of the best practice architecture style, how would one expect success in SOA initiative?

To prevent SOA failure, the two camps should rethink their approach. I think enterprises in the first camp must integrate their EA and SOA work – these two should really be viewed as one. Those in the second camp may have to adopt two things – EA framework and SOA. I don’t think SOA will succeed without proper EA work in place.

There are few EA frameworks out there that can be enhanced to support SOA and has a potential to lead to a success. One of these is TOGAF. TOGAF helps the enterprise to do work on business architecture from which business services can be identified. It provides an architectural process by linking key architectures one after the other, i.e. Business architecture, information architecture and technology architecture. Even though TOGAF provides a mechanism for approaching architecture from business architecture and then to technology architecture, some enhancements are required to use TOGAF for SOA. A guide to help with such enhancement is under development by the Open Group SOA working group. In conclusion, if proper enhancement is done to TOGAF for SOA support, TOGAF can be at the rescue for the enterprise struggling to lift their head out of the architectural mess and do the right SOA. I will try to give some ideas on how TOGAF can be used for SOA work in my future postings. I will be presenting on this topic at the Open Group Architecture Practitioners conference in San Francisco on January 28, 2008.

Awel Dico, PhD

There are many varying definitions of service oriented architecture (SOA) out there. Some define an SOA as simple as collection of services and others equate it to simple exposition of application functionality as web services. As enterprises struggle to adapt a service oriented architecture approach, there are still confusions on the basic concepts- particularly around the terms “Web Service”, “service” and “SOA”. Below is my attempt to clarify concepts around these terms.

1. Web Service, Service and SOA – they are not the same

There is one common mistake out there – the thinking that service is simply exposing functionalities as web services. One cannot achieve service (as defined in SOA) by simply wrapping and exposing methods and functions via web services. All web services enabled applications are not “services” from SOA perspective. This is very dangerous mistake in that it can cost the enterprise lots of money and never get to the vision of SOA. We need to separate the two and understand the boundaries in order to achieve the promise of SOA.

SOA is about achieving agile business through services that are designed with certain principles in mind. It is about empowering the business with the ability to respond quickly and efficiently to change in business requirements and to leverage change for competitive advantage. Thus, at the heart of SOA are services designed for business flexibility.

Web Service is about establishing a standard connectivity to addresses how the services are accessed and consumed. Though there are many other technologies for use in SOA-based service access and connectivity, Web service is the preferred technology. This is because it is standard based, simple to implement and there is big investment in this technology by platform and tool vendors.

The bottom line is to separate SOA and Web Services right from the start. There is no commonality between the two from the fact that you can have SOA without Web Services. However, using Web Services in SOA implementations is a powerful approach that gives you both business flexibility and standard connectivity.

2. An SOA definition

Service Oriented Architecture is an architecture style that helps in realization of business agility by closing a gap between business and IT environments through enablement of:

• service based design of business processes and IT resources,
• flexible infrastructure in which services are deployed, managed, secured, monitored, and registered and discovered, and 
• enterprise wide governance based on polices, processes, and guidelines.

This definition points out three important parts of service oriented architecture. The first part (i.e. service based design) is an SOA approach for designing and implementing business applications. Business services built at right granular level can be assembled to build complete business applications that implement business process. Service granularity is something relative and it should be relevant to the service consumer. That is where the service interface design becomes critical in that the right service granularity can only be achieved by understanding the consumers and the business process.

In addition to business aspect of services, within SOA, the IT assets and information are also viewed as services. What this means is that each element of business process and underlying IT resources are treated as services that can be reused and orchestrated to address changing business requirements.

The second part of SOA is the infrastructure for deploying, running, managing and securing the services. Even with good service design and implementation, if proper infrastructure is not in place the benefits promised by SOA approach may not be realized. It is important to note that no single SOA infrastructure blueprint that fits all enterprise; and each enterprise may have simple or complex infrastructure based on its business requirements.

The third part is enterprise wide governance process to ensure that SOA principles are met through compliance with enterprise guidelines, standards, and polices. Establishing good governance process is a key to maximize the benefits and reduce risks while the enterprise incrementally moves to full SOA based implementations.

With flexible infrastructure and SOA governance in place, an enterprise can close the gap between IT and business by designing and managing business and IT capabilities as services, and reusing and assembling them as required implementing business requirements. This in turn gives the enterprise ability to accomplish business goals more easily and economically in less time.

Awel Dico, PhD

After you have done your pre-work (i.e. made a business case and decided to invest in SOA), where do you start and how do you organize your work to move forward? That is where most of the time is spent with current SOA pursuers. So, how can you get started?

Any SOA initiative need to first understand and prepare to work simultaneously in three major fronts. These are:

•  Exposing business functions as services (building your assets)
• Implementing SOA infrastructure
• Establishing SOA governance

From the surface, this seems simple and may be overlooked leading to unexpected cost and disappointments. Such a mistake arises when you fail to understand the requirements for all of these tasks; and rather think implementing SOA is simply wrapping functions with Web Services interfaces. A successful SOA implementation realizes these three major fronts and progress about implementing them incrementally to meet current and evolving business needs.

Properly implemented SOA allows business to respond quickly and efficiently to changes in business requirements, i.e. business agility. This gives the business the ability to leverage change for competitive advantage. Achieving this requires some discipline and changes in current IT approach to business application development. Currently, business and IT groups within one organization communicate in a vague way and disconnected when it comes to understanding business dynamics. Business requirements do not flow smoothly and unaltered from business to technology implementations. As a consequence changing the current systems (applications) to support changes in business needs (or to accommodate new requirements) is too costly, if not impossible.

SOA approach requires three inter-related elements, namely, business requirement, service and technology – to work together. Understanding the relationship between these three elements is crucial for achieving business agility. That is, business requirements must drive the services and the services drive technology. Services address the needs of the business and technology provides a means for implementing and deploying those services.

The other part is enterprise wide governance process to ensure that SOA principles are met through compliance with enterprise guidelines, standards and polices. Establishing good governance process is a key to maximize the benefits and reduce risks while the enterprise incrementally moves to full SOA based implementations.

This SOA approach is in agreement with the TOGAF framework . The three elements introduced above maps to the first three architectures in TOGAF framework; namely, business architecture, Information architecture and technology architecture. Business architecture deals with business requirements and business processes. Information architecture addresses the design of services and data; while Technology architecture addresses the infrastructure for services development and deployment. If you are familiar with TOGAF framework, Information architecture consists of Data and Application architectures. There is no disagreement if you understand that, in SOA approach, applications are group of services and data is what those service manipulate.

Going back to the three major fronts for SOA implementation introduced above, the following can be summarized:

•  Exposing business functions as services involves work in Business and Information architectures
• Infrastructure implementation involves work in Technology Architecture 
• Governance process involves ensuring SOA principles are met through compliance with enterprise guidelines, standards, and polices.

Awel Dico, PhD