All successful SOA implementations have one thing in common – Governance! Five years ago I shared my thoughts on my blog discussing the three major work streams required for successful SOA implementation. Establishing SOA Governance is one of them. The Open Group SOA Working Group have developed and published an SOA Governance Framework so that organizations adopting SOA can leverage best practice SOA Governance. Organizations will save time and money by adopting those governance best practices provided in the framework – i.e. do not have to re-invent the wheel.

You can download the framework from the Open Group site and also listen to the webinar recording on the topic – click on the link below.

Listen to the webinar recording here

The webinar provides an overview of the SOA Governance Framework and describes the value of adopting a standard method using the SOA Governance reference model and vitality model. You will gain an understanding of the framework processes, guiding principles, roles & responsibilities and benefits. You will also learn how to apply the framework to different organizational circumstances to enhance the usage and reuse of resources (people, technology, processes) and optimize business results.

Let me know what you think of the framework by commenting on this post.

Bye for now -:) Awel Dico

I came across a blog by Richard Veryard on architectural principles. He argues against the importance of SOA/EA principles by stating that “they may have some limited use, but they are not strong enough to bear the weight that is commonly put on them”. This is only true if the architect misunderstand what the architectural principles are or the architect is unable to link the principles with the reality and measure them. How can you measure the applicability or usefulness of architectural principles? I will share with you my 2 cents thought on this.
Enterprise Architecture, SOA or any other architecture, for that matter, needs to be guided by certain architectural principles. Where do these principles come from? Well, in the first place, the architectural principles must support the Enterprise architecture objectives/goals (which in turn must be driven by business objectives or strategies). That means the principles you put together or use to guide your architecture is not just from no where. They have linkages to the business objectives. For example, the business objective may be to be the best in customer service/satisfaction – customer centric. So, the Enterprise Architecture objectives must support that business objective. The principles must be developed to guide to achieving that business objective (customer first). Your architecture will then be guided by those specific principles.
There is a challnge though. Let’s say you have an architecture that is in compliance with the priniples. But, how do you know the architecture get implemented without losing the core priniples? How do you measure the effectiviness of your priniples? Unless you make sure that they are implemented and you are able to measure the effectiveness, those priniples may be disconnected from the reality (i.e. deployed business solution); in which case Richard’s argument may hold. Here is how you may measure:

1. Develop specific guidelines based on those principles. These guidelines are NOT abstract – rather they are concrete guidelines that the solution architects, developers, etc use. The developers may not understand the “abstract” principles. You need to translate the principles into concrete guidelines that they understand. To make sure that a single principle is properly followed, you may have to develop many clear and concrete guidelines that the developers use. By doing so, you are saying that “if those guidelines are followed and implemented accordingly, the principles are followed”.
2. Put together the metrics based on the compliance with those guidelines. Here is where you make sure that your principles are followed through compliance with those concrete guidelines. Here you need a governance process to assess such compliance. How do you measure the effectiveness of the principles? Well, your metrics need to include business related metrics in addition to low level metrics. If the business is happy – you know -:). Your principles are not effective if there are many “exceptions or problems” in complying with your guidelines. In which case you may have to review your principles (or the guidelines that represent them).

If the right principles are developed based on EA objectives (which support business objectives) and the implementation of those principles are followed down to the solution deployment through concrete guidelines (which are meausrable), then priniples are very important for any architecture initiative.

Awel Dico

It is kind of common now to associate SOA with Service Reuse. Yes – reuse is a very good thing – why spend much more money and time if you can reuse. If a service is reusable, in addition to contributing to business agility, it is also “green service” – environment friendly as well. The question is – How exactly is service reuse achieved? Many have been disappointed by the results they see in their SOA initiatives because their primary measure (metrics) for their initiative is Reuse. They lack clear understanding of what the service reuse is and how to get there. They assumed that service reuse is a given thing once you initiate SOA. The reality speaks different language from the assumptions. I used to preach to my team service reuse is one of the primary goal and want the team to deliver reusable service from the start. Overtime, I tracked the services deployed and only few services had the potential for reuse. Why? – I tried to find reasons why Service reuse is such a very good thing but wrapped by illusion.

Some of the reasons I listed include: Service interface and data not propoerly designed, service components not well designed, service level not properly incorporated, requirements not understood creating issue with service granularity and functional limitation, etc… You know what? … these and many more could be a reason to the problem. But, is that really a problem? Can one avoid these pitfalls by doing everything perfect and achieve the service reuse needed? The answer is NO! Yes, for simple cases where you can do perfectly and get reusable service – say across multi-business domains or multi-channels. For large organizations with many lines of businesses (or domains) it is not easy to get reusable services for cross business domain reuse. I came to understand why Service Reuse is such an illusion in many cases and what the organizations have to do to achieve reuse.

Here is my learning in the field -

1. First recognize that achieving (or getting to) service reuse in SOA is a process – i.e. a service may have to go through some iterative cycles to be close to perfect for reuse across multi-channel and multi-business-domain. It is proved to be hard to arrive at reusable service by just one shot! For large enterprise, this is mainly because of the lack of proper understanding of the complete requirements across the lines of businesses. This is the case today as the projects are mainly driven by lines of businesses bounded by cost and time. You may argue that enterprise can invest in completing cross business requirement understanding and absorb the cost for making service reusable. If so, you may have to add a “market factor” to this – the business line has urgent need to enhance its business capability to meet customers’ demand and cannot wait horizontal enterprise requirement analysis.
2. Second, recognize the importance of good service design to achieve enterprise service reuse. The evolution of your service towards reuse ( or in other words, your path and speed to reusable service) depends on the quality of your initial service design. Taking the #1 above into consideration, if your service design is not done well to position for further evolution of the service for reuse, you will have to redesign the service or end-up with duplicate non-reusable services across the enterprise – defeating the overtime “green service” vision of SOA.
3. Third, understand that enterprise service reuse requires strong SOA governance to facilitate the service reuse process. This goes into Service portfolio and service life-cycle management for the enterprise and more…
4. Fourth, recognize that SOA infrastructure can play a major role in facilitating enterprise service reuse … more on this in my next blog postings

I will try to share some detail toughts on this later… Bye for now.

It seems there is still some confusion out there as to what a “Service” means in the context of Service Oriented Architecture. Just recently I came across a discussion where “service” is being communicated as just a simple web services interface. That is why I thought of sharing here the definition I used in a presentation about SOA over a year ago. I used some concepts from OASIS SOA Reference Model and added other aspects of service that I think are important. The diagram below shows the anatomy of SOA service – what it really means when we say “Service” in the context of SOA. The usual common picture people use to communicate SOA based service is by using the triangle that shows service provider, service consumer and service broker or registry. What is more important and most of the time not clearly explained is the other view of the same triangle I overlay in the diagram below. That is, service has a description, service has implementation and service is governed. 

Service Anatomy - What Service means in SOA

In short, the following aspects of “service” are implied or need to be considered when one speaks of service in SOA:

• The usual high level view
  • Service has provider
  • Service has consumer
  • Service has a registry
• The other view which is more important is:
  • Service has description. This includes tasks that the provider has to worry about to meet the consumer need and how service can be used by the consumer (for the definition of these terms please refer to OASIS model linked above):
    • Contract
    • Interface
    • Reachability
    • Metrics
    • Functionality
  • Service has implementation. This includes tasks that the provider has to worry about to meet the consumer expectation of the capability of the service :
    • Design (you know how this is important too -:))
    • Code (you know … this is where the meat is -:))
    • Test (yeah.. you think this is not important? In most of the cases this is where the money is spent – cost. You must deal with on the strategy on how to test SOA services)
    • Package (yeah… you think this is not important? Think again… this is related to service versioning, migration, change management etc.. issues. You better have a good strategy on how to package the services implementation before they get deployed)
    • Deploy (considerations of performance, scalability, availability, and other operational needs)
    • Secured (this is missing from my previous diagram – but i think it is important aspect that must be considered)
    • Monitor (why? you must capture some metrics to evaluate if the service is of value to the business as expected. This metrics comes from the top level business service design because that is where they identified as important factors to capture for business use (mostly referred to as KPI). There are also operational metrics that are collected to meet operational requirements – e.g. capacity planning, trouble-shooting, etc…)
  • Service is governed. This includes tasks that the Enterprise Architecture has to worry about:
    • Standards (i.e. service may have to be compliant with some industry standards. Those standards are best specified by enterprise architecture)
    • Guidelines (i.e. enterprise architecture should have SOA principles from which some measurable guidelines can be derived. Services need to be compliant with these guidelines, otherwise the principle is considered not applied)
    • Compliance assessments (i.e. services should be assessed against the specified standards, guidelines or principles for compliance by the governance team)
    • Exceptions management (i.e. it is a reality that a given standrd or guideline may not be followed in under some circumstances. For example the cost of implementing the guideline or standard may not be justified at the moment; and as a result a non-standard approach may be used to implement the same capability required by the business. This is about weighing the pros and cons of such a decision and if allowed then the governance team need to manage that exception. The exception is normally temporary. However there are cases where the exceptions reveal a defect in the principles or architecture itself.  This is mostly the case if too many exception is granted as a result of violation of a given standard or guideline. In such situations, the architecture or principle need to be reviewed and the broken guideline need to be updated accordingly.)

 In the above discussion, I did not mention where the service we talked about came from – I mean its granularity, usefulness from business needs perspective, etc. I assumed that a “service” we talk about here is based on top level service architecture as a part of enterprise architecture. See my discussion on Service Oriented Enterprise Architecture Framework which puts the “SOA service” discussed here in a bigger context.

From end-to-end perspective, I think when we say “service” in the context of SOA, we need to understand the above aspects of a SOA service – its complete anatomy.

Awel Dico

In part 2 of my blog entry I discussed the five elements of end-to-end SOA work. The focus of that discussion was to introduce and define those elements I think are required for successful SOA initiatives. The diagram I show there is good for small organizations that are not divided into business domains. Large organizations or enterprises have usually more than one independently operated domain area. For such organizations that diagram need to include domain architecture as a part of enterprise architecture.  In this article (part III) I want to discuss the Service Oriented Enterprise Architecture Framework (SOEAF) that can be used in large enterprises that can be divided into multiple business domains.

Below I want to share my thoughts on some details and link to the concept of architecture partitioning approach introduced in TOGAF 9.  I gave an overview of the three architecture levels introduced in TOGAF 9 earlier. The three architecture levels are: Strategic Architecture, Segment (or Domain) Architecture and Capability Architecture. I think there is no misinterpretation of what the first two architecture level types are. The third level (i.e. Capability Architecture) has generated some interesting discussions. For some it could be defined as the architecture activity where capabilities are identified and their dependencies are defined to address the domain business goals. For others it could be interpreted as a commonly know type of solution architecture. I am not going to conclude one is wrong and the other is correct. Regardless of the different interpretations of what it can be, for the purpose of this discussion I interpret the capability architecture as the following: -  the architecture that is done to identify lower level business and IT capabilities within a domain (or segment) in order to optimally support the higher level business capabilities identified in strategic business architecture.

The assumption here is that an enterprise can be grouped into two or more domains/segments and that each domain/segment has one or more capabilities identified and defined to address the domain/segment goals while supporting the enterprise business strategy. All the domain (or segment) architectures together they make up enterprise architecture. The strategic architecture provides direction and guidance to the various domain architectures so that all the domain architecture deliverables can be viewed as a part of the whole enterprise architecture. However, not all domain/segment delivered capabilities are reusable across the enterprise – some may be just domain specific capabilities (or services) while contributing to the over all enterprise business objectives. Some business and IT capabilities can be common across the various domains and as such they can be positioned for reuse at the enterprise level.

Service Oriented Enterprise Architecture Framework (SOEAF)

The diagram above shows the complete picture of Service Oriented Enterprise Architecture Framework (SOEAF). SOEAF is the customizable enterprise framework that is based on TOGAF and SOA approach. SOEAF includes four levels of architecture as shown in the diagram. These are Strategic, Domain, Capability and Solution architectures. These architecture levels are NOT disconnected; rather they are connected from top strategic architecture down to solution architecture – the top architectures giving guidance to the lower level architectures to support the enterprise business strategy. This does not mean that the architectures cannot be done independently. It becomes more specific and concrete as one goes from strategic architecture down to solution architecture after which the implementation is done. In the diagram I only showed details of strategic architecture and solution architecture. Similar details can be shown for Domain and Capability architectures.

The other key component of SOEAF is the SOA Governance and Change management part. This uses the principles, guidelines, standards, and reference models to perform compliance assessment of the various architecture and implementation artifacts. The principles, standards and guidelines are developed at the strategic enterprise level in support of Enterprise Architecture objectives which in turn supports the enterprise business strategy. Thus, the compliance assessment of the deliverables helps the ongoing subsequent incremental deliverables to be aligned with EA objectives and reduce (or remove) unexpected cost and business risks. In addition governance activity encourages reuse and facilitates reuse by providing the necessary enterprise and domain level processes and tools. Some of the key tools of governance are the Enterprise Architecture Repository and SOA Service Registry tools. The various architecture artifacts can be published in an architecture repository based on some enterprise acceptable taxonomy. In addition, the SOA services (both business and technical services) should be published in SOA Service Registry tool to facilitate service reuse. In general, reuse is not as easy as saying it. It is one of the hardest things to achieve even with governance in place. However, I think proper governance processes and tools coupled with skilled resource can help achieve reuse of both architecture and services over time. Architecture reuse is different from SOA service reuse; but they aim for one goal – reduce the cost and time to implementing business capabilities and improve “time to value” of initiatives. If proper SOA governance body, equipped with the necessary tools and processes, is not established the enterprise SOA initiative may not deliver any value to the business as expected.

SOEAF also includes end-to-end security architecture which addresses authentication and access control to deployed business services, intrusion detection, privacy, logging and auditing. I think security should be externalized from SOA based business service implementations. Authentication and authorization to business services can be handled external to the services that are being protected – i.e. services should not be aware of those security functions. The only time the incoming security information is needed by business services is when the business service has to pass some user security context to downstream legacy applications. This is because those legacy applications may expect user information (including password) in the message body. Even then I think there are ways to add those required information to the outgoing message with no intervention from business service implementation if proper SOA infrastructure (e.g. Enterprise Service Bus) is deployed.

For SOEAF to be complete, it must be supported by the following:

• Teams (people): SOEAF involves multiple teams working together at different levels. For example, EA, Domain architecture, Governance, and development and operations/Infrastructure teams are the main teams.
• Processes: SOEAF is supported by three main process types. These include: architectural process (i.e. TOGAF ADM), Governance processes, and SDLC process.
• Tools: Each team mentioned above needs proper tools to do the job.

Finally, SOEAF must include metrics for measuring the value of Enterprise Architecture to the business. As the various architectures are done by the various teams using SOEAF, some key metrics need to be collected at some key governance points. Example of such a metrics is the collection of service reuse, number of exceptions granted, infrastructure reuse, etc.

It is not possible to give a complete detail about SOEAF in one article. Please let me know if you think this type of framework is useful and let me know the part you want me to discuss in some detail here. Thanks for reading – I hope you have enjoyed your stay.

Awel Dico, PhD

 I discussed earlier that SOA and EA should not be seen as separate and competing initiatives. In this post I would like to share my thoughts on the characteristics of a successful SOA initiative. I will first discuss a very useful concept introduced in TOGAF 9. Previously I only discussed an overview of TOGAF 9 here without much detail. TOGAF 9 introduced a way of dealing with EA through architecture partitioning. There are three architecture types defined, namely, Strategic Architecture, Segment (or Domain) Architecture and Capability Architecture.  The relationship between these three architecture types is depicted on the following diagram (taken from TOGAF 9 document).

Source: The Open Group (TOGAF 9 - Architecture levels)

As a member of the Open Group SOA working group, I have been working on SOA-TOGAF practical guide project as a co-chair. The objective of the project is to produce a practical guide that enterprise architects would use to deliver SOA using TOGAF. Well – you got the feeling… how hard this work is. One needs to understand SOA and TOGAF not only from theoretical perspective, but also practical aspect of it. For the success of this project, the understanding of both SOA and TOGAF is critical. The scope of this project is not to merge TOGAF and SOA and rewrite TOGAF. The guide will help the enterprise architects to understand and make the necessary adjustments to TOGAF in such a way that TOGAF can be used for SOA initiatives. The team is working hard on this project and lots of interesting discussions are going on. All the discussions I had on this subject make me think of the importance of bringing SOA and TOGAF together. What this means is that, given the acceptance of SOA in industry, SOA must really get integrated into EA through major enhancement of EA frameworks and methodologies. Particularly, the best of TOGAF and the best of SOA practices should come together to create one common enterprise architecture framework. I refer to this new framework as Service Oriented Enterprise Architecture Framework (SOEAF). I will describe this in some detail soon.

Some Clarification update below:

I just want to clarify what I meant by brining SOA and TOGAF together. This doesn’t mean changing TOGAF to accommodate SOA. TOGAF is very good as it is a generic architecture framework that supports various architectural styles. SOA is just one architectural style. What I meant by brining the two together is applicable to organizations adopting both TOGAF and SOA. Those organizations benefit most if they can bring the strength of the two (SOA and TOGAF) to a common architectural framework – which I referred to as SOEAF.

Awel Dico, PhD

There are many varying definitions of service oriented architecture (SOA) out there. Some define an SOA as simple as collection of services and others equate it to simple exposition of application functionality as web services. As enterprises struggle to adapt a service oriented architecture approach, there are still confusions on the basic concepts- particularly around the terms “Web Service”, “service” and “SOA”. Below is my attempt to clarify concepts around these terms.

1. Web Service, Service and SOA – they are not the same

There is one common mistake out there – the thinking that service is simply exposing functionalities as web services. One cannot achieve service (as defined in SOA) by simply wrapping and exposing methods and functions via web services. All web services enabled applications are not “services” from SOA perspective. This is very dangerous mistake in that it can cost the enterprise lots of money and never get to the vision of SOA. We need to separate the two and understand the boundaries in order to achieve the promise of SOA.

SOA is about achieving agile business through services that are designed with certain principles in mind. It is about empowering the business with the ability to respond quickly and efficiently to change in business requirements and to leverage change for competitive advantage. Thus, at the heart of SOA are services designed for business flexibility.

Web Service is about establishing a standard connectivity to addresses how the services are accessed and consumed. Though there are many other technologies for use in SOA-based service access and connectivity, Web service is the preferred technology. This is because it is standard based, simple to implement and there is big investment in this technology by platform and tool vendors.

The bottom line is to separate SOA and Web Services right from the start. There is no commonality between the two from the fact that you can have SOA without Web Services. However, using Web Services in SOA implementations is a powerful approach that gives you both business flexibility and standard connectivity.

2. An SOA definition

Service Oriented Architecture is an architecture style that helps in realization of business agility by closing a gap between business and IT environments through enablement of:

• service based design of business processes and IT resources,
• flexible infrastructure in which services are deployed, managed, secured, monitored, and registered and discovered, and 
• enterprise wide governance based on polices, processes, and guidelines.

This definition points out three important parts of service oriented architecture. The first part (i.e. service based design) is an SOA approach for designing and implementing business applications. Business services built at right granular level can be assembled to build complete business applications that implement business process. Service granularity is something relative and it should be relevant to the service consumer. That is where the service interface design becomes critical in that the right service granularity can only be achieved by understanding the consumers and the business process.

In addition to business aspect of services, within SOA, the IT assets and information are also viewed as services. What this means is that each element of business process and underlying IT resources are treated as services that can be reused and orchestrated to address changing business requirements.

The second part of SOA is the infrastructure for deploying, running, managing and securing the services. Even with good service design and implementation, if proper infrastructure is not in place the benefits promised by SOA approach may not be realized. It is important to note that no single SOA infrastructure blueprint that fits all enterprise; and each enterprise may have simple or complex infrastructure based on its business requirements.

The third part is enterprise wide governance process to ensure that SOA principles are met through compliance with enterprise guidelines, standards, and polices. Establishing good governance process is a key to maximize the benefits and reduce risks while the enterprise incrementally moves to full SOA based implementations.

With flexible infrastructure and SOA governance in place, an enterprise can close the gap between IT and business by designing and managing business and IT capabilities as services, and reusing and assembling them as required implementing business requirements. This in turn gives the enterprise ability to accomplish business goals more easily and economically in less time.

Awel Dico, PhD

After you have done your pre-work (i.e. made a business case and decided to invest in SOA), where do you start and how do you organize your work to move forward? That is where most of the time is spent with current SOA pursuers. So, how can you get started?

Any SOA initiative need to first understand and prepare to work simultaneously in three major fronts. These are:

•  Exposing business functions as services (building your assets)
• Implementing SOA infrastructure
• Establishing SOA governance

From the surface, this seems simple and may be overlooked leading to unexpected cost and disappointments. Such a mistake arises when you fail to understand the requirements for all of these tasks; and rather think implementing SOA is simply wrapping functions with Web Services interfaces. A successful SOA implementation realizes these three major fronts and progress about implementing them incrementally to meet current and evolving business needs.

Properly implemented SOA allows business to respond quickly and efficiently to changes in business requirements, i.e. business agility. This gives the business the ability to leverage change for competitive advantage. Achieving this requires some discipline and changes in current IT approach to business application development. Currently, business and IT groups within one organization communicate in a vague way and disconnected when it comes to understanding business dynamics. Business requirements do not flow smoothly and unaltered from business to technology implementations. As a consequence changing the current systems (applications) to support changes in business needs (or to accommodate new requirements) is too costly, if not impossible.

SOA approach requires three inter-related elements, namely, business requirement, service and technology – to work together. Understanding the relationship between these three elements is crucial for achieving business agility. That is, business requirements must drive the services and the services drive technology. Services address the needs of the business and technology provides a means for implementing and deploying those services.

The other part is enterprise wide governance process to ensure that SOA principles are met through compliance with enterprise guidelines, standards and polices. Establishing good governance process is a key to maximize the benefits and reduce risks while the enterprise incrementally moves to full SOA based implementations.

This SOA approach is in agreement with the TOGAF framework . The three elements introduced above maps to the first three architectures in TOGAF framework; namely, business architecture, Information architecture and technology architecture. Business architecture deals with business requirements and business processes. Information architecture addresses the design of services and data; while Technology architecture addresses the infrastructure for services development and deployment. If you are familiar with TOGAF framework, Information architecture consists of Data and Application architectures. There is no disagreement if you understand that, in SOA approach, applications are group of services and data is what those service manipulate.

Going back to the three major fronts for SOA implementation introduced above, the following can be summarized:

•  Exposing business functions as services involves work in Business and Information architectures
• Infrastructure implementation involves work in Technology Architecture 
• Governance process involves ensuring SOA principles are met through compliance with enterprise guidelines, standards, and polices.

Awel Dico, PhD