/* Google Analytics ----------------------------------------------- */

Tuesday, July 7, 2009

SOA for real

I'm IT and I will be honored to let the business build the company SOA program.
Let consulting companies help you work on your business processes, do your powerpoints slides or BPMN diagrams, make the right ROI spreadsheets. As everybody says, SOA is about strategy, business vision, blah blah blah... Then, at some time, you will come back to the reality, your IS and then you will discover that you can not do your SOA in less than several years.

In the meantime, you have to make the IT shop rolling and try to make it more flexible, agile, elastic. IT overhaul is a process much more complicated than what consulting companies said. So lets ignore philosophical debate about birth and death of SOA.
Let's move to my basic stupid learned from the ground recommendations.

Organization maturity is key to build your program

I keep thinking that to beging doing SOA you need to meet some pre-conditions:
  1. Your organization should be mature enough. For me at least CMMi level 3.
  2. You need to know what are your business value chains and where you earn money
  3. You need to know how you will do billing and charge back. Whatever you do at some time you will have the question: why do I have to pay for all others? So having a clear billing plan is part of the governance (and good for financial audit).
  4. You need to know how to negotiate, implement or verify Service Level Agreement.
  5. You need to be able to cope with the organizational changes and make all teams able to not think product (silo) anymore but services (offered, required).
If you do not meet those pre-conditions, no worry, you can do a good job. But do not talk about SOA. Use the term IT overhaul or Service Based Architecture or IT/Business Shared Services.

Always begin by the data quality and categorization

The first step is to categorize, create, gather, sometimes aggregate, cleanse and organize your data. For each category of data define the business rules to be applied and their lifecycle.

If you have legacy systems holding data, you may have also to think of implementing a mapping service to cross link your data silo. This is true for all types of data categories listed below. This could be done statically using and ETL or dynamically using an EII solution. This mapping service should be available through Web Service, of course.

Note that some tools were build especially to solve data management issues, and especially the semantic mismatch between data. I was really impressed by Progress DataExtend SI (but in fact they are offering a full suite of tools) and I still love the Informatica Platform. Try not to build your own, it is not cost effective in the long run, nevertheless it could be used as a tactical solution between migration phases.

Try to categorize data in a way that will benefit you Service vision. For example:
  • Master data - cleanse and federate (or centralize) those data. Create management rules for them and define master data stewardship. One good product on the market to help you doing that is EBX.Platform from orchestra network.
  • Identity data - Identity and access management will be key in your Service approach. So define asap how you will cope with security compliance and security processes. Here we can recommend to use OpenSSO from Sun (especially the Web service federation module). For Microsoft shops, as usual, you will find everything you need in their portfolio (here). The main standards to use are WS-security, WS-Federation (Microsoft), SAML V2 (all except Microsoft for now, but this will change). Most advanced users will also use openID, Oauth, WS-Addressing.
  • Transactional data - Should reuse Master data and identity data to create traceable and coherent transactions. If the data coherence could not be enforced, then some reconciliation processes should be put in place.
  • BI Data - In general fed by an ETL and stored as a big denormalized DBMS. All business related data should finish their live here.
  • Configuration data - Always forgetted ... Remember that you want to offer services to your multiple clients. So for each of your client, you will have to store some attributes to specialize the services. Where do you store this information?

Unlock your data silo

When you do SOA (or any other name you can use), you need first to free the data from their current application silo. My approach was to ask each product or application team to build what I called "basic services".

Here, they have to define their business objects to be exchanged (using their own technical format, but if possible reusing the same business terminology to ease semantic mapping later). I love to let developer do their job with the tools they like. I only request SLA to be met. So you can use all the technology you want.

I like particularly the new Sun Java Metro, Apache CXF, Spring WS and SOAFACEs in the J2SE world, Symfony in PHP world and WCF in Microsoft World. Anyway any server based Java tool vendor is offering its SOA suite (WSO2, Jboss, OW2, IBM, Oracle, sopera).

In order to avoid the Bazaar, without creating a Cathedral, you need to create some basic architecture and technical recommendations at design time. You also need a tool to validate them, we used Parasoft SOATest. We decided to create rules at the XML and WSDL layer. That's why we did forbid usage of REST style for external communications. But again it is a question of maturity. Create your standards, and find a way to validate them automatically.

Here are some examples of rules we are looking for:
  • The naming convention associated with the document/literal wrapped pattern should be used.
  • All the operations and their input and output parameters have to be documented using the element. The documentation has to describe what the operation do, what are its typical use cases and what are its pre and post-conditions.
  • All data types definitions have to use references to external schemas.
    The element has to be used for that. Services that use the same data types should use the same schema.
  • Errors have to be reported to the client using the standard SOAP fault message. The faultcode element has to include the type of error : Technical or Business.
    The possible errors that can be returned by an operation has to be documented in the WSDL file.
Son during this first installment we understood than depending organization maturity SOA is the right term or not. Then we looked at how to prepare the company for SOA beginning to work on data, their categorization and their packaging as basic services. The objective here is to offer data as a service.

I will continue to describe my journey into SOA in several coming posts. Stay tuned.