To SOA or Not To SOA

While we are used to developing systems of software, we are fare from confident with software-solutions consisting of multiple systems.

This is probably one of the reasons why large enterprises where the first to embrace SOA - they have lots of systems to integrate. This is probably also the reason why conventional system-development has been slow at embracing SOA - they do not have the integration need - so some COTS-system-producers are even reluctant to provide SOA-ready interfaces to their stand-alone system. What they do not realise; they need a SOA-ready interface just as much as they need their own GUI, otherwise large enterprises will not bye their otherwise great stand-alone-system.

Why was SOA invented? As enterprises grow and information systems where added for each business process to control, their systems portfolio also grow. Then enterprises wanted to have integrated applications or at least integrated data in their stand-alone systems, so they started doing point-to-point integration based on some narrow focused need for specific data (or even worse, they duplicated data in different systems). Sponsors where often owners of individual business processes with no current need nor budget to integrate all the systems. The results; in a couple of years distinct departments had sponsored a point-to-point integration-solution. Their unplanned point-to-point integration-solution turned out to be tremendously expensive to maintain. Especially as the systems continued to evolve, be upgraded, and replaced. Somehow all the point-to-point data-integrations just remained, and the enterprises had no sponsor ready to replace all of their expensive point-to-point data-integrations, and what could they replace it with? There where no COTS ready to help replace their data-integrations. Okay, some COTS did exist a couple of years ago, often unknown. Did you know Ascential Software or DataPower - before IBM bought them?

Finally the enterprises point-to-point data-integrations became a transitive closure of system-connections. Here is an example with just 7 distinct systems, having (n-1)+(n-2)+..1 = (n-1)*(½*n) connections or 21 two-way connections. This data-integration example could include 42 distinct system-interfaces where each system-interface could have its own data-transformation component:

Then enterprises data-integration-solution started to cost more than the individual systems; finally management could see the business case for an improved data-integration architecture. Luckily the COTS now had the proper brand and a sales force to sell them. The international business machine behind the proper brand eyed a large opportunity (or threat if you will) in their share of existing enterprise data-integration budgets.

If enterprises are lucky, the above example will look like this in a couple of years:

This could include one new domain-model of the business, 7 new data-transformation components, and one expensive Enterprise Service Bus (ESB). This new integrated data-exchange-solution will save the enterprises a great deal of money, and as an added bonus let them integrate new systems in no-time. Some also believe this will allow enterprises to rethink their business because the ESB allows them to build and deploy new business processes - in no-time. Well, let us see the ESB's capability before we give them too much credit.

On sale: A Lean and Agile Black Belt Level 5 Scrum - with a Cristal Clear touch of eXtreme

I was unable to re-find the fine article by a very accredited software process professional. The article I am seeking is the one about re-inventing the process model over and over again.

Anyway once you have a broken process, you will never be fully comfortable with performing it. Many people ignore the hiccups and the annoyances, however some one might try to improve on the process. This is fine. Now the trouble starts. The one who actually succeeds to improve the process, thinks the new super process is globally applicable, writes it down in a book, sells some books and talks about this new super process.

Until now, no real harm has been done, except for the $$$ and hours your company and you has spent on the books and talks. The real trouble starts when you and your company does not understand the context of the new super process, and you start to apply the new super process in the context of your own company.

The trouble is, that a model is not a one-size-fits-all. A model is an ideal and filtered image of some past reality, hopefully based on some empiric observations. Thus a model is always some kind of interpretation of the real world (what ever that is).

Once you start to run your own copy of the new super process model, it might act up, especially if it is a complex model. And surely a few feedback loops does not help, neither does any kind of external noise - periodic or not. This might be why science like simple models over complex models.

So what you realise is, that your copy of the new super process does not perform better than your own old process. But once in, the new "super" process cannot be uninstalled. You can try to erase it, distort it, or even try to forget it, but it will newer be the same as before.

Then you live with it, learn to love it, and new generations will learn it and pass it one. Until someone cannot stand the hiccups and the annoyances, and finally decides to try to improve the process - with success. Then writes a book and gives some talks, and you know the rest.

What makes this story fun, is the fact that this kind of knowledge-transfer is highly regarded in our western society (if you pay for the IM copyrights that is). You pay a lot of money for your copy of the new super process. And everyone knows that it is hard to make such a transfer succeed, so you choose the most expensive consultants to help you introduce their copy of the model of the new super process. And of course you end up with a - most of all - expensive process copy.

My wisdom: Learn from other's mistakes. Make your own process improvements (how to succeed on this is a completely other topic - a BI-topic, or business intelligence topic).