I recently re-read parts of Gregor Hohpe’s book on integration patterns, and felt a level of professional jealousy. I have been involved in an enterprise level integration effort for a couple of months now. The project entails replacement of a couple of line of business applications with a single package. Complexities involved in this process include:

•Existing applications are tightly integrated into the existing environment. An example of the level of tight coupling that exists is that some integration relies heavily on point to point integration on the mainframe using Cobol copybook calls. This all needs to be replaced with a loosely coupled Wintel based application

•More than 60 integration points needs to be handled.

•Integration interface types include the whole range: Point to point, Message based (synchronous and asynchronous), Online SOAP (synchronous and asynchronous) and good old batch with CSV file exchanges

•Different middleware is used for batch, online and message based integration

•Due to the sheer volume of work, a phased approach is required. Sounds great, but what this really means is that we need to gradually phase out the tight integrations into the loosely coupled integrations, with periods where both interfaces will be active

So why the professional jealousy towards the work of Gregor?

His book, with the impressive set of integration patterns, creates the impression that Gregor had much more technical challenges and much less business and process related challenges. At this point, my enterprise level integration project is definitely 80% about sorting out business process, project management and contingency related issues rather than the stuff techies dream about. So I thought I will pop Gregor a mail and see what his opinion on this is – is it all the technical fun and games, or is my experience what you should expect with a large scale integration project. Will report back on that at a later stage.

Despite the technical complexity driven level of jealousy, I have to admit to enjoying this project. The following list of why’s

•This is a really big site, with commitment to slowly migrate into the SOA world. It is not text book SOA (yet), as the real world and “business as usual” needs requires a pragmatic approach

•Enterprise Architecture and the TOGAF model of managing enterprise architecture came to real good use in understanding the real “Big Picture” and managing each one of the integration points in the larger context

•Although I’m joking a bit re the technical component of the project, in truth there are more than enough technical complexities to this project.

So I’m learning a lot and enjoying it. In a nutshell, the big lessons I have learned:

•Doing an integration project without understanding the architecture of the enterprise is doomed to fail. A framework such as TOGAF’s is a good tool to model and understand the intricacies of introducing change into the enterprise architecture

•Web services are not as technology neutral in real life as promised.

•Adding proprietary security or conversation rules increases coupling

•Slapping a web service interface over old technology by using middleware has its limitations. You can only teach an old dog a limited number of new tricks. Refer to the tight coupling comment above

•Project managing a project of this nature requires a lot of technical input on a daily basis. A project manager with a technical background and up to date to date knowledge of technology is required; else get a tech lead that can spend a lot of time with project management

•A solution architect with the ability to communicate ideas and concepts between the project stakeholders is critical to project success