Had a thought-provoking late-night talk with Martijn Linssen about the topic of enterprise integration yesterday. We touched upon many topics like the need and the level of standardization, the 80-20 rule, APIs and the role of a message broker within an enterprise IT landscape. Let’s recap…
As we had sort of a rough start (see comment thread of his blog) we decided to first check where we are on common ground talking into account the 30 years of experience we had picked up doing enterprise integration projects.
Our first agreement is that standardization in general eases doing business together and that some level of standardization is a good thing e.g. when it comes to the basic set of business processes and commodity services. In SAP lingo this relates to the understanding of core services (the core); the usual suspects would be purchasing or equipment management or the like. Here, only a few companies may need to do something completely different than the rest of the world, few would see the need to have a differentiation here. (Well, maybe except for those whose primary core business is exactly about purchasing or equipment management – we’ll get back to this in a minute!)
Standardization is our ally on many fronts though, not only on the processes side, but for sure also on the technical side. I don’t want to stress the OSI layers at this point, but instead let’s just note that reducing the number of alternatives on all layers comes in handy when talking to each other. Be it operation systems, runtime containers, programming languages, networking protocols, data protocols, business standards etc…
Here, the idea of open standards come into the play… if there’s a open standard and its serves its purpose it may see adoption. The higher the adoption, the more people have a common understanding of how to talk to each other based on it. Adoption leads to better documentation and more people with related skills, it also leads to competition, which ultimately results in lower costs for all involved parties. That’s the benefit of open standards.
The next thing we agreed upon is that the closer you get to the end user the higher the demand to be more user-centric or individual as in “we do things slightly different here!” And that’s just fair! In fact, isn’t catering to individual needs not key to what we consider good customer service? That I do not have to be 100% precise (adhering to every little detail of all these standards, domain languages and etiquette), but instead rely on the other side to know my context and being able to support me to get the right answer? Sure is.
A fair level of standards evens the playing field for all, yet there will always be valid reasons that speak in favor of a more custom/individual approach. Think of different industries or regional differences. So, there’s not that one standard, but a multitude of standards caused by the needs of individual domains. For most of us for example the process of renting a car could be standardized and we’d be happy; yet for a car rental company it’s their core business and they sure want to differentiate from the competition and have a custom solution that leverages their domain know-how best. This is where people/business want to differentiate and where there is an accelerated speed of innovation and yes… change! That’s what is sometimes referred to as the edge.
Bringing it all together
So far it matches my experiences… I said it many times: there’s no such thing as a one-size-fits-all solution. (That’s what made me picking extensibility concepts as a favorite topic in the first place!) So, yes… at the end of the day every enterprise would love to have an IT landscape that caters to both needs: having a low-cost set of standard processes and ways to easily and quickly alter, enhance and create services as required. For me, this boils down to the vision SAP has been delivering on in its 40 years of existence…
There’s a ‘stable’ core set of services that is constantly renovated without disruption at the core (we used to call it the business process platform) AND there’s the edge where you want tailor-fitted and/or custom-made solutions to complement the set of services. Based on these findings Martijn and me tried to map that to his understanding of what he calls Adaptive Integrated Enterprise. Others may call it timeless software… at the end the underlying idea is largely the same.
Adaptive Integrated Enterprise
Let’s get a bit more technical now – and by the way, that was my main point of critism to Martijn’s blog post – I was missing the technical details.
So as any piece of information (let’s call it message) travels from actor A to actor B it crosses multiple areas. It goes from the core to the edge (through the ESB) and from there through edge to core on the receiving side. From origin to target, from original data format to the translated one on the other side. So, these are the standard characteristics of an ESB: routing, mapping, translations and transformation. From a software architecture perspective such tasks are better encapsulated in individual components. Usual names for such entities would be transformer or adapters (why not speak of agents?)
Similar to the idea of an agent in a call center the principal idea is to get to understand the request and provide the requested service result. As in real life, it may need a series of agents until one is found that is finally able to help us out. So, once a message is received it gets transformed and translated to the inherent data model of an enterprise via a series of agents, then the service request is processed and the result is returned (going through the transformation/translating process again.)
I think this was the central point of diverging perceptions between Martijn and me in the past. For some reason I thought he was talking about an approach that would use generic agents capable to auto-magically route through any message. Yesterday I got to see that his vision is a bit more pragmatic… which is good! That’s another learning from my past SOA projects… being pragmatic helps PLUS it saves costs!
Don’t get me wrong, I see a trend for technologies like smart agents. Based on concepts like the semantic web, predictive analysis and AI that seems doable in the mid to long-term. Maybe sooner if one aims for the famous 80% and then leave the rest for tailor-fitted hand-crafted agents that do a specific job catering to an individual scenario. (LOL, I still hear Martijn saying “What is it with your ‘manual coding’ all the time?!?“)
So, in his vision (at least to what I think I understood) there’s a place for custom hand-coded agents, which cater to exotic formats, unique conditions or special scenarios. By his rational it may be easier (= more cost efficient!) to take a pragmatic approach and cover the remaining 20% with man-power. And it sounds reasonable; taking into account the accelerated speed of innovation (= change!) at the edge even more so!
For that purpose, on the edge (where there are no open standards yet!) you need a platform that allows to quickly create special agents with minimal effort. Some platforms may offer model-driven tools for this purpose, others may provide a scripting environment or SDK. Or what about PaaS? ;)
I sure am biased – but of course I cannot help but think SAP NetWeaver Neo when crossing this very topic. I mean we talk about a cloud platform based on open-standards, which provides a set of commonly required services to build such type of applications like persistence (of course including HANA ✓), connectivity (including on-premise systems), identity management, document storage etc etc. A platform build upon open-source frameworks allowing you to use whatever programming model you want (as long as it is capable of running within a JVM.) The entry barrier is as low as possible… each enterprise can choose from a broad set of open standards to build such agents/extensions/applications. Use whatever frameworks your people are familiar with… or based on whatever know-how you can hire for decent money.
It’s been indeed a very interesting 90 min on Skype yesterday and it seems that Martijn and me are not that far off each other’s perspective. Not at all… We both acknowledge that a modern enterprise should do their homework and establish a set of core services based on a canonical data model that fits its domain (for me that spells like Sustainable Architectures for SOA, but your mileage may vary!) A good level of standardization greatly helps, yet experience has shown that being pragmatic (=KISS) may be less painful than betting on over-engineered 1-size-fits-all solutions.
And this is why I believe that the future is bright indeed for SAP as it offers that stable platform to support your core processes. For varying industries, for varying sizes. SAP did introduce a canonical data model that can be the foundation of your very own SOA implementation at the core of your enterprise. There are also other offers like Rapid Deployment Solutions (RDS) and Line-of-Business applications that address certain niches and special domains.
SAP has been actively encouraging its ecosystem to take up the white spaces and leverage domain-specific expertise to build applications that run on top of the platform or connect to it. PLUS… SAP also heavily invests on the technical capabilities so that you can also build your custom extensions – with fast results and low costs. Consequently – for me – the only conclusion I can draw from our discussion is that either way you put it… SAP is well-positioned!