Had a thought-provoking late-night talk with Martijn Linssen about the topic of enterprise integration yesterday. We touched upon many topics like the need and the level of standardization, the 80-20 rule, APIs and the role of a message broker within an enterprise IT landscape. Let’s recap…

As we had sort of a rough start (see comment thread of his blog) we decided to first check where we are on common ground talking into account the 30 years of experience we had picked up doing enterprise integration projects.

Standardization

Our first agreement is that standardization in general eases doing business together and that some  level of standardization is a good thing e.g. when it comes to the basic set of business processes and commodity services. In SAP lingo this relates to the understanding of core services (the core); the usual suspects would be purchasing or equipment management or the like. Here, only a few companies may need to do something completely different than the rest of the world, few would see the need to have a differentiation here. (Well, maybe  except for those whose primary core business is exactly about purchasing or equipment management – we’ll get back to this in a minute!)

Standardization is our ally on many fronts though, not only on the processes side, but for sure also on the technical side. I don’t want to stress the OSI layers at this point, but instead let’s just note that reducing the number of alternatives on all layers comes in handy when talking to each other. Be it operation systems, runtime containers, programming languages, networking protocols, data protocols, business standards etc…

Here, the idea of open standards come into the play… if there’s a open standard and its serves its purpose it may see adoption. The higher the adoption, the more people have a common understanding of how to talk to each other based on it. Adoption leads to better documentation and more people with related skills, it also leads to competition, which ultimately results in lower costs for all involved parties. That’s the benefit of open standards.

Differentiating Factors

The next thing we agreed upon is that the closer you get to the end user the higher the demand to be more user-centric or individual as in “we do things slightly different here!” And that’s just fair! In fact, isn’t catering to individual needs not key to what we consider good customer service? That I do not have to be 100% precise (adhering to every little detail of all these standards, domain languages and etiquette), but instead rely on the other side to know my context and being able to support me to get the right answer? Sure is.

A fair level of standards evens the playing field for all, yet there will always be valid reasons that speak in favor of a more custom/individual approach. Think of different industries or regional differences. So, there’s not that one standard, but a multitude of standards caused by the needs of individual domains. For most of us for example the process of renting a car could be standardized and we’d be happy; yet for a car rental company it’s their core business and they sure want to differentiate from the competition and have a custom solution that leverages their domain know-how best. This is where people/business want to differentiate and where there is an accelerated speed of innovation and yes… change! That’s what is sometimes referred to as the edge.

Bringing it all together

So far it matches my experiences… I said it many times: there’s no such thing as a one-size-fits-all solution. (That’s what made me picking extensibility concepts as a favorite topic in the first place!) So, yes… at the end of the day every enterprise would love to have an IT landscape that caters to both needs: having a low-cost set of standard processes and ways to easily and quickly alter, enhance and create services as required. For me, this boils down to the vision SAP has been delivering on in its 40 years of existence…

There’s a ‘stable’ core set of services that is constantly renovated without disruption at the core (we used to call it the business process platform) AND there’s the edge where you want tailor-fitted and/or custom-made solutions to complement the set of services. Based on these findings Martijn and me tried to map that to his understanding of what he calls Adaptive Integrated Enterprise. Others may call it timeless software…  at the end the underlying idea is largely the same.

Adaptive Integrated Enterprise

Let’s get a bit more technical now – and by the way, that was my main point of critism to Martijn’s blog post – I was missing the technical details.

So as any piece of information (let’s call it message) travels from actor A to actor B it crosses multiple areas. It goes from the core to the edge (through the ESB) and from there through edge to core on the receiving side. From origin to target, from original data format to the translated one on the other side. So, these are the standard characteristics of an ESB: routing, mapping, translations and transformation. From a software architecture perspective such tasks are better encapsulated in individual components. Usual names for such entities would be transformer or adapters (why not speak of agents?)

Similar to the idea of an agent in a call center the principal idea is to get to understand the request and provide the requested service result. As in real life, it may need a series of agents until one is found that is finally able to help us out. So, once a message is received it gets transformed and translated to the inherent data model of an enterprise via a series of agents, then the service request is processed and the result is returned (going through the transformation/translating process again.)

Coders wanted!

I think this was the central point of diverging perceptions between Martijn and me in the past. For some reason I thought he was talking about an approach that would use generic agents capable to auto-magically route through any message. Yesterday I got to see that his vision is a bit more pragmatic… which is good! That’s another learning from my past SOA projects… being pragmatic helps PLUS it saves costs!

Don’t get me wrong, I see a trend for technologies like smart agents. Based on concepts like the semantic web, predictive analysis and AI that seems doable in the mid to long-term. Maybe sooner if one aims for the famous 80% and then leave the rest for tailor-fitted hand-crafted agents that do a specific job catering to an individual scenario. (LOL, I still hear Martijn saying “What is it with your ‘manual coding’ all the time?!?“)

So, in his vision (at least to what I think I understood) there’s a place for custom hand-coded agents, which cater to exotic formats, unique conditions or special scenarios. By his rational it may be easier (= more cost efficient!) to take a pragmatic approach and cover the remaining 20% with man-power. And it sounds reasonable; taking into account the accelerated speed of innovation (= change!) at the edge even more so!

For that purpose, on the edge (where there are no open standards yet!) you need a platform that allows to quickly create special agents with minimal effort. Some platforms may offer model-driven tools for this purpose, others may provide a scripting environment or SDK. Or what about PaaS? ;)

I sure am biased – but of course I cannot help but think SAP NetWeaver Neo when crossing this very topic. I mean we talk about a cloud platform based on open-standards, which provides a set of commonly required services to build such type of applications like persistence (of course including HANA ✓), connectivity (including on-premise systems), identity management, document storage etc etc. A platform build upon open-source frameworks allowing you to use whatever programming model you want (as long as it is capable of running within a JVM.) The entry barrier is as low as possible… each enterprise can choose from a broad set of open standards to build such agents/extensions/applications. Use whatever frameworks your people are familiar with… or based on whatever know-how you can hire for decent money.

Conclusion

It’s been indeed a very interesting 90 min on Skype yesterday and it seems that Martijn and me are not that far off each other’s perspective. Not at all… We both acknowledge that a modern enterprise should do their homework and establish a set of core services based on a canonical data model that fits its domain (for me that spells like Sustainable Architectures for SOA, but your mileage may vary!) A good level of standardization greatly helps, yet experience has shown that being pragmatic (=KISS) may be less painful than betting on over-engineered 1-size-fits-all solutions.

And this is why I believe that the future is bright indeed for SAP as it offers that stable platform to support your core processes. For varying industries, for varying sizes. SAP did introduce a canonical data model that can be the foundation of your very own SOA implementation at the core of your enterprise. There are also other offers like Rapid Deployment Solutions (RDS) and Line-of-Business applications that address certain niches and special domains.

SAP has been actively encouraging its ecosystem to take up the white spaces and leverage domain-specific expertise to build applications that run on top of the platform or connect to it. PLUS… SAP also heavily invests on the technical capabilities so that you can also build your custom extensions – with fast results and low costs. Consequently – for me – the only conclusion I can draw from our discussion is that either way you put it… SAP is well-positioned!

 

2 Responses to About Enterprise Integration
  1. Hi Matthias,

    Thanks for sharing the results of this discussion, and it’s good to know that you and Martijn found many areas of agreement.

    In my view there are separate discussions about the best ways to manage business data (including enforcing business logic / rules) and the best ways to provide access to this.

    My take on Martijn’s blog was that he was agnostic about how you manage business data, and expected (encouraged?) businesses to select the most appropriate tools for hte job. However when providing access (be that Mobile, A2A, B2B or anything else) using a common (canonical) business model, and routing everything through this layer, has advantages over direct access to native application models.

    For my part, I see SAP’s approach in this area currently as fragmented. On one side we have eSOA for A2A / B2B (supported by Processs Integration / Orchestration) and on another we have REST for Mobile (supported by Gateway / SUP). And then there’s Duet Enterprise, which is A2A, but uses Gateway. Then there’s a whole raft of direct / proprietary A2A approaches ALE / RFC which SAP continues to leverage for integration.

    Martijn seems to see some light at the end of the tunnel – but for my part I’m still seeing lots of tunnels many of which lead to the same destination. The obvious response is that SAP is providing options, thereby promoting choice. However due to architectural realities / restrictions, customers end up having to implement and manage all of them, which just results in a more complex and difficult to manage landscape.

    I’m hopeful that we can eventually reconcile these different integration approaches, but I’m not sure the current strategy (particularly with regard to Gateway and PI/PO) is going to get us there.

    Cheers,
    Jon

  2. Great recap Matthias

    Standardisation is almost mandatory for your tertiary processes: cleaning your company, canteen / catering, exterior decorating etc., the stuff you really don’t care about as long as you get what you paid for – you’ll notice that all service offerings there seem to be similar if not the same. You’re aiming for rules here and don’t need any exceptions

    Going to secondary processes, of course leaving the cold inanimate objects and closing in on the warm human flesh (your employees), you will care a lot more but still not become fussy. Salary pay, HR, leasing, etc – you might want to pay “unreasonably more” to get one or two exceptions to the rule(s)

    Your primary processes? Oh my – you might end up wanting even more exceptions than there are rules, if you don’t constrain yourself

    Funnily enough, standardisation does occur in inanimate objects mostly, and never among live ones – likewise, diversity increases if you move away from the cold iron towards the warm humans

    On AIE

    There are no “agents”, or maybe I misread you.
    The ESB’s routing engine is just a post office: requests from outside are put in mailboxes on the inside, that are retrieved by their respective owners. All the messages need a clearly legible envelope of course, regardless in which syntax as long as it contains the right semantics.
    Read the envelope, route the message – in a straight line to its destination

    Pragmatic is good indeed ;-)

    Semantic web is bullshit, and you and I live on different planets when it comes to tooling. The business / information side of intergation is about 95% of the work, although they can’t do that without serious help from a “translator” as I’d like to call myself sometimes. Building the physical mapping or coding the business rules that enable the routing is hours or at best days work.

    Integration is not about building huge applications, it’s about wrapping back-end business functionality in thin veils that fit the environment they’re wanted in. You basically retrieve the parameters to a function, shuffle them a bit around in the right order, and pass them on to the backend. Whatever the outcome of that, you’ll get a return. Thinly veil that again, and that’s it – zero footprint

    Conclusion

    Yup! Core services, canonical, standardisation / prefab so customers can pick (never give them limited choices, they will behave like a kid in a candy store), one-size-fits-all never existed nor will it, and I see a shining future for SAP here indeed

    Jonathan,

    you are spot on. I don’t care how you come to work (bike, bus, train, plane, car, walk, crawl) or even who you are (intern, perm, temp, male, female, black, gay, Chinese) as long as you provide me with the functionality I asked for. In time, on schedule, within budget

    Point to point (scripting together hand-coded SOAP and REST API’s) is destined to fail to deliver any ROI beyond a 10-app company

    I share your concern regarding the diffuse SAP offering wrt integration, and the irony is that there’s a lot of legacy in SAP’s solution to expose legacy functionality. Is the cure worse than the disease?

    That last one would be a great question for Vishal Sikka btw


[top]

Leave a Reply