Application Integration
Application integration is a broad topic with various aspects and views. Before presenting different views, let us look at the very basic models that drive application integration. They are:
-
Level at which applications are integrated — data, application logic (method), or presentation (user interface)
-
Loose versus tight coupling (synchronous versus asynchronous) between the players
-
Point to point versus hub/bus type integrations
Application Connectivity Types
Figure 1 shows the key players in integration: information suppliers that are custodians of the needed information, information consumers that are the clients of the needed information, and the mediators that enable the consumer-supplier interactions . As shown in this figure, you can interconnect applications at data, method, or presentation (user interface) level by using different types of middleware services. These middleware services comprise the mediators between information suppliers and information consumers. Examples of middleware technologies are:
-
Remote Data Connectors (RDCs)- access of remote data directly, typically through remote SQL and ODBC/JDBC, without invoking any enterprise functionality (e.g., business logic).
-
Remote Method Connectors (RMCs)- access of a remote method (e.g., a C++ procedure that handle payments) through APIs by using a variety of technologies such as CORBA or RPC packages.
-
Remote User Connectors (RUCs) – access of remote information through some presentation services by using Web scrapers or screen scrapers.
Naturally, there are tradeoffs in connecting at different levels. For example, data level interconnectivity means that the supplier has to expose the database schema.
Figure 1: Application Connectivity Type
Synchronous versus Asynchronous Communications : Loose Versus Tight Coupling
Another aspect of application connectivity is the Interaction model that is used between suppliers and consumers. For example, a supplier can communicate with the consumer in:
-
Blocked, also known as synchronous communication, mode (i.e., the client must wait for response from the server)
-
Unblocked, also known as asynchronous communications, mode (i.e., the client can continue processing while the server is busy developing the response)
Synchronous communications are interactive in nature and assume a tight coupling between the clients and servers. The foundation of synchronous communication is the client/server (C/S) model that allows application processes at different sites to interactively exchange messages. Most C/S applications utilize the remote procedure call (RPC) paradigm which extends the scope of a local procedure call. A very common example of the C/S model is the World Wide Web where the browsers are the clients that access remote Web servers in real time. Thus the synchronous communication model is used by the Web browsers and servers — when you click on a link, you are blocked and must wait for a response from the server. This implies that the Web browsers are tightly coupled to the Web servers through HTTP. The synchronous communication model is also used extensively in the distributed objects technologies such as CORBA, DCOM, RMI, and SOAP (e.g., SOAP RPC).
Synchronous communications offer maximum synchronization capabilities between partners. These are the types of technologies that application vendors such as SAP use to integrate components of their application suites. The drawback to this approach is that it hampers the autonomy of collaborating organizations and typically requires combinations of joint development, shared client/server logic and a central architecture. These approaches work best for centrally architected solutions but are inappropriate for large scale enterprise-wide and B2B solutions.
In asynchronous communications, the clients are not blocked when a message is sent to the server—the clients can put a message on the queue and then continue processing. Another segment of client logic can continue to monitor the output queue for any responses and process the outputs received. Asynchronous processing has the following benefits:
-
Failure of server does not stop the clients from continuing to process as much as possible.
-
Clients can be more efficient and responsive because they do not have to wait for the server to respond.
-
It is suitable for detachable (e.g., wireless) computers that need to access data asynchronously without having to be in constant session with the hosts.
-
Clients and servers can use “deferred message” or call back features if a response is not available.
-
Clients and servers can operate in a publish-subscribe model where events are posted by the publisher on a channel. The subscribers listen to events of their choice and act accordingly.
Which one is better? In most enterprise level integrations, it is better to use an asynchronous model because it implies loose coupling. The suppliers and consumers do not have to be blocked from further processing. This model also supports a publish-subscribe model that is very suitable for large scale integrations. This is the main reason why most integration platforms are based on asynchronous processing. The best known example of asynchronous middleware is MOM (message oriented middleware).
The synchronous/asynchronous communications can be used at the data, method or presentation levels discussed in the previous section. For example, an order processing applicant can access the data of an inventory management application by using synchronous or asynchronous communications.
Point-to-Point Versus Hub/Bus Models
Integrations for a number of years were done on a point to point basis (see Figure 2).In these cases, the suppliers and consumers directly interfaced with each other through mediation technologies. As shown in the following figure, the point to point communication may work well for small number of partners. For example, you can use point to point communication when integrating two or three systems. However, this approach does not scale well because you need N(N-1)/2 interfaces between N partners.
Hubs provide a better approach for integrating different applications. Hub-based application integration is useful because:
-
It cuts down on the number of interfaces that need to be supported between the applications. Thus instead of point to point interconnections, all applications connect to the hub that organizes and mediates between applications.
-
New applications can be added to the hub easily and without disrupting other applications. This is especially useful in trading networks where different partners can appear and disappear on a regular basis.
Bus-based integration is a variant of hub-based integration because they can be used to combine the point-to-point with hub-based architectures. The parties can communicate directly with each other over the bus or can use a control engine (a “process engine”) that can provide basis for flow of information between different players. For example, A can directly communicate with B because they are part of the same organization but go through a control engine when talking to C, an external entity. Process engines are typically workflow managers.
Combining the Models — The Enterprise Service Bus
From the above discussion, it is obvious that asynchronous, hub/bus-oriented systems that support integration at the data, method as well as presentation level are needed for large scale enterprise integration. This is what the Enterprise Service Bus (ESB) provides.
An Enterprise Service Bus (ESB) is a key component of SOA and provides the main mechanism for integrating the enterprise applications. A SOA ESB provides a collection of technologies (middleware such as Web Services, adapters/gateways for protocol conversion, data transformers, transaction managers, and work/process flow systems) that allow diverse applications to communicate with each other. Figure 3 shows a conceptual view of an ESB. The ESB platform is not a new technology – rather, it is a combination of “well-known” technologies that can integrate multiple applications. All applications (business components) provide services that are invoked through well defined interfaces. In reality, ESB is an extension of the well known EAI (Enterprise Application Integration) platforms that were based on the older CORBA and DCOM technologies.
Figure 3: Enterprise Service Bus (ESB) – Conceptual View
An ESB is expected to possess the following common set of characteristics:
Communication through a Broker
- Intelligent Routing through Directory Services
- Message transformation. ESBs typically provide off-the-shelf adapters that are used for message and protocol translations.
- Basic Web services support
- Additional features provided by some ESB vendors (e.g., security, administration, software development, validation, logging, and auditing capabilities).
Although ESB platforms may use different types of middleware technologies (e.g., Message Queuing, etc), Web Services are the most popular technologies of ESBs at the time of this writing. ESBs may also exist as EAI (enterprise application integration) platforms or message brokers. ESB software is commercially available from vendors such as IBM (Websphere ESB), Microsoft (Biztalk Server 2010) and others (e.g., Mule, Sonic Software, Systinet, Tibco, Fiorano, IONA). An ESB basically provides “adapters” that allow different applications to connect at data, method or presentation level over the bus.
These platforms commonly use the publish/subscribe model for asynchronous (unblocked) communication for loosely connecting a diverse array of applications over a bus. In a publish/subscribe model the messages can be “published” into a “channel”. This is similar to the notion of a television news channel. Thus a channel publisher simply posts (“pushes”) events of interest to the channel. The publisher is unaware of the actual recipients (subscribers). The subscribers retrieve (“pull”) the information from the channel. This provides a useful mechanism for decoupling applications from one another. A trading system, for instance, can simply publish a new item to be traded on a “new items” channel and various traders can respond to this event. Publish/subscribe is currently available through MOM, JMS, and Web Services. The ESB in effect is the channel. An ESB can be used to interconnect applications for internal as well as for B2B trade.
Source: Umar, A., “Enterprise Application Integration Using SOA”, NGE Solutions, 2nd edition, 2013.
Arif Randy Agustian Blog's 