I was attending a lunch and learn today where someone asked the following question about publish-subscribe: if the publisher is delayed, how are the subscribers affected? I hear this type of questions often from IT people who have spent their career working on mainframe systems. There are used to processing data in batch, and in some cases, in large quantities. The client is currently implementing Oracle E-Business Suite (EBS) to replace some business applications on the mainframe. According to the current process, a job is scheduled to run daily or twice daily to send a batch file to a business partner. If you have worked on mainframe systems before, you know that it is not that uncommon for a batch job to be delayed due to dependencies as an example. Thus, this question is a very legitimate one.
When I hear a question along these lines, my first reaction is to promote (near) real-time processing. In this specific case, a business event can be configured in EBS to publish a change or an event as it occurs as an example. The business event can be published to the Event Delivery Network (EDN) in SOA Suite and (to keep this example simple) dispatched to the subscribers. Obviously, SOA Suite and Oracle Service Bus (OSB) provide a lot of capabilities to do much more than that.
The first argument I generally hear with this model is about a business partner’s expectations. More specifically, the business partner expects a batch file at certain times every day as an example. I often suggest contacting the business partner and inquiring if they would be interested in moving to a (near) real-time processing model. Sometimes, I am told that it would be preferable not to affect the business partners.
One technique we used on a project was to leverage an intermediary store to accumulate the data for some or all the business partners. This can be a database or a message queue as examples. In other words, a special subscriber receives the business event from EDN, completes all the processing required and saves the data to the intermediary store. A job is scheduled to run as per the original scheduled to extract the data from the intermediary store, batch it into a file and transfer the file to the business partner(s).
Moving from a mainframe system to a distributed system that involves Oracle Fusion Middleware requires a paradigm shift. This is just an example of a solution we implemented in a past project to forward data in batch mode. Once the data is published out of a business system such as EBS in (near) real-time, there are plenty of subscribers who greatly benefit from that model.
