PADRES: A Reliable Publish/Subscribe Middleware

PADRES (Publish/Subscribe Applied to Distributed Resource Scheduling) is an enterprise-grade event management infrastructure that is designed for large-scale event management applications. Ongoing research seeks to add and improve enterprise-grade qualities of the middleware.

The PADRES system is a distributed content-based publish/subscribe middleware with features built with enterprise applications in mind. These features include

• Intelligent and scalable rule-based routing protocol and matching algorithm
• Powerful correlation of future and historic event
• Failure detection, recovery and dynamic load balancing
• Distributed system administration and monitoring

As well, the PADRES project studies application concerns above the infrastructure layer, such as

• Distributed transformation, deployment and execution
• Distributed monitoring and control
• Goal-oriented resource discovery and scheduling
• Secure, decentralized choreography and orchestration

A publish/subscribe middleware provides many benefits to enterprise applications. Content-based interaction simplifies the IT development and maintenance by decoupling enterprise components. As well, the expressive PADRES subscription language supports sophisticated interactions among components, and allows fine-grained queries and event management functions. Furthermore, scalability is achieved with in-network filtering and processing capabilities.

The technical architecture and features supported by the PADRES system are described in more detail in the architecture overview section.

The PADRES research project is conducted by the Middleware Systems Research Group (MSRG) at the University of Toronto and is a collaboration involving various industry partners and Canadian funding agencies.

Vision

The overall system architecture consists of several layers that together realize a distributed workflow execution engine.

The network layer consists of a heterogeneous set of distributed computing and network resources, which are virtualized by a distributed content-based publish/subscribe routing layer. The publish/subscribe overlay exposes a set of communication abstractions including publish/subscribe and point-to-point interfaces. Using these interfaces, a distributed workflow execution engine is deployed, and finally various management and monitoring capabilities are provided to deploy, control or observe the various layers in the system.

A rich event mangement framework allows each layer in the architecture to report detailed information about its characteristics and behaviour. This data from across the layers can be filtered and correlated using a powerful subscription language in a flexible and efficient manner. The data from the event management framework is used both internally by the system to observe and optimize itself, and to report statistics externally to applications and users.

Publications

Amer Farroukh, Elias Ferzli, Naweed Tajuddin, and Hans-Arno Jacobsen. Parallel Event Processing for Content-Based Publish/Subscribe Systems (Extended Version). Distributed Event-Based Systems (DEBS), July 2009, Nashville, Tennessee, USA. [Paper]

Wei Yan, Songlin Hu, Vinod Muthusamy, Hans-Arno Jacobsen, and Li Zha. Efficient Event-based Resource Discovery. Distributed Event-Based Systems (DEBS), July 2009, Nashville, Tennessee, USA. [Paper]

Hans-Arno Jacobsen, Alex Cheung, Guoli Li, Balasubramaneyam Maniymaran, Vinod Muthusamy, and Reza Sherafat Kazemzadeh. The PADRES Publish/Subscribe System. Book chapter of Handbook of Research on Advanced Distributed Event-Based Systems, Publish/Subscribe and Message Filtering Technologies. To be published in 2009 [Paper]

Songlin Hu, Vinod Muthusamy, Guoli Li, and Hans-Arno Jacobsen. Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems. International Conference on Distributed Computing Systems (ICDCS), June 2009, Montreal, Canada. [Paper]

Guoli Li, Vinod Muthusamy, and Hans-Arno Jacobsen. Adaptive Content-based Routing in General Overlay Topologies. ACM Middleware, Leuven, Belgium, 2008. [Paper]

Songlin Hu, Vinod Muthusamy, Guoli Li, and Hans-Arno Jacobsen. Distributed Automatic Service Composition in Large-Scale Systems. Distributed Event-Based Systems (DEBS), July 2008, Rome, Italy. [Paper]

Guoli Li, Shuang Hou, and Hans-Arno Jacobsen. Routing of XML and XPath Queries in Data Dissemination Networks. 28th International Conference on Distributed Computing Systems (ICDCS). Beijing, China 2008. [Paper]

Reza Sherafat Kazemzadeh, and Hans-Arno Jacobsen. δ-Fault-Tolerant Publish/Subscribe Systems. Technical Report CSRG-570, Middleware Systems Research Group, University of Toronto, November 2007. [Paper]

Alex Wun and H.-A. Jacobsen. A Policy Framework for Content-based Publish/Subscribe Middleware (Extended Version). ACM Middleware, November 2007, Newport Beach, California, USA. [Paper]

Guoli Li, Vinod Muthusamy and Hans-Arno Jacobsen. Subscribing to the past in content-based publish/subscribe. Middleware Systems Research Group Technical Report, CSRG-585, October 2007. [Paper]

Guoli Li, Vinod Muthusamy and Hans-Arno Jacobsen. Adaptive Content-based Routing In General Overlay Topologies. Middleware Systems Research Group Technical Report, October 2007. [Paper]

Alex Wun, Alex Cheung and H.-A. Jacobsen. A Taxonomy for Denial of Service Attacks in Content-based Publish/Subscribe Systems. DEBS 2007, June 2007, Toronto, Canada. [Paper]

Alex Wun and H.-A. Jacobsen. Modeling Performance Optimizations for Content-based Publish/Subscribe. DEBS 2007, June 2007, Toronto, Canada. [Paper]

Guoli Li, Alex Cheung, Shuang Huo, Vinod Muthusamy, Reza Sherafat Kazemzadeh, Alex Wun, H.-A. Jacobsen, and Serge Manovski. Historic Data Access in Publish/Subscribe. DEBS 2007, June 2007, Toronto, Canada. [Paper]

Guoli Li, Vinod Muthusamy and Hans-Arno Jacobsen. Ninos: A Distributed Service Oriented Architecture for Business Process Execution. Middleware Systems Research Group Technical Report, July 2007. [Paper]

Alex Cheung and Hans-Arno Jacobsen. Dynamic Load Balancing in Distributed Content-based Publish/Subscribe. Middleware 2006, December, Melborne, Australia. [Paper]

Guoli Li, Vinod Muthusamy, Hans-Arno Jacobsen and Serge Mankovski. Decentralized Execution of Event-Driven Scientific Workflows. International Workshop on Event-driven Architecture, Processing and Systems (EDA-PS), 2006. [Paper]

Guoli Li, Vinod Muthusamy and Hans-Arno Jacobsen. Adaptive Content-based Routing In General Overlay Topologies. Middleware Systems Research Group Technical Report, April 2006.

Eli Fidler, Hans-Arno Jacobsen, Guoli Li and Serge Mankovski. The PADRES Distributed Publish/Subscribe System. Feature Interactions in Telecommunications and Software Systems VIII. p. 12-30. Leicester, UK, July 2005. [Paper]

Guoli Li, Shuang Hou, and Hans-Arno Jacobsen. A Unified Approach to Routing, Covering and Merging in Publish/Subscribe Systems based on Modified Binary Decision Diagrams. 25th International Conference on Distributed Computing Systems (ICDCS). Columbus, Ohio, 2005. [Paper]

Guoli Li and Hans-Arno Jacobsen. Composite Subscriptions in Content-based Publish/Subscribe Systems. ACM Middleware, Grenoble, France, 2005. [Paper]

Related Publications

Milenko Petrovic, Vinod Muthusamy, and Hans-Arno Jacobsen. Managing Automation Data Flows in Sensor/Actuator Networks. Middleware Systems Research Group Technical Report, November 2007. [Paper]

Haifeng Liu, Milenko Petrovic, and Hans-Arno Jacobsen. Efficient Filtering of RSS Documents on Computer Cluster. Middleware Systems Research Group, Technical Report, November 2007. [Paper]

Zhengdao Xu and Hans-Arno Jacobsen. Expressive Location-based Continuous Query Evaluation With Binary Decision Diagrams Middleware Systems Research Group Technical Report, Nov. 2007. [Paper]

Guoli Li, Vinod Muthusamy and Hans-Arno Jacobsen. Routing of XML and XPath Queries in Data Dissemination Networks Middleware Systems Research Group Technical Report, Nov. 2007. [Paper]

Alex Wun, Milenko Petrovic and H.-A. Jacobsen. A System for Semantic Data Fusion in Sensor Networks. DEBS 2007, June 2007, Toronto, Canada. [Paper]

Vinod Muthusamy and Hans-Arno Jacobsen. Infrastructure-less Content-Based Publish/Subscribe. Middleware Systems Research Group Technical Report, March 2007. [Paper]

Additional related publications can be found at Publications page of the Middleware Systems Research Group.

Theses (directly related to the project)

Pengcheng Wan. Decentralized Business Process Execution in Content-based Publish/Subscribe System. M.Sc. Thesis, September 2004 to May 2006.

Eli Fidler. PADRES: A Distributed Content-based Publish/Subscribe System. M.A.Sc. Thesis, September 2002 to January 2006.

Alex Cheung. Dynamic Load Balancing in Distributed Content-Based Publish/Subscribe. M.A.Sc. Thesis, May 2003 to December 2005.

Guoli Li. Efficient Content-Based Publish/Subscribe for Distributed Workflow Management. M.Sc. Thesis, September 2003 to January 2005.

David Matheson. A Rule-Based Approach for Publish/Subscribe. M.A.Sc. Thesis, January 2002 to April 2004.

Theses (partially related to the project)

Vinod Muthusamy. Infrastructureless Data Dissemination: A Distributed Hash Table Based Publish/Subscribe System. M.A.Sc. Thesis, September 2002 to January 2005.

Ferdous Jewel. A Fast Filtering Engine for Publish/Subscribe System. M.Sc. Thesis, January 2002 to April 2004.

In the News

2007-06-21: Our sponsor CA announces a $1 million committment to Canadian universities.

2007-07-06: ITWorld Canada reports on CA's $1 million research grants.

People

Principle Researcher

• Hans-Arno Jacobsen

Students & Guests

• Alex Cheung (PhD student)
• Eli Fidler (MASc)
• Gerald Chen (B.S.)
• Ferdous Jewel (MSc)
• Guoli Li (PhD student)
• Jian Li (Post-doctoral Fellow)
• David Matheson (MASc student)
• Vinod Muthusamy (PhD student)
• Pengcheng Wan (MSc)
• Alex Wun (MASc student)
• Shuang Hou (Research Programmer)
• Songlin Hu (Research Associate)
• Reza Sherafat (PhD student)
• Naweed Tajuddin (Research Programmer)

Partners

• Serge Mankovski (Computer Associates)
• Lynne Zucker (Sun Microsystems of Canada)

Project History

The Middleware Systems Research Group (MSRG) has been conducting research on publish/subscribe and messagging middleware for many years. This research effort has become known as the Toronto Publish/Subscribe System family of projects (also known as ToPSS).

In 2002, MSRG was approached by Cybermation, Inc to investigate the use of content-based publish/subscribe for building a distributed job scheduling and workload management infrastructure to form the basis of Cybermation's future product line targeting distributed, cross-enterprise job scheduling, workload management, and resource monitoring.

From 2003 to 2005, supported by Cybermation, Inc, NSERC, and CITO, the Middleware Systems Research Group built the first PADRES prototype, experimenting with a rule-based matching engine design, composite subscription routing and composite event detection, and historic query access.

From 2006 to 2007, supported by Cybermation, Inc (acquired in April 2006 by Computer Associates, Inc), Sun Microsystems of Canada. and OCE/CITO, the Middleware Systems Research Group is hardening the PADRES prototype and extends PADRES with enterprise-grade features, such as load balancing, failure resiliance, and security.

Logos

Which logo do you like best?

Acknowledgements of sponsors