RabbitMQ Integration with Magento 2.4

Previously, Magento was using MySQL adapters for messaging and cron jobs to ensure the delivery of messages. This wasn’t reliable and scalable. To combat this, you can integrate RabbitMQ, which adds asynchronous messaging capabilities to Magento and is also scalable and reliable.

For example, RabbitMQ could be used where Magento needs to communicate with an ERP System. RabbitMQ decouples an ERP integration process from the customer order fulfillment process. With the consumer now running as a background process, we have a faster order fulfillment process and better error handling. This thereby improves the overall end-user experience.

What is RabbitMQ

RabbitMQ is the open-source message broker, an intermediary for messaging. It gives applications a common platform to send and receive messages, and a safe place for messages to live until received.

RabbitMQ is the middleman for messaging and supports several protocols including Advanced Message Queuing Protocol (AMQP) for processing communications between two applications. It’s a middleware where queues are defined, and applications (consumers) can connect to the queues to pull messages. A queue could be any kind of information, such as a process or task that should start on another application or even a simple text message.

The Benefits of RabbitMQ

There are several benefits with utilizing RabbitMQ, such as:

• Asynchronous Messaging
• Clustering
• Federation
• Highly Available Queues
• Multi-protocol
• Management UI
• Plugin System

There are a number of tasks/activities performing within a commerce website’s architecture as a customer places an order from the website, such as:

• Generating the order
• Taking payment
• Stock management
• Enabling email and text communications
• Sending/Receiving order data to SAP/CRM/ERP systems
• Sending data to analytics

When all these activities are performed in real-time, more time is added to completing the order process, which can negatively affect a business. RabbitMQ can help transfer most of these tasks into queues and get them processed afterward through asynchronous messaging, thereby allowing web servers to respond quickly.

Basic Architecture of a Message Queue

Inspired Digital Experiences for Manufacturing & Automotive

Whether you’re just beginning your digital transformation journey or are well on your way, we invite you to explore our partnership with Adobe and our diverse capabilities in manufacturing and automotive.

Download Lookbook

The client applications, called producers, create messages and deliver them to the broker (RabbitMQ). The applications, called consumers, connect to the queue and subscribe to the messages to be processed.

Installation and Configuration

Before you can build your messaging application, you need to set up a RabbitMQ server to handle receiving and sending messages.

There are several ways the RabbitMQ server can be setup. Follow this outline of best practices to set up the RabbitMQ server using Docker images or on-base RHEL/CentOS machines.

Method 1: RHEL/CentOS Machines

• Install packages. Make sure the below listed dependent packages are also installed with the rabbitmq-server package
  • erlang
  • socat
  • logrotate
• Custom configuration. Configure RabbitMQ for any custom requirement, if any. For example, It’s recommended to encrypt the communication for security reasons, especially in a production environment. To do this, we can configure a Proxy and let RabbitMQ run on default configurations or we can configure RabbitMQ for TLS enabled connections. To enable TLS in RabbitMQ, we need to tell the node, which is the RabbitMQ standalone server, or nodes in a clustered environment to use encrypted connections by enabling TLS related parameters and providing the SSL certificate-key pair and CA signed certificate.
• Enable the plugins. We can enable plugins per project requirements. We usually need to enable the management plugin so that the RabbitMQ server is accessible through a management UI.
• Remove the default user. The default user and password for RabbitMQ is publicly known, so it’s recommended to remove the default user and create a custom username and password.
• Perform Tests. Start the rabbitmq-server and perform tests to check if the server is working as expected, such as ssl handshake test or a port connectivity test, etc.
• Publish and Consume Messages. Once the test results in the desired output, we can proceed with publishing and consuming the queues (messages).

Method 2: Docker Image

This is the simpler way to host a RabbitMQ server as the docker image is prebuilt with all the requirements for the service to run properly, we are just left with the plugins and custom configuration part.

• Build Docker Image & Run Container. Build a RabbitMQ Docker image using DockerFile and Run container using previously created image through docker-compose.yml file. Pull the alpine image so that the image size is the least possible.
• Custom configuration. Configure RabbitMQ for any custom requirement, if any. As mentioned above, the same custom configuration can be applied in the default rabbitmq configuration file and added in the docker image or can be included as an environment variable file in the docker-compose.yml file.
• Enable the plugins. As mentioned above, similar plugins can be enabled during image creation or later after the container is up and running.
• Remove the default user. Once the container is up and running, remove the default user, and a custom user should be created. This can be done either from CLI or even from the Management UI, as well as by adding custom user credentials in the default RabbitMQ configuration file.
• Perform Tests. Perform tests and check if the server is working as expected, such as a SSL handshake test or port connectivity test, etc.
• Publish and Consume Messages. Once the test results desired output, we can proceed and test publishing and consuming the queues (messages).

Integration with Magento 2.4

Once the RabbitMQ server is up and ready to accept client connections, developers need to configure Magento to be able to communicate with RabbitMQ.

Either provide the below-listed RabbitMQ associated environment variables via the command-line or append them to the env.php file:

• Host
• Port
• User
• Password
• Virtualhost
• Cafile
• Certfile
• Keyfile

We can now configure message queues in RabbitMQ. Per the official Magento documentation, configuring the message queue topology involves creating and modifying the following configuration files.

• communication.xml – defines aspects of the message queue system that all communication types have in common
• queue_consumer.xml – defines the relationship between an existing queue and its consumer
• queue_topology.xml – defines the message routing rules and declares queues and exchanges
• queue_publisher.xml – defines the exchange where a topic is published

Depending on the project’s needs, developers may only need to create and configure communication.xml and one or two of these files.

• If we only want to publish to an existing queue created by a third-party system, we will only need the queue_publisher.xml file.
• If we only want to consume from an existing queue, we will only need the queue_consumer.xml config file.
• In cases where we want to configure the local queue and publish it for third-party systems to consume, we will need the queue_publisher.xml and queue_topology.xml files.
• When we want to configure the local queue and consume messages published by third-party system, we will need the queue_topology.xml and queue_consumer.xml files.

Complete these steps once the configuration is complete:

• Run “php bin/magento setup:upgrade” to install custom module in Magento
• Run “php bin/magento queue:consumers:list” and verify consumer-defined in queue_consumer.xml is listed in the output
• Run “php bin/magento queue:consumers:start ConsumerName” to run the consumer

Let RabbitMQ be Your Middleman

These processes outlined above will help you easily integrate RabbitMQ with Magento 2.4 for publishing and consuming message queues. For more information on these technical integrations, contact our experts today. As always, happy messaging.