How to install RabbitMQ on Kubernetes Cluster

Introduction

Message queuing systems play a critical role in building scalable, distributed applications. RabbitMQ, an open-source message broker, offers a robust and flexible solution for managing message queues. In this article, we will discuss the fundamentals of message queues, delve into the unique features of RabbitMQ, and compare it with other message queue solutions.

What is Message Queue?

Message queues are intermediary systems that enable asynchronous communication between distributed applications. They store and transmit messages between producers (applications that create messages) and consumers (applications that process messages). This asynchronous communication allows applications to operate independently, without waiting for immediate responses, thereby improving overall system performance.

Producers and Consumers

In a message queuing system, producers create messages and send them to the message queue. Consumers then retrieve and process these messages from the queue. This separation of concerns allows each component to function independently, even when the other is not available or is experiencing high load.

Message persistence and durability options

Message queues can be configured to store messages persistently or transiently. Persistent messages are stored on disk, ensuring that they are not lost in case of a system crash or restart. Transient messages, on the other hand, are stored in memory and may be lost if the system goes down.

Use Cases

Examples of when to use message queues Message queues are useful in various scenarios, such as:

• Offloading resource-intensive tasks to background workers
• Distributing tasks among multiple instances of an application for load balancing
• Ensuring fault tolerance by allowing messages to be reprocessed in case of errors

Using Message Queue in Applications

Load Balancing

Message queues distribute workload among multiple instances of an application, ensuring efficient resource utilization and preventing bottlenecks.

Fault Tolerance

Message queues contribute to fault tolerance by allowing messages to be reprocessed in case of errors, ensuring a high level of application reliability.

Decoupling Components

Message queues decouple components of a distributed system, allowing each component to evolve independently and scale according to demand.

RabbitMQ

RabbitMQ was developed by Rabbit Technologies Ltd. in 2007 as an implementation of the Advanced Message Queuing Protocol (AMQP). Since then, it has grown in popularity and is now maintained by VMware Tanzu, with a large and active community contributing to its development.

Unique features and benefits

Protocol Support
RabbitMQ was designed around AMQP but also supports other messaging protocols, such as MQTT and STOMP, making it suitable for various use cases.

Flexibility and Extensibility
RabbitMQ is highly configurable, allowing developers to tailor it to their specific needs. Its plugin-based architecture enables the addition of custom functionality, further extending its capabilities.

Clustering and high availability
RabbitMQ supports clustering, allowing multiple instances of the broker to work together to distribute the load. It also supports high availability through mirrored queues, ensuring that messages are not lost even in case of node failures.

Comparison with another solutions

There are many Message Queue solutions in the market right now. Each with its uniqueness and functionality. Here are some glimpse when we compare RabbitMQ with another tools like Apache Kafka or AWS SQS.

Strengths and weaknesses

• RabbitMQ: Offers a flexible and extensible solution with support for multiple messaging protocols, clustering, and high availability. However, it may not be as fast as other solutions like Apache Kafka for specific high-throughput scenarios.
• Apache Kafka: Provides high throughput and fault tolerance, making it ideal for large-scale data streaming applications. However, it has a more complex setup and administration compared to RabbitMQ.
• Amazon SQS: Offers a fully managed, scalable, and serverless message queue service, which can be an advantage for organizations looking to minimize infrastructure management. However, it is limited by the feature set and pricing model of the AWS ecosystem.
• ActiveMQ: Supports multiple messaging protocols and provides a wide range of features, including message prioritization and filtering. However, it might not be as scalable as other solutions like RabbitMQ or Apache Kafka.
• Redis: Although not primarily a message queue solution, Redis can be used as a lightweight, in-memory message broker. It offers high performance but lacks some of the advanced messaging features available in dedicated message queue solutions like RabbitMQ.

Install RabbitMQ on Kubernetes Cluster

Before proceeding with the installation of RabbitMQ on your Kubernetes cluster, it's important to ensure that Helm is installed and properly configured. Helm is a package manager for Kubernetes, which simplifies the deployment and management of applications on a cluster.

In this guide, we will be using the RabbitMQ Helm chart provided by Bitnami. Bitnami is a well-known provider of open-source software stacks and maintains up-to-date, reliable Helm charts for various applications, including Rabbit MQ.

~$ helm repo add bitnami https://charts.bitnami.com/bitnami
~$ helm repo update

Create dedicated namespace for RabbitMQ

~$ kubectl create namespace rabbitmq

A minimum of 10 GB storage disk is required. You can either utilize your own Persistent Volume Claim (PVC) or opt for Network File System (NFS) as detailed in this article:

How to setup persistent storage with NFS on GKE (Google Kubernetes Engine), the proper way

A Persistent Volume (PV) in Kubernetes is a storage resource abstraction that provides a way to store and manage persistent data for…

Medium8grams

Additionally, an SSL certificate is necessary for the RabbitMQ Dashboard Ingress since we are publicly exposing the RabbitMQ Dashboard in this tutorial. To learn how to obtain an SSL certificate, refer to this article:

How to get Free SSL Certificate for Kubernetes Cluster using Let’s Encrypt

Using SSL is essential for any website that wants to protect its users’ sensitive information. Without SSL, data can be intercepted and stolen by hackers, putting users’ privacy and security at risk.

8grams - DevOps Consultant8grams

Create values.yaml as Helm values. Adjust it with your own values.

Install with Helm

~$ helm install rabbitmq bitnami/rabbitmq --namespace rabbitmq -f values.yaml

Verify Installation

~$ kubectl get statefulset -n rabbitmq

NAME       READY   AGE
rabbitmq   1/1     1m

Looks good! Now you can open https://rabbitmq-dashboard.example.com , the URL we set on Ingress configuration, to access RabbitMQ Dashboard.

Login with username and password you set on values.yaml.

Troubleshooting

Occasionally, you may encounter issues logging into the RabbitMQ Dashboard due to various reasons. To address this problem, you can simply create a new user in RabbitMQ.

Go to RabbitMQ Pod

~$ exec kubectl exec -i -t -n rabbitmq rabbitmq-0 -c rabbitmq -- sh -c "clear; (bash || ash || sh)"

To create a new user, in this example named foo with the password bar, follow these steps. Optionally, you can also grant the user administrator rights and assign proper permissions:

~$ rabbitmqctl add_user foo bar

~$ rabbitmqctl set_user_tags foo administrator

~$ rabbitmqctl set_permissions -p / foo ".*" ".*" ".*"

About 8grams

We are a small DevOps Consulting Firm that has a mission to empower businesses with modern DevOps practices and technologies, enabling them to achieve digital transformation, improve efficiency, and drive growth.

Ready to transform your IT Operations and Software Development processes? Let's join forces and create innovative solutions that drive your business forward.

Subscribe to our newsletter for cutting-edge DevOps practices, tips, and insights delivered straight to your inbox!