For my setup, I chose to deploy the MariaDB Galera Cluster and RabbitMQ Cluster on dedicated VMs instead of running them within the Kubernetes cluster. A few key considerations drove this decision:
- Simplicity over complexity: The anticipated application load is relatively low, so I avoided over-engineering the architecture.
- Upfront resource overprovisioning: Allocating more resources early on helps ensure stability without complicating scaling efforts.
- Scalability, when needed: Horizontal scaling can be handled easily if required, though its demand would not be frequent or critical at this stage.
- Reliable storage: I use Ceph with RBD-backed volumes, which offer high reliability.
- Snapshot safety net: Ceph’s point-in-time RBD snapshots provide an additional layer of protection.
- Lower network overhead: Running these core services outside of Kubernetes helps reduce unnecessary network complexity within the cluster.
I opted for MariaDB version 11.8 due to its support for Vector DB integration.
Launch 3 Debian 12 VMs (8 vCPU, 32 G RAM).
Install the Mariadb server in all the nodes and stop the service.
sudo apt-get install apt-transport-https curl
sudo mkdir -p /etc/apt/keyrings
sudo curl -o /etc/apt/keyrings/mariadb-keyring.pgp 'https://mariadb.org/mariadb_release_signing_key.pgp'
sudo apt-get update
sudo apt-get install mariadb-server mariadb-backup
systemctl stop mariadb.service
Each VM has 3 x 10G NICs. The first is for VM Management, the second is for applications to access MariaDB, and the third is for the Galera cluster. In my case, the IP addresses associated with the Galera clustering of the three nodes are 10.0.3.16, 10.0.3.17, and 10.0.3.18.
Edit /etc/mysql/mariadb.conf.d/50-server.cnf in all nodes and ensure the bind address is 0.0.0.0. (All nodes)
Create /etc/mysql/conf.d/galera.cnf with the following contents (All nodes)
[mysqld]
binlog_format=ROW
default-storage-engine=innodb
#Innodb specific configurations
# Setting aside 2G of OS
innodb_buffer_pool_size=23G
#Recommended 1/4th or 1/8th of innodb_buffer_pool_size
innodb_log_file_size=4G
#General recommendation 2
innodb_log_files_in_group=2
innodb_flush_method=O_DIRECT
#Automatic=0, recommendation vCPU * 2
innodb_thread_concurrency=16
aria_pagecache_buffer_size=2G
aria_sort_buffer_size=1G
bind-address=0.0.0.0
log-error=/var/log/mysql/mysql.err
#Galera Provider Configuration
wsrep_on=ON
wsrep_provider=/usr/lib/galera/libgalera_smm.so
#Galera Cluster Configuration
wsrep_cluster_name="dccluster"
wsrep_cluster_address="gcomm://10.0.3.16,10.0.3.17,10.0.3.18"
#Galera Synchronization Configuration
wsrep_sst_method=mariabackup
#Galera Node Configuration
wsrep_node_address="10.0.3.16"
wsrep_node_name="msynch1"
In one of the nodes, initiate the galera_cluster
# galera_new_cluster
In the other two nodes
systemctl start mariadb
Verify cluster status
MariaDB [(none)]> show status like "wsrep_cluster%";
+----------------------------+--------------------------------------+
| Variable_name | Value |
+----------------------------+--------------------------------------+
| wsrep_cluster_weight | 3 |
| wsrep_cluster_capabilities | |
| wsrep_cluster_conf_id | 31 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | 52bc4c69-00dc-11f0-8e21-02fe63fa97a5 |
| wsrep_cluster_status | Primary |
+----------------------------+--------------------------------------+
6 rows in set (0.001 sec)
MariaDB [(none)]>