Understanding EMQ X broker cluster concepts

EMQX a distributed, highly available and highly scalable message broker. EMQX is designed and built using Erlang/OTP. In this article I will explain how to setup EMQ Cluster manually.

Understanding Erlang and OTP

Erlang is a functional programming language and runtime to build massively scalable and high available systems.

OTP (Open Telecom Platform) is a collection of useful Erlang tools, libraries and design principals to build softwares using Erlang programming language.

Understanding Erlang/OTP Distributed Runtime and Erlang Runtime Node

Erlang/OTP Distributed system is made of multiple Erlang runtime systems known as node. These Erlang runtime nodes communicate with each other by passing messages to each other using TCP/IP sockets.

Erlang runtime system / node is idenitified with a email like name e.g. node_name@host or node@127.0.0.1epmd(Erlang port mapper daemon) maps the node name with TCP Sockets. Authentication among Erlang nodes happens using magic cookie. Following snippet shows how you can set Erlang node cookie. Please note that the word COOKIE in following snippet is just a placeholder, you need to replace it with actual contents for Erlang node cookie.

erl -setcookie COOKIE

EMQ X Broker Cluster Architecture and Design

The architecture of EMQ X Broker Cluster is based on Erlang/OTP and Mnesiadatabase.

EMQ X Broker Cluster Design Rules

  1. MQTT Client** *SUBSCRIBE a TOPIC on a EMQ X Broker Cluster Node, this node will inform all other nodes in current Cluster.
  2. MQTT Client** *PUBLISH a MESSAGE to a EMQ X Broker Cluster Node, this node will forward this MESSAGE to all the nodes present in Topic Table.

Global Route table of TOPIC -> NODE

global route table with topic and nodes mappings is replicated to all nodes in cluster. Following a sample illustration of global route table.

Topic NameNodes
Topic1node1, node2, node3,node4
Topic2node1, node3
Topic3node2
Topic4node2, node3, node4

That’s it, hope you enjoyed it. You like this article, have any questions or suggestions please let us know in the comments section.

Thanks and Happy Learning!

Shoket Mahmood Ahmed

13 thoughts on “Understanding EMQ X broker cluster concepts

  1. Seriously thats precisely the advice that I have been hunting to get! You actually reached appreciate Google just for delivering us straight to your site!

  2. I want to thank you for the efforts you have put in writing this blogpost. I am hoping the same best blog post from you in the future as well. In fact your creative writing abilities has inspired me to start my own blog now. Truly the blogging is spreading its wings rapidly. Your write up is a fine example of it.

  3. My boyfriend bookmarked this link and I came here accidentally. Thanks a lot with this post! Really a good read. I dont have a lot of time right now to create a lot of my blathering opinions,(my lil girl needs attention, lol), however when I return tomorrow evening, I will explain in depth why I really dig with this article.

  4. really liked what that you published actually. it really isnt that simple to discover great posts to read (you know really READ and not simply browsing through it like some zombie before going to yet another post to just ignore), so cheers mate for really not wasting my time!

  5. Whoever said junk food tastes better than its healthier counterpart has obviously not feasted their eyes (and mouth!) on Cooking Light magazine. This monthly publication scores low-fat brownie points for providing recipes that are healthy AND tasty at the same time. Yes, Healthy And tasty.

  6. I love your web site.. very nice colors & theme. Did you create this internet site yourself or did you hire somebody to do it for you? Plz reply back as I’m looking to create my own blog and would like to know where u got this from. thanks

  7. Hi there, MegaCool blog mate, I really loved this page. I’ll be sure to talk about this to my cousin who would, odds are, love to check out this post too. Found this sites post through the Bing search engine by the way, incase you were curious. Many thanks for the wonderful read!

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top