RabbitMQ Reverse Topic Exchange Type

This plugin adds a reverse topic exchange type to RabbitMQ. The exchange type is x-rtopic.

The idea is to be able to specify routing patterns when publishing messages. With the default topic exchange patterns are only accepted when binding queues to exchanges.

With this plugin you can decide which queues receive the message at publishing time. With the default topic exchange the decision is made during queue binding.

With this exchange your routing keys will be words separated by dots, and the binding keys will be words separated by dots as well, with the difference that on the routing keys you can provide special characters like the # or the *. The hash # will match zero or more words. The star * will match one word.

Usage

If we have the following setup, (we assume the exchange is of type rtopic):

• Queue A bound to exchange rtopic with routing key "server1.app1.mod1.info".
• Queue B bound to exchange rtopic with routing key "server1.app1.mod1.error".
• Queue C bound to exchange rtopic with routing key "server1.app2.mod1.info".
• Queue D bound to exchange rtopic with routing key "server2.app2.mod1.warning".
• Queue E bound to exchange rtopic with routing key "server1.app1.mod2.info".
• Queue F bound to exchange rtopic with routing key "server2.app1.mod1.info".

Then we execute the following message publish actions.

%% Parameter order is: message, exchange name and routing key.

basic_publish(Msg, "rtopic", "server1.app1.mod1.info").
%% message is only received by queue A.

basic_publish(Msg, "rtopic", "*.app1.mod1.info").
%% message is received by queue A and F.

basic_publish(Msg, "rtopic", "#.info").
%% message is received by queue A, C, E and F.

basic_publish(Msg, "rtopic", "#.mod1.info").
%% message is received by queue A, C, and F.

basic_publish(Msg, "rtopic", "#").
%% message is received by every queue bound to the exchange.

basic_publish(Msg, "rtopic", "server1.app1.mod1.*").
%% message is received by queues A and B.

basic_publish(Msg, "rtopic", "server1.app1.#").
%% message is received by queues A, B and E.

The exchange type used when declaring an exchange is x-rtopic.

Supported RabbitMQ Versions

The most recent release of this plugin targets RabbitMQ 3.12.x.

Supported Erlang/OTP Versions

This plugin requires Erlang 25.0 or later versions, same as RabbitMQ 3.12.x.

Installation and Binary Builds

This plugin is now available from the RabbitMQ community plugins page. Please consult the docs on how to install RabbitMQ plugins.

Then enable the plugin:

rabbitmq-plugins enable rabbitmq_rtopic_exchange

Building from Source

See Plugin Development guide.

TL;DR: running

make dist

will build the plugin and put build artifacts under the ./plugins directory.

Examples and Tests

To run the tests use make tests.

The test suite can also be used for code examples.

Performance

Internally the plugin uses a trie like data structure, so the following has to be taken into account when binding either queues or exchanges to it.

The following applies if you have thousands of queues. After some benchmarks I could see that performance degraded for +1000 bindings. So if you have say, 100 bindings to this exchange, then performance should be acceptable in most cases. In any case, running your own benchmarks wont hurt. The file rabbit_rtopic_perf.erl has some precarious tools to run benchmarks that I ought to document at some point.

A trie performs better when doing prefix searches than suffix searches. For example we have the following bindings:

a0.b0.c0.d0
a0.b0.c1.d0
a0.b1.c0.d0
a0.b1.c1.d0
a0.b0.c2.d1
a0.b0.c2.d0
a0.b0.c2.d1
a0.b0.c3.d0
a1.b0.c0.d0
a1.b1.c0.d0

If we publish a message with the following routing key: "a0.#", it's the same as asking "find me all the routing keys that start with "a0". After the algorithm descended on level in the trie, then it needs to visit every node in the trie. So the longer the prefix, the faster the routing will behave. That is, queries of the kind "find all string with prefix", will go faster, the longer the prefix is.

On the other hand if we publish a message with the routing key "#.d0", it's the same as asking "find me all the bindings with suffix "d0". That would be terribly slow to do with a trie, but there's a trick. If you need to use this exchange for this kind of routing, then you can build your bindings in reverse, therefore you could do a "all prefixes" query instead of a "all suffixes" query.

If you have the needs for routing "a0.#.c0.d0.#.f0.#" then again, with a small amount of binding keys it should be a problem, but keep in mind that the longer the gaps represented by the # character, the slower the algorithm will run. AFAIK there's no easy solution for this problem.

Creating a Release

1. Update broker_version_requirements in helpers.bzl & Makefile (Optional)
2. Update the plugin version in MODULE.bazel
3. Push a tag (i.e. v3.12.0) with the matching version
4. Allow the Release workflow to run and create a draft release
5. Review and publish the release

License

See LICENSE.

Copyright

(c) 2007-2022 VMware, Inc. or its affiliates.

Originally developed by Álvaro Videla.