No failure of publish in case of broker down

[FStefanni]

Hi,

I have read the following "old" issue:

#1987

which shows also a code snippet on how to workaround the publish never failing if the broker goes down.
Here I report the proposed snippet:

client.publishAsyncSafe = async function(topic, message, options, timeout = 5000) {
    return Promise.race([
        client.publishAsync(topic, message, options),
        new Promise((resolve, reject) => setTimeout(() => reject(new Error('MQTT publishAsync() timeout')), timeout)),
    ]);
};

But this is my doubt:
in case of timeout, since the message is actually still enqueued for a publish, isn't this a memory leak?
In case of memory constraint, couldn't this lead to a always growing memory which is never cleaned?
Or even, since the application code fails, it will never know whether whether message has been actually sent (for example, if the connection turns up again).

Thank you for the clarification.
Regards

[robertsLando]

Hi Francesco, as I explained in that topic this is how MQTT protocol works, there is nothing on protocol specs that mentions timeouts on publish or a way to cancel outgoing messages. So yes even if the timeout triggers when and if the client connection will go up that message will be sent anyway.

[FStefanni]

Hi,

thank you for the reply.
It is clear to me that the MQTT protocol does not have options related to this issue.
But I am wondering if the library MQTT.js provides a way (also by manually clearing the queue for example) to avoid to saturate the memory.
This is the scenario I am working on:

• every 100ms the local app try to send data to the cloud via MQTT
• the data is stored in a local db, and it is delated from it only after having sent it successfully and received an "ack" from the cloud
• we must support the case that the connection goes down for a couple of days...

In this scenario we must avoid to make the memory "explode" by storing into the queue an unlimited number of packets, and also, in this case all these packets will be also duplicated (data is sent "in order")

So there is a way to minimize this issue? Maybe, via a manual timeout as in the linked issue, plus a manual mqtt queue clearing? Or maybe something better?

Thank you,
regards

[robertsLando]

I personally handle offline buffering using https://github.com/robertsLando/mqtt-jsonl-store and node:sqlite. Before publishing a message I check if client is connected, if not I push the packet to sqlite, otherwise I send it. It could be that sometime the client cannot deliver some packets so them will be queued by the jsonl store in that case, when client comes back online I publish all messages I have in node:sqlite (better if you can do this in batches instead of one by one).

If you want you could also watch the jsonl store or whatever store you provide to mqttjs and manage the packets stored there using provided methods

[FStefanni]

Hi,

ok thank you. I'll probably try something similar.

Regards