Message gets redelivered in loop during the time of high CPU processing on the client side

[coder5492]

This is the flow of the issue

• A queue message which requires high amount of CPU processing reaches the client.
• Client is not able to respond to the heartbeat from the broker.
• Broker redelivers the same message and also gives "unexpected ACK" error as the ack Id changes on redelivery of same message and previous ack id becomes invalid
• This seems to happen again and again for a while
• I receive connect time out error around - 250000 times after the first message acknowledges
• Consumer count shows up as 0 in broker console, for all queues in the nodeJs application, not only the one which the messages were being sent to
• Consumers are not able to reconnect back, broker always shows consumers as 0

When I disable both heartbeat everything seems to work fine
When I disable the broker to client heartbeat, message occasionally redelivers but the consumers still go to 0 in the broker console
I can't disable the heartbeat to make it work as it has it's own advantages

In all of these cases I am receiving high amounts of connect time out error.
During low or average CPU loads on the nodejs server, everthing seems to work fine. Issue is only when the load is high

This is the code for connection

const stompit = require('stompit')
const keys = require('./keys')
const logger = require('../utils/logger.util')
const reconnectOptions = {
  useExponentialBackOff: true
}
let ssl = !!parseInt(keys.activeMQ.ssl)
const serverPrimary = {
  host: keys.activeMQ.host,
  port: keys.activeMQ.port,
  ssl: ssl,
  connectHeaders: {
    host: '/',
    login: keys.activeMQ.username,
    passcode: keys.activeMQ.password,
    'heart-beat': '5000,5000',
  },
}

const serverFailover = {
  host: keys.activeMQ.host1,
  port: keys.activeMQ.port,
  ssl: ssl,

  connectHeaders: {
    host: '/',
    login: keys.activeMQ.username,
    passcode: keys.activeMQ.password,
    'heart-beat': '5000,5000',
  },
}
connManager = new stompit.ConnectFailover(
  [serverPrimary, serverFailover],
  reconnectOptions,
)

connManager.on('error', function (e) {
  const connectArgs = e.connectArgs
  const address = connectArgs.host + ':' + connectArgs.port
  logger.error({ error: e, customMessage: address })
})

channelPool = new stompit.ChannelPool(connManager)
module.exports = channelPool

@gdaws Could you please help me out on this. I have been stuck on this for weeks

Main issues being

• Redelivery in loop of the same message on loss of heartbeat from the application during high load
• Consumers becoming 0 in broker console. Hence new messages piling up on the broker
• Consumer Connection doesn't reestablish after broker console becomes 0.

[medchkir]

Hi,
I have the same problem. I reproduced with a timeout after 20s (> heartbeat).

[medchkir]

Hello, i tried to migrate to activemq artemis version and it seems to work fine

[hanseld28]

I have the exact same problem with the classic and artemis version. Is this problem related to STOMP protocol, stompit library, Node.js or the broker?