No connection to MQTT due to vernemq/HiveMQ pod timeout

️ This problem might also occur on HiveMQ from version 0.9.10 onwards. Let us know if you are having this problem and if this article could still help you

Problem

No connection to MQTT is possible, when looking at the pods in UMHLens, you will see that both mqttkafkabridge and vernemq pods are not working properly. When clicking on the vernemq pod, you will see the following errors:

Readiness probe failed: Get ”<IP>/health”: contect deadline exceeded (Client. Timeout exceeded while awaiting headers)

Solution

If there is a lot of data in vernemqs local database, it can occur that vernemq can’t handle starting up before it gets killed due to the readiness probe of Kubernetes.

To fix that, you need to go into the StatefulSets of the vernemq pod and change the initialDelaySeconds for livenessProbe and readinessProbe. The easiest way to fix it is to just add a 0 to both of them at the end.