Using Node-RED with Kafka messages

In this guide we are going to show you how to use the npm package node-red-contrib-kafkajs in Node-RED to receive and send Kafka messages.

First, we are going to need to install the package. For that open the node-RED browser interface (through <IP of the installation device>:1880/nodered when you are using the United Manufacturing Hub).

Press on the 3 horizontal lines on the upper right corner to open the menu and select Manage palette.

This should display a new window, select the Install Tab.

In the search bar, type in node-red-contrib-kafkajs, select the package and press on install.

Now you should have two new nodes displayed in your node library: kafkajs-consumer and kafkajs-producer.

Kafkajs-consumer can listen in on kafka topics and relay the messages to linked nodes.

Kafkajs-producer will produce messages on the designated topic and a payload of what the upstream nodes feed the producer node.

When using either of these nodes, you will have to configure the kafkajs-client node. Important is that you put in the correct address of the kafka broker from which you want to receive messages.

One simple setup would look like this, the blue injection node, would send messages to the producer node for it to publish them.

In the inject node, you will need to configure your topic and payload. In the producer node, you need to make sure the node has the topic of the incoming message for it to publish it to the kafka broker.

kafkajs-consumer works in a similar way, you set up a topic for it to listen in on and then forward it when it receives a message. You can then set the message up to be processed to your desires and then either send it back out as an MQTT message with the mqtt-out node or with a kafkajs-producer as a Kafka message.