Monitoring Kafka

Kafka with Plumber

In this guide, we will walk you through how to use Kafka with Plumber.

Installation

To get started, you will first need to install Plumber. You can do this using the following command:

$ brew tap streamdal/public
$ brew install plumber

Reading

Plumber can be used to read messages from Kafka. Here is a basic example of how you can use the plumber read command with Kafka

    --address="broker1.domain.com:9092 --continuous

Writing

Plumber can also be used to write messages to Kafka. Here is an example of how you can use the plumber write command with Kafka:

    --address "broker1.domain.com:9092" \
    --address "broker2.domain.com:9092" \
    --address "broker3.domain.com:9092" \
    --input="{\"order_id\": \"A-3458-654-1\", \"status\": \"processed\"}"

Relaying

Plumber also supports relay mode, which allows you to read messages from one backend and write them to another. Here is an example of how you can use the plumber relay command with Kafka:

    -e PLUMBER_RELAY_TOKEN="$YOUR-STREAMDAL-TOKEN-HERE"
    -e PLUMBER_RELAY_KAFKA_ADDRESS="pkc-4kgmg.us-west-2.aws.confluent.cloud:9092,pkc-5kgmg.us-west-2.aws.confluent.cloud:9092"
    -e PLUMBER_RELAY_KAFKA_TOPIC="$YOUR_TOPIC"
    -e PLUMBER_RELAY_KAFKA_INSECURE_TLS="true"
    -e PLUMBER_RELAY_KAFKA_USERNAME="$YOUR_CONFLUENT_API_KEY"
    -e PLUMBER_RELAY_KAFKA_PASSWORD="$YOUR_CONFLUENT_API_SECRET"
    -e PLUMBER_RELAY_KAFKA_SASL_TYPE="plain"
    streamdal/plumber plumber relay kafka

More Examples

https://github.com/streamdal/plumber/blob/master/docs/examples.md