The Kafka Connect API

Jun 23 2020 Connect>Kafka

Apart from core APIs which requires external systems to install Kafka client for the integration, Kafka also supports Kafka Connect API with REST API for the more flexibility in communicating with different systems. This note is to collect list of basic APIs on frequent useage.

Viewing Connectors

1	curl http://localhost:8083/connector-plugins \| python -m json.tool

The | python -m json.tool above simply takes the output of the curl command and prints the JSON nicely. You can omit this if you’d like!

Create a Connector

curl -X POST -H 'Content-Type: application/json' -d '{
    "name": "first-connector",
    "config": {
        "connector.class": "FileStreamSource",
        "tasks.max": 1,
        "file": "/var/log/journal/confluent-kafka-connect.service.log",
        "topic": "kafka-connect-logs"
    }
  }' \
  http://localhost:8083/connectors

List connectors

1	curl http://localhost:8083/connectors \| python -m json.tool

Detailing connectors

1	curl http://localhost:8083/connectors/first-connector \| python -m json.tool

Pausing connectors

Sometimes its desirable to pause or restart connectors:

To pause

1	curl -X PUT http://localhost:8083/connectors/first-connector/pause

To restart

1	curl -X POST http://localhost:8083/connectors/first-connector/restart

Deleting connectors

1	curl -X DELETE http://localhost:8083/connectors/first-connector

#kafka