Kafka Topics UI

Kafka-topics-ui interacts with the Kafka rest-proxy to allow browsing data from Kafka Topics. Gain visibility to explore your Kafka data, Kafka Schemas, Kafka connectors and more

Aug 06, 2016

Hey there, You are looking at an older version. Go to Lenses to view data in Kafka.

Kafka is now the de-facto platform for streaming architectures, and it’s eco-system is maturing, but is not just yet as Enterprise Ready as many people in Big | Fast Data would like it to be. Lenses.io is a London based start-up that wants to drive Kafka faster to the future, and thus…

We are announcing the kafka-topics-ui a User Interface that allows browsing data from Kafka Topics and a lot more

About Kafka-Topics-UI

The kafka-topics-ui is a user interface that interacts with the Kafka rest-proxy to allow browsing data from Kafka Topics. At the same time it brings visibility by providing a single entry point to explore i) Kafka data, ii) Kafka Schemas, iii) Kafka connectors and a lot more, such as partitions per topic, replication factor per topic and topic configurations.

Kafka REST Proxy

The Kafka REST Proxy - is an excellent state-less micro-service by Confluent that allows anyone to easily:

Create a consumer or producer via a POST request
Consume or Produce Avro | JSON | Binary data on a Kafka Topic
Delete a consumer

Utilizing it’s capabilities for building a UI was a straight-forward processes after overcoming the initial difficulties. First and foremost we had to tackle the fact you can’t tell a priory what type of data a particular topic contains. Once we figured out that, we can

hard-code the type of particular config-topics
automatically detect Avro type by fetching the _schemas topic
in case type is still unknown iterate through Avro->JSON->Binary consumers till one succeeds

Then things got so much more interesting. Another hidden gem was the flag - that allows us to efficiently stream just enough data to feed the UI and achieve the above process in just 2-3 seconds.

?max_bytes=50000

Kafka-Topics-UI features

Currently

Identifies and visualizes Kafka topics and config overrides
Automatically detects data types (Avro|JSON|Binary)
Base64 decodes and parses of binary topics
Provides Table and JSON views
Allows downloading data from Kafka topics
Displays number of partitions and replication factor per topic
Docker container

Using it

Check https://github.com/landoop/kafka-topics-ui and follow instructions (Tip: Remember to enable CORS). The easiest way to get it running is using docker:

docker pull landoop/kafka-topics-ui

And run the container, by specifying the url of the kafka-rest service:

Quick preview

Conclusions

Hey there, This is an old version. Check Lenses to view data in Kafka or the new version for Kafka Topic UI here.

Lenses.io - similarly to you continuously brings to light best practices while thriving for operational excellence. Together with the open-source community we are aiming to enhance kafka-topics-ui with new features and capabilities: