Kafka stream processing via Lenses SQL, scale with Kafka - part 3

#LensesHowTos
photo of Andrew Stevenson
Andrew Stevenson

As mentioned in previous post, Lenses SQL leverages Kafka Streams to process data and currently provides 3 execution modes to run Lenses SQL processors. In this video we demonstrate how to scale out using CONNECT mode as well as how to manage the Lenses SQL processors via the Lenses web interface or the CLI tool.

About Lenses SQL Processor modes

IN_PROC is the default execution mode and the processors are executed locally within Lenses. This can have scalability issues and poses a risk to the running application, and can affect stability. IN_PROC is recommended only for testing.

CONNECT is the execution mode that solves these limitations and provides availability guarantees and scalability. Lenses can deploy your Lenses SQL processors in Kafka Connect. Kafka Connect provides a distributed, fault tolerant and scalable framework as part of the core Apache Kafka distribution.

KUBERNETES, which we’ll see in an upcoming blog post, is an execution mode that provides scalability by deploying Lenses SQL runners into Kubernetes clusters. Lenses can deploy and monitior SQL runner deployments created through Lenses or existing tools such as Helm or kubectl.

Additional Resources

Find out more about Lenses & Lenses SQL Processors: Lenses Documentation

Download Lenses Now at Downloads Page

Relevant Blogs


Share this article

Did you like this article?

Subscribe to get new blogs, updates & news

Follow us for news, updates and releases!
@LandoopLtd
LENSES
For Apache Kafka ®
Download Now
Share this article


2 Minute Overview


Discover awesome features


Community


Join us at Landoop Community


Resources


Repos, Docs, Trainings, Tutorials


Free Download


ALL-IN-ONE free for developers!