## Zilla MQTT support is now GA
At Current 2023, in a session titled [“Handle Millions of IoT Devices Connected to Kafka via MQTT,”](https://www.confluent.io/events/current/2023/handle-millions-of-iot-devices-connected-to-kafka-via-mqtt/) we presented an exciting development to the data streaming community: [Zilla](https://github.com/aklivity/zilla) —our multi-protocol, Kafka-native proxy— added support for MQTT. Today, we’re thrilled to share that Zilla’s MQTT capabilities have been rounded out and are readily available!
With Zilla, you can now turn Kafka into an MQTT broker, enabling IoT clients to consume and produce Kafka event streams without any coding, integration pipelines, or dedicated MQTT brokers. Industry 4.0, smart cities, connected cars, and other exciting IoT use cases can be streamlined like never before!
### TL;DR
- Zilla turns Kafka into a fully-fledged MQTT broker, simplifying IoT deployments by eliminating the need for a standalone MQTT broker and integration pipelines.
- Zilla supports major MQTT versions (v3.1.1 and v5.0) and has advanced MQTT routing capabilities.
- Zilla is stateless and handles MQTT connection off-loading, ensuring IoT-scale use cases are not a problem.
## Message Queuing Telemetry Transport (MQTT)
MQTT is a lightweight messaging protocol that uses a publish/subscribe model, catering to devices with limited resources. Its efficiency, support for various Quality of Service levels, and features like retained messages and session persistence make it widely used in IoT applications.
While MQTT is a great protocol for managing IoT clients, it was not designed for data processing and integration. To optimally respond to IoT inputs and apply them to business workflows, MQTT messages must land in a modern data streaming platform such as Apache Kafka.
## Apache Kafka for MQTT Data Processing and Integration
Apache Kafka is designed for high-throughput, scalable, and fault-tolerant data streaming. It has a rich ecosystem of data transformation, enrichment, and integration tools, enabling it as a central data storage and processing hub.
Like MQTT, Kafka also has its own messaging pub/sub protocol, and as a result, one might be inclined to interface IoT clients directly with it, bypassing MQTT altogether. But this isn’t recommended, as Kafka is not built for tens of thousands of connections and requires clients to run on a stable network. So, to make the most of an IoT deployment, MQTT and Kafka must be combined.
## Unifying MQTT and Apache Kafka with Zilla
Zilla is an [open-source](https://github.com/aklivity/zilla), multi-protocol proxy for connecting clients and services to Apache Kafka. It natively supports the Kafka wire protocol and uses novel protocol mediation techniques to establish stateless API entry points into Kafka. Supported protocols/APIs include REST, Server-Sent Events (SSE), gRPC, and now MQTT!
### Implementing MQTT Specs
MQTT comes in two main flavors: [v3.1.1](http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html) and [v5.0](https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html). While v5.0 is significantly enhanced over its predecessor (and an upgrade is strongly advised for existing MQTT clients), in IoT scenarios with deeply remote clients, doing so is not always possible. Recognizing this, we’ve ensured Zilla supports both MQTT versions so it can be applied in legacy and modern IoT deployments.
### Turning Kafka into an MQTT Broker
As mentioned, Zilla turns Kafka into an MQTT broker. Zilla uses Kafka as a persistence layer for messages and managing MQTT client state, meaning it doesn't just dump MQTT messages onto Kafka topics. Zilla mediates between the protocols, converting to MQTT in both directions. This enables MQTT clients to interact with Kafka with all of the features and guarantees that a dedicated MQTT broker would provide, such as Keep-Alive, Last Will and Testament, and all three Quality of Service (QOS) agreements. Additionally, Kafka clients can interact with the same topics receiving messages from and delivering messages to MQTT clients. Either type of message consumer, MQTT or Kafka client, can manage the correlated request-response data to successfully deliver a result back to the original producer. This is all through the MQTT protocol so MQTT clients are unaware they’re interfaced with Kafka.
#### A Note on Topic Spaces
One of the challenges of turning Kafka into an MQTT broker is their differing approaches to topic spaces.
Both MQTT and Kafka have the concept of a “topic” — a logical grouping of messages that clients publish and subscribe to. But whereas in Kafka topics are explicitly separate, in MQTT they are more closely coupled. This is because in MQTT a topic is characterized as a UTF-8 string that consists of one or more nested levels separated by a forward slash, e.g. `usa/ca/palo_alto/aklivity_hq/pets` is an MQTT topic for the current number of pets in our office…the answer is three if you’re wondering. Therefore, an MQTT topic space can be viewed as a single space where topic names are treated as filters.
Mapping an exact MQTT topic to a corresponding Kafka topic is straightforward, but splitting an MQTT topic space over different Kafka topics is complex because Kafka doesn’t have server-side filtering. Zilla addresses this with advanced routing and pattern-matching capabilities, ensuring MQTT and Kafka topic topologies are properly aligned.
### Handling Millions of Connections
IoT deployments are notorious for client connection counts that can reach into the
millions. While Zilla is stateless and able to scale out linearly, in order to effectively broker connectivity to Kafka it cannot afford to create a new connection for each connected MQTT client. Kafka is constrained by the number of connections it can handle, and so a key development in Zilla to ensure proper scalability is MQTT to Kafka connection offloading.
## Get Started with Zilla
The best way to get started with Zilla and try out its latest MQTT capabilities is by heading over to our docs. If you want to quickly see Zilla’s protocol mediation capabilities in action, jump to the [Quickstart](https://docs.aklivity.io/zilla/next/tutorials/metrics/prometheus-intro.html). For a more in-depth experience that walks you through deploying Zilla in front of your own Kafka broker, follow the [MQTT-Kafka Proxying Guide.](https://docs.aklivity.io/zilla/next/how-tos/mqtt/mqtt.kafka.broker.html)
We also have a fun [taxi demo](https://github.com/aklivity/zilla-demos/tree/main/taxi) showcasing how Zilla and Kafka can track a hypothetical fleet of taxi cabs hailed to and from local bars in San Jose, California.
## That's a Wrap!
MQTT is the de facto standard protocol for IoT. Kafka is the powerhouse data streaming platform that has found its way into almost every corner of IT and data engineering. These two worlds were separate and connected only as an afterthought with plugins and additional components. Now, with Zilla, they are seamlessly brought together!
If you’re excited about Zilla and our effort in building out event-native API infrastructure, please support the project by starring on [GitHub](https://github.com/aklivity/zilla). Zilla is open-source, so we welcome all input and participation! If you have questions about the latest updates or just want to chat with our team, join our [Aklivity Community on Slack](https://www.aklivity.io/slack).
