About three-fifths of the world’s railroads have a distance of 4 feet 8.5 inches (1435mm) between the inside faces of their tracks. While seemingly trivial, this measure is actually one of the most significant technological innovations of the 19th century. It's known as the Standard Railway Gauge.
Before railways were standardized, countries and even individual cities would set gauges at will, resulting in:
- **Disjunction:** Trains of various gauges could not travel from one railway line to another, limiting the efficiency and reach of railway transport.
- **Increased Cost and Complexity:** The need for multiple sets of rolling stock and track equipment for different gauge lines increased costs and operational complexity.
- **Hindered Trade and Economic Growth:** Goods and passengers had to be transferred between trains of different gauges, causing delays and increasing costs.
Just as standards in track gauges led to interoperability, reduced costs, and increased efficiency, the same has been true for APIs.
API standards, such as the OpenAPI and more recent AsyncAPI specifications, have been critical to the latest API development and management efforts. With their help, developers quickly create documentation, generate mock servers, and validate API interactions.
Today, I’m thrilled to share that the Zilla proxy is OpenAPI and AsyncAPI compatible! With Zilla, you can validate API requests and message streams based on OpenAPI and AsyncAPI schemas. Moreover, you can use your existing OpenAPI and AsyncAPI definitions to generate non-Kafka interfaces to your Kafka broker.
This latest development not only further simplifies configuring Zilla, as you can just drop in an existing OpenAPI/AsyncAPI specification, but because Zilla can now expose standard API interfaces, it can seamlessly integrate with API management workflows and tooling. The worlds of API Management and data streaming are brought closer together than ever before!
To demonstrate all of this, below we have an “eventful” take on the classic OpenAPI/Swagger Petstore. But before jumping into the demo, let’s take a closer look at what’s behind the the latest Zilla release.
### TL;DR
- The Zilla proxy now supports OpenAPI and AsyncAPI specifications for configuration and validation.
- With just a few lines of YAML Zilla can make OpenAPI service definitions asynchronous and event-driven by mapping them to Kafka (see the Zilla Petstore demo below).
- Zilla can enforce OpenAPI and AsyncAPI schemas pulled from a remote schema registry such as Apicurio. Integrations with Karapace and Confluent Schema Registry for Avro, JSON, and Protobuf schemas have also arrived.
- Zilla is now better aligned with existing API management tooling and workflows.
## OpenAPI and Zilla
Zilla is a multiprotocol edge and service proxy that can mediate between different network and data protocols. Notably, Zilla supports Kafka’s wire protocol as well as HTTP, gRPC, and MQTT. This allows it to be configured as a Kafka proxy that lets non-native Kafka clients, applications, and services consume and produce data streams via their own APIs of choice.
In the case of HTTP-Kafka proxying, Zilla enables exposing Kafka via an application-centric REST API (as opposed to a [system-level API](https://www.aklivity.io/post/bring-your-own-rest-apis-for-apache-kafka) from the likes of the Confluent REST proxy). Zilla’s Kafka REST APIs not only make Kafka look like a traditional REST server, with conventional URIs and CRUD operations, but also support correlated request-response over topics. This enables the creation of asynchronous REST APIs and unlocks CQRS design patterns.
Before the OpenAPI support, setting up Zilla as a REST-Kafka proxy required explicitly defining API endpoints and routing them to Kafka topics. While this was done declaratively via YAML and was already quite approachable, it was a manual effort and potentially redundant one if an existing REST API was being ported. Now, to configure a Zilla REST-Kafka proxy, you can just reference an existing OpenAPI definition, including one that’s maintained in an API catalog, such as the Apicurio Registry.
If a net new Kafka-REST API is being created, it can still be directly declared in Zilla or you can use your favorite API design tool (Swagger, Postman, Stoplight, etc.) to describe an OpenAPI specification that can then be referenced.
Looking at a side-by-side comparison of a Zilla REST-Kafka proxy configured explicitly and one with OpenAPI, you can see that the latter requires 4x fewer lines of YAML!
To highlight the difference even further, let’s pull up the [Zilla VSCode extension](https://marketplace.visualstudio.com/items?itemName=aklivity.zilla-vscode-ext) and visualize how much simpler the Zilla YAML has become.
A Zilla configuration without OpenAPI:
...and one with:
## The AsyncAPI Initiative
Looking at the updated Zilla configuration above, the keen-eyed might have noticed that not only are REST API routes no longer explicitly defined, but there’s no mention of Kafka topics either! This is because they are addressed with an AsyncAPI specification, which Zilla is parsing alongside the OpenAPI counterpart.
The [AsyncAPI Initiative](https://www.asyncapi.com/en) is a project dedicated to standardizing the description and documentation of event-driven APIs, akin to what OpenAPI does for RESTful ones. Like OpenAPI, AsyncAPI has a specification, but whereas the former has to address only one type of protocol (HTTP) and API (REST), the latter covers many — Kafka, MQTT, WebSocket and AMQP. [HERE’s](https://www.asyncapi.com/docs/tutorials/getting-started/coming-from-openapi) a great resource to see how the two compare.
While AysncAPI is a more recent effort, its adoption and tooling ecosystems are rapidly growing. As software architectures become increasingly event-driven and data stacks standardize on streaming, the importance of AsyncAPI will continue to rise quickly.
## Mapping OpenAPI to AsyncAPI
Just as in OpenAPI, the AsyncAPI specification has a concept of an “operation” that represents a particular action or interaction an application can perform. Because AsyncAPI has to cover different protocols, an additional concept of a “channel” exists, which defines how messages are organized and transmitted. Depending on the protocol addressed, channels can be defined as a topic, queue, routing key, path, or subject.
To map OpenAPI to AsyncAPI and achieve correlated request-response over Kafka topics, we needed a way to carry out multiple request-reply operations over the same AsyncAPI channel. Unfortunately, this wasn’t available in earlier versions of the AsyncAPI specification, and it was only until just a few months ago, with AsyncAPI v3, that we finally had everything needed to achieve the mapping!
## The Eventful Petstore Zilla Demo
The [Petstore API](https://petstore.swagger.io/) is a synchronous RESTful service for managing pet data. It showcases how tools such as Swagger can be used to document, visualize, and interact with an OpenAPI server. With Zilla, let’s make the Petstore example more lively by mapping it onto Kafka and unlocking asynchronous interactions.
Follow the recording or run the [DEMO](https://github.com/aklivity/zilla-demos/tree/main/petstore) yourself.
[](https://youtu.be/FemWt31R7jg)
## Mapping AsyncAPI to AsyncAPI
One of Zilla’s most powerful features is its ability to turn Kafka into a full-fledged MQTT broker. It does this by mediating between the MQTT and Kafka wire protocols while maintaining MQTT client state across Kafka topics.
Previously, just as with REST-Kafka proxying, Zilla’s MQTT-Kafka proxy had to be directly configured. Now, besides mapping OpenAPI to AsyncAPI, Zilla can also cross-map different AsyncAPI specifications.
Once again, configuring Zilla with existing API specifications is not only easier and faster, but it enables alignment with API management practices and tooling. Solutions such as the [AsyncAPI Studio](https://studio.asyncapi.com/) for generating documentation and the [Apicurio Registry](https://www.apicur.io/registry/) for schema governance and maintenance, make supporting IoT deployments with Zilla, MQTT, and Kafka more controlled, secure, and agile.
You can see how Zilla maps MQTT to Kafka AsyncAPIs by checking out the [Zilla IoT Taxi Demo](https://github.com/aklivity/zilla-demos/tree/main/taxi).
## Support for Registries
Registries are a vital component in any distributed data system, ensuring data consistency, compatibility, and efficiency in environments where schemas must be shared and enforced across multiple applications and services.
It’s no surprise that support for schema registries was a highly anticipated Zilla feature, and it's also included in the latest release! As mentioned and demonstrated above, Zilla can validate against OpenAPI and AsyncAPI schemas fetched from an Apicurio registry. For other types of schemas, including JSON, Avro, and Protobuf, Zilla now integrates with Karapace and Confluent Schema Registry.
## What's Next?
This latest release is one of Zilla’s biggest, but even more is coming! Especially since Aklivity is now an AsyncAPI Initiative sponsor!
First, we plan to expand our AsyncAPI capabilities to include protocols beyond MQTT and Kafka, such as WebSocket. Next, we'll contribute to the specification by expanding its coverage for Server Sent Event (SSE)—another key protocol supported by Zilla. Last but not least, we’ll be flushing out more traditional API Gateway features such as throttling and rate limiting. Ultimately, we’re working towards making Zilla THE gateway for AsyncAPIs.
Our public roadmap is [HERE](https://github.com/orgs/aklivity/projects/4). We'd love to hear your requests, recommendations, or feedback! Just start a discussion on the [Zilla GitHub repo](https://github.com/aklivity/zilla/discussions) or within the [Aklivity Slack Community](https://www.aklivity.io/slack).
## That's a Wrap!
If you’re excited about Zilla and our effort in building out event-native API infrastructure, please support the project by starring on [GitHub](https://github.com/aklivity/zilla). Zilla is open-source, so we welcome all input and participation! If you have questions about the latest updates or just want to chat with our team, join our [Aklivity Slack Community](https://www.aklivity.io/slack).
