Establish connectivity between remote Kafka® clients and MSK clusters with ease. Zilla Plus+ can be configured as an MSK Proxy that automates the configuration of AWS PrivateLink access to your MSK cluster. This one-time setup continues to work even after your cluster scales out with new Kafka® brokers.
An advantage of Amazon MSK over other managed Kafka® solutions is that it is always deployed within your own VPC. However, while you have complete control over how co-located Kafka® clients access your MSK clusters, extending access to Lambdas and other clients running in a remote VPC is operationally complex. This makes alignment with AWS Well-Architected security requirements increasingly difficult.
Organize workloads in separate accounts, and group accounts based on function, compliance requirements, or a common set of controls.
Provide secure cross-account VPC connectivity between Kafka clients and Amazon MSK clusters.
Existing approaches for cross-VPC connectivity to MSK clusters have limitations.
A Kafka® client and an MSK cluster may be located in different VPCs. To establish connectivity, their VPCs can be Peered, but this leads to an increased security surface area and requires setting up fine-grained network access controls.
AWS PrivateLink is an alternative to VPC peering that provides fine-grained network access control. When an MSK cluster is exposed over PrivateLink, a Network Load Balancer (NLB) is used to distribute client connections across the Kafka® brokers in the cluster. An NLB though cannot direct a client to a specific broker.
For a client to reach a specific Kafka® broker, either multiple NLBs with varying IP addresses must be deployed or the port numbers of the NLB listeners must be varied for a single NLB. Neither approach is resilient to a dynamically changing number of Kafka brokers in an auto-scaling Amazon MSK cluster.
Aklivity enables PrivateLink connectivity between Kafka® clients and auto-scaling MSK clusters.
Zilla Plus simplifies deploying a VPC Endpoint Service as well as corresponding Interface Endpoints for your MSK cluster. It eliminates the need for multiple NLBs and supports a dynamically changing number of MSK brokers. Zilla Plus MSK Proxy instances are stateless and can be scaled independently of the number of Kafka® brokers in the MSK cluster.
With Zilla Plus, Kafka® clients configured to use MSK directly from within the same VPC can now be incrementally migrated to a consuming VPC and continue to work across PrivateLink without any changes.
The Aklivity Private MSK Proxy is an AWS Qualified solution.
