
LinkedIn Replaces ZooKeeper with Kafka and xDS for Service Discovery
TL;DR
LinkedIn's engineering team has updated their service discovery architecture by replacing ZooKeeper with Apache Kafka and the xDS protocol, aiming for increased scalability and performance.
LinkedIn Enhances Its Service Discovery Platform
The LinkedIn engineering team has replaced its old service discovery system, ZooKeeper, with a new architecture that utilizes Apache Kafka and the xDS protocol. This change aims to enhance the scalability and performance of the platform.
What is ZooKeeper and Its Limitations
ZooKeeper is a distributed coordination tool used to manage configurations and services in complex environments. However, its architecture showed limitations in appropriately scaling to meet the growing demand of applications on LinkedIn.
The New Architecture with Kafka and xDS
Addressing the shortcomings of the old system, the implementation of Kafka enables high-speed data processing, while the xDS protocol provides a flexible interface for various services. This combination results in eventual consistency and supports system migration without causing interruptions.
Results Achieved with the Update
After the transition, LinkedIn has significantly reduced latency, improving overall performance. With this new architecture, the platform efficiently manages hundreds of thousands of application instances.
Impact of the Update on User Experience
With the new structure, LinkedIn can offer a more agile and stable experience for its users. The speed in accessing and executing services is crucial in a scenario where the demand for digital services continues to increase.
Future Perspectives
The replacement of ZooKeeper may mark a new direction for LinkedIn's service infrastructure, inspiring other companies to explore alternatives to traditional service architectures. Continuous investments in technology are crucial for maintaining competitiveness in the sector.
Content selected and edited with AI assistance. Original sources referenced above.


