Learnitweb

Introduction to Service Discovery and Service Registration in microservices

—

by

Editorial Team
in

1. Introduction

In this tutorial, we’ll discuss why we need service discovery and service registration in microservices. Let us first understand the problem which service discovery and service registration tries to solve.

Suppose we have three microservices: ‘Product’, ‘Order’ and ‘Delivery’. These microservices handle storing, fetching, and processing data, executing business logic, and responding to application requests accordingly. We will deploy all our microservices within a shared network, referred to as the microservice network (or a name of your choice). A firewall will secure this network, ensuring that all external clients access the microservices through a single entry point.

The API Gateway serves as the single entry point for all external traffic to the microservices network. This setup ensures there are no other access points, allowing us to handle non-functional requirements like security, auditing, and logging centrally. External clients must route their requests through the gateway, which validates the traffic before forwarding it to the respective microservices. This gateway is a crucial component of the microservices architecture. We’ll discuss this component in later tutorials.

In this section, we focus on internal communication between microservices. For example, if an external request reaches the ‘Product’ microservice via the API Gateway, it may need to interact with the ‘Order’ and ‘Delivery’ microservices to provide a complete response. This interaction within the microservices network is termed internal communication, as opposed to external communication involving external clients. We will explore the challenges microservices face during internal communication within the network.

If one instance of a microservice needs to connect with another, how does it establish the connection?

Each microservice has its own unique endpoint and port number. While it’s possible for one service to use these details to connect to another, this approach has limitations in a microservices environment.

Unlike monolithic applications, where components are relatively static, microservices are deployed in containers that can be created, destroyed, or scaled dynamically based on demand. As a result, endpoints frequently change, making direct connections unreliable without a mechanism to handle this dynamic nature.

How does a new service instance join the microservices network and communicate with others?

For example, in a production deployment, we may initially have one instance each of the product, order, and delivery microservices. If traffic increases, we might scale up to five instances of each service.

When this scaling occurs dynamically, how do these new instances join the network and share their details with other microservices? For instance, the product service might assume there’s only one order service, but behind the scenes, four additional order instances were created. How will the product service learn about these new instances?

This challenge also arises when replacing unhealthy instances. A new container will have a different IP address, making it crucial for it to communicate its details to others. Without this communication, the new instance won’t be invoked, making scaling or replacement ineffective.

Additionally, with multiple instances of a microservice running, load balancing becomes critical. For example, if the product service calls the order service, which has five instances, how does it decide which instance to use? We can’t have one instance handling all traffic while the others sit idle.

To address these challenges, we use patterns like service discovery, service registration, and load balancing. These ensure the microservice network dynamically adapts to changes and operates efficiently. If these concepts seem complex at first, don’t worry—they are the key to solving these problems effectively.

2. Why not traditional load balancers for microservices?

Without service discovery and registration, relying on traditional approaches in a microservices environment creates significant challenges:

  • Static Configuration: Services require fixed IP addresses or DNS names to communicate. In dynamic microservices setups, where instances are created, scaled, or destroyed frequently, maintaining static configurations is impractical.
  • Scalability Issues: When scaling up a service (e.g., adding new loans service instances), other services like accounts would not automatically detect the new instances, leading to incomplete or inefficient communication.
  • Resilience Problems: If an unhealthy instance is replaced with a new one having a different IP or DNS name, services depending on the old instance fail until configurations are manually updated.
  • High Maintenance Overhead: Constantly updating IPs or DNS records across multiple services becomes error-prone and time-consuming.

These challenges highlight the inefficiency of traditional methods in dynamic microservices environments.

3. Challenges of Using Traditional Load Balancers in Microservices

  • Dynamic IPs and DNS Records:
    • Microservices often scale dynamically, creating or destroying instances with new IPs.
    • Maintaining accurate DNS-to-IP mappings manually is impractical due to the ephemeral nature of containers.
  • Static Configuration:
    • Traditional load balancers require static IPs or predefined routing tables, which fail in dynamic environments.
    • Updates to routing tables for scaling or replacing instances need manual intervention.
  • Limited Scalability:
    • Scaling horizontally in traditional setups is limited because IPs must be known and preconfigured.
    • This restricts dynamic scale-up/down required in microservices architectures.
  • Single Point of Failure:
    • While secondary load balancers can act as backups, failure of both results in a total breakdown of traffic flow.
    • Traditional load balancers don’t support clustering for resilience.
  • High Cost:
    • Licensing and setup for traditional load balancers add financial overhead, making them less ideal for cloud-native systems.
  • Complex Management:
    • Manual management of routing tables and configurations is error-prone and time-intensive.
    • They aren’t container-friendly due to frequent lifecycle changes of Docker containers.

Note: The biggest challenge with traditional load balancers is that someone has to manually maintain the routing tables which is an impossible task inside the microservice network because containers are ephemeral in nature.

4. Overcoming Limitations with Service Discovery and Registration

Traditional load balancers and static configurations fail in dynamic microservices environments where instances frequently scale or change.

Service discovery addresses these challenges by:

  • Service Registry:
    • Maintains a real-time record of all running service instances.
    • Automatically updates when new instances start or existing ones shut down.
  • Dynamic Instance Tracking:
    • Services register themselves with the central registry when they start, providing details like IP and port.
    • Instances periodically send health check heartbeats to confirm availability. If a heartbeat is missed, the registry removes the instance.
  • Communication and Load Balancing:
    • Other services query the registry to discover active instances for communication.
    • If multiple instances exist, load-balancing strategies distribute traffic efficiently.
  • Client-Side vs. Server-Side Discovery:
    • Client-Side Discovery: Clients query the registry directly to find service instances and handle load balancing.
    • Server-Side Discovery: A load balancer (or orchestrator like Kubernetes) handles service discovery and distributes traffic.

4.1 Components of Service Discovery

  • Centralized Server: Maintains the service registry with details of all active instances, similar to a configuration server.
  • Registration Process:
    • Services self-register with the registry at startup and deregister on shutdown.
    • Includes regular health checks via heartbeats to ensure up-to-date status.
  • Dynamic Resolution:
    • Services query the central server for instance details dynamically instead of relying on static configurations.

5. Client-side service discovery and load balancing

In client-side service discovery, applications are responsible for registering themselves with a service registry during startup and unregistering when shutting down. When an application needs to communicate with a backing service, it queries the service registry for the associated IP address. If multiple instances of the service are available, the registry returns a list of IP addresses. The client application then selects one based on its own load-balancing strategy.

6. Client-Side vs Server-Side Load Balancing

Both client-side load balancing and server-side load balancing aim to distribute traffic across multiple service instances, but they do so in different ways.

  1. Client-Side Load Balancing:
    • How it works: The client (or service making the request) is responsible for choosing the appropriate instance and managing load balancing.
    • Process:
      • The client queries the service registry to get the list of available instances.
      • Based on the list, the client selects which instance to send a request to, using algorithms like Round Robin, Least Connections, or Random.
    • Advantages:
      • More control for the client (or calling service) over load balancing strategies.
      • Reduces the dependency on a centralized load balancer.
      • Works well when the client has detailed information about available instances (via service discovery).
    • Disadvantages:
      • Clients need to have the logic to handle load balancing, which can lead to complexity.
      • Can result in inefficiency if clients don’t handle load balancing optimally.
      • In case of large networks, maintaining and updating the list of available services can add overhead.
  2. Server-side Load Balancing:
    • How it works: A centralized server or load balancer is responsible for distributing requests to available service instances.
    • Process:
      • The client sends a request to a load balancer.
      • The load balancer checks the health and availability of service instances and routes the request to an appropriate instance.
    • Advantages:
      • Centralized management of load balancing, simplifying client-side logic.
      • Load balancing is managed externally, offloading work from clients and microservices.
      • Easier to scale since the load balancer can be responsible for handling traffic distribution across services.
    • Disadvantages:
      • The centralized load balancer becomes a potential single point of failure.
      • Can become a bottleneck if the load balancer isn’t scaled appropriately.
      • May introduce additional network latency as requests pass through the load balancer.