Gateway Routing Pattern

Introduction

The Gateway Routing Pattern is a design pattern used in microservices architecture to route client requests through a single entry point, known as an API Gateway. The gateway handles authentication, request routing, load balancing, monitoring, and more. This pattern enhances security, reduces direct client interaction with internal microservices, and simplifies the architecture.

The pattern is useful when you want to:

Unified Access to Multiple Services: Clients interact with a single endpoint, and the gateway dynamically routes requests to the appropriate service.
Load Balancing and High Availability: Multiple instances of the same service can be exposed through a single endpoint, ensuring improved distribution of requests and fault tolerance.
Version Management: Different versions of a service can be routed through the gateway, allowing for smooth upgrades and controlled traffic allocation.

Context and Problem

When a client needs to consume multiple services, multiple service instances or a combination of both, the client must be updated when services are added or removed. Consider the following scenarios.

Multiple disparate services – An e-commerce application might provide services such as search, reviews, cart, checkout, and order history. Each service has a different API that the client must interact with, and the client must know about each endpoint in order to connect to the services. If an API changes, the client must be updated as well. If you refactor a service into two or more separate services, the code must change in both the service and the client.
Multiple instances of the same service – The system can require running multiple instances of the same service in the same or different regions. Running multiple instances can be done for load balancing purposes or to meet availability requirements. Each time an instance is spun up or down to match demand, the client must be updated.
Multiple versions of the same service – As part of the deployment strategy, new versions of a service can be deployed along side existing versions. This is known as blue green deployments. In these scenarios, the client must be updated each time there are changes to the percentage of traffic being routed to the new version and existing endpoint.

Solution

Place a gateway in front of a set of applications, services, or deployments. Use application Layer 7 routing to route the request to the appropriate instances.

With this pattern, the client application only needs to know about a single endpoint and communicate with a single endpoint.

The gateway routing pattern is useful in this scenario where a client is consuming multiple services. If a service is consolidated, decomposed or replaced, the client doesn’t necessarily require updating. It can continue making requests to the gateway, and only the routing changes.

A gateway also lets you abstract backend services from the clients, allowing you to keep client calls simple while enabling changes in the backend services behind the gateway. Client calls can be routed to whatever service or services need to handle the expected client behavior, allowing you to add, split, and reorganize services behind the gateway without changing the client.

Multiple instances of the same service

Elasticity is key in cloud computing. Services can be spun up to meet increasing demand or spun down when demand is low to save money. The complexity of registering and unregistering service instances is encapsulated in the gateway. The client is unaware of an increase or decreases in the number of services.

Service instances can be deployed in a single or multiple regions. The Geode pattern details how a multi-region, active-active deployment can improve latency and increase availability of a service.

Multiple versions of the same service

This pattern can be used for deployments, by allowing you to manage how updates are rolled out to users. When a new version of your service is deployed, it can be deployed in parallel with the existing version. Routing lets you control what version of the service is presented to the clients, giving you the flexibility to use various release strategies, whether incremental, parallel, or complete rollouts of updates. Any issues discovered after the new service is deployed can be quickly reverted by making a configuration change at the gateway, without affecting clients.

Issues and considerations

The gateway service can introduce a single point of failure. Ensure it’s properly designed to meet your availability requirements. Consider resiliency and fault tolerance capabilities in the implementation.
The gateway service can introduce a bottleneck. Ensure the gateway has adequate performance to handle load and can easily scale in line with your growth expectations.
Perform load testing against the gateway to ensure you don’t introduce cascading failures for services.
Gateway routing is level 7. It can be based on IP, port, header, or URL.
Gateway services can be global or regional. Azure Front Door is a global gateway, while Azure Application Gateway is regional. Use a global gateway if your solution requires multi-region deployments of services. Consider using Application Gateway if you have a regional workload that requires granular control how traffic is balanced. For example, you want to balance traffic between virtual machines.
The gateway service is the public endpoint for services it sits in front of. Consider limiting public network access to the backend services, by making the services only accessible via the gateway or via a private virtual network.

When to use this pattern

Use this pattern when:

A client needs to consume multiple services that can be accessed behind a gateway.
You want to simplify client applications by using a single endpoint.
You need to route requests from externally addressable endpoints to internal virtual endpoints, such as exposing ports on a VM to cluster virtual IP addresses.
A client needs to consume services running in multiple regions for latency or availability benefits.
A client needs to consume a variable number of service instances.
You want to implement a deployment strategy where clients access multiple versions of the service at the same time.