Skip to main content

4 posts tagged with "microservice"

View All Tags

Microservice - Spring Cloud Gateway

· 16 min read
  1. Introduction
  2. Why We Need API Gateway
  3. Build API Gateway
  4. Setting Up API Gateway
  5. Route Configuration
  6. Security and JWT Authentication
  7. Dockerfile Configuration
  8. Helm Chart Configuration
  9. Project Structure
  10. API Gateway Architecture
  11. Sequence Diagrams
  12. HTTPS Configuration for API Gateway
  13. Difference Between lb:// and http:// Protocols
  14. Conclusion

Introduction​

In a microservices architecture, an API Gateway acts as a single entry point for client requests, routing them to the appropriate microservices within the Kubernetes cluster. This article will delve into the setup and configuration of a Spring Cloud API Gateway, focusing on JWT authentication validation, route management, and deployment using Docker and Helm.


Why We Need API Gateway​

Single Entry Point​

An API Gateway provides a single entry point for all client requests, simplifying client-side code and reducing the number of endpoints clients need to interact with.

Routing and Load Balancing​

The gateway routes requests to the appropriate microservices and handles load balancing across multiple instances of each service.

Security​

It centralizes security concerns such as authentication and authorization, reducing the need for each microservice to implement these features independently.

Protocol Translation​

The gateway can handle protocol translation, allowing clients to use a single protocol (e.g., HTTP/2) while communicating with microservices using different protocols.


Build API Gateway​

Is there an official API Gateway image?

  • Currently, Spring Cloud Gateway does not offer an official pre-built Docker image, developers build a Spring Boot application with spring-cloud-starter-gateway and custom configurations (routes, JWT filter, etc.).

  • As gateway services are often customized for specific routing, load balancing, security and potentially other cross-cutting concerns and needs. It acts as the main entry point for all external requests to your microservices. The gateway can either pull routing configurations from a service discovery mechanism(like Spring Cloud Kubernetes Discovery) or have the configurations set explicitly.

Steps to Build a Production-Ready API Gateway

  1. Route Configuration: Define routing rules either in application.yml or using Java annotations.
  2. Security Setup: Add JWT validation (e.g., JwtAuthenticationFilter) and configure security policies in the gateway.
  3. Service Discovery: Use Spring Cloud Kubernetes for service discovery, allowing the gateway to dynamically discover services.
  4. Containerization and Deployment: Build a Docker image and deploy it to Kubernetes with a Helm chart for scalability and configuration management.

Setting Up API Gateway​

Step 1: Add Dependencies​

Add the necessary dependencies to your pom.xml file.

<dependencies>
<!-- Spring Boot Starter -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>

<!-- Spring Cloud Gateway -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

<!-- Spring Security -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>

<!-- JWT -->
<dependency>
<groupId>io.jsonwebtoken</groupId>
<artifactId>jjwt</artifactId>
<version>0.9.1</version>
</dependency>
</dependencies>

Step 2: Create the Main Application Class​

Create the main application class for the Spring Cloud Gateway.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class ApiGatewayApplication {
public static void main(String[] args) {
SpringApplication.run(ApiGatewayApplication.class, args);
}
}

Service Discovery and Routing Configuration​

YAML-based Configuration​

Configure routes in the application.yml file.

spring:
cloud:
gateway:
discovery:
locator:
enabled: true
lower-case-service-id: true # Ensures lowercase naming for services
routes:
- id: profile_service
uri: lb://profile-svc
predicates:
- Path=/profile/**
- id: inventory_service
uri: lb://inventory-svc
predicates:
- Path=/inventory/**
- id: order_service
uri: lb://order-svc
predicates:
- Path=/order/**
- id: shipment_service
uri: lb://shipment-svc
predicates:
- Path=/shipment/**

kubernetes:
discovery:
all-namespaces: true # Discover services across all namespaces
service-labels: # Optional: filter based on labels
environment: production
  • lb:// */ scheme: Enables load balancing across multiple instances of each microservice.
  • Service Discovery: The gateway will detect microservices like inventory-service or shipment-service using Kubernetes service discovery, eliminating the need for hardcoding service addresses.

Annotation-based Configuration​

Alternatively, configure routes programmatically using annotations.

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class GatewayRoutesConfig {

@Bean
public RouteLocator gatewayRoutes(RouteLocatorBuilder builder) {
return builder.routes()
.route("profile_service_route", r -> r.path("/profile/**")
.uri("lb://profile-svc"))
.route("inventory_service_route", r -> r.path("/inventory/**")
.uri("lb://inventory-svc"))
.route("order_service_route", r -> r.path("/order/**")
.uri("lb://order-svc"))
.route("shipment_service_route", r -> r.path("/shipment/**")
.uri("lb://shipment-svc"))
.build();
}
}

Enable Service Discovery To enable service discovery, you can use the @EnableDiscoveryClient annotation. This annotation is typically placed in a configuration class.

import org.springframework.cloud.client.discovery.EnableDiscoveryClient;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableDiscoveryClient
public class DiscoveryConfig {
// Additional configuration for service discovery can be added here if needed
}

application.yml Configuration for Service Discovery​

Ensure that the following configuration is present in your application.yml or application.properties file to enable service discovery and other settings:

spring:
cloud:
gateway:
discovery:
locator:
enabled: true
lower-case-service-id: true
kubernetes:
discovery:
all-namespaces: true
service-labels:
environment: production

Summary​

  • Annotation-based Configuration: The GatewayRoutesConfig class defines the routes using the RouteLocatorBuilder.
  • Service Discovery: The DiscoveryConfig class enables service discovery using the @EnableDiscoveryClient annotation.
  • YAML Configuration: The application.yml file contains the necessary settings for service discovery and gateway configuration.

Note: The discovery settings, such as enabling discovery locator and configuring Kubernetes discovery, cannot be directly configured using annotations in the configuration class. These settings are typically configured in the application.yml or application.properties file.


Security and JWT Authentication​

Add Dependencies Include spring-boot-starter-oauth2-resource-server and spring-cloud-starter-gateway in your pom.xml:

<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
</dependencies>

Gateway Application Properties Configuration: Configure application.yml for JWT validation. This includes specifying the JWT issuer URI and enabling authorization.

spring:
cloud:
gateway:
discovery:
locator:
enabled: true
routes:
- id: inventory-service
uri: lb://inventory-service
predicates:
- Path=/inventory/**
filters:
- TokenRelay # Forward the token to the downstream services
security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://<your-auth-provider>/realms/your-realm
  • TokenRelay Filter: Passes the JWT token along with requests to the backend services.
  • Issuer URI: This points to your authentication provider, e.g., an OpenID Connect provider like Keycloak or Auth0.

Enable JWT Authentication: Set up SecurityConfig to enable JWT authentication and configure routes for authorized access.

@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {

@Bean
public SecurityWebFilterChain securityWebFilterChain(ServerHttpSecurity http) {
http
.authorizeExchange()
.pathMatchers("/inventory/**", "/shipment/**").authenticated() // Require auth for specified paths
.anyExchange().permitAll()
.and()
.oauth2ResourceServer()
.jwt(); // Enable JWT authentication

return http.build();
}
}
  • This configuration restricts access to /inventory/** and /shipment/** endpoints to authenticated users, while allowing others as specified.

Annotation-based Configuration for Routes and Filters

import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.web.server.ServerHttpSecurity;
import org.springframework.security.web.server.SecurityWebFilterChain;

@Configuration
public class GatewayRoutesConfig {

@Bean
public RouteLocator gatewayRoutes(RouteLocatorBuilder builder) {
return builder.routes()
.route("inventory-service", r -> r.path("/inventory/**")
.filters(f -> f.filter(new JwtAuthenticationFilter("ROLE_USER,ROLE_ADMIN")))
.uri("lb://inventory-service"))
.build();
}

@Bean
public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
http
.authorizeExchange()
.anyExchange().authenticated()
.and()
.oauth2ResourceServer()
.jwt()
.jwkSetUri("https://example-issuer.com/.well-known/jwks.json")
.and()
.and()
.csrf().disable();
return http.build();
}
}

>**Note**: **Annotation-based Configuration**: The `GatewayRoutesConfig` class defines the routes and filters using the `RouteLocatorBuilder`. It also configures security settings using Spring Security annotations.

JWT Authentication Filter​

Create a JWT authentication filter to verify incoming requests.

import org.springframework.security.authentication.AuthenticationManager;
import org.springframework.security.web.server.authentication.AuthenticationWebFilter;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.security.oauth2.jwt.ReactiveJwtDecoder;

@Configuration
public class SecurityConfig {

@Autowired
private JwtDecoder jwtDecoder;

@Bean
public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
AuthenticationWebFilter jwtAuthFilter = new AuthenticationWebFilter(authenticationManager());
http
.authorizeExchange()
.pathMatchers("/oauth2/**").permitAll()
.pathMatchers("/inventory/**", "/shipment/**").authenticated().permitAll()
.anyExchange().authenticated()
.and().addFilterAt(jwtAuthFilter, SecurityWebFiltersOrder.AUTHENTICATION);

return http.build();
}

@Bean
public AuthenticationManager authenticationManager() {
return new JwtAuthenticationManager(jwtDecoder);
}
}

JWT Decoder Configuration​

Configure the JWT decoder to validate tokens.

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.oauth2.jwt.JwtDecoder;
import org.springframework.security.oauth2.jwt.NimbusJwtDecoder;

@Configuration
public class JwtConfig {

@Bean
public JwtDecoder jwtDecoder() {
return NimbusJwtDecoder.withPublicKey(publicKey).build();
}
}

Service-to-Service Communication via Gateway​

To communicate between services, such as shipment-service needing to communicate with inventory-service, microservices can send requests through the API Gateway.

This method simplifies routing, as services only need to know the gateway address and the service paths (e.g., /inventory/**). This approach enhances security and centralizes cross-cutting concerns like authentication, logging, and rate limiting.

Benefits of Gateway-based Communication​

  1. Centralized Security: JWT and other auth mechanisms can be handled at the gateway.
  2. Simplified Routing: Each service interacts with others via a single entry point.
  3. Enhanced Scalability: Kubernetes manages load balancing for each service, and the gateway balances requests across pods.

Service Discovery and Routing Workflow​

[External Client] --> [API Gateway]
|
| (Discovers routes from Kubernetes Discovery)
|
[inventory-service] <--+-- "lb://inventory-service"
[shipment-service] <--+-- "lb://shipment-service"
[other-service] <--+-- Discovered automatically

Sequence Diagram: Inventory and Shipment Communication via Gateway

  1. Shipment Service sends a request to API Gateway at /inventory/items.
  2. API Gateway validates the JWT token, looks up inventory-service route, and forwards the request.
  3. Inventory Service responds with the requested data.
  4. API Gateway relays the response to Shipment Service.
Shipment Service --> API Gateway: Request /inventory/items with JWT
API Gateway --> inventory-service: Validate JWT and route to inventory
inventory-service --> API Gateway: Return data
API Gateway --> Shipment Service: Relay data response

Example: Sending an Authenticated Request to Another Microservice via the Gateway

In shipment-service, use Spring's WebClient to call the inventory-service through the API Gateway.

  1. Define WebClient Bean for Consistency Across Calls

    @Bean
    public WebClient.Builder webClientBuilder() {
    return WebClient.builder();
    }
  2. Send Authenticated Request through Gateway

    @Autowired
    private WebClient.Builder webClientBuilder;

    public InventoryResponse fetchInventoryItems(String token) {
    return webClientBuilder
    .build()
    .get()
    .uri("http://api-gateway/inventory/items")
    .header(HttpHeaders.AUTHORIZATION, "Bearer " + token)
    .retrieve()
    .bodyToMono(InventoryResponse.class)
    .block();
    }
    • Token Forwarding: The token is forwarded with the request, allowing the gateway to validate it before routing to the target service.
    • Gateway URI: Requests go through http://api-gateway/inventory/items, routing based on service discovery.

In this example:

  • The gateway is deployed separately in Kubernetes and handles dynamic service discovery and routing via lb:// URIs.
  • Service-to-service calls leverage the API Gateway for ease of scaling, simplified routing, and centralized security. Each service only needs to know the API Gateway’s address and service paths.

Comparison: Gateway vs. Direct Service-to-Service Calls​

AspectAPI Gateway RoutingDirect Service-to-Service Calls
RoutingCentralized, via API GatewayDirect, each service must know other services
JWT ValidationCentralized in GatewayEach service needs to validate tokens
ScalingGateway handles scaling and balancingEach service scales independently
SecurityJWT validated once at the gatewayJWT validation needed in each service
Use CaseBest for external and internal callsIdeal for internal-only, tightly integrated

Dockerfile Configuration​

Create a Dockerfile for building the API Gateway image.

# Start with the base image for Java
FROM openjdk:17-jdk-slim

# Set working directory
WORKDIR /app

# Copy the API Gateway JAR to the image
COPY target/api-gateway.jar api-gateway.jar

# Expose port 8443 for the API Gateway
EXPOSE 8443

# Run the API Gateway application
ENTRYPOINT ["java", "-jar", "api-gateway.jar"]

Build and Push the Docker Image​

docker build -t myregistry/api-gateway:latest .
docker push myregistry/api-gateway:latest

Helm Chart Configuration​

Helm Chart Structure​

api-gateway/
├── Chart.yaml
├── values.yaml
└── templates/
├── deployment.yaml
├── service.yaml
└── ingress.yaml

Chart.yaml​

apiVersion: v2
name: api-gateway
version: 0.1.0
description: API Gateway for Microservices

values.yaml​

image:
repository: myregistry/api-gateway
tag: latest
pullPolicy: IfNotPresent

replicaCount: 2

service:
type: ClusterIP
port: 8443

ingress:
enabled: true
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
hosts:
- host: api.example.com
paths:
- path: /
pathType: Prefix
tls:
- secretName: api-gateway-tls
hosts:
- api.example.com

templates/deployment.yaml​

apiVersion: apps/v1
kind: Deployment
metadata:
name: api-gateway
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
app: api-gateway
template:
metadata:
labels:
app: api-gateway
spec:
containers:
- name: api-gateway
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
ports:
- containerPort: 8443

templates/service.yaml​

apiVersion: v1
kind: Service
metadata:
name: api-gateway
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: 8443
selector:
app: api-gateway

templates/ingress.yaml​

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-gateway-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- api.example.com
secretName: api-gateway-tls
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-gateway
port:
number: 8443

Project Structure​

api-gateway/
├── src/
│ ├── main/
│ │ ├── java/
│ │ │ └── com/
│ │ │ └── example/
│ │ │ └── apigateway/
│ │ │ ├── ApiGatewayApplication.java
│ │ │ ├── config/
│ │ │ │ ├── GatewayRoutesConfig.java
│ │ │ │ ├── SecurityConfig.java
│ │ │ │ └── JwtConfig.java
│ │ └── resources/
│ │ └── application.yml
├── Dockerfile
├── pom.xml
└── helm/
└── api-gateway/
├── Chart.yaml
├── values.yaml
└── templates/
├── deployment.yaml
├── service.yaml
└── ingress.yaml

API Gateway Architecture​

graph TD
A[Client] -->|HTTPS| B[API Gateway]
B -->|lb://| C[Profile Service]
B -->|lb://| D[Inventory Service]
B -->|lb://| E[Order Service]
B -->|lb://| F[Shipment Service]
B -->|lb://| G[OAuth2 Service]

Explanation​

  1. Client: Sends requests to the API Gateway over HTTPS.
  2. API Gateway: Routes requests to the appropriate microservice using lb:// protocol.
  3. Microservices: Each service handles requests and returns responses to the API Gateway.

Sequence Diagrams​

Sequence Diagram for Client Request​

sequenceDiagram
participant Client
participant API Gateway
participant Profile Service
participant Inventory Service
participant Order Service
participant Shipment Service

Client->>API Gateway: HTTPS Request
API Gateway->>Profile Service: lb://profile-svc
API Gateway->>Inventory Service: lb://inventory-svc
API Gateway->>Order Service: lb://order-svc
API Gateway->>Shipment Service: lb://shipment-svc
Profile Service-->>API Gateway: Response
Inventory Service-->>API Gateway: Response
Order Service-->>API Gateway: Response
Shipment Service-->>API Gateway: Response
API Gateway-->>Client: HTTPS Response

Sequence Diagram for JWT Authentication​

sequenceDiagram
participant Client
participant API Gateway
participant OAuth2 Service

Client->>API Gateway: HTTPS Request with JWT
API Gateway->>OAuth2 Service: Validate JWT
OAuth2 Service-->>API Gateway: JWT Valid
API Gateway-->>Client: HTTPS Response

HTTPS Configuration for API Gateway​

Step 1: Generate SSL Certificate​

Generate a self-signed SSL certificate or obtain one from a Certificate Authority (CA).

# Generate a self-signed certificate (for testing purposes)
keytool -genkeypair -alias api-gateway -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore api-gateway-keystore.p12 -validity 365

Step 2: Configure HTTPS in application.yml​

Add the keystore settings to application.yml.

server:
port: 8443
ssl:
enabled: true
key-store: classpath:api-gateway-keystore.p12
key-store-password: your_password_here
key-store-type: PKCS12

Step 3: Update Routes to Use lb://​

Ensure that the routes use lb:// to leverage service discovery and load balancing.

spring:
cloud:
gateway:
routes:
- id: profile_service
uri: lb://profile-svc
predicates:
- Path=/profile/**
- id: inventory_service
uri: lb://inventory-svc
predicates:
- Path=/inventory/**
- id: order_service
uri: lb://order-svc
predicates:
- Path=/order/**
- id: shipment_service
uri: lb://shipment-svc
predicates:
- Path=/shipment/**

Step 4: Update Kubernetes Ingress for HTTPS​

Configure the Kubernetes Ingress to redirect traffic to the API Gateway’s HTTPS endpoint.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api-gateway-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- api.example.com
secretName: api-gateway-tls
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-gateway
port:
number: 8443

Difference Between lb:// and http:// Protocols​

lb:// Protocol​

  • Service Discovery: Uses Kubernetes service discovery to find the appropriate service instance.
  • Load Balancing: Automatically load balances requests across multiple instances of the service.
  • Dynamic Routing: Routes can be dynamically updated based on service availability.

Example:

spring:
cloud:
gateway:
routes:
- id: profile_service
uri: lb://profile-svc
predicates:
- Path=/profile/**

http:// Protocol​

  • Direct Routing: Routes requests directly to a specific URL.
  • No Load Balancing: Does not provide load balancing or service discovery.
  • Static Routing: Routes are static and do not change based on service availability.

Example:

spring:
cloud:
gateway:
routes:
- id: profile_service
uri: http://profile-svc:8080
predicates:
- Path=/profile/**

When to Use Each Protocol​

  • Use lb:// for routes within Kubernetes service discovery, allowing the gateway to leverage load balancing.
  • Use http:// for direct routes to a specific URL or when no service discovery is available.

The other option

Here’s a step-by-step setup for building the API Gateway with JWT authentication and routing configurations:

Step 1: Configure API Gateway with JWT Authentication and Routing​

# application.yml
spring:
cloud:
gateway:
routes:
- id: inventory-service
uri: lb://inventory-service
predicates:
- Path=/inventory/**
filters:
- name: JwtAuthenticationFilter
args:
roles: "ROLE_USER,ROLE_ADMIN" # Add roles if role-based access is needed
discovery:
locator:
enabled: true # Enables dynamic route lookup from discovery service
default-filters:
- StripPrefix=1 # Adjusts the path for downstream services

security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://example-issuer.com # Point to the JWT issuer
jwk-set-uri: https://example-issuer.com/.well-known/jwks.json # Set up for validation

Step 2: Implement JWT Authentication Filter for Gateway​

The custom JWT filter ensures each request includes a valid JWT token.

// JwtAuthenticationFilter.java
@Component
public class JwtAuthenticationFilter implements GatewayFilter {

private final JwtDecoder jwtDecoder;

public JwtAuthenticationFilter(JwtDecoder jwtDecoder) {
this.jwtDecoder = jwtDecoder;
}

@Override
public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
String authHeader = exchange.getRequest().getHeaders().getFirst(HttpHeaders.AUTHORIZATION);
if (authHeader != null && authHeader.startsWith("Bearer ")) {
String token = authHeader.substring(7);
try {
Jwt jwt = jwtDecoder.decode(token); // Validates the JWT
return chain.filter(exchange); // If valid, proceed with the request
} catch (JwtException e) {
exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
return exchange.getResponse().setComplete();
}
}
exchange.getResponse().setStatusCode(HttpStatus.UNAUTHORIZED);
return exchange.getResponse().setComplete();
}
}

Add the filter in application.yml for it to be applied to specific routes (like inventory-service in this case).

Step 3: Building and Deploying the API Gateway as a Docker Image​

  1. Dockerfile for the API Gateway:

    # Dockerfile for api-gateway
    FROM openjdk:17-jdk-slim
    ARG JAR_FILE=target/api-gateway.jar
    COPY ${JAR_FILE} api-gateway.jar
    ENTRYPOINT ["java", "-jar", "/api-gateway.jar"]
  2. Build the Docker Image:

    docker build -t myregistry/api-gateway:latest .
    docker push myregistry/api-gateway:latest # Push to your registry
  3. Kubernetes Deployment Configuration:

    # api-gateway-deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: api-gateway
    labels:
    app: api-gateway
    spec:
    replicas: 2
    selector:
    matchLabels:
    app: api-gateway
    template:
    metadata:
    labels:
    app: api-gateway
    spec:
    containers:
    - name: api-gateway
    image: myregistry/api-gateway:latest
    ports:
    - containerPort: 8080
    env:
    - name: SPRING_PROFILES_ACTIVE
    value: k8s

Step 4: Service-to-Service Communication via Gateway​

Now, other services can call the gateway (using api-gateway service DNS) and append the target service’s route (/inventory/*) to the request path.

Sample Code for a Downstream Service Request Using WebClient with JWT​

// ShipmentService.java
@Service
public class ShipmentService {

private final WebClient webClient;

public ShipmentService(WebClient.Builder webClientBuilder) {
this.webClient = webClientBuilder
.baseUrl("http://api-gateway") // Gateway URL in Kubernetes DNS
.defaultHeader(HttpHeaders.AUTHORIZATION, "Bearer " + jwtToken)
.build();
}

public Inventory getInventory(String id) {
return webClient.get()
.uri("/inventory/" + id)
.retrieve()
.bodyToMono(Inventory.class)
.block();
}
}

Comparison: Gateway vs. Interceptor-Based JWT Validation​

AspectGateway with JWT FilterService-Level JWT Interceptors
Centralized ControlAll JWT validation centralized at gatewayEach service has independent JWT validation
Configuration ComplexityEasier for microservices, managed in gatewayMore complex across multiple services
Performance OverheadCan cause bottleneck at gateway if overloadedDistributed load, but duplicate processing
ResilienceSingle point of failure without redundancyMore resilient, distributed across services
Authorization FlexibilityOffers consistent authentication policyEach service can implement role-based restrictions

Microservice meet with Spring Cloud and Kubernetes

· 13 min read

Introduction​

The objective is to design and deploy a group of microservices: Profile Service, Inventory Service, Order Service, Shipment Service, and an OAuth2 Service for centralized authentication. Key requirements include robust service discovery, API Gateway setup, secure communication, and efficient load balancing for high availability and fault tolerance in Kubernetes.

High-Level Architecture​

Each microservice will be deployed in a Kubernetes cluster with multiple replicas, leveraging Spring Cloud Kubernetes Discovery for service registration and Spring Cloud LoadBalancer for client-side load balancing. An API Gateway will handle external requests, providing routing, load balancing, and JWT-based authentication. Communication between services is secured with JWT tokens generated by the OAuth2 Service.

Service Design​

Each microservice is structured independently, deployed as a standalone Spring Boot application with individual responsibilities and access rules. Here’s an overview of the key services:

1. Profile Service​

Manages customer profiles, providing endpoints for CRUD operations on profile data. Accessible to services like Order and Shipment for customer information retrieval.

2. Inventory Service​

Tracks inventory levels and provides endpoints to update, retrieve, and manage inventory data. Called by the Order Service for real-time inventory status updates.

3. Order Service​

Processes customer orders, calculates totals, and interfaces with Inventory and Profile Services. This service is central to the architecture, relying on other services for a smooth customer experience.

4. Shipment Service​

Manages shipment tracking and logistics for orders, called by the Order Service once an order is processed and confirmed.

5. OAuth2 Service​

Acts as the authentication provider, issuing and validating JWT tokens for secure communication. The API Gateway uses these tokens to authenticate requests.

Order-Service Auth-Service Inventory-Service
| | |
|--- Request JWT ------------->| |
| | |
|<-- Receive JWT --------------| |
| | |
|--- Call Inventory ---------->| |
| (with JWT) | |
| |--- Validate JWT------------>|
| | |
| |<-- JWT Valid/Invalid--------|
| | |
|<-- Receive Inventory Data -- | |

Kubernetes Deployment and Service Discovery​

Each service will be deployed to Kubernetes as a Deployment with multiple replicas and exposed via a Service to enable discovery. Spring Cloud Kubernetes is used for service registration and discovery.

Configuring Kubernetes Service Discovery for Each Microservice​

To set up service discovery in Kubernetes, add the following dependencies to your pom.xml:

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-discovery</artifactId>
</dependency>

Configure the application.yml in each service to enable discovery:

spring:
cloud:
kubernetes:
discovery:
enabled: true #Enable service discovery
config:
enabled: true
loadbalancer:
enabled: true #Enable loadbalancer

This configuration registers each service with Kubernetes' internal DNS, allowing them to be discoverable by service name within the namespace. This way, each microservice is registered under its own Kubernetes service name, and can resolve other services (e.g., http://inventory-service:8080) when communicating with them. This configuration, combined with Kubernetes DNS, helps maintain an efficient, loosely coupled architecture.

Note: The Kubernetes Service Discovery configuration should indeed be included in each individual microservice’s configuration to ensure they can all register themselves with the Kubernetes DNS and be discoverable by other services in the namespace. This setup allows each microservice to dynamically locate and communicate with other services via Kubernetes' internal DNS naming conventions.

Why Configure Service Discovery in Each Microservice?

Each microservice is an independently deployed unit, so configuring Spring Cloud Kubernetes Discovery in each service's application.yml allows it to:

  1. Register with Kubernetes DNS under a unique service name, enabling internal routing across services.
  2. Access other services by simply using their service names, as Kubernetes DNS provides a consistent way to locate services within the cluster.

Note Here you might be noticed that both the discovery and load balancer components are configured within each microservice. This design allows each microservice to dynamically discover the current instances of other services (using the spring-cloud-kubernetes-discovery integration) and balance requests among those instances using the spring-cloud-loadbalancer component.

Service Registration​

Each microservice is deployed as a separate Deployment and Service in Kubernetes. Here’s an example of the inventory-service Kubernetes configuration:

To deploy and register microservices like an Inventory Service in Kubernetes, we use Spring Cloud’s Kubernetes Discovery capabilities, allowing services to register with Kubernetes’ DNS-based service registry automatically.

Step 1: Create a Kubernetes Service and Deployment for each microservice:

YAML files for defining the service and deployment:

# inventory-service-deployment.yml
apiVersion: v1
kind: Service
metadata:
name: inventory-service
spec:
selector:
app: inventory
ports:
- protocol: TCP
port: 8080
targetPort: 8080
type: ClusterIP # Internal access within the cluster

---
apiVersion: apps/v1
kind: Deployment
metadata:
name: inventory-deployment
spec:
replicas: 2 # Number of instances (pods) of the Inventory Service
selector:
matchLabels:
app: inventory
template:
metadata:
labels:
app: inventory
spec:
containers:
- name: inventory
image: myregistry/inventory-service:latest
ports:
- containerPort: 8080

This configuration deploys inventory-service with two replicas, registered under the service name inventory-service. Other services can communicate with it via http://inventory-service:8080.

More comprehensive explainations When deploying an inventory-deployment with two replicas, you will indeed see two pods running in parallel in the specified namespace. Kubernetes names these pods based on the deployment name, followed by a unique suffix (not just sequential numbers), so you may see pod names like inventory-deployment-abc12 and inventory-deployment-def34.

In Kubernetes, these two pods are managed by a single inventory-service which acts as a layer of abstraction. When the client load balancer targets inventory-service, the service uses Kubernetes' built-in load balancing to forward requests to one of the available inventory-deployment pods.

How Kubernetes Service Load Balancing Works:

  1. Service Abstraction: The inventory-service has a Cluster IP that exposes the service within the cluster.
  2. Endpoints Selection: The service maintains an updated list of active pod IPs (called "endpoints") that match the deployment selector.
  3. Round-Robin Load Balancing: Kubernetes uses a round-robin approach by default. When a request reaches inventory-service, it selects one of the pod IPs associated with the deployment, balancing requests across pods automatically. The service doesn't keep state; each request is independently forwarded to one of the pod replicas.

Example Flow:

  1. Client Request: A client load balancer sends a request to the inventory-service endpoint.
  2. Service to Pod Routing: Kubernetes service routing picks an endpoint (pod IP) from the inventory-deployment pods and forwards the request to it.

For better control and observability, custom load balancer configurations can also be implemented to fine-tune how the requests are distributed, especially in setups requiring sticky sessions or advanced routing.

Step 2: Register Service with Kubernetes Discovery:

Spring Cloud Kubernetes automatically registers inventory-service based on the deployment labels. Enable Spring Cloud Discovery in your Spring Boot application via @EnableDiscoveryClient:

@EnableDiscoveryClient
@SpringBootApplication
public class InventoryServiceApplication {
public static void main(String[] args) {
SpringApplication.run(InventoryServiceApplication.class, args);
}
}

Step 3: Configure Discovery in application.yml:

spring:
cloud:
kubernetes:
discovery:
enabled: true
all-namespaces: true # Optional: enables discovery across namespaces

Load Balancing with Spring Cloud LoadBalancer​

To efficiently distribute incoming requests across available instances, Spring Cloud LoadBalancer provides a client-side load-balancing solution, automatically distributing traffic among instances of a service.

Enable Spring Cloud LoadBalancer by adding Spring Cloud LoadBalancer dependency:

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Configure a ServiceInstanceListSupplier bean for custom load balancing behavior if required:

import org.springframework.cloud.client.loadbalancer.ServiceInstanceListSupplier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class LoadBalancerConfig {

@Bean
public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier() {
return ServiceInstanceListSupplier.builder()
.withDiscoveryClient()
.withCaching()
.build();
}
}

Set Up Load Balancer with WebClient:

  • Configure a WebClient bean with load balancing to distribute requests among inventory-service instances.
import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.web.reactive.function.client.WebClient;

@Configuration
public class WebClientConfig {

@Bean
@LoadBalanced
public WebClient.Builder loadBalancedWebClientBuilder() {
return WebClient.builder();
}
}

Use Load-Balanced WebClient in the calling service:

@Service
public class InventoryClient {

private final WebClient.Builder webClientBuilder;

public InventoryClient(WebClient.Builder webClientBuilder) {
this.webClientBuilder = webClientBuilder;
}

public Mono<Product> getProductById(String productId) {
return webClientBuilder.build()
.get()
.uri("http://inventory-service/products/{id}", productId)
.retrieve()
.bodyToMono(Product.class);
}
}

How Load Balancing Works:

  • Spring Cloud LoadBalancer, by default, uses a round-robin strategy to distribute requests.
  • It balances requests between the two inventory-service pods and automatically updates when new instances are added or removed.

  1. By default, Spring Cloud LoadBalancer will round-robin requests to different instances, or customize by implementing a custom LoadBalancerRequestTransformer.

API Gateway Setup​

The Spring Cloud API Gateway is deployed as a separate microservice, managing traffic and providing JWT-based authentication. The gateway routes requests to the appropriate services and load-balances among available replicas.

Routing and Load Balancing​

[External Client] --> [API Gateway]
|
| (Discovers routes from Kubernetes Discovery)
|
[inventory-service] <--+-- "lb://inventory-service"
[shipment-service] <--+-- "lb://shipment-service"
[order-service] <--+-- "lb://order-service"
[profile-service] <--+-- "lb://profile-service"
[other-service] <--+-- Discovered automatically

Configure the gateway’s application.yml to define routes to each service:

spring:
cloud:
gateway:
routes:
- id: profile
uri: http://profile-service
predicates:
- Path=/profile/**
- id: inventory
uri: http://inventory-service
predicates:
- Path=/inventory/**
default-filters:
- TokenRelay

The TokenRelay filter enables the gateway to forward JWT tokens to downstream services for authenticated requests.

Configuring JWT Authentication​

For security, JWT validation can be implemented in the gateway or within each service. Here’s how to configure JWT validation in the gateway:

  1. Add OAuth2 dependencies to the gateway:

    <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
    </dependency>
  2. Configure JWT validation in the gateway’s application.yml:

    spring:
    security:
    oauth2:
    resourceserver:
    jwt:
    issuer-uri: https://auth-server/oauth2/default

Microservice Communication​

Microservices communicate through Spring Cloud LoadBalancer and WebClient.

Using WebClient with Load Balancer​

To call the inventory-service from the order-service, configure WebClient with load balancing:

@Configuration
public class WebClientConfig {

@LoadBalanced
@Bean
public WebClient.Builder webClientBuilder() {
return WebClient.builder();
}
}

Example: Order Service Calls Profile Service​

Here’s how order-service might call profile-service:

@RestController
public class OrderController {

private final WebClient.Builder webClientBuilder;

@Autowired
public OrderController(WebClient.Builder webClientBuilder) {
this.webClientBuilder = webClientBuilder;
}

@GetMapping("/order/{orderId}")
public Mono<OrderDetails> getOrderDetails(@PathVariable String orderId) {
return webClientBuilder.build()
.get()
.uri("http://profile-service/profile/{orderId}", orderId) // The uri can be either http or lb(load balancer)
.retrieve()
.bodyToMono(OrderDetails.class);
}
}

Securing Microservices​

Secure Communication with mTLS and JWT

mTLS (mutual TLS) and JWT (JSON Web Token) are two primary ways to secure communication between microservices.

  • mTLS: Ensures that both the client and server authenticate each other, useful in a zero-trust network.
  • JWT: Adds stateless authentication, where each request includes a signed token with user and session data.

Each microservice validates JWT tokens received from the gateway, ensuring secure and authenticated communication.

JWT Authentication with API Gateway​

JWT Authentication is often managed at the API Gateway, validating tokens and authorizing requests before forwarding them to services.

In this approach, JWT validation is centralized in the gateway. Tokens are verified at the gateway and forwarded to services, reducing the validation load on individual services.

  1. Add Spring Security Dependency for JWT support:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
  1. Configure JWT Verification in Gateway:

Validate JWTs at the gateway level, allowing only authorized requests to reach downstream services.

@Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {

@Override
protected void configure(HttpSecurity http) throws Exception {
http
.authorizeRequests()
.antMatchers("/api/**").authenticated()
.and()
.oauth2ResourceServer().jwt();
}
}

Alternative: JWT Validation in Interceptors​

Alternatively, JWT validation can be handled within each microservice using a custom Interceptor, within services, add an Interceptor to validate JWTs before processing requests.

import org.springframework.stereotype.Component;
import org.springframework.web.servlet.HandlerInterceptor;

@Component
public class JwtInterceptor implements HandlerInterceptor {

@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
String token = request.getHeader("Authorization");
// Verify the JWT token
// Decode and validate claims
return isValidToken(token);
}
}

Pros and Cons of JWT and mTLS​

ApproachProsCons
mTLSHigh security with mutual authentication.Requires certificate management, adds complexity.
JWTStateless, easily scalable, widely supported.Token expiration handling; replay attacks if not carefully implemented.

<a name="kubernetes-deployment-and-auto-scaling"></a>
## Kubernetes Deployment and Auto-scaling

Scaling services dynamically is fundamental in Kubernetes. Kubernetes provides an effective way to handle **auto-scaling** of microservices based on resource utilization, e.g. CPU, memory, or custom metrics.

Each microservice is balanced using **Spring Cloud LoadBalancer**, ensuring even distribution across replicas.

> **Step 1**: Configuring Horizontal Pod Autoscaler (HPA)

1. Enable auto-scaling in Kubernetes based on CPU usage by defining an HPA resource.

**inventory-hpa.yaml**
```yaml
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: inventory-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: inventory-deployment
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 75
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50

Step 2: Implementing Cluster Scaling

Use Kubernetes’ Cluster Autoscaler for scaling nodes as workload demand grows, ensuring optimal resource use without manual intervention.


Ensuring Secure Microservice Communication​

Securing communication between services is critical. Use mTLS (mutual TLS) and JWT tokens for service-to-service authentication within Kubernetes clusters.

Step 1: Configure Mutual TLS (mTLS)​

Istio or Linkerd can help enforce mTLS between services:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: your-namespace
spec:
mtls:
mode: STRICT

Summary​

  1. Use ConfigMaps and Secrets for configuration.
  2. Implement autoscaling with HPA.
  3. Secure communication with mTLS and JWT.
  4. Monitor with readiness/liveness probes and Prometheus/Grafana.

This guide provides an architecture blueprint for deploying secure, scalable, and efficient microservices in Kubernetes using Spring Cloud. The API Gateway simplifies external access, service discovery, and JWT-based authentication, while Kubernetes ensures high availability and resilience across services.

Deep Dive into Memory Cache in Spring Boot Application

· 13 min read

Table of Contents​


Introduction​

In modern microservice architectures, performance and scalability are critical. Memory caching is a powerful technique to improve application performance by reducing the load on databases and other backend services. This article delves into the design and implementation of memory caches in Spring Boot applications, focusing on real-life project scenarios and the latest technology stack.


Understanding Memory Cache​

A memory cache is a high-speed data storage layer that stores a subset of data, typically transient in nature, so that future requests for that data are served faster. Caching can significantly improve the performance and scalability of applications by reducing the need to access slower underlying storage layers.

Key Concepts​

  • Cache Hit: When the requested data is found in the cache.
  • Cache Miss: When the requested data is not found in the cache.
  • Cache Eviction: The process of removing data from the cache to make room for new data.

Cache Design Principles​

1. Cache Hit and Miss​

  • Cache Hit: When the requested data is found in the cache.
  • Cache Miss: When the requested data is not found in the cache.

2. Cache Eviction Policies​

  • Least Recently Used (LRU): Removes the least recently used items first.
  • First In, First Out (FIFO): Removes the oldest items first.
  • Least Frequently Used (LFU): Removes the least frequently used items first.

3. Cache Invalidation​

  • Time-to-Live (TTL): Data is automatically invalidated after a certain period.
  • Event-Driven Invalidation: Data is invalidated based on specific events (e.g., data updates).

Cache Implementation in Spring Boot​

Spring Boot Cache Abstraction​

Spring Boot provides a powerful cache abstraction that allows you to apply caching to your methods with minimal configuration. The core annotation is @Cacheable, which caches the result of a method invocation.

Caffeine Cache Integration​

Caffeine is a high-performance, near-optimal caching library for Java. It is widely used in Spring Boot applications due to its efficiency and ease of integration.

Cache Configuration​

To configure Caffeine cache in a Spring Boot application, you need to add the following dependencies:

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-cache</artifactId>
</dependency>
<dependency>
<groupId>com.github.ben-manes.caffeine</groupId>
<artifactId>caffeine</artifactId>
</dependency>

Next, enable caching in your Spring Boot application by adding the @EnableCaching annotation to your main application class:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cache.annotation.EnableCaching;

@SpringBootApplication
@EnableCaching
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}

YAML Configuration​

You can also configure Caffeine cache using YAML:

spring:
cache:
type: caffeine
cache-names:
- userCache
- productCache
caffeine:
spec: maximumSize=500,expireAfterAccess=600s

Detailed Logic in Caffeine Cache​

When you annotate a method with @Cacheable in Spring Boot, the cache manager will first check if the requested data is already present in the specified cache. If the data is found, it will be returned immediately, bypassing the method's execution. If the data is not found, the method will be executed, and the result will be stored in the cache for future requests.

Cache Lookup​

Caffeine uses a ConcurrentHashMap to store cache entries. When a lookup is performed, Caffeine checks if the key exists in the map:

public V getIfPresent(Object key) {
Node<K, V> node = data.get(key);
if (node == null) {
return null;
}
// Additional logic for expiration and eviction checks
return node.getValue();
}

Cache Miss and Method Execution​

If the key is not found in the cache, the method is executed, and the result is stored in the cache:

public V get(K key, Function<? super K, ? extends V> mappingFunction) {
V value = getIfPresent(key);
if (value != null) {
return value;
}
value = mappingFunction.apply(key);
put(key, value);
return value;
}

Cache Update​

When the result is fetched from the database, it is stored in the cache:

public void put(K key, V value) {
Node<K, V> node = new Node<>(key, value);
data.put(key, node);
// Additional logic for eviction and expiration
}

Microservice Architecture and Cache​

Cache Consistency​

Ensuring cache consistency is crucial in distributed systems. Techniques like cache invalidation, event-driven updates, and distributed locking can be used to maintain consistency.

Cache Eviction Strategies​

Choosing the right eviction strategy is important. LRU is commonly used, but in scenarios with frequent updates, LFU or custom eviction policies might be more appropriate.


Complex Scenario 1: Flow Control and Backpressure Handling​

Problem Statement​

In high-traffic scenarios, the cache might become a bottleneck if not properly managed. Flow control and backpressure handling are essential to prevent the cache from being overwhelmed.

Solution Architecture​

  1. Rate Limiting: Implement rate limiting to control the number of requests hitting the cache.
  2. Circuit Breaker: Use a circuit breaker to handle failures gracefully and prevent cascading failures.
  3. Backpressure Handling: Implement backpressure mechanisms to manage the flow of data between services.

Flow Chart​

graph TD
A[Client Request] --> B{Rate Limiter}
B -->|Allowed| C[Cache Lookup]
B -->|Denied| D[Fallback to Database]
C --> E{Cache Hit?}
E -->|Yes| F[Return Cached Data]
E -->|No| G[Fetch from Database]
G --> H[Update Cache]
H --> F
D --> G

Sample Code​

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Service
public class UserService {

@Cacheable(value = "userCache", key = "#userId")
public User getUserById(Long userId) {
// Fetch user from database
return fetchUserFromDatabase(userId);
}

private User fetchUserFromDatabase(Long userId) {
// Database query logic
return new User();
}
}

Complex Scenario 2: Cache Invalidation and Data Synchronization​

Problem Statement​

Ensuring that the cache remains consistent with the underlying data store is challenging, especially in distributed systems.

Solution Architecture​

  1. Event-Driven Invalidation: Use event-driven mechanisms to invalidate the cache when data changes.
  2. Distributed Locking: Use distributed locking to ensure that only one service updates the cache at a time.
  3. TTL-Based Invalidation: Use Time-to-Live (TTL) to automatically invalidate cache entries after a certain period.

Sequence Diagram​

sequenceDiagram
participant Client
participant Service
participant Cache
participant Database

Client->>Service: Update User
Service->>Database: Update User in Database
Database-->>Service: User Updated
Service->>Cache: Invalidate User Cache
Cache-->>Service: Cache Invalidated
Service-->>Client: User Updated

Sample Code​

import org.springframework.cache.annotation.CacheEvict;
import org.springframework.stereotype.Service;

@Service
public class UserService {

@CacheEvict(value = "userCache", key = "#userId")
public void updateUser(Long userId, User user) {
// Update user in database
updateUserInDatabase(userId, user);
}

private void updateUserInDatabase(Long userId, User user) {
// Database update logic
}
}

Complex Scenario 3: Cache Stampede Prevention​

Problem Statement​

A cache stampede occurs when multiple requests for the same data hit the cache simultaneously, leading to a spike in database load.

Solution Architecture​

  1. Probabilistic Early Expiration: Use probabilistic early expiration to spread out cache expiration times.
  2. Locking Mechanism: Implement a locking mechanism to ensure that only one request populates the cache at a time.
  3. Background Refresh: Use background refresh to keep the cache up-to-date without impacting the user experience.

Flow Chart​

graph TD
A[Client Request] --> B{Cache Hit?}
B -->|Yes| C[Return Cached Data]
B -->|No| D{Lock Acquired?}
D -->|Yes| E[Fetch from Database]
D -->|No| F[Wait for Lock]
E --> G[Update Cache]
G --> C
F --> B

Sample Code​

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;

@Service
public class UserService {

private final Lock lock = new ReentrantLock();

@Cacheable(value = "userCache", key = "#userId")
public User getUserById(Long userId) {
try {
lock.lock();
// Double-check to ensure the cache is still empty
User cachedUser = fetchUserFromCache(userId);
if (cachedUser != null) {
return cachedUser;
}
// Fetch user from database
User user = fetchUserFromDatabase(userId);
// Update cache
updateCache(userId, user);
return user;
} finally {
lock.unlock();
}
}

private User fetchUserFromCache(Long userId) {
// Cache lookup logic
return null;
}

private User fetchUserFromDatabase(Long userId) {
// Database query logic
return new User();
}

private void updateCache(Long userId, User user) {
// Cache update logic
}
}

Real-Life Production Use Cases​

1. User Profile Caching​

In a social media application, user profiles are frequently accessed. Caching user profiles can significantly reduce the load on the database and improve response times.

2. Product Catalog Caching​

In an e-commerce application, product catalogs are often large and frequently accessed. Caching product data can improve performance and reduce database load.

3. API Response Caching​

In microservices, caching API responses can reduce the number of calls to downstream services, improving overall system performance.

4. Session Data Caching​

In applications with user sessions, caching session data can reduce the load on session storage systems and improve scalability.


Sample Code​

Caffeine Cache Configuration​

import com.github.benmanes.caffeine.cache.Caffeine;
import org.springframework.cache.CacheManager;
import org.springframework.cache.caffeine.CaffeineCacheManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.concurrent.TimeUnit;

@Configuration
public class CacheConfig {

@Bean
public CacheManager cacheManager() {
CaffeineCacheManager cacheManager = new CaffeineCacheManager("userCache", "productCache");
cacheManager.setCaffeine(caffeineCacheBuilder());
return cacheManager;
}

Caffeine<Object, Object> caffeineCacheBuilder() {
return Caffeine.newBuilder()
.maximumSize(500)
.expireAfterAccess(600, TimeUnit.SECONDS);
}
}

User Service with Caching​

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Service
public class UserService {

@Cacheable(value = "userCache", key = "#userId")
public User getUserById(Long userId) {
// Fetch user from database
return fetchUserFromDatabase(userId);
}

private User fetchUserFromDatabase(Long userId) {
// Database query logic
return new User();
}
}

Rate Limiting for Cache Access​

Why Rate Limiting is Needed​

Reading from a memory cache is generally very fast and should not be a bottleneck in terms of performance. However, there are several reasons why you might still want to implement rate limiting for cache access, even if the cache is in memory:

  1. Resource Management: Even though reading from memory is fast, excessive concurrent requests can still consume a significant amount of CPU and memory resources. Rate limiting helps manage these resources more effectively, ensuring that the system remains responsive and does not become overwhelmed.

  2. Preventing Cache Stampede: While reading from the cache is fast, if the cache is not populated (cache miss), all concurrent requests will end up fetching the data from the database or another slow data source. This can lead to a cache stampede, where the database or other backend services are overwhelmed with requests. Rate limiting can help mitigate this by controlling the number of concurrent requests that can hit the cache.

  3. Graceful Degradation: In high-traffic scenarios, rate limiting ensures that the system can gracefully degrade its performance. By limiting the number of requests that can access the cache, you can prevent the system from becoming unresponsive or crashing under heavy load.

  4. Security and Denial of Service (DoS) Prevention: Rate limiting can also be a security measure to prevent Denial of Service (DoS) attacks. By limiting the number of requests from a single client or IP address, you can protect the system from being overwhelmed by malicious requests.

  5. Ensuring Fairness: Rate limiting ensures that all clients have fair access to the cache. Without rate limiting, a few clients with high request rates could monopolize the cache, leading to poor performance for other clients.

  6. Monitoring and Analytics: Rate limiting provides valuable data on the usage patterns of the cache. This data can be used for monitoring, analytics, and capacity planning.

Example: Rate Limiting with Spring Boot​

Here's an example of how you might implement rate limiting for cache access in a Spring Boot application using the Bucket4j library:

Add Dependencies​

<dependency>
<groupId>com.github.vladimir-bukhtoyarov</groupId>
<artifactId>bucket4j-core</artifactId>
<version>7.5.0</version>
</dependency>

Rate Limiting Configuration​

import io.github.bucket4j.Bandwidth;
import io.github.bucket4j.Bucket;
import io.github.bucket4j.Bucket4j;
import io.github.bucket4j.Refill;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.time.Duration;

@Configuration
public class RateLimitingConfig {

@Bean
public Bucket cacheRateLimiter() {
Bandwidth limit = Bandwidth.classic(100, Refill.greedy(100, Duration.ofMinutes(1)));
return Bucket4j.builder().addLimit(limit).build();
}
}

Service Implementation with Rate Limiting​

import io.github.bucket4j.Bucket;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Service
public class UserService {

@Autowired
private Bucket cacheRateLimiter;

@Cacheable(value = "userCache", key = "#userId")
public User getUserById(Long userId) {
if (cacheRateLimiter.tryConsume(1)) {
// Fetch user from database
return fetchUserFromDatabase(userId);
} else {
throw new RuntimeException("Rate limit exceeded");
}
}

private User fetchUserFromDatabase(Long userId) {
// Database query logic
return new User();
}
}

Explanation​

  1. Bucket4j Configuration: The Bucket4j library is used to configure a rate limiter with a limit of 100 requests per minute.
  2. Service Implementation: The getUserById method checks if the rate limit has been exceeded before fetching the user from the database. If the limit is exceeded, it throws an exception.

Conclusion​

While reading from a memory cache is fast and should not be a bottleneck, implementing rate limiting can provide several benefits, including resource management, preventing cache stampedes, ensuring fairness, and enhancing security. By using libraries like Bucket4j, you can easily implement rate limiting in your Spring Boot application to manage cache access effectively.


Conclusion​

Memory caching is a powerful tool for improving the performance and scalability of Spring Boot applications in a microservice architecture. By understanding the principles of cache design and implementing best practices, you can effectively manage cache performance, consistency, and reliability in real-life projects.

This article provides a comprehensive guide to memory cache design and implementation in Spring Boot, covering various scenarios and best practices. Use the provided sample code and diagrams to implement efficient caching in your microservice architecture.

Approaches to Rolling Back or Canceling Transactions in Microservice

· 7 min read

Handling rollback or cancellation of transactions in a Microservice Architecture when multiple writes are involved is a complex process because each microservice typically has its own database, and there is no central point of coordination like in monolithic applications. Traditional database transactions (ACID) are difficult to apply in microservices, especially when services span across multiple databases, message brokers, and external APIs. Therefore, rollback in microservices needs a more sophisticated approach.

Key Approaches to Rolling Back or Canceling Transactions in Microservice Architecture​

There are several strategies to handle rollback or cancellation in microservices when multiple services and writes are involved. These include:


1. Saga Pattern​

The Saga Pattern is the most popular strategy for handling distributed transactions in microservices. It breaks a large transaction into a sequence of smaller, independent transactions, each handled by a microservice. If any part of the sequence fails, compensating actions (i.e., rollbacks) are triggered to undo the work done by previous steps.

There are two common approaches to implementing the Saga Pattern:

  • Choreography-based Saga
  • Orchestration-based Saga

a. Choreography-based Saga​

In a Choreography-based Saga, each service involved in the transaction emits an event upon completion of its task. Other services listen to these events and take action accordingly. If an error occurs, the service responsible for the failed transaction emits a failure event, and other services listening for that failure event perform compensating actions.

Example: Imagine an e-commerce system where you place an order that spans multiple microservices:

  1. Order Service: Creates the order.
  2. Inventory Service: Reserves the product.
  3. Payment Service: Charges the customer.
  4. Shipping Service: Ships the product.

If the Payment Service fails after reserving the inventory, it can emit a failure event, and the Inventory Service would listen to this and release the reserved product (compensating action).

Advantages:

  • Fully decentralized and scalable.
  • Easy to add new services without affecting others.

Challenges:

  • Harder to track and manage because there is no central coordinator.
  • More complex error handling logic in each service.

b. Orchestration-based Saga​

In an Orchestration-based Saga, a central coordinator (Orchestrator) is responsible for managing the workflow of the transaction. The Orchestrator sends commands to each service to execute a step in the transaction and handles failures by issuing compensating commands if any service fails.

Example:

  • The Orchestrator sends a command to the Order Service to create an order.
  • After receiving success, the Orchestrator asks the Inventory Service to reserve the product.
  • If the Payment Service fails, the Orchestrator issues a compensating command to the Inventory Service to release the product.

Advantages:

  • Easier to implement because the Orchestrator handles the entire workflow.
  • Centralized error handling and compensations.

Challenges:

  • The Orchestrator can become a single point of failure or bottleneck.
  • Needs to manage state and retries, making it potentially complex.

2. Compensation Mechanism​

The Compensation Mechanism is the core of both the Saga Pattern and other eventual consistency models. A compensation transaction is essentially the "undo" operation that reverses the effects of a previously completed step in case of failure.

How It Works: Each microservice is responsible for defining and implementing compensating actions for its operations. For example:

  • If the Inventory Service reserves stock but the Payment Service fails, the Inventory Service must implement logic to release the stock.
  • If the Payment Service successfully charges a customer but the Shipping Service fails, the Payment Service must issue a refund.

Key Considerations:

  • Compensation is not always straightforward: Rolling back a transaction isn't always as simple as undoing a change. Some operations might be irreversible (e.g., shipping a product), requiring compensating actions that mitigate the issue (e.g., issuing a refund).
  • State-driven compensation: Services must maintain state information to understand which transactions require compensation.

3. Two-Phase Commit (2PC)​

Two-Phase Commit (2PC) is a protocol used for coordinating distributed transactions across multiple systems. It's traditionally used in tightly coupled distributed systems but is less popular in microservice architectures due to performance and complexity concerns.

How 2PC Works:​

  1. Prepare Phase: The transaction coordinator asks each microservice if they are ready to commit the transaction. The services reply with "yes" or "no."
  2. Commit/Rollback Phase: If all services reply with "yes," the coordinator instructs them to commit. If any service replies with "no," the coordinator sends a rollback command to all participating services.

Pros:

  • Strong consistency guarantees.
  • Ensures atomicity across distributed systems.

Cons:

  • Performance overhead: 2PC introduces a blocking wait time for services to prepare and commit, which can hurt performance in a microservices architecture.
  • Risk of blocking: If one service crashes, it can leave other services waiting indefinitely, potentially causing system-wide issues.

Given these challenges, 2PC is rarely used in modern, large-scale microservice architectures.


4. Eventual Consistency​

In distributed microservice architectures, eventual consistency is often the goal rather than strong consistency (ACID). Eventual consistency ensures that after some time, all participants in a distributed transaction will reach a consistent state, even if there are temporary inconsistencies.

How to Handle Rollbacks with Eventual Consistency:​

  • Delayed reconciliation: Some systems may use periodic reconciliation processes to detect inconsistencies and trigger compensating actions to ensure data integrity.
  • State-based updates: Services track their state, and if a failure is detected, each service is responsible for ensuring consistency through compensation.

Example: If a Payment Service charges a customer but the Shipping Service fails to ship the item, the system might temporarily be in an inconsistent state (payment made, no shipment). Over time, a reconciliation job or event listener detects the issue and initiates a refund to bring the system back to consistency.

Pros:

  • High scalability and resilience.
  • Tolerates temporary inconsistencies while ensuring long-term data integrity.

Cons:

  • Complex to reason about, as temporary inconsistencies can confuse users and developers.
  • Some actions (e.g., shipping a product) may be difficult to compensate.

5. Idempotency and Retries​

In microservice architectures, idempotency and retries play a critical role in handling rollbacks and retries, ensuring that repeated operations do not result in unwanted side effects.

Idempotency:​

An operation is idempotent if performing it multiple times has the same result as performing it once. For example, if a Payment Service is asked to process a payment twice, it should charge the customer only once.

Why It Matters:

  • When rolling back a transaction, services may need to retry operations (e.g., issuing a refund). These retries must be idempotent to prevent issues like charging a customer multiple times or reserving inventory repeatedly.
  • Retry policies are often implemented in communication mechanisms (e.g., message brokers) to ensure reliability.

Best Practices for Idempotency:​

  • Use unique transaction IDs for requests so that services can detect duplicate requests.
  • Design APIs and actions to ensure they are safe for retries.

6. Conclusion​

Rolling back or canceling transactions in a Microservice Architecture requires careful design to handle distributed data, asynchronous operations, and eventual consistency. The Saga Pattern (both choreography and orchestration) is the most widely used approach, allowing each service to independently handle parts of the transaction and apply compensating actions in case of failure. Other techniques like compensation mechanisms, eventual consistency, and idempotency ensure that microservices can maintain integrity, even across complex, distributed environments.

When choosing a rollback strategy, it's essential to balance the need for consistency with the performance and scalability requirements of the system.