Skip to main content

Microservice meet with Spring Cloud and Kubernetes

· 13 min read

Introduction​

The objective is to design and deploy a group of microservices: Profile Service, Inventory Service, Order Service, Shipment Service, and an OAuth2 Service for centralized authentication. Key requirements include robust service discovery, API Gateway setup, secure communication, and efficient load balancing for high availability and fault tolerance in Kubernetes.

High-Level Architecture​

Each microservice will be deployed in a Kubernetes cluster with multiple replicas, leveraging Spring Cloud Kubernetes Discovery for service registration and Spring Cloud LoadBalancer for client-side load balancing. An API Gateway will handle external requests, providing routing, load balancing, and JWT-based authentication. Communication between services is secured with JWT tokens generated by the OAuth2 Service.

Service Design​

Each microservice is structured independently, deployed as a standalone Spring Boot application with individual responsibilities and access rules. Here’s an overview of the key services:

1. Profile Service​

Manages customer profiles, providing endpoints for CRUD operations on profile data. Accessible to services like Order and Shipment for customer information retrieval.

2. Inventory Service​

Tracks inventory levels and provides endpoints to update, retrieve, and manage inventory data. Called by the Order Service for real-time inventory status updates.

3. Order Service​

Processes customer orders, calculates totals, and interfaces with Inventory and Profile Services. This service is central to the architecture, relying on other services for a smooth customer experience.

4. Shipment Service​

Manages shipment tracking and logistics for orders, called by the Order Service once an order is processed and confirmed.

5. OAuth2 Service​

Acts as the authentication provider, issuing and validating JWT tokens for secure communication. The API Gateway uses these tokens to authenticate requests.

Order-Service Auth-Service Inventory-Service
| | |
|--- Request JWT ------------->| |
| | |
|<-- Receive JWT --------------| |
| | |
|--- Call Inventory ---------->| |
| (with JWT) | |
| |--- Validate JWT------------>|
| | |
| |<-- JWT Valid/Invalid--------|
| | |
|<-- Receive Inventory Data -- | |

Kubernetes Deployment and Service Discovery​

Each service will be deployed to Kubernetes as a Deployment with multiple replicas and exposed via a Service to enable discovery. Spring Cloud Kubernetes is used for service registration and discovery.

Configuring Kubernetes Service Discovery for Each Microservice​

To set up service discovery in Kubernetes, add the following dependencies to your pom.xml:

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-discovery</artifactId>
</dependency>

Configure the application.yml in each service to enable discovery:

spring:
cloud:
kubernetes:
discovery:
enabled: true #Enable service discovery
config:
enabled: true
loadbalancer:
enabled: true #Enable loadbalancer

This configuration registers each service with Kubernetes' internal DNS, allowing them to be discoverable by service name within the namespace. This way, each microservice is registered under its own Kubernetes service name, and can resolve other services (e.g., http://inventory-service:8080) when communicating with them. This configuration, combined with Kubernetes DNS, helps maintain an efficient, loosely coupled architecture.

Note: The Kubernetes Service Discovery configuration should indeed be included in each individual microservice’s configuration to ensure they can all register themselves with the Kubernetes DNS and be discoverable by other services in the namespace. This setup allows each microservice to dynamically locate and communicate with other services via Kubernetes' internal DNS naming conventions.

Why Configure Service Discovery in Each Microservice?

Each microservice is an independently deployed unit, so configuring Spring Cloud Kubernetes Discovery in each service's application.yml allows it to:

  1. Register with Kubernetes DNS under a unique service name, enabling internal routing across services.
  2. Access other services by simply using their service names, as Kubernetes DNS provides a consistent way to locate services within the cluster.

Note Here you might be noticed that both the discovery and load balancer components are configured within each microservice. This design allows each microservice to dynamically discover the current instances of other services (using the spring-cloud-kubernetes-discovery integration) and balance requests among those instances using the spring-cloud-loadbalancer component.

Service Registration​

Each microservice is deployed as a separate Deployment and Service in Kubernetes. Here’s an example of the inventory-service Kubernetes configuration:

To deploy and register microservices like an Inventory Service in Kubernetes, we use Spring Cloud’s Kubernetes Discovery capabilities, allowing services to register with Kubernetes’ DNS-based service registry automatically.

Step 1: Create a Kubernetes Service and Deployment for each microservice:

YAML files for defining the service and deployment:

# inventory-service-deployment.yml
apiVersion: v1
kind: Service
metadata:
name: inventory-service
spec:
selector:
app: inventory
ports:
- protocol: TCP
port: 8080
targetPort: 8080
type: ClusterIP # Internal access within the cluster

---
apiVersion: apps/v1
kind: Deployment
metadata:
name: inventory-deployment
spec:
replicas: 2 # Number of instances (pods) of the Inventory Service
selector:
matchLabels:
app: inventory
template:
metadata:
labels:
app: inventory
spec:
containers:
- name: inventory
image: myregistry/inventory-service:latest
ports:
- containerPort: 8080

This configuration deploys inventory-service with two replicas, registered under the service name inventory-service. Other services can communicate with it via http://inventory-service:8080.

More comprehensive explainations When deploying an inventory-deployment with two replicas, you will indeed see two pods running in parallel in the specified namespace. Kubernetes names these pods based on the deployment name, followed by a unique suffix (not just sequential numbers), so you may see pod names like inventory-deployment-abc12 and inventory-deployment-def34.

In Kubernetes, these two pods are managed by a single inventory-service which acts as a layer of abstraction. When the client load balancer targets inventory-service, the service uses Kubernetes' built-in load balancing to forward requests to one of the available inventory-deployment pods.

How Kubernetes Service Load Balancing Works:

  1. Service Abstraction: The inventory-service has a Cluster IP that exposes the service within the cluster.
  2. Endpoints Selection: The service maintains an updated list of active pod IPs (called "endpoints") that match the deployment selector.
  3. Round-Robin Load Balancing: Kubernetes uses a round-robin approach by default. When a request reaches inventory-service, it selects one of the pod IPs associated with the deployment, balancing requests across pods automatically. The service doesn't keep state; each request is independently forwarded to one of the pod replicas.

Example Flow:

  1. Client Request: A client load balancer sends a request to the inventory-service endpoint.
  2. Service to Pod Routing: Kubernetes service routing picks an endpoint (pod IP) from the inventory-deployment pods and forwards the request to it.

For better control and observability, custom load balancer configurations can also be implemented to fine-tune how the requests are distributed, especially in setups requiring sticky sessions or advanced routing.

Step 2: Register Service with Kubernetes Discovery:

Spring Cloud Kubernetes automatically registers inventory-service based on the deployment labels. Enable Spring Cloud Discovery in your Spring Boot application via @EnableDiscoveryClient:

@EnableDiscoveryClient
@SpringBootApplication
public class InventoryServiceApplication {
public static void main(String[] args) {
SpringApplication.run(InventoryServiceApplication.class, args);
}
}

Step 3: Configure Discovery in application.yml:

spring:
cloud:
kubernetes:
discovery:
enabled: true
all-namespaces: true # Optional: enables discovery across namespaces

Load Balancing with Spring Cloud LoadBalancer​

To efficiently distribute incoming requests across available instances, Spring Cloud LoadBalancer provides a client-side load-balancing solution, automatically distributing traffic among instances of a service.

Enable Spring Cloud LoadBalancer by adding Spring Cloud LoadBalancer dependency:

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Configure a ServiceInstanceListSupplier bean for custom load balancing behavior if required:

import org.springframework.cloud.client.loadbalancer.ServiceInstanceListSupplier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class LoadBalancerConfig {

@Bean
public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier() {
return ServiceInstanceListSupplier.builder()
.withDiscoveryClient()
.withCaching()
.build();
}
}

Set Up Load Balancer with WebClient:

  • Configure a WebClient bean with load balancing to distribute requests among inventory-service instances.
import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.context.annotation.Bean;
import org.springframework.web.reactive.function.client.WebClient;

@Configuration
public class WebClientConfig {

@Bean
@LoadBalanced
public WebClient.Builder loadBalancedWebClientBuilder() {
return WebClient.builder();
}
}

Use Load-Balanced WebClient in the calling service:

@Service
public class InventoryClient {

private final WebClient.Builder webClientBuilder;

public InventoryClient(WebClient.Builder webClientBuilder) {
this.webClientBuilder = webClientBuilder;
}

public Mono<Product> getProductById(String productId) {
return webClientBuilder.build()
.get()
.uri("http://inventory-service/products/{id}", productId)
.retrieve()
.bodyToMono(Product.class);
}
}

How Load Balancing Works:

  • Spring Cloud LoadBalancer, by default, uses a round-robin strategy to distribute requests.
  • It balances requests between the two inventory-service pods and automatically updates when new instances are added or removed.

  1. By default, Spring Cloud LoadBalancer will round-robin requests to different instances, or customize by implementing a custom LoadBalancerRequestTransformer.

API Gateway Setup​

The Spring Cloud API Gateway is deployed as a separate microservice, managing traffic and providing JWT-based authentication. The gateway routes requests to the appropriate services and load-balances among available replicas.

Routing and Load Balancing​

[External Client] --> [API Gateway]
|
| (Discovers routes from Kubernetes Discovery)
|
[inventory-service] <--+-- "lb://inventory-service"
[shipment-service] <--+-- "lb://shipment-service"
[order-service] <--+-- "lb://order-service"
[profile-service] <--+-- "lb://profile-service"
[other-service] <--+-- Discovered automatically

Configure the gateway’s application.yml to define routes to each service:

spring:
cloud:
gateway:
routes:
- id: profile
uri: http://profile-service
predicates:
- Path=/profile/**
- id: inventory
uri: http://inventory-service
predicates:
- Path=/inventory/**
default-filters:
- TokenRelay

The TokenRelay filter enables the gateway to forward JWT tokens to downstream services for authenticated requests.

Configuring JWT Authentication​

For security, JWT validation can be implemented in the gateway or within each service. Here’s how to configure JWT validation in the gateway:

  1. Add OAuth2 dependencies to the gateway:

    <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
    </dependency>
  2. Configure JWT validation in the gateway’s application.yml:

    spring:
    security:
    oauth2:
    resourceserver:
    jwt:
    issuer-uri: https://auth-server/oauth2/default

Microservice Communication​

Microservices communicate through Spring Cloud LoadBalancer and WebClient.

Using WebClient with Load Balancer​

To call the inventory-service from the order-service, configure WebClient with load balancing:

@Configuration
public class WebClientConfig {

@LoadBalanced
@Bean
public WebClient.Builder webClientBuilder() {
return WebClient.builder();
}
}

Example: Order Service Calls Profile Service​

Here’s how order-service might call profile-service:

@RestController
public class OrderController {

private final WebClient.Builder webClientBuilder;

@Autowired
public OrderController(WebClient.Builder webClientBuilder) {
this.webClientBuilder = webClientBuilder;
}

@GetMapping("/order/{orderId}")
public Mono<OrderDetails> getOrderDetails(@PathVariable String orderId) {
return webClientBuilder.build()
.get()
.uri("http://profile-service/profile/{orderId}", orderId) // The uri can be either http or lb(load balancer)
.retrieve()
.bodyToMono(OrderDetails.class);
}
}

Securing Microservices​

Secure Communication with mTLS and JWT

mTLS (mutual TLS) and JWT (JSON Web Token) are two primary ways to secure communication between microservices.

  • mTLS: Ensures that both the client and server authenticate each other, useful in a zero-trust network.
  • JWT: Adds stateless authentication, where each request includes a signed token with user and session data.

Each microservice validates JWT tokens received from the gateway, ensuring secure and authenticated communication.

JWT Authentication with API Gateway​

JWT Authentication is often managed at the API Gateway, validating tokens and authorizing requests before forwarding them to services.

In this approach, JWT validation is centralized in the gateway. Tokens are verified at the gateway and forwarded to services, reducing the validation load on individual services.

  1. Add Spring Security Dependency for JWT support:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
  1. Configure JWT Verification in Gateway:

Validate JWTs at the gateway level, allowing only authorized requests to reach downstream services.

@Configuration
public class SecurityConfig extends WebSecurityConfigurerAdapter {

@Override
protected void configure(HttpSecurity http) throws Exception {
http
.authorizeRequests()
.antMatchers("/api/**").authenticated()
.and()
.oauth2ResourceServer().jwt();
}
}

Alternative: JWT Validation in Interceptors​

Alternatively, JWT validation can be handled within each microservice using a custom Interceptor, within services, add an Interceptor to validate JWTs before processing requests.

import org.springframework.stereotype.Component;
import org.springframework.web.servlet.HandlerInterceptor;

@Component
public class JwtInterceptor implements HandlerInterceptor {

@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
String token = request.getHeader("Authorization");
// Verify the JWT token
// Decode and validate claims
return isValidToken(token);
}
}

Pros and Cons of JWT and mTLS​

ApproachProsCons
mTLSHigh security with mutual authentication.Requires certificate management, adds complexity.
JWTStateless, easily scalable, widely supported.Token expiration handling; replay attacks if not carefully implemented.

<a name="kubernetes-deployment-and-auto-scaling"></a>
## Kubernetes Deployment and Auto-scaling

Scaling services dynamically is fundamental in Kubernetes. Kubernetes provides an effective way to handle **auto-scaling** of microservices based on resource utilization, e.g. CPU, memory, or custom metrics.

Each microservice is balanced using **Spring Cloud LoadBalancer**, ensuring even distribution across replicas.

> **Step 1**: Configuring Horizontal Pod Autoscaler (HPA)

1. Enable auto-scaling in Kubernetes based on CPU usage by defining an HPA resource.

**inventory-hpa.yaml**
```yaml
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: inventory-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: inventory-deployment
minReplicas: 2
maxReplicas: 10
targetCPUUtilizationPercentage: 75
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 50

Step 2: Implementing Cluster Scaling

Use Kubernetes’ Cluster Autoscaler for scaling nodes as workload demand grows, ensuring optimal resource use without manual intervention.


Ensuring Secure Microservice Communication​

Securing communication between services is critical. Use mTLS (mutual TLS) and JWT tokens for service-to-service authentication within Kubernetes clusters.

Step 1: Configure Mutual TLS (mTLS)​

Istio or Linkerd can help enforce mTLS between services:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: your-namespace
spec:
mtls:
mode: STRICT

Summary​

  1. Use ConfigMaps and Secrets for configuration.
  2. Implement autoscaling with HPA.
  3. Secure communication with mTLS and JWT.
  4. Monitor with readiness/liveness probes and Prometheus/Grafana.

This guide provides an architecture blueprint for deploying secure, scalable, and efficient microservices in Kubernetes using Spring Cloud. The API Gateway simplifies external access, service discovery, and JWT-based authentication, while Kubernetes ensures high availability and resilience across services.