API gateways are probably one of the first few keywords you hear when you enter the world of cloud computing. You must have yourself or seen someone drop a box casually in a system design interview & mark it as API gateway. Though aside from the fact that it is considered as standard solution for allowing clients to interface with your services, it serves a bunch of other functionalities. Covering all of them will require a post about each of these functionalities(maybe these go into my to-do for future posts). So I will cover two aspects of API gateways in this post i.e. request routing & authentication. This also fits nicely into the theme of the image used in the post which gives you vibes from the Severance.
Setting the stage
Lets first go through a brief introduction to API gateway & why we essentially need them. So consider your application which was initially built as a monolith. Now you are bumping into scaling challenges & you decide to jump on the microservice bandwagon(Not always the right choice but you-do-you). Earlier in the monolith all your endpoints were secured through JWT & clients initially called a /login endpoint to get the JWT & then use it in their Authorization headers to invoke service endpoints. The service controllers can make a function call to validate function & verify if JWT is still valid before processing the request.
Now that you are in the microservice world, the request enters your service boundary & then each service will need to invoke the user-service to validate their tokens. This means any change in user-service ends up propagating to all your services. Another challenge is request routing. Earlier your monolith application served all your endpoints whereas in microservice setup, each service serves a set of endpoints. Clients will now need to know which service serves what set of endpoints & where these services(or their corresponding load balancers) are deployed.
Understanding the problem
You can now see the cracks forming in this architecture. With increase in number of services, the amount of service mapping data also increases. Also each service is now tightly coupled with the API contract of the user-service as it provides authentication mechanism.
Lets try to understand this better with a concrete example. Consider an e-commerce application consisting of order-service serving functionalities around orders & an order-pickup-service serving pickups. Both these services require authenticated access so we have created user-service. You can find the starter code for these baseline services on this branch.
How does API gateway solves this?
We have seen the pain points around authentication & service routing which we need to address if we expose these services directly. Now lets try to understand how an API gateway solves these problems. An API gateway will be the entry point for your application from client’s perspective so that your clients don’t need to know the exact location of your services or even that their requests are served through individual services. API gateway abstracts this & routes the request to correct service based upon the request path. So requests for /api/v1/orders will get routed to order-service while api/v1/pickups will get routed to order-pickup-service.
The authentication also happens through the API gateway & only authenticated requests are routed to the services so they don’t need to be concerned about the user-service. Any change in API contract for user-service needs to be now addressed only at API gateway level & doesn’t impacts all the microservices in your application. The request workflow looks something like the one described in following image:
Now lets try to see the API gateway in action through a demo. We will see how we can implement an API gateway using Spring Cloud Gateway & through Ingress in the Kubernetes world. Lets dive in.
Spring Cloud API Gateway
Spring ecosystem provides great support for implementing API gateway out of the box. It comes with sane defaults & provides support for customization such as request routing based upon custom rules. It also can be integrated with service discovery so that when you scale up/down instances of any service in your application, its location can be registered with the API gateway. Lets tackle the problems we saw initially & understand how does Spring cloud gateway helps in solving them. The updated diagram with Spring cloud gateway is below
Request routing
In order to route the request to correct service, we just need to define the request routes in our gateway service’s config after which any request that matches a specific route will be routed to the corresponding service. Clients will only interact with the API gateway & are not concerned about the actual location of the service. The routing looks as below where we have routes & the corresponding service to which the request needs to be forwarded. We are also adding a custom header X-From-Gateway so that the service knows that the request was forwarded from the API gateway(Not sufficient for production use case).
spring:
application:
name: gateway
cloud:
gateway:
mvc:
routes:
- id: order-service
uri: http://localhost:8001
predicates:
- Path=/api/v1/orders/**
filters:
- AddRequestHeader=X-From-Gateway, true
- id: pickup-service
uri: http://localhost:8002
predicates:
- Path=/api/v1/pickups/**
filters:
- AddRequestHeader=X-From-Gateway, true
- id: user-service
uri: http://localhost:8003
predicates:
- Path=/auth/login,/auth/register
server:
port: 8888
Authentication
Now that our API gateway serves as entry point for our requests, we can implement the authentication logic here itself & terminate the requests which don’t have proper authentication setup. Users can invoke /login or /register endpoint when they receive a 401/403 status to either signup or refresh their JWT.
As we are using Spring framework, we can achieve this by an interceptor that processes the request for authentication before routing to the actual service. This piece of code works before we hit the routing rules defined in our config. As part of this we extract the JWT & invoke /validate method on user-service. If the token is valid we allow the request to move forward else we return a 401/403 status code.
@Component
public class JwtValidationInterceptor implements HandlerInterceptor {
private static final String AUTH_SERVICE_URL = "http://localhost:8003/auth/validate";
private final RestTemplate restTemplate;
public JwtValidationInterceptor(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
String jwtToken = request.getHeader("Authorization");
if (jwtToken == null || !jwtToken.startsWith("Bearer ")) {
response.sendError(HttpStatus.UNAUTHORIZED.value(), "Missing or invalid Authorization header");
return false;
}
String token = jwtToken.substring(7);
Boolean isValid;
try {
isValid = restTemplate.postForObject(AUTH_SERVICE_URL, token, Boolean.class);
} catch (HttpClientErrorException e) {
if (e.getStatusCode() == HttpStatus.UNAUTHORIZED) {
response.sendError(HttpStatus.UNAUTHORIZED.value(), "Invalid JWT token");
} else {
response.sendError(HttpStatus.INTERNAL_SERVER_ERROR.value(), "Error validating JWT token");
}
return false;
}
if (isValid == null || !isValid) {
response.sendError(HttpStatus.UNAUTHORIZED.value(), "Invalid JWT token");
return false;
}
return true;
}
}
Demo
Lets now see these components in action. We will see the following workflow where user invokes gateway endpoint running on port 8888 while services run on different ports:
- Non-registered user tries to invoke the
/loginendpoint ->403 - User registers by invoking
/registerendpoint ->200with JWT - User tries the
/loginendpoint again ->200with JWT - User tries to create an order without any authorization headers ->
401 - User retries order creation with authorization headers ->
200 - User retrieves the created order by id with authorization headers ->
200 - User retries retrieval request without authorization headers ->
401
You can find all the code for this demo on this branch.
API Gateway through Kubernetes Ingress
In the Kubernetes world, we have different options to achieve the functionality that was provided by the Spring cloud gateway. This can be achieved through Gateway API or Ingress rules. The Gateway API provides more comprehensive feature set related to API gateways. For this demo we will use the Ingress rules to achieve the routing & authentication functionality. The request now enters the Kubernetes cluster through our Ingress implementation & is then routed to internal services which are not directly exposed to outside world by specifying their service type as ClusterIP.
On implementation level, we have removed the Spring cloud gateway project from the codebase & deployed our services to Kubernetes through deployment workload. Manifest for one of the services is as below:
# deployment/order-service-deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: order-service
namespace: api-gateway
spec:
replicas: 1 # The number of replicas is kept to 1 as we are using an in-memory database.
selector:
matchLabels:
app: order-service
template:
metadata:
labels:
app: order-service
spec:
containers:
- name: order-service
image: varunu2892/order-service:latest
imagePullPolicy: Never
ports:
- containerPort: 8001
---
apiVersion: v1
kind: Service
metadata:
name: order-service
namespace: api-gateway
spec:
selector:
app: order-service
ports:
- protocol: TCP
port: 80
targetPort: 8001
type: ClusterIP
Request routing
We do request routing by specifying the prefix for service endpoints. Consider the ingress rule for user-service where we are routing traffic for /auth/register & /auth/login endpoints to the user-service deployed on port 8003. We do routing for other services in the similar manner.
# User service ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: auth-service-ingress
namespace: api-gateway
spec:
ingressClassName: nginx
rules:
- host: api-gateway-demo.com
http:
paths:
- path: /auth/login
pathType: Prefix
backend:
service:
name: user-service
port:
number: 8003
- path: /auth/register
pathType: Prefix
backend:
service:
name: user-service
port:
number: 8003
Authentication
Now for authentication you can specify an auth-url in the ingress annotation which the ingress rule will invoke once it receives a request. Note that we have not specified these annotations in the manifest of user-service as we want clients to be able to invoke login/register endpoint without providing any JWT. This is why we have also specified ingress rule for user-service in a separate manifest so that the auth annotations don’t get applied on them. Now when a request enters our cluster, the ingress rule will first invoke /validate endpoint with a GET operation & if the validation is successful, it forwards the request to the services.
We also specify additional request header which will be passed on to the downstream services so that they can confirm that the request is being routed through ingress.
# deployment/protected-services-protected-services-ingress.yml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: protected-services-ingress
namespace: api-gateway
annotations:
nginx.ingress.kubernetes.io/auth-url: "http://user-service.api-gateway.svc.cluster.local/auth/validate"
nginx.ingress.kubernetes.io/auth-method: "GET"
nginx.ingress.kubernetes.io/auth-response-headers: "X-From-Gateway"
spec:
ingressClassName: nginx
rules:
- host: api-gateway-demo.com
http:
paths:
- path: /api/v1/orders
pathType: Prefix
backend:
service:
name: order-service
port:
number: 8001
- path: /api/v1/pickups
pathType: Prefix
backend:
service:
name: order-pickup-service
port:
number: 8002
Demo
For demo, we will see a similar workflow as we saw in case of Spring cloud gateway demo. We first verify that our service pods are running & then test the endpoints through a REST client.
Lets first take a look at the services deployed in the cluster. The services are deployed in the api-gateway namespace which we have created for this demo. We also see the 2 different ingress rules defined respectively for user-service providing authentication & an ingress rule for the other 2 services.
Now lets see the workflow in action
You can find all the code for this demo on this branch.
Conclusion
API gateways are very powerful & can be utilized to solve a variety of problems in a cloud ecosystem. Again it comes down to the actual use case you are trying to solve & how well does a particular technology fits into that context. Hope this was helpful. Next I will cover circuit breakers to see how to keep your systems operational in case of failures. Till then happy learning.