Rate Limiting in Spring Cloud Netflix Zuul

1. Introduction

Spring Cloud Netflix Zuul is an open source gateway that wraps Netflix Zuul. It adds some specific features for Spring Boot applications. Unfortunately, rate limiting is not provided out of the box.

In this tutorial, we will explore Spring Cloud Zuul RateLimit which adds support for rate limiting requests.

2. Maven Configuration

In addition to the Spring Cloud Netflix Zuul dependency, we need to add Spring Cloud Zuul RateLimit to our application’s pom.xml:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
<dependency>
    <groupId>com.marcosbarbero.cloud</groupId>
    <artifactId>spring-cloud-zuul-ratelimit</artifactId>
    <version>2.2.0.RELEASE</version>
</dependency>

3. Example Controller

Firstly, let’s create a couple of REST endpoints on which we will apply the rate limits.

Below is a simple Spring Controller class with two endpoints:

@Controller
@RequestMapping("/greeting")
public class GreetingController {

    @GetMapping("/simple")
    public ResponseEntity<String> getSimple() {
        return ResponseEntity.ok("Hi!");
    }

    @GetMapping("/advanced")
    public ResponseEntity<String> getAdvanced() {
        return ResponseEntity.ok("Hello, how you doing?");
    }
}

As we can see, there is no code specific to rate limit the endpoints. This is because we’ll configure that in our Zuul properties within the application.yml file. Thus, keeping our code decoupled.

4. Zuul Properties

Secondly, let’s add the following Zuul properties in our application.yml file:

zuul:
  routes:
    serviceSimple:
      path: /greeting/simple
      url: forward:/
    serviceAdvanced:
      path: /greeting/advanced
      url: forward:/
  ratelimit:
    enabled: true
    repository: JPA
    policy-list:
      serviceSimple:
        - limit: 5
          refresh-interval: 60
          type:
            - origin
      serviceAdvanced:
        - limit: 1
          refresh-interval: 2
          type:
            - origin
  strip-prefix: true

Under zuul.routes we provide the endpoint details. And under zuul.ratelimit.policy-list, we provide the rate limit configurations for our endpoints. The limit property specifies the number of times the endpoint can be called within the refresh-interval.

As we can see, we added a rate limit of 5 requests per 60 seconds for the serviceSimple endpoint. In contrast, serviceAdvanced has a rate limit of 1 request per 2 seconds.

The type configuration specifies which rate limit approach we want to follow. Here are the possible values:

• origin – rate limit based on the user origin request
• url – rate limit based on the request path of the downstream service
• user – rate limit based on the authenticated username or ‘anonymous’
• No value – acts as a global configuration per service. To use this approach just don’t set param ‘type’

5. Testing the Rate Limit

5.1. Request Within the Rate Limit

Next, let’s test the rate limit:

@Test
public void whenRequestNotExceedingCapacity_thenReturnOkResponse() {
    ResponseEntity<String> response = restTemplate.getForEntity(SIMPLE_GREETING, String.class);
    assertEquals(OK, response.getStatusCode());

    HttpHeaders headers = response.getHeaders();
    String key = "rate-limit-application_serviceSimple_127.0.0.1";

    assertEquals("5", headers.getFirst(HEADER_LIMIT + key));
    assertEquals("4", headers.getFirst(HEADER_REMAINING + key));
    assertThat(
      parseInt(headers.getFirst(HEADER_RESET + key)),
      is(both(greaterThanOrEqualTo(0)).and(lessThanOrEqualTo(60000)))
    );
}

Here we make a single call to the endpoint /greeting/simple. The request is successful since it is within the rate limit.

Another key point is that with each response we get back headers providing us with further information on the rate limit. For above request, we would get following headers:

X-RateLimit-Limit-rate-limit-application_serviceSimple_127.0.0.1: 5
X-RateLimit-Remaining-rate-limit-application_serviceSimple_127.0.0.1: 4
X-RateLimit-Reset-rate-limit-application_serviceSimple_127.0.0.1: 60000

In other words:

• X-RateLimit-Limit-[key]: the limit configured for the endpoint
• X-RateLimit-Remaining-[key]: the remaining number of attempts to call the endpoint
• X-RateLimit-Reset-[key]: the remaining number of milliseconds of the refresh-interval configured for the endpoint

In addition, if we immediately fire the same endpoint again, we could get:

X-RateLimit-Limit-rate-limit-application_serviceSimple_127.0.0.1: 5
X-RateLimit-Remaining-rate-limit-application_serviceSimple_127.0.0.1: 3
X-RateLimit-Reset-rate-limit-application_serviceSimple_127.0.0.1: 57031

Notice the decreased remaining number of attempts and remaining number of milliseconds.

5.2. Request Exceeding the Rate Limit

Let’s see what happens when we exceed the rate limit:

@Test
public void whenRequestExceedingCapacity_thenReturnTooManyRequestsResponse() throws InterruptedException {
    ResponseEntity<String> response = this.restTemplate.getForEntity(ADVANCED_GREETING, String.class);
    assertEquals(OK, response.getStatusCode());
    
    for (int i = 0; i < 2; i++) {
        response = this.restTemplate.getForEntity(ADVANCED_GREETING, String.class);
    }

    assertEquals(TOO_MANY_REQUESTS, response.getStatusCode());

    HttpHeaders headers = response.getHeaders();
    String key = "rate-limit-application_serviceAdvanced_127.0.0.1";

    assertEquals("1", headers.getFirst(HEADER_LIMIT + key));
    assertEquals("0", headers.getFirst(HEADER_REMAINING + key));
    assertNotEquals("2000", headers.getFirst(HEADER_RESET + key));

    TimeUnit.SECONDS.sleep(2);

    response = this.restTemplate.getForEntity(ADVANCED_GREETING, String.class);
    assertEquals(OK, response.getStatusCode());
}

Here we call the endpoint /greeting/advanced twice in quick successions. Since we have configured rate limit as one request per 2 seconds, the second call will fail. As a result, the error code 429 (Too Many Requests) is returned to the client.

Below are the headers returned when the rate limit is reached:

X-RateLimit-Limit-rate-limit-application_serviceAdvanced_127.0.0.1: 1
X-RateLimit-Remaining-rate-limit-application_serviceAdvanced_127.0.0.1: 0
X-RateLimit-Reset-rate-limit-application_serviceAdvanced_127.0.0.1: 268

After that, we sleep for 2 seconds. This is the refresh-interval configured for the endpoint. Finally, we fire the endpoint again and get a successful response.

6. Custom Key Generator

We can customize the keys sent in the response header using a custom key generator. This is useful because the application might need to control the key strategy beyond the options offered by the type property.

For instance, this can be done by creating a custom RateLimitKeyGenerator implementation. We can add further qualifiers or something entirely different:

@Bean
public RateLimitKeyGenerator rateLimitKeyGenerator(RateLimitProperties properties, 
  RateLimitUtils rateLimitUtils) {
    return new DefaultRateLimitKeyGenerator(properties, rateLimitUtils) {
        @Override
        public String key(HttpServletRequest request, Route route, 
          RateLimitProperties.Policy policy) {
            return super.key(request, route, policy) + "_" + request.getMethod();
        }
    };
}

The code above appends the REST method name to the key. For example:

X-RateLimit-Limit-rate-limit-application_serviceSimple_127.0.0.1_GET: 5

Another key point is that the RateLimitKeyGenerator bean will be automatically configured by spring-cloud-zuul-ratelimit.

7. Custom Error Handling

The framework supports various implementations for rate limit data storage. For instance, Spring Data JPA and Redis are provided. By default, failures are just logged as errors using the DefaultRateLimiterErrorHandler class.

When we need to handle the errors differently, we can define a custom RateLimiterErrorHandler bean:

@Bean
public RateLimiterErrorHandler rateLimitErrorHandler() {
    return new DefaultRateLimiterErrorHandler() {
        @Override
        public void handleSaveError(String key, Exception e) {
            // implementation
        }

        @Override
        public void handleFetchError(String key, Exception e) {
            // implementation
        }

        @Override
        public void handleError(String msg, Exception e) {
            // implementation
        }
    };
}

Similar to the RateLimitKeyGenerator bean, the RateLimiterErrorHandler bean will also be automatically configured.

8. Conclusion

In this article, we saw how to rate limit APIs using Spring Cloud Netflix Zuul and Spring Cloud Zuul RateLimit.