3. Spring Cloud LoadBalancer

Spring Cloud provides its own client-side load-balancer abstraction and implementation. For the load-balancing mechanism, ReactiveLoadBalancer interface has been added and a Round-Robin-based and Random implementations have been provided for it. In order to get instances to select from reactive ServiceInstanceListSupplier is used. Currently we support a service-discovery-based implementation of ServiceInstanceListSupplier that retrieves available instances from Service Discovery using a Discovery Client available in the classpath.spring-doc.cn

It is possible to disable Spring Cloud LoadBalancer by setting the value of spring.cloud.loadbalancer.enabled to false.

3.1. Switching between the load-balancing algorithms

The ReactiveLoadBalancer implementation that is used by default is RoundRobinLoadBalancer. To switch to a different implementation, either for selected services or all of them, you can use the custom LoadBalancer configurations mechanism.spring-doc.cn

For example, the following configuration can be passed via @LoadBalancerClient annotation to switch to using the RandomLoadBalancer:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    ReactorLoadBalancer<ServiceInstance> randomLoadBalancer(Environment environment,
            LoadBalancerClientFactory loadBalancerClientFactory) {
        String name = environment.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RandomLoadBalancer(loadBalancerClientFactory
                .getLazyProvider(name, ServiceInstanceListSupplier.class),
                name);
    }
}
The classes you pass as @LoadBalancerClient or @LoadBalancerClients configuration arguments should either not be annotated with @Configuration or be outside component scan scope.

3.2. Spring Cloud LoadBalancer integrations

In order to make it easy to use Spring Cloud LoadBalancer, we provide ReactorLoadBalancerExchangeFilterFunction that can be used with WebClient and BlockingLoadBalancerClient that works with RestTemplate. You can see more information and examples of usage in the following sections:spring-doc.cn

3.3. Spring Cloud LoadBalancer Caching

Apart from the basic ServiceInstanceListSupplier implementation that retrieves instances via DiscoveryClient each time it has to choose an instance, we provide two caching implementations.spring-doc.cn

3.3.1. Caffeine-backed LoadBalancer Cache Implementation

If you have com.github.ben-manes.caffeine:caffeine in the classpath, Caffeine-based implementation will be used. See the LoadBalancerCacheConfiguration section for information on how to configure it.spring-doc.cn

If you are using Caffeine, you can also override the default Caffeine Cache setup for the LoadBalancer by passing your own Caffeine Specification in the spring.cloud.loadbalancer.cache.caffeine.spec property.spring-doc.cn

WARN: Passing your own Caffeine specification will override any other LoadBalancerCache settings, including General LoadBalancer Cache Configuration fields, such as ttl and capacity.spring-doc.cn

3.3.2. Default LoadBalancer Cache Implementation

If you do not have Caffeine in the classpath, the DefaultLoadBalancerCache, which comes automatically with spring-cloud-starter-loadbalancer, will be used. See the LoadBalancerCacheConfiguration section for information on how to configure it.spring-doc.cn

To use Caffeine instead of the default cache, add the com.github.ben-manes.caffeine:caffeine dependency to classpath.

3.3.3. LoadBalancer Cache Configuration

You can set your own ttl value (the time after write after which entries should be expired), expressed as Duration, by passing a String compliant with the Spring Boot String to Duration converter syntax. as the value of the spring.cloud.loadbalancer.cache.ttl property. You can also set your own LoadBalancer cache initial capacity by setting the value of the spring.cloud.loadbalancer.cache.capacity property.spring-doc.cn

The default setup includes ttl set to 35 seconds and the default initialCapacity is 256.spring-doc.cn

You can also altogether disable loadBalancer caching by setting the value of spring.cloud.loadbalancer.cache.enabled to false.spring-doc.cn

Although the basic, non-cached, implementation is useful for prototyping and testing, it’s much less efficient than the cached versions, so we recommend always using the cached version in production. If the caching is already done by the DiscoveryClient implementation, for example EurekaDiscoveryClient, the load-balancer caching should be disabled to prevent double caching.
When you create your own configuration, if you use CachingServiceInstanceListSupplier make sure to place it in the hierarchy directly after the supplier that retrieves the instances over the network, for example, DiscoveryClientServiceInstanceListSupplier, before any other filtering suppliers.

3.4. Zone-Based Load-Balancing

To enable zone-based load-balancing, we provide the ZonePreferenceServiceInstanceListSupplier. We use DiscoveryClient-specific zone configuration (for example, eureka.instance.metadata-map.zone) to pick the zone that the client tries to filter available service instances for.spring-doc.cn

You can also override DiscoveryClient-specific zone setup by setting the value of spring.cloud.loadbalancer.zone property.
For the time being, only Eureka Discovery Client is instrumented to set the LoadBalancer zone. For other discovery client, set the spring.cloud.loadbalancer.zone property. More instrumentations coming shortly.
To determine the zone of a retrieved ServiceInstance, we check the value under the "zone" key in its metadata map.

The ZonePreferenceServiceInstanceListSupplier filters retrieved instances and only returns the ones within the same zone. If the zone is null or there are no instances within the same zone, it returns all the retrieved instances.spring-doc.cn

In order to use the zone-based load-balancing approach, you will have to instantiate a ZonePreferenceServiceInstanceListSupplier bean in a custom configuration.spring-doc.cn

We use delegates to work with ServiceInstanceListSupplier beans. We suggest using a DiscoveryClientServiceInstanceListSupplier delegate, wrapping it with a CachingServiceInstanceListSupplier to leverage LoadBalancer caching mechanism, and then passing the resulting bean in the constructor of ZonePreferenceServiceInstanceListSupplier.spring-doc.cn

You can use this sample configuration to set it up:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                    .withDiscoveryClient()
                    .withCaching()
                    .withZonePreference()
                    .build(context);
    }
}

3.5. Instance Health-Check for LoadBalancer

It is possible to enable a scheduled HealthCheck for the LoadBalancer. The HealthCheckServiceInstanceListSupplier is provided for that. It regularly verifies if the instances provided by a delegate ServiceInstanceListSupplier are still alive and only returns the healthy instances, unless there are none - then it returns all the retrieved instances.spring-doc.cn

This mechanism is particularly helpful while using the SimpleDiscoveryClient. For the clients backed by an actual Service Registry, it’s not necessary to use, as we already get healthy instances after querying the external ServiceDiscovery.
This supplier is also recommended for setups with a small number of instances per service in order to avoid retrying calls on a failing instance.
If using any of the Service Discovery-backed suppliers, adding this health-check mechanism is usually not necessary, as we retrieve the health state of the instances directly from the Service Registry.
The HealthCheckServiceInstanceListSupplier relies on having updated instances provided by a delegate flux. In the rare cases when you want to use a delegate that does not refresh the instances, even though the list of instances may change (such as the DiscoveryClientServiceInstanceListSupplier provided by us), you can set spring.cloud.loadbalancer.health-check.refetch-instances to true to have the instance list refreshed by the HealthCheckServiceInstanceListSupplier. You can then also adjust the refretch intervals by modifying the value of spring.cloud.loadbalancer.health-check.refetch-instances-interval and opt to disable the additional healthcheck repetitions by setting spring.cloud.loadbalancer.health-check.repeat-health-check to false as every instances refetch will also trigger a healthcheck.

HealthCheckServiceInstanceListSupplier uses properties prefixed with spring.cloud.loadbalancer.health-check. You can set the initialDelay and interval for the scheduler. You can set the default path for the healthcheck URL by setting the value of the spring.cloud.loadbalancer.health-check.path.default property. You can also set a specific value for any given service by setting the value of the spring.cloud.loadbalancer.health-check.path.[SERVICE_ID] property, substituting [SERVICE_ID] with the correct ID of your service. If the [SERVICE_ID] is not specified, /actuator/health is used by default. If the [SERVICE_ID] is set to null or empty as a value, then the health check will not be executed. You can also set a custom port for health-check requests by setting the value of spring.cloud.loadbalancer.health-check.port. If none is set, the port under which the requested service is available at the service instance.spring-doc.cn

If you rely on the default path (/actuator/health), make sure you add spring-boot-starter-actuator to your collaborator’s dependencies, unless you are planning to add such an endpoint on your own.

In order to use the health-check scheduler approach, you will have to instantiate a HealthCheckServiceInstanceListSupplier bean in a custom configuration.spring-doc.cn

We use delegates to work with ServiceInstanceListSupplier beans. We suggest passing a DiscoveryClientServiceInstanceListSupplier delegate in the constructor of HealthCheckServiceInstanceListSupplier.spring-doc.cn

You can use this sample configuration to set it up:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                    .withDiscoveryClient()
                    .withHealthChecks()
                    .build(context);
        }
    }
For the non-reactive stack, create this supplier with the withBlockingHealthChecks(). You can also pass your own WebClient or RestTemplate instance to be used for the checks.
HealthCheckServiceInstanceListSupplier has its own caching mechanism based on Reactor Flux replay(). Therefore, if it’s being used, you may want to skip wrapping that supplier with CachingServiceInstanceListSupplier.
When you create your own configuration, HealthCheckServiceInstanceListSupplier, make sure to place it in the hierarchy directly after the supplier that retrieves the instances over the network, for example, DiscoveryClientServiceInstanceListSupplier, before any other filtering suppliers.

3.6. Same instance preference for LoadBalancer

You can set up the LoadBalancer in such a way that it prefers the instance that was previously selected, if that instance is available.spring-doc.cn

For that, you need to use SameInstancePreferenceServiceInstanceListSupplier. You can configure it either by setting the value of spring.cloud.loadbalancer.configurations to same-instance-preference or by providing your own ServiceInstanceListSupplier bean — for example:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                    .withDiscoveryClient()
                    .withSameInstancePreference()
                    .build(context);
        }
    }
This is also a replacement for Zookeeper StickyRule.

3.7. Request-based Sticky Session for LoadBalancer

You can set up the LoadBalancer in such a way that it prefers the instance with instanceId provided in a request cookie. We currently support this if the request is being passed to the LoadBalancer through either ClientRequestContext or ServerHttpRequestContext, which are used by the SC LoadBalancer exchange filter functions and filters.spring-doc.cn

For that, you need to use the RequestBasedStickySessionServiceInstanceListSupplier. You can configure it either by setting the value of spring.cloud.loadbalancer.configurations to request-based-sticky-session or by providing your own ServiceInstanceListSupplier bean — for example:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                    .withDiscoveryClient()
                    .withRequestBasedStickySession()
                    .build(context);
        }
    }

For that functionality, it is useful to have the selected service instance (which can be different from the one in the original request cookie if that one is not available) to be updated before sending the request forward. To do that, set the value of spring.cloud.loadbalancer.sticky-session.add-service-instance-cookie to true.spring-doc.cn

By default, the name of the cookie is sc-lb-instance-id. You can modify it by changing the value of the spring.cloud.loadbalancer.instance-id-cookie-name property.spring-doc.cn

This feature is currently supported for WebClient-backed load-balancing.

3.8. Spring Cloud LoadBalancer Hints

Spring Cloud LoadBalancer lets you set String hints that are passed to the LoadBalancer within the Request object and that can later be used in ReactiveLoadBalancer implementations that can handle them.spring-doc.cn

You can set a default hint for all services by setting the value of the spring.cloud.loadbalancer.hint.default property. You can also set a specific value for any given service by setting the value of the spring.cloud.loadbalancer.hint.[SERVICE_ID] property, substituting [SERVICE_ID] with the correct ID of your service. If the hint is not set by the user, default is used.spring-doc.cn

3.9. Hint-Based Load-Balancing

We also provide a HintBasedServiceInstanceListSupplier, which is a ServiceInstanceListSupplier implementation for hint-based instance selection.spring-doc.cn

HintBasedServiceInstanceListSupplier checks for a hint request header (the default header-name is X-SC-LB-Hint, but you can modify it by changing the value of the spring.cloud.loadbalancer.hint-header-name property) and, if it finds a hint request header, uses the hint value passed in the header to filter service instances.spring-doc.cn

If no hint header has been added, HintBasedServiceInstanceListSupplier uses hint values from properties to filter service instances.spring-doc.cn

If no hint is set, either by the header or by properties, all service instances provided by the delegate are returned.spring-doc.cn

While filtering, HintBasedServiceInstanceListSupplier looks for service instances that have a matching value set under the hint key in their metadataMap. If no matching instances are found, all instances provided by the delegate are returned.spring-doc.cn

You can use the following sample configuration to set it up:spring-doc.cn

public class CustomLoadBalancerConfiguration {

    @Bean
    public ServiceInstanceListSupplier discoveryClientServiceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                    .withDiscoveryClient()
                    .withCaching()
                    .withHints()
                    .build(context);
    }
}

3.10. Transform the load-balanced HTTP request

You can use the selected ServiceInstance to transform the load-balanced HTTP Request.spring-doc.cn

For RestTemplate, you need to implement and define LoadBalancerRequestTransformer as follows:spring-doc.cn

@Bean
public LoadBalancerRequestTransformer transformer() {
    return new LoadBalancerRequestTransformer() {
        @Override
        public HttpRequest transformRequest(HttpRequest request, ServiceInstance instance) {
            return new HttpRequestWrapper(request) {
                @Override
                public HttpHeaders getHeaders() {
                    HttpHeaders headers = new HttpHeaders();
                    headers.putAll(super.getHeaders());
                    headers.add("X-InstanceId", instance.getInstanceId());
                    return headers;
                }
            };
        }
    };
}

For WebClient, you need to implement and define LoadBalancerClientRequestTransformer as follows:spring-doc.cn

@Bean
public LoadBalancerClientRequestTransformer transformer() {
    return new LoadBalancerClientRequestTransformer() {
        @Override
        public ClientRequest transformRequest(ClientRequest request, ServiceInstance instance) {
            return ClientRequest.from(request)
                    .header("X-InstanceId", instance.getInstanceId())
                    .build();
        }
    };
}

If multiple transformers are defined, they are applied in the order in which Beans are defined. Alternatively, you can use LoadBalancerRequestTransformer.DEFAULT_ORDER or LoadBalancerClientRequestTransformer.DEFAULT_ORDER to specify the order.spring-doc.cn

3.11. Spring Cloud LoadBalancer Starter

We also provide a starter that allows you to easily add Spring Cloud LoadBalancer in a Spring Boot app. In order to use it, just add org.springframework.cloud:spring-cloud-starter-loadbalancer to your Spring Cloud dependencies in your build file.spring-doc.cn

Spring Cloud LoadBalancer starter includes Spring Boot Caching and Evictor.

3.12. Passing Your Own Spring Cloud LoadBalancer Configuration

You can also use the @LoadBalancerClient annotation to pass your own load-balancer client configuration, passing the name of the load-balancer client and the configuration class, as follows:spring-doc.cn

@Configuration
@LoadBalancerClient(value = "stores", configuration = CustomLoadBalancerConfiguration.class)
public class MyConfiguration {

    @Bean
    @LoadBalanced
    public WebClient.Builder loadBalancedWebClientBuilder() {
        return WebClient.builder();
    }
}
In order to make working on your own LoadBalancer configuration easier, we have added a builder() method to the ServiceInstanceListSupplier class.
You can also use our alternative predefined configurations in place of the default ones by setting the value of spring.cloud.loadbalancer.configurations property to zone-preference to use ZonePreferenceServiceInstanceListSupplier with caching or to health-check to use HealthCheckServiceInstanceListSupplier with caching.

You can use this feature to instantiate different implementations of ServiceInstanceListSupplier or ReactorLoadBalancer, either written by you, or provided by us as alternatives (for example ZonePreferenceServiceInstanceListSupplier) to override the default setup.spring-doc.cn

You can see an example of a custom configuration here.spring-doc.cn

The annotation value arguments (stores in the example above) specifies the service id of the service that we should send the requests to with the given custom configuration.

You can also pass multiple configurations (for more than one load-balancer client) through the @LoadBalancerClients annotation, as the following example shows:spring-doc.cn

@Configuration
@LoadBalancerClients({@LoadBalancerClient(value = "stores", configuration = StoresLoadBalancerClientConfiguration.class), @LoadBalancerClient(value = "customers", configuration = CustomersLoadBalancerClientConfiguration.class)})
public class MyConfiguration {

    @Bean
    @LoadBalanced
    public WebClient.Builder loadBalancedWebClientBuilder() {
        return WebClient.builder();
    }
}
The classes you pass as @LoadBalancerClient or @LoadBalancerClients configuration arguments should either not be annotated with @Configuration or be outside component scan scope.
When you create your own configuration, if you use CachingServiceInstanceListSupplier or HealthCheckServiceInstanceListSupplier, makes sure to use one of them, not both, and make sure to place it in the hierarchy directly after the supplier that retrieves the instances over the network, for example, DiscoveryClientServiceInstanceListSupplier, before any other filtering suppliers.

3.13. Spring Cloud LoadBalancer Lifecycle

One type of bean that it may be useful to register using Custom LoadBalancer configuration is LoadBalancerLifecycle.spring-doc.cn

The LoadBalancerLifecycle beans provide callback methods, named onStart(Request<RC> request), onStartRequest(Request<RC> request, Response<T> lbResponse) and onComplete(CompletionContext<RES, T, RC> completionContext), that you should implement to specify what actions should take place before and after load-balancing.spring-doc.cn

onStart(Request<RC> request) takes a Request object as a parameter. It contains data that is used to select an appropriate instance, including the downstream client request and hint. onStartRequest also takes the Request object and, additionally, the Response<T> object as parameters. On the other hand, a CompletionContext object is provided to the onComplete(CompletionContext<RES, T, RC> completionContext) method. It contains the LoadBalancer Response, including the selected service instance, the Status of the request executed against that service instance and (if available) the response returned to the downstream client, and (if an exception has occurred) the corresponding Throwable.spring-doc.cn

The supports(Class requestContextClass, Class responseClass, Class serverTypeClass) method can be used to determine whether the processor in question handles objects of provided types. If not overridden by the user, it returns true.spring-doc.cn

In the preceding method calls, RC means RequestContext type, RES means client response type, and T means returned server type.
If you are using custom HTTP status codes, you will be getting exceptions. In order to prevent this, you can set the value of spring.cloud.loadbalancer.use-raw-status-code-in-response-data. It will cause raw status codes to be used instead of HttpStatus enums. The httpStatus field in ResponseData will then be used, but you’ll be able to get the raw status code from the rawHttpStatus field.

3.14. Spring Cloud LoadBalancer Statistics

We provide a LoadBalancerLifecycle bean called MicrometerStatsLoadBalancerLifecycle, which uses Micrometer to provide statistics for load-balanced calls.spring-doc.cn

In order to get this bean added to your application context, set the value of the spring.cloud.loadbalancer.stats.micrometer.enabled to true and have a MeterRegistry available (for example, by adding Spring Boot Actuator to your project).spring-doc.cn

MicrometerStatsLoadBalancerLifecycle registers the following meters in MeterRegistry:spring-doc.cn

  • loadbalancer.requests.active: A gauge that allows you to monitor the number of currently active requests for any service instance (service instance data available via tags);spring-doc.cn

  • loadbalancer.requests.success: A timer that measures the time of execution of any load-balanced requests that have ended in passing a response on to the underlying client;spring-doc.cn

  • loadbalancer.requests.failed: A timer that measures the time of execution of any load-balanced requests that have ended with an exception;spring-doc.cn

  • loadbalancer.requests.discard: A counter that measures the number of discarded load-balanced requests, i.e. requests where a service instance to run the request on has not been retrieved by the LoadBalancer.spring-doc.cn

Additional information regarding the service instances, request data, and response data is added to metrics via tags whenever available.spring-doc.cn

For some implementations, such as BlockingLoadBalancerClient, request and response data might not be available, as we establish generic types from arguments and might not be able to determine the types and read the data.
The meters are registered in the registry when at least one record is added for a given meter.
You can further configure the behavior of those metrics (for example, add publishing percentiles and histograms) by adding MeterFilters.

3.15. Configuring Individual LoadBalancerClients

Individual Loadbalancer clients may be configured individually with a different prefix spring.cloud.loadbalancer.clients.<clientId>. where clientId is the name of the loadbalancer. Default configuration values may be set in the spring.cloud.loadbalancer. namespace and will be merged with the client specific values taking precedencespring-doc.cn

Example 5. application.yml
spring:
  cloud:
    loadbalancer:
      health-check:
        initial-delay: 1s
      clients:
        myclient:
          health-check:
            interval: 30s

The above example will result in a merged health-check @ConfigurationProperties object with initial-delay=1s and interval=30s.spring-doc.cn

The per-client configuration properties work for most of the properties, apart from the following global ones:spring-doc.cn

  • spring.cloud.loadbalancer.enabled - globally enables or disables load-balancingspring-doc.cn

  • spring.cloud.loadbalancer.retry.enabled - globally enables or disables load-balanced retries. If you enable it globally, you can still disable retries for specific clients using the client-prefixed properties, but not the other way roundspring-doc.cn

  • spring.cloud.loadbalancer.cache.enabled - globally enables or disables LoadBalancer caching. If you enable it globally, you can still disable caching for specific clients by creating a custom configuration that does not include the CachingServiceInstanceListSupplier in the ServiceInstanceListSupplier delegates hierarchy, but not the other way round.spring-doc.cn

  • spring.cloud.loadbalancer.stats.micrometer.enabled - globally enables or disables LoadBalancer Micrometer metricsspring-doc.cn

For the properties where maps where already used, where you can specify a different value per-client without using the clients keyword (for example, hints, health-check.path), we have kept that behaviour in order to keep the library backwards compatible. It will be modified in the next major release.
Starting with 3.1.7, we have introduced the callGetWithRequestOnDelegates flag in LoadBalancerProperties. If this flag is set to true, ServiceInstanceListSupplier#get(Request request) method will be implemented to call delegate.get(request) in classes assignable from DelegatingServiceInstanceListSupplier that don’t already implement that method, with the exclusion of CachingServiceInstanceListSupplier and HealthCheckServiceInstanceListSupplier, which should be placed in the instance supplier hierarchy directly after the supplier performing instance retrieval over the network, before any request-based filtering is done. For 3.1.x the flag is set to false by default, however, since 4.1.0 it’s going to be set to true by default.