Spring Cloud Sleuth Integrations
1. Asynchronous Communication
In this section, we describe how to customize asynchronous communication with Spring Cloud Sleuth.
1.1. @Async
Annotated methods
This feature is available for all tracer implementations.
In Spring Cloud Sleuth, we instrument async-related components so that the tracing information is passed between threads.
You can disable this behavior by setting the value of spring.sleuth.async.enabled
to false
.
If you annotate your method with @Async
, we automatically modify the existing Span as follows:
-
If the method is annotated with
@SpanName
, the value of the annotation is the Span’s name. -
If the method is not annotated with
@SpanName
, the Span name is the annotated method name. -
The span is tagged with the method’s class name and method name.
Since we’re modifying the existing span, if you want to maintain its original name (e.g. a span created by receiving an HTTP request)
you should wrap your @Async
annotated method with a @NewSpan
annotation or create a new span manually.
1.2. @Scheduled
Annotated Methods
This feature is available for all tracer implementations.
In Spring Cloud Sleuth, we instrument scheduled method execution so that the tracing information is passed between threads.
You can disable this behavior by setting the value of spring.sleuth.scheduled.enabled
to false
.
If you annotate your method with @Scheduled
, we automatically create a new span with the following characteristics:
-
The span name is the annotated method name.
-
The span is tagged with the method’s class name and method name.
If you want to skip span creation for some @Scheduled
annotated classes, you can set the spring.sleuth.scheduled.skipPattern
with a regular expression that matches the fully qualified name of the @Scheduled
annotated class.
1.3. Executor, ExecutorService, and ScheduledExecutorService
This feature is available for all tracer implementations.
We provide LazyTraceExecutor
, TraceableExecutorService
, and TraceableScheduledExecutorService
.
Those implementations create spans each time a new task is submitted, invoked, or scheduled.
The following example shows how to pass tracing information with TraceableExecutorService
when working with CompletableFuture
:
CompletableFuture<Long> completableFuture = CompletableFuture.supplyAsync(() -> {
// perform some logic
return 1_000_000L;
}, new TraceableExecutorService(beanFactory, executorService,
// 'calculateTax' explicitly names the span - this param is optional
"calculateTax"));
Sleuth does not work with parallelStream() out of the box.
If you want to have the tracing information propagated through the stream, you have to use the approach with supplyAsync(…) , as shown earlier.
|
If there are beans that implement the Executor
interface that you would like to exclude from span creation, you can use the spring.sleuth.async.ignored-beans
property where you can provide a list of bean names.
You can disable this behavior by setting the value of spring.sleuth.async.enabled
to false
.
1.3.1. Customization of Executors
Sometimes, you need to set up a custom instance of the AsyncExecutor
.
The following example shows how to set up such a custom Executor
:
@Configuration(proxyBeanMethods = false)
@EnableAutoConfiguration
@EnableAsync
// add the infrastructure role to ensure that the bean gets auto-proxied
@Role(BeanDefinition.ROLE_INFRASTRUCTURE)
public static class CustomExecutorConfig extends AsyncConfigurerSupport {
@Autowired
BeanFactory beanFactory;
@Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
// CUSTOMIZE HERE
executor.setCorePoolSize(7);
executor.setMaxPoolSize(42);
executor.setQueueCapacity(11);
executor.setThreadNamePrefix("MyExecutor-");
// DON'T FORGET TO INITIALIZE
executor.initialize();
return new LazyTraceExecutor(this.beanFactory, executor);
}
}
To ensure that your configuration gets post processed, remember to add the @Role(BeanDefinition.ROLE_INFRASTRUCTURE) on your
@Configuration class
|
2. HTTP Client Integration
Features from this section can be disabled by setting the spring.sleuth.web.client.enabled
property with value equal to false
.
2.1. Synchronous Rest Template
This feature is available for all tracer implementations.
We inject a RestTemplate
interceptor to ensure that all the tracing information is passed to the requests.
Each time a call is made, a new Span is created.
It gets closed upon receiving the response.
To block the synchronous RestTemplate
features, set spring.sleuth.web.client.enabled
to false
.
You have to register RestTemplate as a bean so that the interceptors get injected.
If you create a RestTemplate instance with a new keyword, the instrumentation does NOT work.
|
2.2. Asynchronous Rest Template
This feature is available for all tracer implementations.
Starting with Sleuth 2.0.0 , we no longer register a bean of AsyncRestTemplate type.
It is up to you to create such a bean.
Then we instrument it.
|
To block the AsyncRestTemplate
features, set spring.sleuth.web.async.client.enabled
to false
.
To disable creation of the default TraceAsyncClientHttpRequestFactoryWrapper
, set spring.sleuth.web.async.client.factory.enabled
to false
.
If you do not want to create AsyncRestClient
at all, set spring.sleuth.web.async.client.template.enabled
to false
.
2.2.1. Multiple Asynchronous Rest Templates
Sometimes you need to use multiple implementations of the Asynchronous Rest Template.
In the following snippet, you can see an example of how to set up such a custom AsyncRestTemplate
:
@Configuration(proxyBeanMethods = false)
public static class TestConfig {
@Bean(name = "customAsyncRestTemplate")
public AsyncRestTemplate traceAsyncRestTemplate() {
return new AsyncRestTemplate(asyncClientFactory(), clientHttpRequestFactory());
}
private ClientHttpRequestFactory clientHttpRequestFactory() {
ClientHttpRequestFactory clientHttpRequestFactory = new CustomClientHttpRequestFactory();
// CUSTOMIZE HERE
return clientHttpRequestFactory;
}
private AsyncClientHttpRequestFactory asyncClientFactory() {
AsyncClientHttpRequestFactory factory = new CustomAsyncClientHttpRequestFactory();
// CUSTOMIZE HERE
return factory;
}
}
2.2.2. WebClient
This feature is available for all tracer implementations.
We inject a ExchangeFilterFunction
implementation that creates a span and, through on-success and on-error callbacks, takes care of closing client-side spans.
To block this feature, set spring.sleuth.web.client.enabled
to false
.
You have to register WebClient as a bean so that the tracing instrumentation gets applied.
If you create a WebClient instance with a new keyword, the instrumentation does NOT work.
|
2.2.3. Traverson
This feature is available for all tracer implementations.
If you use the Traverson library, you can inject a RestTemplate
as a bean into your Traverson object.
Since RestTemplate
is already intercepted, you get full support for tracing in your client.
The following pseudo code shows how to do that:
@Autowired RestTemplate restTemplate;
Traverson traverson = new Traverson(URI.create("https://some/address"),
MediaType.APPLICATION_JSON, MediaType.APPLICATION_JSON_UTF8).setRestOperations(restTemplate);
// use Traverson
2.2.4. Apache HttpClientBuilder
and HttpAsyncClientBuilder
This feature is available for Brave tracer implementation.
We instrument the HttpClientBuilder
and HttpAsyncClientBuilder
so that tracing context gets injected to the sent requests.
To block these features, set spring.sleuth.web.client.enabled
to false
.
2.2.5. Netty HttpClient
This feature is available for all tracer implementations.
We instrument the Netty’s HttpClient
.
To block this feature, set spring.sleuth.web.client.enabled
to false
.
You have to register HttpClient as a bean so that the instrumentation happens.
If you create a HttpClient instance with a new keyword, the instrumentation does NOT work.
|
2.2.6. UserInfoRestTemplateCustomizer
This feature is available for all tracer implementations.
We instrument the Spring Security’s UserInfoRestTemplateCustomizer
.
To block this feature, set spring.sleuth.web.client.enabled
to false
.
3. HTTP Server Integration
Features from this section can be disabled by setting the spring.sleuth.web.enabled
property with value equal to false
.
3.1. HTTP Filter
This feature is available for all tracer implementations.
Through the TracingFilter
, all sampled incoming requests result in creation of a Span.
You can configure which URIs you would like to skip by setting the spring.sleuth.web.skipPattern
property.
If you have ManagementServerProperties
on classpath, its value of contextPath
gets appended to the provided skip pattern.
If you want to reuse the Sleuth’s default skip patterns and just append your own, pass those patterns by using the spring.sleuth.web.additionalSkipPattern
.
By default, all the spring boot actuator endpoints are automatically added to the skip pattern.
If you want to disable this behaviour set spring.sleuth.web.ignore-auto-configured-skip-patterns
to true
.
To change the order of tracing filter registration, please set the
spring.sleuth.web.filter-order
property.
To disable the filter that logs uncaught exceptions you can disable the
spring.sleuth.web.exception-throwing-filter-enabled
property.
3.2. HandlerInterceptor
This feature is available for all tracer implementations.
Since we want the span names to be precise, we use a TraceHandlerInterceptor
that either wraps an existing HandlerInterceptor
or is added directly to the list of existing HandlerInterceptors
.
The TraceHandlerInterceptor
adds a special request attribute to the given HttpServletRequest
.
If the the TracingFilter
does not see this attribute, it creates a “fallback” span, which is an additional span created on the server side so that the trace is presented properly in the UI.
If that happens, there is probably missing instrumentation.
In that case, please file an issue in Spring Cloud Sleuth.
3.3. Async Servlet support
This feature is available for all tracer implementations.
If your controller returns a Callable
or a WebAsyncTask
, Spring Cloud Sleuth continues the existing span instead of creating a new one.
3.4. WebFlux support
This feature is available for all tracer implementations.
Through TraceWebFilter
, all sampled incoming requests result in creation of a Span.
That Span’s name is http:
+ the path to which the request was sent.
For example, if the request was sent to /this/that
, the name is http:/this/that
.
You can configure which URIs you would like to skip by using the spring.sleuth.web.skipPattern
property.
If you have ManagementServerProperties
on the classpath, its value of contextPath
gets appended to the provided skip pattern.
If you want to reuse Sleuth’s default skip patterns and append your own, pass those patterns by using the spring.sleuth.web.additionalSkipPattern
.
In order to achieve best results in terms of performance and context propagation we suggest that you switch the spring.sleuth.reactor.instrumentation-type
to MANUAL
.
In order to execute code with the span in scope you can call WebFluxSleuthOperators.withSpanInScope
.
Example:
@GetMapping("/simpleManual")
public Mono<String> simpleManual() {
return Mono.just("hello").map(String::toUpperCase).doOnEach(WebFluxSleuthOperators
.withSpanInScope(SignalType.ON_NEXT, signal -> log.info("Hello from simple [{}]", signal.get())));
}
To change the order of tracing filter registration, please set the
spring.sleuth.web.filter-order
property.
4. Messaging
Features from this section can be disabled by setting the spring.sleuth.messaging.enabled
property with value equal to false
.
4.1. Spring Integration
This feature is available for all tracer implementations.
Spring Cloud Sleuth integrates with Spring Integration.
It creates spans for publish and subscribe events.
To disable Spring Integration instrumentation, set spring.sleuth.integration.enabled
to false
.
You can provide the spring.sleuth.integration.patterns
pattern to explicitly provide the names of channels that you want to include for tracing.
By default, all channels but hystrixStreamOutput
channel are included.
When using the Executor to build a Spring Integration IntegrationFlow , you must use the untraced version of the Executor .
Decorating the Spring Integration Executor Channel with TraceableExecutorService causes the spans to be improperly closed.
|
If you want to customize the way tracing context is read from and written to message headers, it’s enough for you to register beans of types:
-
Propagator.Setter<MessageHeaderAccessor>
- for writing headers to the message -
Propagator.Getter<MessageHeaderAccessor>
- for reading headers from the message
4.1.2. Customizing messaging spans
In order to change the default span names and tags, just register a bean of type MessageSpanCustomizer
. You can also
override the existing DefaultMessageSpanCustomizer
to extend the existing behaviour.
@Component
class MyMessageSpanCustomizer extends DefaultMessageSpanCustomizer {
@Override
public Span customizeHandle(Span spanCustomizer,
Message<?> message, MessageChannel messageChannel) {
return super.customizeHandle(spanCustomizer, message, messageChannel)
.name("changedHandle")
.tag("handleKey", "handleValue")
.tag("channelName", channelName(messageChannel));
}
@Override
public Span.Builder customizeSend(Span.Builder builder,
Message<?> message, MessageChannel messageChannel) {
return super.customizeSend(builder, message, messageChannel)
.name("changedSend")
.tag("sendKey", "sendValue")
.tag("channelName", channelName(messageChannel));
}
}
4.2. Spring Cloud Function and Spring Cloud Stream
This feature is available for all tracer implementations.
Spring Cloud Sleuth can instrument Spring Cloud Function.
The way to achieve it is to provide a Function
or Consumer
or Supplier
that takes in a Message
as a parameter e.g. Function<Message<String>, Message<Integer>>
.
If the type is not Message
then instrumentation will not take place.
Out of the box instrumentation will not take place when dealing with Reactor based streams - e.g. Function<Flux<Message<String>>, Flux<Message<Integer>>>
.
Since Spring Cloud Stream reuses Spring Cloud Function, you’ll get the instrumentation out of the box.
You can disable this behavior by setting the value of spring.sleuth.function.enabled
to false
.
In order to work with reactive Stream functions you can leverage the MessagingSleuthOperators
utility class that allows you to manipulate the input and output messages in order to continue the tracing context and to execute custom code within the tracing context.
class SimpleReactiveManualFunction implements Function<Flux<Message<String>>, Flux<Message<String>>> {
private static final Logger log = LoggerFactory.getLogger(SimpleReactiveFunction.class);
private final BeanFactory beanFactory;
SimpleReactiveManualFunction(BeanFactory beanFactory) {
this.beanFactory = beanFactory;
}
@Override
public Flux<Message<String>> apply(Flux<Message<String>> input) {
return input.map(message -> (MessagingSleuthOperators.asFunction(this.beanFactory, message))
.andThen(msg -> MessagingSleuthOperators.withSpanInScope(this.beanFactory, msg, stringMessage -> {
log.info("Hello from simple manual [{}]", stringMessage.getPayload());
return stringMessage;
})).andThen(msg -> MessagingSleuthOperators.afterMessageHandled(this.beanFactory, msg, null))
.andThen(msg -> MessageBuilder.createMessage(msg.getPayload().toUpperCase(), msg.getHeaders()))
.andThen(msg -> MessagingSleuthOperators.handleOutputMessage(this.beanFactory, msg)).apply(message));
}
}
4.3. Spring RabbitMq
This feature is available for Brave tracer implementation.
We instrument the RabbitTemplate
so that tracing headers get injected into the message.
To block this feature, set spring.sleuth.messaging.rabbit.enabled
to false
.
4.4. Spring Kafka
This feature is available for Brave tracer implementation.
We instrument the Spring Kafka’s ProducerFactory
and ConsumerFactory
so that tracing headers get injected into the created Spring Kafka’s
Producer
and Consumer
.
To block this feature, set spring.sleuth.messaging.kafka.enabled
to false
.
4.5. Spring Kafka Streams
This feature is available for Brave tracer implementation.
We instrument the KafkaStreams
KafkaClientSupplier
so that tracing headers get injected into the Producer
and Consumer`s. A `KafkaStreamsTracing
bean allows for further instrumentation through additional TransformerSupplier
and
ProcessorSupplier
methods.
To block this feature, set spring.sleuth.messaging.kafka.streams.enabled
to false
.
4.6. Spring JMS
This feature is available for Brave tracer implementation.
We instrument the JmsTemplate
so that tracing headers get injected into the message.
We also support @JmsListener
annotated methods on the consumer side.
To block this feature, set spring.sleuth.messaging.jms.enabled
to false
.
We don’t support baggage propagation for JMS |
5. OpenFeign
This feature is available for all tracer implementations.
By default, Spring Cloud Sleuth provides integration with Feign through TraceFeignClientAutoConfiguration
.
You can disable it entirely by setting spring.sleuth.feign.enabled
to false
.
If you do so, no Feign-related instrumentation take place.
Part of Feign instrumentation is done through a FeignBeanPostProcessor
.
You can disable it by setting spring.sleuth.feign.processor.enabled
to false
.
If you set it to false
, Spring Cloud Sleuth does not instrument any of your custom Feign components.
However, all the default instrumentation is still there.
6. OpenTracing
This feature is available for all tracer implementations.
Spring Cloud Sleuth is compatible with OpenTracing.
If you have OpenTracing on the classpath, we automatically register the OpenTracing Tracer
bean.
If you wish to disable this, set spring.sleuth.opentracing.enabled
to false
7. Quartz
This feature is available for all tracer implementations.
We instrument quartz jobs by adding Job/Trigger listeners to the Quartz Scheduler.
To turn off this feature, set the spring.sleuth.quartz.enabled
property to false
.
8. Reactor
This feature is available for all tracer implementations.
We have the following modes of instrumenting reactor based applications that can be set via spring.sleuth.reactor.instrumentation-type
property:
-
DECORATE_QUEUES
- With the new Reactor queue wrapping mechanism (Reactor 3.4.3) we’re instrumenting the way threads are switched by Reactor. This should lead to feature parity withON_EACH
with low performance impact. -
DECORATE_ON_EACH
- wraps every Reactor operator in a trace representation. Passes the tracing context in most cases. This mode might lead to drastic performance degradation. -
DECORATE_ON_LAST
- wraps last Reactor operator in a trace representation. Passes the tracing context in some cases thus accessing MDC context might not work. This mode might lead to medium performance degradation. -
MANUAL
- wraps every Reactor in the least invasive way without passing of tracing context. It’s up to the user to do it.
Current default is ON_EACH
for backward compatibility reasons, however we encourage the users to migrate to the MANUAL
instrumentation and profit from WebFluxSleuthOperators
and MessagingSleuthOperators
.
The performance improvement can be substantial.
Example:
@GetMapping("/simpleManual")
public Mono<String> simpleManual() {
return Mono.just("hello").map(String::toUpperCase).doOnEach(WebFluxSleuthOperators
.withSpanInScope(SignalType.ON_NEXT, signal -> log.info("Hello from simple [{}]", signal.get())));
}
9. Redis
This feature is available for Brave tracer implementation.
We set tracing
property to Lettuce ClientResources
instance to enable Brave tracing built in Lettuce.
Spring Cloud Sleuth will provide a traced version of the ClientResources
bean. If you have your own implementation of that bean, remember to customize the ClientResources.Builder
with a stream of `ClientResourcesBuilderCustomizer`s like presented below:
@Bean(destroyMethod = "shutdown")
DefaultClientResources myLettuceClientResources(ObjectProvider<ClientResourcesBuilderCustomizer> customizer) {
DefaultClientResources.Builder builder = DefaultClientResources.builder();
// setting up the builder manually
customizer.stream().forEach(c -> c.customize(builder));
return builder.build();
}
To disable Redis support, set the spring.sleuth.redis.enabled
property to false
.
10. Runnable and Callable
This feature is available for all tracer implementations.
If you wrap your logic in Runnable
or Callable
, you can wrap those classes in their Sleuth representative, as shown in the following example for Runnable
:
Runnable runnable = new Runnable() {
@Override
public void run() {
// do some work
}
@Override
public String toString() {
return "spanNameFromToStringMethod";
}
};
// Manual `TraceRunnable` creation with explicit "calculateTax" Span name
Runnable traceRunnable = new TraceRunnable(this.tracer, spanNamer, runnable, "calculateTax");
The following example shows how to do so for Callable
:
Callable<String> callable = new Callable<String>() {
@Override
public String call() throws Exception {
return someLogic();
}
@Override
public String toString() {
return "spanNameFromToStringMethod";
}
};
// Manual `TraceCallable` creation with explicit "calculateTax" Span name
Callable<String> traceCallable = new TraceCallable<>(tracer, spanNamer, callable, "calculateTax");
That way, you ensure that a new span is created and closed for each execution.
11. RPC
This feature is available for Brave tracer implementation.
Sleuth automatically configures the RpcTracing
bean which serves as a foundation for RPC instrumentation such as gRPC or Dubbo.
If a customization of client / server sampling of the RPC traces is required, just register a bean of type brave.sampler.SamplerFunction<RpcRequest>
and name the bean sleuthRpcClientSampler
for client sampler and
sleuthRpcServerSampler
for server sampler.
For your convenience the @RpcClientSampler
and @RpcServerSampler
annotations can be used to inject the proper beans or to reference the bean names via their static String NAME
fields.
Ex. Here’s a sampler that traces 100 "GetUserToken" server requests per second. This doesn’t start new traces for requests to the health check service. Other requests will use the global sampling configuration.
@Configuration(proxyBeanMethods = false)
class Config {
@Bean(name = RpcServerSampler.NAME)
SamplerFunction<RpcRequest> myRpcSampler() {
Matcher<RpcRequest> userAuth = and(serviceEquals("users.UserService"), methodEquals("GetUserToken"));
return RpcRuleSampler.newBuilder().putRule(serviceEquals("grpc.health.v1.Health"), Sampler.NEVER_SAMPLE)
.putRule(userAuth, RateLimitingSampler.create(100)).build();
}
}
11.1. Dubbo RPC support
Via the integration with Brave, Spring Cloud Sleuth supports Dubbo.
It’s enough to add the brave-instrumentation-dubbo
dependency:
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-instrumentation-dubbo</artifactId>
</dependency>
You need to also set a dubbo.properties
file with the following contents:
dubbo.provider.filter=tracing
dubbo.consumer.filter=tracing
11.2. gRPC
Spring Cloud Sleuth provides instrumentation for gRPC via the Brave tracer.
You can disable it entirely by setting spring.sleuth.grpc.enabled
to false
.
11.2.1. Variant 1
Dependencies
The gRPC integration relies on two external libraries to instrument clients and servers and both of those libraries must be on the class path to enable the instrumentation. |
Maven:
<dependency>
<groupId>io.github.lognet</groupId>
<artifactId>grpc-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-instrumentation-grpc</artifactId>
</dependency>
Gradle:
compile("io.github.lognet:grpc-spring-boot-starter")
compile("io.zipkin.brave:brave-instrumentation-grpc")
Server Instrumentation
Spring Cloud Sleuth leverages grpc-spring-boot-starter to register Brave’s gRPC server interceptor with all services annotated with @GRpcService
.
Client Instrumentation
gRPC clients leverage a ManagedChannelBuilder
to construct a ManagedChannel
used to communicate to the gRPC server.
The native ManagedChannelBuilder
provides static methods as entry points for construction of ManagedChannel
instances, however, this mechanism is outside the influence of the Spring application context.
Spring Cloud Sleuth provides a SpringAwareManagedChannelBuilder that can be customized through the Spring application context and injected by gRPC clients.
This builder must be used when creating ManagedChannel instances.
|
Sleuth creates a TracingManagedChannelBuilderCustomizer
which inject Brave’s client interceptor into the SpringAwareManagedChannelBuilder
.
11.2.2. Variant 2
Grpc Spring Boot Starter automatically detects the presence of Spring Cloud Sleuth and Brave’s instrumentation for gRPC and registers the necessary client and/or server tooling.
12. RxJava
This feature is available for all tracer implementations.
We registering a custom RxJavaSchedulersHook
that wraps all Action0
instances in their Sleuth representative, which is called TraceAction
.
The hook either starts or continues a span, depending on whether tracing was already going on before the Action was scheduled.
To disable the custom RxJavaSchedulersHook
, set the spring.sleuth.rxjava.schedulers.hook.enabled
to false
.
You can define a list of regular expressions for thread names for which you do not want spans to be created.
To do so, provide a comma-separated list of regular expressions in the spring.sleuth.rxjava.schedulers.ignoredthreads
property.
The suggested approach to reactive programming and Sleuth is to use the Reactor support. |
13. Spring Cloud CircuitBreaker
This feature is available for all tracer implementations.
If you have Spring Cloud CircuitBreaker on the classpath, we will wrap the passed command Supplier
and the fallback Function
in its trace representations.
In order to disable this instrumentation set spring.sleuth.circuitbreaker.enabled
to false
.