Cloud Trace
Google Cloud Platform provides a managed distributed tracing service called Cloud Trace, and Spring Cloud Sleuth can be used with it to easily instrument Spring Boot applications for observability.
Typically, Spring Cloud Sleuth captures trace information and forwards traces to services like Zipkin for storage and analysis. However, on GCP, instead of running and maintaining your own Zipkin instance and storage, you can use Cloud Trace to store traces, view trace details, generate latency distributions graphs, and generate performance regression reports.
This Spring Cloud GCP starter can forward Spring Cloud Sleuth traces to Cloud Trace without an intermediary Zipkin server.
Maven coordinates, using Spring Cloud GCP BOM:
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>spring-cloud-gcp-starter-trace</artifactId>
</dependency>Gradle coordinates:
dependencies {
implementation("com.google.cloud:spring-cloud-gcp-starter-trace")
}You must enable Cloud Trace API from the Google Cloud Console in order to capture traces. Navigate to the Cloud Trace API for your project and make sure it’s enabled.
|
If you are already using a Zipkin server capturing trace information from multiple platform/frameworks, you can also use a Stackdriver Zipkin proxy to forward those traces to Cloud Trace without modifying existing applications. |
Tracing
Spring Cloud Sleuth uses the Brave tracer to generate traces.
This integration enables Brave to use the StackdriverTracePropagation propagation.
A propagation is responsible for extracting trace context from an entity (e.g., an HTTP servlet request) and injecting trace context into an entity.
A canonical example of the propagation usage is a web server that receives an HTTP request, which triggers other HTTP requests from the server before returning an HTTP response to the original caller.
In the case of StackdriverTracePropagation, first it looks for trace context in the x-cloud-trace-context key (e.g., an HTTP request header).
The value of the x-cloud-trace-context key can be formatted in three different ways:
-
x-cloud-trace-context: TRACE_ID -
x-cloud-trace-context: TRACE_ID/SPAN_ID -
x-cloud-trace-context: TRACE_ID/SPAN_ID;o=TRACE_TRUE
TRACE_ID is a 32-character hexadecimal value that encodes a 128-bit number.
SPAN_ID is an unsigned long.
Since Cloud Trace doesn’t support span joins, a new span ID is always generated, regardless of the one specified in x-cloud-trace-context.
TRACE_TRUE can either be 0 if the entity should be untraced, or 1 if it should be traced.
This field forces the decision of whether or not to trace the request; if omitted then the decision is deferred to the sampler.
If a x-cloud-trace-context key isn’t found, StackdriverTracePropagation falls back to tracing with the X-B3 headers.
Spring Boot Starter for Cloud Trace
Spring Boot Starter for Cloud Trace uses Spring Cloud Sleuth and auto-configures a StackdriverSender that sends the Sleuth’s trace information to Cloud Trace.
All configurations are optional:
Name |
Description |
Required |
Default value |
|
Auto-configure Spring Cloud Sleuth to send traces to Cloud Trace. |
No |
|
|
Overrides the project ID from the Spring Cloud GCP Module |
No |
|
|
Overrides the credentials location from the Spring Cloud GCP Module |
No |
|
|
Overrides the credentials encoded key from the Spring Cloud GCP Module |
No |
|
|
Overrides the credentials scopes from the Spring Cloud GCP Module |
No |
|
|
Number of threads used by the Trace executor |
No |
4 |
|
HTTP/2 authority the channel claims to be connecting to. |
No |
|
|
Name of the compression to use in Trace calls |
No |
|
|
Call deadline in milliseconds |
No |
|
|
Maximum size for inbound messages |
No |
|
|
Maximum size for outbound messages |
No |
|
|
Waits for the channel to be ready in case of a transient failure |
No |
|
|
Timeout in seconds before pending spans will be sent in batches to GCP Cloud Trace. (previously |
No |
1 |
|
Server response timeout in millis. |
No |
|
|
(Experimental) Auto-configure Pub/Sub instrumentation for Trace. |
No |
|
You can use core Spring Cloud Sleuth properties to control Sleuth’s sampling rate, etc. Read Sleuth documentation for more information on Sleuth configurations.
For example, when you are testing to see the traces are going through, you can set the sampling rate to 100%.
spring.sleuth.sampler.probability=1 # Send 100% of the request traces to Cloud Trace.
spring.sleuth.web.skipPattern=(^cleanup.*|.+favicon.*) # Ignore some URL paths.
spring.sleuth.scheduled.enabled=false # disable executor 'async' traces
By default, Spring Cloud Sleuth auto-configuration instruments executor beans, which may cause recurring traces with the name async to appear in Cloud Trace if your application or one of its dependencies introduces scheduler beans into Spring application context. To avoid this noise, please disable automatic instrumentation of executors via spring.sleuth.scheduled.enabled=false in your application configuration.
|
Spring Cloud GCP Trace does override some Sleuth configurations:
-
Always uses 128-bit Trace IDs. This is required by Cloud Trace.
-
Does not use Span joins. Span joins will share the span ID between the client and server Spans. Cloud Trace requires that every Span ID within a Trace to be unique, so Span joins are not supported.
-
Uses
StackdriverHttpRequestParserby default to populate Stackdriver related fields.
Overriding the auto-configuration
Spring Cloud Sleuth supports sending traces to multiple tracing systems as of version 2.1.0.
In order to get this to work, every tracing system needs to have a Reporter<Span> and Sender.
If you want to override the provided beans you need to give them a specific name.
To do this you can use respectively StackdriverTraceAutoConfiguration.REPORTER_BEAN_NAME and StackdriverTraceAutoConfiguration.SENDER_BEAN_NAME.
Customizing spans
You can add additional tags and annotations to spans by using the brave.SpanCustomizer, which is available in the application context.
Here’s an example that uses WebMvcConfigurer to configure an MVC interceptor that adds two extra tags to all web controller spans.
@SpringBootApplication
public class Application implements WebMvcConfigurer {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
@Autowired
private SpanCustomizer spanCustomizer;
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(new HandlerInterceptor() {
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
spanCustomizer.tag("session-id", request.getSession().getId());
spanCustomizer.tag("environment", "QA");
return true;
}
});
}
}You can then search and filter traces based on these additional tags in the Cloud Trace service.
Integration with Logging
Integration with Cloud Logging is available through the Cloud Logging Support.
If the Trace integration is used together with the Logging one, the request logs will be associated to the corresponding traces.
The trace logs can be viewed by going to the Google Cloud Console Trace List, selecting a trace and pressing the Logs → View link in the Details section.
Pub/Sub Trace Instrumentation (Experimental)
You can enable trace instrumentation and propagation for Pub/Sub messages by using the spring.cloud.gcp.trace.pubsub.enabled=true property.
It’s set to false by default, but when set to true, trace spans will be created and propagated to Cloud Trace whenever the application sends or receives messages through PubSubTemplate or any other integration that builds on top of PubSubTemplate, such as the Spring Integration channel adapters, and the Spring Cloud Stream Binder.
# Enable Pub/Sub tracing using this property
spring.cloud.gcp.trace.pubsub.enabled=true
# You should disable Spring Integration instrumentation by Sleuth as it's unnecessary when Pub/Sub tracing is enabled
spring.sleuth.integration.enabled=falseSample
A sample application and a codelab are available.