Cloud Logging

Maven coordinates, using Spring Framework on Google Cloud BOM:

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>spring-cloud-gcp-starter-logging</artifactId>
</dependency>

Gradle coordinates:

dependencies {
    implementation("com.google.cloud:spring-cloud-gcp-starter-logging")
}

Cloud Logging is the managed logging service provided by Google Cloud.

This module provides support for associating a web request trace ID with the corresponding log entries. It does so by retrieving the X-B3-TraceId value from the Mapped Diagnostic Context (MDC), which is set by Micrometer. If Micrometer isn’t used, the configured TraceIdExtractor extracts the desired header value and sets it as the log entry’s trace ID. This allows grouping of log messages by request, for example, in the Google Cloud Console Logs viewer.

Due to the way logging is set up, the Google Cloud project ID and credentials defined in application.properties are ignored. Instead, you should set the GOOGLE_CLOUD_PROJECT and GOOGLE_APPLICATION_CREDENTIALS environment variables to the project ID and credentials private key location, respectively. You can do this easily if you’re using the Google Cloud SDK, using the gcloud config set project [YOUR_PROJECT_ID] and gcloud auth application-default login commands, respectively.

Web MVC Interceptor

For use in Web MVC-based applications, TraceIdLoggingWebMvcInterceptor is provided that extracts the request trace ID from an HTTP request using a TraceIdExtractor and stores it in a thread-local, which can then be used in a logging appender to add the trace ID metadata to log messages.

If Spring Framework on Google Cloud Trace is enabled, the logging module disables itself and delegates log correlation to Micrometer.

LoggingWebMvcConfigurer configuration class is also provided to help register the TraceIdLoggingWebMvcInterceptor in Spring MVC applications.

Applications hosted on the Google Cloud include trace IDs under the x-cloud-trace-context header, which will be included in log entries. However, if Micrometer is used the trace ID will be picked up from the MDC.

Logback Support

Currently, only Logback is supported and there are 2 possibilities to log to Cloud Logging via this library with Logback: via direct API calls and through JSON-formatted console logs.

Log via API

A Cloud Logging appender is available using com/google/cloud/spring/logging/logback-appender.xml. This appender builds a Cloud Logging log entry from a JUL or Logback log entry, adds a trace ID to it and sends it to Cloud Logging.

STACKDRIVER_LOG_NAME and STACKDRIVER_LOG_FLUSH_LEVEL environment variables can be used to customize the STACKDRIVER appender.

Your configuration may then look like this:

<configuration>
  <include resource="com/google/cloud/spring/logging/logback-appender.xml" />

  <root level="INFO">
    <appender-ref ref="STACKDRIVER" />
  </root>
</configuration>

If you want to have more control over the log output, you can further configure the appender. The following properties are available (see java-logging-logback project for the full list):

Property Default Value Description

Property	Default Value	Description
`log`	`spring.log`	The Cloud Logging Log name. This can also be set via the `STACKDRIVER_LOG_NAME` environmental variable.
`flushLevel`	`WARN`	If a log entry with this level is encountered, trigger a flush of locally buffered log to Cloud Logging. This can also be set via the `STACKDRIVER_LOG_FLUSH_LEVEL` environmental variable.
`enhancer`		Fully qualified class name for customizing a logging entry; must implement `com.google.cloud.logging.LoggingEnhancer`.
`loggingEventEnhancer`		Fully qualified class name for customizing a logging entry given an `ILoggingEvent`; must implement `com.google.cloud.logging.logback.LoggingEventEnhancer`.

log

spring.log

The Cloud Logging Log name. This can also be set via the STACKDRIVER_LOG_NAME environmental variable.

flushLevel

WARN

If a log entry with this level is encountered, trigger a flush of locally buffered log to Cloud Logging. This can also be set via the STACKDRIVER_LOG_FLUSH_LEVEL environmental variable.

enhancer

Fully qualified class name for customizing a logging entry; must implement com.google.cloud.logging.LoggingEnhancer.

loggingEventEnhancer

Fully qualified class name for customizing a logging entry given an ILoggingEvent; must implement com.google.cloud.logging.logback.LoggingEventEnhancer.

Asynchronous Logging

If you would like to send logs asynchronously to Cloud Logging, you can use the AsyncAppender.

Your configuration may then look like this:

<configuration>
  <include resource="com/google/cloud/spring/logging/logback-appender.xml" />

  <appender name="ASYNC_STACKDRIVER" class="ch.qos.logback.classic.AsyncAppender">
    <appender-ref ref="STACKDRIVER" />
  </appender>

  <root level="INFO">
    <appender-ref ref="ASYNC_STACKDRIVER" />
  </root>
</configuration>

Log via Console

For Logback, a com/google/cloud/spring/logging/logback-json-appender.xml file is made available for import to make it easier to configure the JSON Logback appender.

Your configuration may then look something like this:

<configuration>
  <include resource="com/google/cloud/spring/logging/logback-json-appender.xml" />

  <root level="INFO">
    <appender-ref ref="CONSOLE_JSON" />
  </root>
</configuration>

If your application is running on Google Kubernetes Engine, Google Compute Engine or Google App Engine Flexible, your console logging is automatically saved to Google Cloud Logging. Therefore, you can just include com/google/cloud/spring/logging/logback-json-appender.xml in your logging configuration, which logs JSON entries to the console. The trace id will be set correctly.

If you want to have more control over the log output, you can further configure the appender. The following properties are available:

Property Default Value Description

Property	Default Value	Description
`projectId`	If not set, default value is determined in the following order: `SPRING_CLOUD_GCP_LOGGING_PROJECT_ID` Environmental Variable. Value of `DefaultGcpProjectIdProvider.getProjectId()`	This is used to generate fully qualified Cloud Trace ID format: `projects/[PROJECT-ID]/traces/[TRACE-ID]`. This format is required to correlate trace between Cloud Trace and Cloud Logging. If `projectId` is not set and cannot be determined, then it’ll log `traceId` without the fully qualified format.
`traceIdMdcField`	`traceId`	The MDC field name for retrieving a trace id
`spanIdMdcField`	`spanId`	the MDC field name for retrieving a span id
`includeTraceId`	`true`	Should the trace id be included
`includeSpanId`	`true`	Should the span id be included
`includeLevel`	`true`	Should the severity be included
`includeThreadName`	`true`	Should the thread name be included
`includeMDC`	`true`	Should all MDC properties be included. The MDC properties `X-B3-TraceId`, `X-B3-SpanId` and `X-Span-Export` provided by Micrometer will get excluded as they get handled separately
`includeLoggerName`	`true`	Should the name of the logger be included
`includeFormattedMessage`	`true`	Should the formatted log message be included.
`includeExceptionInMessage`	`true`	Should the stacktrace be appended to the formatted log message. This setting is only evaluated if `includeFormattedMessage` is `true`
`includeContextName`	`true`	Should the logging context be included
`includeMessage`	`false`	Should the log message with blank placeholders be included
`includeException`	`false`	Should the stacktrace be included as a own field
`serviceContext`	none	Define the Stackdriver service context data (service and version). This allows filtering of error reports for service and version in the Google Cloud Error Reporting View.
`customJson`	none	Defines custom json data. Data will be added to the json output.
`loggingEventEnhancer`	none	Name of a class implementing `JsonLoggingEventEnhancer` which modifies the JSON logging output. This tag is repeatable. Examples are provided in the extensions package. - Logstash Enhancer

projectId

If not set, default value is determined in the following order:

SPRING_CLOUD_GCP_LOGGING_PROJECT_ID Environmental Variable.
Value of DefaultGcpProjectIdProvider.getProjectId()

This is used to generate fully qualified Cloud Trace ID format: projects/[PROJECT-ID]/traces/[TRACE-ID].

This format is required to correlate trace between Cloud Trace and Cloud Logging.

If projectId is not set and cannot be determined, then it’ll log traceId without the fully qualified format.

traceIdMdcField

traceId

The MDC field name for retrieving a trace id

spanIdMdcField

spanId

the MDC field name for retrieving a span id

includeTraceId

true

Should the trace id be included

includeSpanId

true

Should the span id be included

includeLevel

true

Should the severity be included

includeThreadName

true

Should the thread name be included

includeMDC

true

Should all MDC properties be included. The MDC properties X-B3-TraceId, X-B3-SpanId and X-Span-Export provided by Micrometer will get excluded as they get handled separately

includeLoggerName

true

Should the name of the logger be included

includeFormattedMessage

true

Should the formatted log message be included.

includeExceptionInMessage

true

Should the stacktrace be appended to the formatted log message. This setting is only evaluated if includeFormattedMessage is true

includeContextName

true

Should the logging context be included

includeMessage

false

Should the log message with blank placeholders be included

includeException

false

Should the stacktrace be included as a own field

serviceContext

none

Define the Stackdriver service context data (service and version). This allows filtering of error reports for service and version in the Google Cloud Error Reporting View.

customJson

none

Defines custom json data. Data will be added to the json output.

loggingEventEnhancer

none

Name of a class implementing JsonLoggingEventEnhancer which modifies the JSON logging output. This tag is repeatable.

Examples are provided in the extensions package.

- Logstash Enhancer

This is an example of such an Logback configuration:

<configuration >
  <property name="projectId" value="${projectId:-${GOOGLE_CLOUD_PROJECT}}"/>

  <appender name="CONSOLE_JSON" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
      <layout class="com.google.cloud.spring.logging.StackdriverJsonLayout">
        <projectId>${projectId}</projectId>

        <!--<traceIdMdcField>traceId</traceIdMdcField>-->
        <!--<spanIdMdcField>spanId</spanIdMdcField>-->
        <!--<includeTraceId>true</includeTraceId>-->
        <!--<includeSpanId>true</includeSpanId>-->
        <!--<includeLevel>true</includeLevel>-->
        <!--<includeThreadName>true</includeThreadName>-->
        <!--<includeMDC>true</includeMDC>-->
        <!--<includeLoggerName>true</includeLoggerName>-->
        <!--<includeFormattedMessage>true</includeFormattedMessage>-->
        <!--<includeExceptionInMessage>true</includeExceptionInMessage>-->
        <!--<includeContextName>true</includeContextName>-->
        <!--<includeMessage>false</includeMessage>-->
        <!--<includeException>false</includeException>-->
        <!--<serviceContext>
              <service>service-name</service>
              <version>service-version</version>
            </serviceContext>-->
        <!--<customJson>{"custom-key": "custom-value"}</customJson>-->
        <!--<loggingEventEnhancer>your.package.YourLoggingEventEnhancer</loggingEventEnhancer> -->
      </layout>
    </encoder>
  </appender>
</configuration>

Sample

A Sample Spring Boot Application is provided to show how to use the Cloud logging starter.