Tracing
The OpenTelemetry Java Agent instruments the application at JVM startup. The agent is attached to the JVM using the -javaagent option and configured through OpenTelemetry properties.
Installing the OpenTelemetry Java Agent in Kubernetes
In Kubernetes, the OpenTelemetry Java Agent is installed automatically by the OpenTelemetry Operator, which has to be installed separately.
The application deployment does not need to manually provide the -javaagent JVM option or mount the agent JAR. The operator injects the agent into the application pod and configures the JVM so that the Java application starts with the OpenTelemetry agent attached.
To enable tracing in Kubernetes:
- Make sure the OpenTelemetry Operator is installed in the cluster.
- Configure the application workload for Java auto-instrumentation according to the cluster’s OpenTelemetry Operator setup.
- Configure the OpenTelemetry Collector or tracing backend endpoint.
- Restart or redeploy the application so that the operator can inject the agent.
After the application starts, the OpenTelemetry Java Agent automatically produces traces for supported frameworks, libraries, HTTP clients, database calls, and other instrumented components. Some Sophora applications, like the Sophora Server, also create custom spans for relevant internal method calls.
In Kubernetes, traces are usually sent to an OpenTelemetry Collector running in the cluster. The collector then forwards the traces to the configured observability backend.
Installing the OpenTelemetry Java Agent locally
For local development, place the OpenTelemetry Java Agent JAR file and its configuration file in the project directory.
Example:
- sophora-server/agents/otel-agent.jar
- sophora-server/agents/otel.properties
otel.properties:
otel.service.name=sophora-server
otel.traces.exporter=otlp
otel.exporter.otlp.traces.endpoint=http://localhost:4318/v1/traces
otel.logs.exporter=none
otel.metrics.exporter=none
The agent must be attached when the JVM starts.
This can be done using JAVA_TOOL_OPTIONS:
export JAVA_TOOL_OPTIONS="-javaagent:$PROJECT_DIR/sophora-server/agents/otel-agent.jar"
export OTEL_JAVAAGENT_CONFIGURATION_FILE="$PROJECT_DIR/sophora-server/agents/otel.properties"
Then start the application as usual.
Alternatively, pass the options directly to the JVM:
java \
-javaagent:$PROJECT_DIR/sophora-server/agents/otel-agent.jar \
-Dotel.javaagent.configuration-file=$PROJECT_DIR/sophora-server/agents/otel.properties \
-jar sophora-server.jar
For local tracing, an OpenTelemetry Collector or another OTLP-compatible tracing backend must be running locally and accepting traces.
Enable Structured Logging
Sophora supports structured logging with JSON across all applications. To enable structured logging, include the LogstashEncoder appender in your logback.xml file:
<appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
Add Trace Information to Logs
To correlate application logs with distributed traces, include the OpenTelemetry trace and span identifiers in the logging output.
When the OpenTelemetry Java Agent is active, it can inject trace context into the logging framework’s MDC. This allows log messages to include values such as:
- trace_id
- span_id
- trace_flags
This makes it possible to search logs by trace ID and correlate log entries with traces in the observability backend. The OpenTelemetry Java Agent must be enabled at JVM startup for trace IDs and span IDs to be available in the logging context.
Default Encoder Pattern
logback.xml. It automatically includes the trace information.Include MDC values in the logging pattern:
<pattern>
%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] [%X{trace_id:-},%X{span_id:-}] %logger{36} - %msg%n
</pattern>
Example log output:
2026-06-02 12:34:56.789 INFO [http-nio-8080-exec-1] [4bf92f3577b34da6a3ce929d0e0e4736,00f067aa0ba902b7] com.example.Service - Processing request
Kubernetes Probes
For Sophora applications deployed on Kubernetes, we recommend configuring liveness and readiness probes that include the connection to the Sophora server. For applications using Spring Boot and Spring Boot Sophora Commons, adding a check for the Sophora connection to the standard Spring Boot probe endpoints can be done by setting the respective options in the application configuration, e.g. in the application.yml:
management.endpoint.health.group.liveness.include=livenessState,sophora
management.endpoint.health.group.readiness.include=readinessState,sophora
The probe endpoints can then be used in the Kubernetes deployment configuration. We recommend using a timeout of at least two minutes (failureThreshold x periodSeconds / 12 x 10 seconds) for the liveness probe.
Example Kubernetes deployment configuration:
livenessProbe:
httpGet:
path: "/actuator/health/liveness"
port: <actuator-port>
failureThreshold: 12
readinessProbe:
httpGet:
path: "/actuator/health/readiness"
port: <actuator-port>