Observability helps you understand what is happening inside your system by looking at the signals it produces. When you run applications on Kubernetes, this becomes especially important: your workloads are distributed across pods, nodes, services, and namespaces, and issues can appear in many different layers of the platform.
With a good observability setup, you can quickly answer questions like:
- Is my application healthy?
- Where is this error coming from?
- Why did latency increase?
- What changed before the incident started?
- Which service, pod, or dependency should I investigate first?
Our recommended observability stack is designed for Kubernetes-based environments. Kubernetes already gives you useful building blocks such as labels, namespaces, events, health checks, resource metrics, and workload metadata. By using these features consistently, you can connect application behavior with the underlying infrastructure and get a clear picture of how your system is performing.
The goal is not only to collect data, but to make that data useful when you need it most: during development, operations, debugging, and incident response.
Stack overview
This section introduces our recommended observability tech stack. ou can choose from many alternatives, but we focus on tools that work well together, are proven in Kubernetes environments, and are supported by strong open-source communities.
We recommend this stack because it gives you a practical starting point: it is widely adopted, well-documented, and flexible enough to grow with your platform as your observability needs to become more advanced.
This stack gives you a practical foundation for observing Kubernetes-based systems:
- Grafana is your main interface for dashboards, exploration, and alerting.
- Prometheus collects and queries metrics from applications and infrastructure.
- Loki centralizes logs and makes them searchable using Kubernetes metadata.
- OpenTelemetry standardizes instrumentation and helps you trace requests across services.
Monitoring with Grafana
Grafana is where you go to see what is happening in your system. It gives you dashboards, visualizations, alerts, and exploration tools in one place.
You can use Grafana to:
- Monitor the health of your applications and Kubernetes clusters
- Build dashboards for namespaces, workloads, services, pods, and nodes
- Set up alerts for important conditions
- Compare metrics, logs, and traces during troubleshooting
- Give teams a shared view of system behavior
In this stack, Grafana acts as your main observability entry point. When something looks wrong, Grafana is usually where you start investigating. You can read more about using Grafana for monitoring Sophora here.
Metrics with Prometheus
Prometheus collects and stores metrics from your applications and infrastructure. It is widely used across cloud-native systems.
You can use Prometheus to track things like:
- CPU, memory, disk, and network usage
- Pod, deployment, node, and cluster health
- Request rate, error rate, and response latency
- Application-specific technical and business metrics
- Service-level indicators and alert conditions
Metrics help you understand trends, spot abnormal behavior, and define alerts before small issues become larger incidents. The sophora server comes with an integrated Prometheus metrics endpoint.
Logging with Loki
Loki gives you a central place to collect and search logs. It works especially well with Kubernetes metadata, so you can find logs by namespace, workload, pod, container, or application label.
You can use Loki to:
- Collect logs from your applications and platform components
- Search logs during debugging and incident response
- Filter log output using Kubernetes labels
- Connect logs with Grafana dashboards
- Understand what happened around a specific error or event
Loki is designed to be efficient in Kubernetes environments. Instead of indexing every word in every log line, it focuses on indexing labels, which makes it easier to organize logs around the way your workloads are deployed.
Tracing with OpenTelemetry
OpenTelemetry helps you understand how requests move through your system. This is especially useful when you have multiple services, APIs, queues, databases, or external dependencies involved in a single user action.
You can use OpenTelemetry to:
- Instrument your applications consistently
- Generate distributed traces across services
- See where time is spent during a request
- Identify slow dependencies or failing service calls
- Understand how services interact with each other
Tracing gives you the request-level detail that metrics and logs alone often cannot provide. When latency increases or a request fails somewhere in a chain of services, traces help you find the exact part of the flow that needs attention. You can read more about configuring Tracing in your Kubernetes environment here.