As part of our commitment to open standards, Google Cloud is deeply invested in making OpenTelemetry the universal client, data format, and set of standards for telemetry data.
Last year we announced support in Cloud Observability for sending traces using OpenTelemetry Protocol (OTLP). Today, we’re excited to announce the next step toward our goal of OpenTelemetry everywhere: Cloud Observability now supports OTLP for metrics in Cloud Monitoring!
OTLP for metrics: More than just a new standard
Using OpenTelemetry and OTLP lets you generate and send metric data to Google Cloud with a completely provider-agnostic pipeline: You can create OTLP metrics using the OpenTelemetry SDK, collect and transform them using the OpenTelemetry collector, and send that data directly to Cloud Monitoring in OpenTelemetry format.
By default, this data gets stored in the same format as Managed Service for Prometheus data, at the same low price. This data is queryable using the same interfaces available to query any other data in Cloud Monitoring.
Using OTLP also lets you take advantage of several highly-requested new features, such as:
- DELTA-type metrics: Send the amount that a monotonic counter changed between the last export and the current export, instead of tracking all counters in memory and always sending the latest value of the counter. This allows clients to flush memory in between exports, which significantly reduces resource consumption on the client-side and better supports collecting short-lived or infrequently incremented time series.
- Exponential (dynamic) histograms: Classic Histograms require you to explicitly set their bucket widths based on the projected data distribution. If that projection doesn’t match the actual data distribution, you can end up with all the observations lumped into a few low buckets, or with the most interesting observations smeared across a “lower than infinity” bucket. OpenTelemetry Exponential Histograms instead dynamically change the bucket boundaries based on the range of values actually seen, so you no longer have to guess and check histogram buckets. Just set it and forget it!
- Dots and slashes in metric names and dots in label keys: Cloud Monitoring now has full support for querying URL-style names using PromQL. Additionally, Cloud Monitoring now supports the . character in label keys, which enables support for OpenTelemetry’s semantic conventions.
- Sending metrics directly from the SDK to Cloud Monitoring with no collector: For extremely high-volume, high-cardinality metric sources such as Envoy (which reports pod-pod and service-service traffic) or customer-run load balancer processes, it can be prohibitively expensive to have an OpenTelemetry collector in the pipeline. Collectors can get overloaded with excessive volume of metrics, and horizontally or vertically scaling them is a lot of work for developers. With OTLP, you can point metrics exported by the OpenTelemetry SDK directly at Cloud Observability’s Telemetry API for metrics, letting you rely on Google to handle your volume rather than having to run and scale an intermediary process yourself.
- Zero-code auto-instrumentation for metrics and traces: Use OpenTelemetry to automatically instrument compatible workloads, which generates standardized traces and Golden Signal metrics without requiring any code, and then send data to Cloud Observability in OTLP format. No longer do you need to mix application and instrumentation code to get Golden Signal metrics — and that’s if you even remember to consistently instrument every RPC in your code.
Managed OpenTelemetry for Google Kubernetes Engine
Running an OpenTelemetry collector yourself can be a lot of work, requiring you to manually deploy, configure, and scale OpenTelemetry collectors. But for typical workloads with typical observability profiles, all you typically need is a simple in-cluster endpoint for receiving and enriching OTLP signals.
That’s why we’re excited to also announce Managed OpenTelemetry for GKE, a fully-managed, “one-click” pipeline for generating and collecting OTLP traces, metrics, and logs on Google Kubernetes Engine. Let Google handle the collector lifecycle, upgrades, and scaling, so you can focus on your application code, not your observability infrastructure.
Managed OpenTelemetry is the first fully managed trace solution for GKE. Tracing is critical for application performance monitoring and powers features like the application topology map, a dynamic, actionable view of your application’s dependencies.
You can also use Managed OpenTelemetry for GKE to automatically configure and instrument workloads that use the OpenTelemetry SDK. With a single Custom Resource, you can get Golden Signals in Cloud Observability for all your OpenTelemetry-enabled applications — including AI agents built with frameworks that support OpenTelemetry such as the Agent Development Kit (ADK).
How to get started
OTLP for metrics is currently in preview, is open to all customers, and is supported when using OpenTelemetry versions 0.140.0 and higher. To get started with OTLP metrics using the OpenTelemetry SDK or the OpenTelemetry Collector, see the OTLP metric ingestion documentation. When running your own collector, we recommend using the Google-built OpenTelemetry Collector whenever possible.
Managed OpenTelemetry for GKE is currently in preview, is open to all customers, and is available for GKE cluster versions 1.34.1-gke.2178000 or later and gcloud CLI versions 551.0.0 or later. To get started, see the Managed OpenTelemetry for GKE documentation.
Finally, to get started with zero-code auto-instrumentation for Java workloads on GKE using a self-deployed OpenTelemetry Collector, see the zero-code documentation.
