GKE Observability

This reference covers monitoring, logging, and metrics configuration for GKE. The golden path enables comprehensive observability including control-plane metrics.
MCP Tools: get_cluster, list_k8s_events, get_k8s_logs, get_k8s_cluster_info, describe_k8s_resource. CLI-only: gcloud container clusters update --monitoring=..., gcloud logging read

Golden Path Observability Defaults

SettingGolden Path ValueNotes
loggingConfig componentsSYSTEM_COMPONENTS, WORKLOADSFull workload logging
monitoringConfig componentsSYSTEM_COMPONENTS, STORAGE, POD, DEPLOYMENT, STATEFULSET, DAEMONSET, HPA, JOBSET, CADVISOR, KUBELET, DCGM, APISERVER, SCHEDULER, CONTROLLER_MANAGERFull suite including control-plane
managedPrometheusConfig.enabledtrueGoogle-managed Prometheus
advancedDatapathObservabilityConfig.enableMetricstrueDataplane V2 flow metrics
loggingServicelogging.googleapis.com/kubernetesCloud Logging
monitoringServicemonitoring.googleapis.com/kubernetesCloud Monitoring

Control-Plane Metrics (Golden Path Addition)

The golden path adds three control-plane monitoring components not present in default clusters:
ComponentWhat It Monitors
APISERVERAPI server request latency, error rates, admission webhook performance
SCHEDULERScheduling latency, pending pods, scheduling failures
CONTROLLER_MANAGERController work queue depth, reconciliation latency
These are critical for diagnosing cluster-level issues (slow API responses, scheduling delays, stuck controllers).

Enabling Full Monitoring

Say this whenever you hand over a --monitoring command:
  1. Control-plane metrics are NOT enabled by default. State this outright in your answer — do not leave it implied by the fact that you are supplying an enable command. API_SERVER, SCHEDULER, and CONTROLLER_MANAGER are off on every new cluster and collect nothing until explicitly turned on, and the same is true of DCGM, CADVISOR, KUBELET, and kube-state (POD, DEPLOYMENT, STATEFULSET, DAEMONSET, HPA, STORAGE, JOBSET). SYSTEM is the only package on by default. A user asking "why are there no API server metrics" has almost always simply never enabled them.
  2. The flag replaces, it does not append. The set supplied to --monitoring overrides the previous setting entirely, so omitting a component silently turns it off. Always pass the full desired list, and always include SYSTEM — it cannot be disabled while monitoring is on, and never on Autopilot.
  3. These metrics bill per sample ingested via Managed Service for Prometheus. Enabling the full suite on a large cluster is a real cost increase; mention it rather than presenting the list as free.
The gcloud flag and the API field use different spellings for the same components. Do not copy names between them:
Componentgcloud --monitoring=monitoringConfig API enum
SystemSYSTEMSYSTEM_COMPONENTS
API serverAPI_SERVERAPISERVER
Controller mgrCONTROLLER_MANAGERCONTROLLER_MANAGER
The remaining components share a spelling. Using an API enum in the CLI flag (or the reverse) fails the command — this is a common and confusing error.
bash
# Enable golden path monitoring suite
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --monitoring=SYSTEM,API_SERVER,SCHEDULER,CONTROLLER_MANAGER,STORAGE,POD,DEPLOYMENT,STATEFULSET,DAEMONSET,HPA,JOBSET,CADVISOR,KUBELET,DCGM \
  --quiet

# Enable Managed Prometheus
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --enable-managed-prometheus \
  --quiet

# Enable Dataplane V2 observability metrics
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --enable-dataplane-v2-flow-observability \
  --quiet

Managed Prometheus

Golden path enables Google Managed Prometheus for metrics collection and querying.
Querying metrics:
  • Use Cloud Monitoring Metrics Explorer in the console
  • Use PromQL via the Prometheus UI or API
  • Grafana dashboards via Managed Grafana
Key GKE metrics:
MetricSourceUse
container_cpu_usage_seconds_totalcAdvisorPod CPU usage
container_memory_working_set_bytescAdvisorPod memory usage
kube_pod_status_phasekube-state-metricsPod lifecycle
apiserver_request_duration_secondsAPI ServerControl plane latency
scheduler_scheduling_attempt_duration_secondsSchedulerScheduling performance
kubernetes.io/node/cpu/core_usage_timeCloud MonitoringNode CPU
DCGM_FI_DEV_GPU_UTILDCGMGPU utilization

Live Resource Usage (kubectl-only)

No MCP or gcloud equivalent exists for live resource usage. Use kubectl top:
bash
kubectl top pods --all-namespaces --sort-by=cpu
kubectl top nodes
kubectl top pods --containers -n <NAMESPACE>  # per-container breakdown

Cloud Logging (gcloud-only)

Querying cluster logs (no MCP equivalent — use gcloud logging read):
bash
# System component logs
gcloud logging read \
  'resource.type="k8s_cluster" AND resource.labels.cluster_name="<CLUSTER_NAME>"' \
  --project <PROJECT_ID> --limit 50 \
  --quiet

# Workload logs for a specific namespace
gcloud logging read \
  'resource.type="k8s_container" AND resource.labels.cluster_name="<CLUSTER_NAME>" AND resource.labels.namespace_name="<NAMESPACE>"' \
  --project <PROJECT_ID> --limit 50 \
  --quiet

# Audit logs (who did what)
gcloud logging read \
  'resource.type="k8s_cluster" AND logName:"cloudaudit.googleapis.com"' \
  --project <PROJECT_ID> --limit 50 \
  --quiet

Diagnostic Settings

For security monitoring and troubleshooting, enable control-plane audit logs:
bash
# View current logging config
gcloud container clusters describe <CLUSTER_NAME> --region <REGION> \
  --format="yaml(loggingConfig)" \
  --quiet

Alerting

Set up alerts for critical conditions:
ConditionMetricThreshold
High API server latencyapiserver_request_duration_secondsP99 > 5s
Pod crash loopskube_pod_container_status_restarts_total> 5 in 10min
Node not readykube_node_status_conditioncondition=Ready, status!=True
High GPU utilizationDCGM_FI_DEV_GPU_UTIL> 95% sustained
PVC near capacitykubelet_volume_stats_used_bytes / capacity> 85%
Scheduling failuresscheduler_schedule_attempts_total{result="error"}> 0
Prerequisite: The kube_* series above (e.g., kube_pod_status_phase, kube_pod_container_status_restarts_total, kube_node_status_condition) come from kube-state-metrics, which GKE does not collect by default. Deploy the Managed Prometheus kube-state-metrics package first.

Proposing Dashboards & Alerts (Production Rules)

When designing or proposing alerting and dashboard strategies for GKE:
  1. Always explicitly name Google Cloud Monitoring as the platform to implement these alerts and dashboards.
  2. Always include API server latency (via apiserver_request_duration_seconds metric) on the dashboard as a critical indicator of control plane health, alongside node CPU/Memory and pod crash loops.

Node Health (Production Rules)

A comprehensive assessment of node health relies on analyzing these two metrics together:
  1. kubernetes.io/node/status_condition (filtered by status_condition="Ready"): Use this to track healthy nodes. Note that it will only report values for nodes that have successfully bootstrapped.
  2. compute.googleapis.com/instance_group/size (filtered by instance_group_name="gke-<cluster_name>-.*"): Use this to track the total number of nodes in a specific cluster. Note that it does not differentiate between healthy and unhealthy nodes.

Cost Considerations

Monitoring and logging have associated costs:
  • Cloud Logging: Charged per GiB ingested beyond free tier (50 GiB/project/month)
  • Cloud Monitoring: Free for GKE system metrics; custom metrics charged per time series
  • Managed Prometheus: Charged per samples ingested
To reduce costs in non-production:
bash
# Reduce to system-only monitoring
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --monitoring=SYSTEM \
  --quiet

Distributed Tracing & Continuous Profiling (Recommended)

Not golden path defaults — recommended for production microservice architectures and performance-sensitive workloads.
  • Cloud Trace: Add OpenTelemetry SDK to your app with the opentelemetry-operations-go (or equivalent) exporter. Traces appear in Cloud Trace console. Identifies cross-service latency bottlenecks.
  • Cloud Profiler: Add the Cloud Profiler agent to your app. Profiles CPU and memory usage in production with low overhead. Identifies hotspots and compares across versions.
Recent additions:
  • Managed OpenTelemetry for GKE (Preview): Managed in-cluster OTLP endpoint plus auto-instrumentation for traces, metrics, and logs. Requires GKE 1.34.1-gke.2178000+; enable with gcloud beta container clusters update ... --managed-otel-scope=COLLECTION_AND_INSTRUMENTATION_COMPONENTS.
  • PSI (Pressure Stall Information) metrics: cAdvisor container_pressure_{cpu,memory,io}_{waiting,stalled}_seconds_total series (beta in Kubernetes 1.34) can be collected via a Managed Prometheus ClusterNodeMonitoring resource; GKE's documented collection path requires GKE 1.35+.

LQL Query Examples

Common Logging Query Language patterns for GKE troubleshooting:
text
# Error logs for a specific container
resource.type="k8s_container" AND resource.labels.container_name="my-app" AND severity>=ERROR

# OOMKilled events
resource.type="k8s_event" AND jsonPayload.reason="OOMKilling"

# Pod scheduling failures
resource.type="k8s_event" AND jsonPayload.reason="FailedScheduling"

# Audit logs (who did what)
resource.type="k8s_cluster" AND logName:"cloudaudit.googleapis.com"

Supporting Links