Alert: ObservabilityDataDiskHigh

Symptom

/var/lib/observability-data on observability VM is above 70% full for 30 minutes.

Immediate action

ssh observability 'df -h /var/lib/observability-data && du -xh /var/lib/observability-data/* 2>/dev/null | sort -h | tail -15'

Diagnosis

1. Loki chunk growth (audit 6yr tier).
2. Prometheus TSDB size.
3. Compactor retention lag.

Escalation

Ryan before disk hits 90% critical threshold.

Resolution history

Date	Notes