The Sins of High Cardinality

High cardinality is a sin in observability metrics collection. Cardinality in the form of granular labels or other metadata can cause exponential growth in your observability time-series storage and compute resources; costing money and slowing down queries. It's not always economical nor practical to collect all possible metrics with all possible labels and then worry about how to extract value using queries after the fact. Sure, we want as much granularity as possible in our observability data, but it's a trade-off, we need to be strategic in using metric cardinality to get the granularity needed to discover and remediate problems.

This talk will focus on presenting different strategies to constrain the impact of high metrics cardinality referencing applicable open source Prometheus metrics collection examples.