WIUNIX ACADEMY
Prometheus high cardinality: find the source and test a useful recording rule
Page 3 of 3. Review the earlier steps and prerequisites before continuing.
YOUR SYSTEM. YOUR STEPS.
Where are you working?
Choose your OS and version. The prerequisites, commands and troubleshooting follow your environment.
USE YOUR OWN VALUES
Configure this tutorial
Set the example values for your environment; related explanations and copy panels update together. Never enter passwords, private keys or tokens. Values stay in this tab and are not sent to the server or share links. Check actual DNS, certificates and network policy before running commands.
Different IP addresses, domains or ports? Open the settings below, change the values and select “Apply to tutorial”, then copy the updated commands.
Set values for my environment4 editable values
Do not destroy histogram meaning while aggregating
Classic histogram quantiles require retaining le while aggregating bucket rates; counters and native histograms have different expression contracts. Do not sum precomputed quantiles across instances. Dropping route/status can hide the failure you wanted to observe. Review an SLO expression independently rather than adapting this request-rate rule by renaming the metric.
Rollback a rule without deleting TSDB data
If evaluation cost/errors rise, revert the reviewed rule/source reference through the normal reload process and restore dependent dashboards to their prior expression. Derived historical series remain until retention; there is no need to delete WAL or TSDB blocks. A source-label fix reduces future combinations, not all historical series instantly. Record head-series trend, scrape health, rule duration and dashboard latency before/after at comparable load.
GO TO THE SOURCE
Official references
Each environment states how it was reviewed. A documentation review does not claim a hands-on test of every version.
KEEP LEARNING