Data management policy

The way manages and stores the data that comes from your various data source integrations must be understood so that one can clearly interpert the calculations, values and charts displayed throughout the platform. Since at any given time might be ingesting very large amounts of datapoints, storing all this data indefinitely is not viable. As such, applies a downsampling policy to historical time series data so that we can scale alongside our customers.

Data types

We can generically categorize user data in two ways:
  • Raw data: data that is being ingested directly from a user's data sources.
  • SLO data: data that is generated as a consequence of monitoring an SLO such as SLI time series data, error budget and burn rate data, etc... only stores raw data if it's being actively used in the calculation of a given SLI. This type of data is subject to down-sampling. Otherwise, requests this data directly to the user's data source integrations when it's needed and does not store it. An important exception are push integrations such as Prometheus Remote Write for which handles raw data a bit differently.
SLO data is always stored by and is not subject to down-sampling.

Data retention

Both raw data and SLO data is stored by up to 1 year. If you require a larger data retention period, you can contact a representative directly to work on a custom data retention plan.

Push integrations

Raw data sent to through push integrations such as Prometheus Remote Write has a TTL of 14 days and is down-sampled to 1 hour intervals after 1 day has elapsed.

Raw data downsampling

Down-sampling is the mechanism through which high granularity time series data sets are transformed into a more coarse and manageable set of data. For example, aggregating a time series with 1 minute intervals into a lower frequency time series of 1 hour intervals.
As time moves forward, historical raw data is down-sampled according to the following table:
Elapsed Time
Maximum Time Series Interval
<= 1 day
5 minutes
<= 3 days
15 minutes
<= 7 days
30 minutes
<= 2 weeks
1 hour
<= 1 month (30 days)
6 hours
> 1 month
1 day