Task #7636: Find out current retention period for monitoring servers and ensure that data is kept for 5 years - Open Infrastructure - ungleich redmine

Actions

Copy link

Task #7636

closed

Find out current retention period for monitoring servers and ensure that data is kept for 5 years

Added by Nico Schottelius over 4 years ago. Updated about 3 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Target version:

Start date:

01/20/2020

Due date:

% Done:

Estimated time:

PM Check date:

Description

Looking at monitoring.place6 I see data for less than 90 days.
My expectation is to be able to zoom out to 5 years so that we can see changes we did over years
My assumption is that prometheus is configured with some storage size and it auto drops/rotates
So probably the solution is just to add/extend the hdd space on monitoring.placeX by a factor of Y

Actions

Copy link

Updated by Dominique Roux over 4 years ago

it's configured in __dcl_monitoring_server type

...
__prometheus_server \
    --config "$prometheus_config" \
    --retention-days 30 \
    --rule-files "$__type/files/*.rules" \
    --storage-path /data/prometheus
...

on the __prometheus_server you see:

...
FLAGS="--storage.tsdb.path $storage_path --storage.tsdb.retention $((retention_days*24))h --web.listen-address [::]:9090" 
...

See here for the configuration
https://www.robustperception.io/configuring-prometheus-storage-retention
I suggest to use the flag --storage.tsdb.retention.size 90GB (not sure if 90GB works) and get rid of the --storage.tsdb.retention flag

Here some interesting information about prometheus storage handling https://prometheus.io/docs/prometheus/1.8/storage/

Actions

Copy link