Project

General

Profile

Actions

Task #7636

closed

Find out current retention period for monitoring servers and ensure that data is kept for 5 years

Added by Nico Schottelius over 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
01/20/2020
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

  • Looking at monitoring.place6 I see data for less than 90 days.
  • My expectation is to be able to zoom out to 5 years so that we can see changes we did over years
  • My assumption is that prometheus is configured with some storage size and it auto drops/rotates
  • So probably the solution is just to add/extend the hdd space on monitoring.placeX by a factor of Y
Actions #1

Updated by Dominique Roux over 2 years ago

it's configured in __dcl_monitoring_server type

...
__prometheus_server \
    --config "$prometheus_config" \
    --retention-days 30 \
    --rule-files "$__type/files/*.rules" \
    --storage-path /data/prometheus
...

on the __prometheus_server you see:

...
FLAGS="--storage.tsdb.path $storage_path --storage.tsdb.retention $((retention_days*24))h --web.listen-address [::]:9090" 
...

See here for the configuration
https://www.robustperception.io/configuring-prometheus-storage-retention
I suggest to use the flag --storage.tsdb.retention.size 90GB (not sure if 90GB works) and get rid of the --storage.tsdb.retention flag

Here some interesting information about prometheus storage handling https://prometheus.io/docs/prometheus/1.8/storage/

Actions #2

Updated by Nico Schottelius over 2 years ago

Go with it

writes:

Actions #3

Updated by Timothée Floure over 1 year ago

  • Assignee deleted (Timothée Floure)
Actions #4

Updated by Nico Schottelius over 1 year ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF