Project

General

Profile

Task #7191

Setup time servers and monitor time differences

Added by Nico Schottelius 8 months ago. Updated 7 months ago.

Status:
Seen
Priority:
Low
Assignee:
Target version:
-
Start date:
09/30/2019
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

  • Setup ntpd on black* and red*
  • Configure hosts to synchronise clock with the local time servers
  • Raise an alert if time is > 10 second different to the average of the time servers
    • node_time_seconds in promtheus

History

#1

Updated by Ahmed Bilal 8 months ago

  • Status changed from New to Seen
#2

Updated by Ahmed Bilal 7 months ago

  • Assignee changed from Ahmed Bilal to Nico Schottelius

What is red* and black*?

Can I add their time difference monitoring at https://monitoring.place11.ungleich.ch

#3

Updated by Nico Schottelius 7 months ago

Ah - the time difference should probably better monitored on
monitoring1.place{5,6}, not on monitoring.place11.

monitoring.place11 should be mainly configured as "monitoring us from
outside".

writes:

#4

Updated by Ahmed Bilal 7 months ago

  • Assignee changed from Nico Schottelius to Ahmed Bilal
#5

Updated by Ahmed Bilal 7 months ago

Install ntp

apt install ntp
service ntp start

To enable, ntp collector run the following command on targets only if the file is not all commented out.

cat > /etc/default/prometheus-node-exporter << EOF
ARGS="--collector.ntp --collector.ntp.server ::1"
EOF

service prometheus-node-exporter restart

#6

Updated by Ahmed Bilal 7 months ago

I think that the prometheus query node_ntp_offset would be the one we needed. For more details, please look https://github.com/prometheus/node_exporter/blob/master/docs/TIME.md.

#7

Updated by Ahmed Bilal 7 months ago

  • Assignee changed from Ahmed Bilal to ll nu

Please verify whether the alerts are correct according to above mentioned criteria https://monitoring.place6.ungleich.ch/d/D4dhTzgik/ceph-cluster?orgId=1&panelId=59&fullscreen&edit&tab=alert

#8

Updated by Ahmed Bilal 7 months ago

  • Assignee changed from ll nu to Ahmed Bilal

Also available in: Atom PDF