Project

General

Profile

Actions

Task #7590

closed

Expect everything to fail (uncloud)

Added by Ahmed Bilal almost 5 years ago. Updated almost 3 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
01/09/2020
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

Especially external components, such as
1. etcd
2. netbox
3. otp.ungleich.ch


Related issues 1 (0 open1 closed)

Related to Open Infrastructure - Task #7583: Handle etcd leader change or temporary unavailability gracefully in uncloudRejected01/07/2020

Actions
Actions #1

Updated by Ahmed Bilal almost 5 years ago

How do we plan to handle failures in etcd

Failures can be temporarily (leadership change) or permanent (etcd cluster dies)

We use four type of functions of etcd
1. get
2. put
3. get_prefix
4. watch_prefix

get

can output the following:
  • EtcdEntry
  • Exception (KeyError if key not found, and a pleothera of other depending on situations)

TODO: KeyError is recently introduced so would require us to handle it in all stretches of uncloud other errors are also not handled.
TODO: Handle Exception thrown by get.

put

can output the following:
  • Response from etcd library
  • Exception

TODO: See whether the response from etcd library has success/failure indication.
TODO: Handle exception in case something goes wrong.

get_prefix

can output/yield the following:

  • EtcdEntry
  • Exception

TODO: Handle exception in case something goes wrong.

watch_prefix

can output/yield the following:

  • EtcdEntry
  • an empty iter in case of exception
Actions #2

Updated by Ahmed Bilal almost 5 years ago

  • Related to Task #7583: Handle etcd leader change or temporary unavailability gracefully in uncloud added
Actions #3

Updated by Ahmed Bilal almost 5 years ago

I am little uncertain how to handle failures in etcd. For Example, put every etcd function call in try/except block or do something else.

Actions #4

Updated by Ahmed Bilal almost 4 years ago

  • Assignee deleted (Ahmed Bilal)
Actions #5

Updated by Nico Schottelius almost 3 years ago

  • Status changed from New to Rejected
Actions

Also available in: Atom PDF