Project

General

Profile

Task #7649

Sketch a VM backup & restore

Added by Nico Schottelius over 1 year ago. Updated over 1 year ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
-
Start date:
01/27/2020
Due date:
% Done:

0%

Estimated time:
(Total: 0.00 h)
PM Check date:

Description

  • User wants to have possibility to roll back in time

Implementation

  • We can use ceph snapshots
  • We might need/want to signal qemu before taking the snapshot
    • how?
    • Difference between uncloud and opennebula?

User facing API / CLI

  • User can set a maximum amount of snapshots (?)
    • Useful for auto rotate
  • User can configure a snapshot
  • User can list all snapshots
  • User can delete snapshots

Ideas for an CLI

ungleich-cli --username xx --password yy list-vms
ungleich-cli --username xx --password yy vm-snaphot-create <vmid>
ungleich-cli --username xx --password yy vm-snaphot-list <vmid>
ungleich-cli --username xx --password yy vm-snaphot-restore <vmid> <snapshot>
ungleich-cli --username xx --password yy vm-snaphot-delete <vmid> <snapshot>

Billing

Regular pricing - it's the same usage

Billed per gigabyte per second (?) basis (?)

Per second calculation: SSD

>>> (3.5/10.)/30/24/3600
1.3503086419753083e-07
>>> per_gb_per_second=((3.5/10.)/30/24/3600) 
>>> 1.36*10**(-7) * 10 * 86400 * 30
3.5251200000000003
>>> "{:0.9f}".format(1.36*10**(-7))
'0.000000136'
>>> 1.4*10**(-7) * 10 * 86400 * 30
3.628799999999999

  • Could offer it for 0.00000014 CHF per GB per second
  • Monthly would stay the same
  • User wants to have possibility to roll back in time

Implementation

  • We can use ceph snapshots
  • We might need/want to signal qemu before taking the snapshot
    • how?
    • Difference between uncloud and opennebula?

User facing API / CLI

  • User can set a maximum amount of snapshots (?)
    • Useful for auto rotate
  • User can configure a snapshot
  • User can list all snapshots
  • User can delete snapshots

Billing

Regular pricing - it's the same usage

Billed per gigabyte per second (?) basis (?)

Per second calculation: SSD

>>> (3.5/10.)/30/24/3600
1.3503086419753083e-07
>>> per_gb_per_second=((3.5/10.)/30/24/3600) 
>>> 1.36*10**(-7) * 10 * 86400 * 30
3.5251200000000003
>>> "{:0.9f}".format(1.36*10**(-7))
'0.000000136'
>>> 1.4*10**(-7) * 10 * 86400 * 30
3.628799999999999

  • Could offer it for 0.00000014 CHF per GB per second
  • Monthly would stay the same

Per second calculation: HDD

>>> per_gb_per_second=((1.5/100.)/30/24/3600) 
>>> per_gb_per_second
5.787037037037037e-09
>>> 
>>> "{:0.10f}".format(per_gb_per_second)
'0.0000000058'
>>> 0.0000000058* 100 * 86400 * 30
1.5033599999999998
  • Could offer for 0.0000000058 per gb per second
  • However units will be smaller than 100gb packages

Testing

  • Using my private VM
  • Locating the disk image
[10:43:59] red2.place5:~# rbd -p one ls | grep 10761
one-60-10761-0
[10:44:14] red2.place5:~# 
  • Does not show all attached disks -> need to go via opennebula api

Per second calculation: HDD

>>> per_gb_per_second=((1.5/100.)/30/24/3600) 
>>> per_gb_per_second
5.787037037037037e-09
>>> 
>>> "{:0.10f}".format(per_gb_per_second)
'0.0000000058'

Testing

  • Using my private VM
  • Locating the disk image
[10:43:59] red2.place5:~# rbd -p one ls | grep 10761
one-60-10761-0
[10:44:14] red2.place5:~# 
  • Does not show all attached disks -> need to go via opennebula api

Steps

  • Create a PoC for some VMs
    • record creation and deletion time is needed for billing later
  • Implement a CLI to be used by staff (i.e. no customer auth)
  • Integrate it into billing

Subtasks

Task #7654: Get VMs info from Opennebula and save it in etcdNewAhmed Bilal

Actions

History

#1

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)

Ceph

It seems ceph even has a limit support:

    snap create (snap add)      Create a snapshot.
    snap limit clear            Remove snapshot limit.
    snap limit set              Limit the number of snapshots.
    snap list (snap ls)         Dump list of image snapshots.
    snap protect                Prevent a snapshot from being deleted.
    snap purge                  Delete all snapshots.
    snap remove (snap rm)       Delete a snapshot.
    snap rename                 Rename a snapshot.
    snap rollback (snap revert) Rollback image to snapshot.
    snap unprotect              Allow a snapshot to be deleted.

[10:48:18] red2.place5:~# rbd snap create one/one-60-10761-0@20200124_1048
[10:49:07] red2.place5:~# rbd snap ls one/one-60-10761-0
SNAPID NAME            SIZE TIMESTAMP                
   124 20200124_1048 500GiB Fri Jan 24 10:49:07 2020 
[10:49:14] red2.place5:~# 

#2

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)
#3

Updated by Nico Schottelius over 1 year ago

Snapshotting:

[10:52:32] red2.place5:~# rbd rm one/nicotest
2020-01-24 11:14:57.886218 7fc527460700 -1 librbd::image::RemoveRequest: 0x56525faae5f0 check_image_snaps: image has snapshots - not removing
Removing image: 0% complete...failed.
rbd: image has snapshots - these must be deleted with 'rbd snap purge' before the image can be removed.
[11:14:57] red2.place5:~# rbd snap purge one/nicotest
Removing all snapshots: 100% complete...done.
[11:15:41] red2.place5:~# rbd rm one/nicotest
Removing image: 100% complete...done.
[11:15:43] red2.place5:~# 
#4

Updated by Nico Schottelius over 1 year ago

OpenNebula testing

[11:17:33] server1.place5:~# onevm list | grep nico-vm
 10761 nico.schottelius@ungleich.ch             users    nico-vm.schotte runn 5.05    2.1G server1.place5.ungleich.ch     918d 16h56
[11:18:07] server1.place5:~# onevm snapshot-create 10761
[11:18:17] server1.place5:~# 
  • Did not create anything
  • Using the webui creating a snapshot per disk works
#5

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)
#6

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)
#7

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)
#8

Updated by Nico Schottelius over 1 year ago

  • Description updated (diff)
#9

Updated by Ahmed Bilal over 1 year ago

  • Assignee changed from Nico Schottelius to Ahmed Bilal
#10

Updated by Ahmed Bilal over 1 year ago

The functionality part is complete beside `ungleich-cli` and can be found at https://code.ungleich.ch/ahmedbilal/vm-cli-services. I am ready to give a test-run tour whenever you want.

Also available in: Atom PDF