Project

General

Profile

Actions

Task #12343

open

Evaluate cloud-hypervisor (Q12024)

Added by Nico Schottelius 3 months ago. Updated 3 months ago.

Status:
In Progress
Priority:
Normal
Target version:
-
Start date:
01/07/2024
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

  • Might be a lightweight option for running in k8s
  • Requires quite some work around it
  • No management tooling
  • Storage
    • For ceph probably need to use RBD mapped, rook supports that
    • For creating thin provisioning, probably need to create a wrapper/controller
  • live migration
    • seems to be supported
  • Networking unclear
    • macvtap support, see below
"tap=<if_name>,ip=<ip_addr>,mask=<net_mask>,mac=<mac_addr>,fd=<fd1:fd2...>,iommu=on|off,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,id=<device_id>,vhost_user=<vhost_user_enable>,socket=<vhost_user_socket_path>,vhost_mode=client|server,bw_size=<bytes>,bw_one_time_burst=<bytes>,bw_refill_time=<ms>,ops_size=<io_ops>,ops_one_time_burst=<io_ops>,ops_refill_time=<ms>" 

live migration

  • examples use same host migration
% cloud-hypervisor \
        --kernel ./hypervisor-fw \
        --disk path=focal-server-cloudimg-amd64.raw \
        --cpus boot=2 \
        --memory size=1024M \
        --net "tap=,mac=,ip=,mask=" --api-socket=/tmp/api1

[17:42] nb3:~% cloud-hypervisor --api-socket=/tmp/api2

# receive VM
ch-remote --api-socket=/tmp/api2 receive-migration unix:/tmp/sock

# send VM - fails
% ch-remote --api-socket=/tmp/api1 send-migration --local unix:/tmp/sock 
Error running command: Server responded with an error: InternalServerError: ApiError(VmSendMigration(MigrateSend(Local migration requires shared memory or hugepages enabled)))

Sketch for running VMs with cloud-hypervisor

  • Manage networking outside
    • pod running potentially in hostnetwork
    • creating bridge depending on which customer it is
    • Potentially running IPAM on a per customer basis
    • Could potentially utilise netbox as a backend, but needs to be written
  • Console access
    • read only via pod
    • serial forwarding unclear
  • Disk management
    • Thin provisioning / templates needs to be built
    • Growing disks might be supported native by k8s/rook
Actions #1

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #2

Updated by Nico Schottelius 3 months ago

running simple VM

cloud-hypervisor \
    --kernel ./hypervisor-fw \
    --disk path=focal-server-cloudimg-amd64.raw \
    --cpus boot=2 \
    --memory size=1024M \
    --net "tap=,mac=,ip=,mask=" 

  • Comes up with a login after a few seconds
Actions #3

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #4

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #5

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #6

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #7

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #8

Updated by Nico Schottelius 3 months ago

  • Description updated (diff)
Actions #9

Updated by Nico Schottelius 3 months ago

  • Project changed from 45 to Open Infrastructure
Actions

Also available in: Atom PDF