Task #12343
Updated by Nico Schottelius 4 months ago
* Might be a lightweight option for running in k8s * Requires quite some work around it * No management tooling * Storage ** For ceph probably need to use RBD mapped, rook supports that ** For creating thin provisioning, probably need to create a wrapper/controller * live migration ** seems to be supported * Networking unclear ** macvtap support, see below <pre> "tap=<if_name>,ip=<ip_addr>,mask=<net_mask>,mac=<mac_addr>,fd=<fd1:fd2...>,iommu=on|off,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,id=<device_id>,vhost_user=<vhost_user_enable>,socket=<vhost_user_socket_path>,vhost_mode=client|server,bw_size=<bytes>,bw_one_time_burst=<bytes>,bw_refill_time=<ms>,ops_size=<io_ops>,ops_one_time_burst=<io_ops>,ops_refill_time=<ms>" </pre> h2. live migration * examples use same host migration <pre> % cloud-hypervisor \ --kernel ./hypervisor-fw \ --disk path=focal-server-cloudimg-amd64.raw \ --cpus boot=2 \ --memory size=1024M \ --net "tap=,mac=,ip=,mask=" --api-socket=/tmp/api1 [17:42] nb3:~% cloud-hypervisor --api-socket=/tmp/api2 # receive VM ch-remote --api-socket=/tmp/api2 receive-migration unix:/tmp/sock # send VM - fails % ch-remote --api-socket=/tmp/api1 send-migration --local unix:/tmp/sock Error running command: Server responded with an error: InternalServerError: ApiError(VmSendMigration(MigrateSend(Local migration requires shared memory or hugepages enabled))) </pre> h2. Sketch for running VMs with cloud-hypervisor * Manage networking outside ** pod running potentially in hostnetwork ** creating bridge depending on which customer it is ** Potentially running IPAM on a per customer basis ** Could potentially utilise netbox as a backend, but needs to be written * Console access ** read only via pod ** serial forwarding unclear * Disk management ** Thin provisioning / templates needs to be built ** Growing disks might be supported native by k8s/rook