Open Infrastructure

• Might be a lightweight option for running in k8s
• Requires quite some work around it
• No management tooling
• Storage
  • For ceph probably need to use RBD mapped, rook supports that
  • For creating thin provisioning, probably need to create a wrapper/controller
• live migration
  • seems to be supported
• Networking unclear
  • macvtap support, see below

"tap=<if_name>,ip=<ip_addr>,mask=<net_mask>,mac=<mac_addr>,fd=<fd1:fd2...>,iommu=on|off,num_queues=<number_of_queues>,queue_size=<size_of_each_queue>,id=<device_id>,vhost_user=<vhost_user_enable>,socket=<vhost_user_socket_path>,vhost_mode=client|server,bw_size=<bytes>,bw_one_time_burst=<bytes>,bw_refill_time=<ms>,ops_size=<io_ops>,ops_one_time_burst=<io_ops>,ops_refill_time=<ms>" 

live migration¶

• examples use same host migration

% cloud-hypervisor \
        --kernel ./hypervisor-fw \
        --disk path=focal-server-cloudimg-amd64.raw \
        --cpus boot=2 \
        --memory size=1024M \
        --net "tap=,mac=,ip=,mask=" --api-socket=/tmp/api1

[17:42] nb3:~% cloud-hypervisor --api-socket=/tmp/api2

# receive VM
ch-remote --api-socket=/tmp/api2 receive-migration unix:/tmp/sock

# send VM - fails
% ch-remote --api-socket=/tmp/api1 send-migration --local unix:/tmp/sock 
Error running command: Server responded with an error: InternalServerError: ApiError(VmSendMigration(MigrateSend(Local migration requires shared memory or hugepages enabled)))

Sketch for running VMs with cloud-hypervisor¶

• Manage networking outside
  • pod running potentially in hostnetwork
  • creating bridge depending on which customer it is
  • Potentially running IPAM on a per customer basis
  • Could potentially utilise netbox as a backend, but needs to be written
• Console access
  • read only via pod
  • serial forwarding unclear
• Disk management
  • Thin provisioning / templates needs to be built
  • Growing disks might be supported native by k8s/rook