Task #6872
Updated by Nico Schottelius over 5 years ago
This service will run on EVERY vm host and will do the heavy lifting / creating of VMs For v1, implement the following: * Watch etcd on key prefix /v1/vm/ * If there is a change, check whether ** an existing VM on the current host should be killed ** a new VM should be started ** On event emit an email to sre@ (mail should be injected by env) -- just initially, later this will be disabled * Create a VM ** if there is a new json entry in /v1/vm with a VM that is designated for this host and has not been started ** Use qemu ... for it (sample below) * Delete a VM ** If the status json entry of a VM is TO_STOP, stop it ** After stopping was successful, put it into STOP ** We need to register the runtime somewhere (v2) * Monitor a VM ** If it is still running ** If it is not running anymore, notify sysadmins *** Write to /v1/log/ *** And/or send email to sre@ *** And restart the VM For v2 implement: * Every 5 minutes poll etcd * Check whether the VMs that are supposed to run on this host are running ** If one missing: send mail to sre@ * Check that not OTHER VMs are running on this host ** If there is one that doesn't belong here: send mail to sre@ h2. Sample qemu commands from production * This is way too long, we can trim it down. * We will also use the -name attribute * We will also use ceph and smp * We will also use uuid * We will run VNC ** v1: tcp ** v4: unix sockets <pre> oneadmin 3641289 0.5 0.1 2577776 407572 ? Sl Jun21 59:10 /usr/bin/qemu-system-x86_64 -name guest=one-24442,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-24-one-24442/master-key.aes -machine pc-i440fx-2.8,accel=kvm,usb=off,dump-guest-core=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d902efb7-2c66-4685-8914-95d38a717c90 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-24-one-24442/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -object secret,id=virtio-disk0-secret0,data=PR218Q61gRpXffWtRuIgQntokENHbAaBJGDDG66u3lg=,keyid=masterKey0,iv=XUHyWqKdHWGuva4eX+5qPw==,format=base64 -drive file=rbd:one/one-449-24442-0:id=libvirt:auth_supported=cephx\;none:mon_host=ceph1\:6789\;ceph2\:6789\;ceph3\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/one//datastores/100/24442/disk.1,format=raw,if=none,id=drive-ide0-0-0,readonly=on -device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=37 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:00:b3:39:79:75,bus=pci.0,addr=0x3 -vnc [::]:4162 -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on </pre>