ucloud (To Be Continued)¶

• Table of contents
• ucloud (To Be Continued)
  • Preface
  • Administrator Manual
    • Requirements
    • Base installation
    • Env file installation
  • User Manual
    • Creating a VM
    • Viewing the console of the VM
    • Deleting a VM
  • API reference
    • ucloud-api
    • ucloud-scheduler
  • ucloud-vmIt is responsible for creating (running) / deleting (stopping) / monitoring virtual machines

Preface¶

The main development work is described in ticked #6869.

Notations:

• $DOMAIN: refers to a dns domain that you will use for running ucloud

Administrator Manual¶

(being written: how to setup the ucloud infrastructure)

Requirements¶

ucloud is an opinionated cloud management framework. It is made to solve the problem of "virtual machine hosting" including migrations, scale out, etc. While there are many different ways on how to run, store and distribute VMs, there are not that many ways that work reliable and scale out. For this reason ucloud is based on existing software that has proven to be working for virtual machine hosting.

The following components are required for running ucloud:

1. Python 3.7, pip, pipenv (the programming language of choice)
2. etcd (for data storage)
3. ceph (for storage of VMs)
4. Guacamole (for console access)

Setting up these components is out of scope of this manual.

Base installation¶

Note: This installation guide assumes that the ~ is /root

• Install QEMU
  

  sudo apt install qemu qemu-kvm
  

• Create Directory Structure
  

  mkdir /var/vm/
  mkdir /var/www/
  

• Create User Directories
  

  mkdir /var/www/ahmedbilal-admin
  mkdir /var/www/nico
  mkdir /var/www/kjg
  cd ~
  

• Download and place Alpine Linux image in /var/www/ahmedbilal-admin
  

  wget https://www.dropbox.com/s/5eyryhxun6847hx/alpine.zip?dl=1 -O alpine.zip
  unzip alpine.zip
  mv alpine.qcow2 /var/www/ahmedbilal-admin
  

• Clone repos
  

  git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-api.git
  git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-scheduler.git
  git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-vm.git
  git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-file-scan.git
  git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-image-scanner.git
  

Env file installation¶

• Run "ucloud-api" as daemon
  

  cd ucloud-api
  pipenv install
  pipenv run gunicorn --bind [::]:80 main:app --daemon
  wget https://www.dropbox.com/s/lyu3atymxyw3846/setup.py?dl=1 -O setup.py
  pipenv run python setup.py
  cd ~
  

• Run "ucloud-scheduler" as daemon
  

  cd ucloud-scheduler
  pipenv install
  pipenv run python main.py &>/dev/null &
  cd ~
  

• Run "ucloud-vm" as daemon
  

  cd ucloud-vm
  pipenv install
  pipenv run python main.py /v1/host/1 --vm True &>/dev/null &
  cd ~
  

• Install ucloud-file-scan
  

  cd ucloud-file-scan
  pipenv install
  cd ~
  

• Install ucloud-image-scanner
  

  cd ucloud-image-scanner
  pipenv install
  cd ~
  

• Run crontab -e and put these entries there and save by pressing Ctrl+x then y then enter
  

  */1 * * * * (cd /root/ucloud-file-scan && /usr/local/bin/pipenv run python main.py)
  */1 * * * * (cd /root/ucloud-image-scanner/ && /usr/local/bin/pipenv run python main.py)
  

User Manual¶

(being written: how to use ucloud as a user)

Creating a VM¶

(to be filled in by ahmed)

Viewing the console of the VM¶

• Go to console.$DOMAIN
• Login with your username password
• Select the VM that you want to view

Deleting a VM¶

API reference¶

(to be moved here by Ahmed)

ucloud-api¶

Outside world (indirect) communication with the internal systems.

Its responsibilities are

• Create VM (Done) POST /vm/create
  

  {
      "name": "username",
      "realm": "user_realm",
      "seed": "user_seed",
      "specs": {
          "cpu": 16,
          "ram": 256,
          "hdd": 2000,
          "ssd": 256
      }
  }
  

• Delete VM (Done) POST /vm/delete
  

  {
      "name": "username",
      "realm": "user_realm",
      "seed": "user_seed",
      "vmid": "id_of_virtual_machine" 
  }
  

• Status VM (Done) GET /vm/status
  

  {
      "id": "id_of_virtual_machine" 
  }
  

• Create new network
• Attach network to VM
• Detach network from VM
• Delete network

ucloud-scheduler¶

It schedules/reschedules VM that are created using ucloud-api. How does it schedules? It does by watching any changes under the /v1/vm/ prefix. Basically, it deals with two ETCD entries

1. Virtual Machines Entries that looks like

/v1/vm/1
{
  "owner": "ahmedbilal-admin",
  "specs": {
    "cpu": 20,
    "ram": 2,
    "hdd": 10,
    "ssd": 10
  },
  "hostname": "",
  "status": "REQUESTED_NEW" 
}

2. Hosts Entries that look like

/v1/host/1
{
    "cpu": 32,
    "ram": 128,
    "hdd": 1024,
    "ssd": 0
    "status": "UP" 
}

It loop through list of host entries found under /v1/host/ and check whether we can run the incoming VM on that host. if we can, then it (ucloud-scheduler) sets the hostname key of incoming VM to that host. Suppose, our scheduler decides that we can run the earlier shown VM /v1/vm/1 on /v1/host/1. Then, our updated entry would look like

/v1/vm/1
{
  "owner": "ahmedbilal-admin",
  "specs": {
    "cpu": 20,
    "ram": 2,
    "hdd": 10,
    "ssd": 10
  },
  "hostname": "/v1/host/1",
  "status": "REQUESTED_NEW" 
}

Note the hostname key/value pair.

ucloud-vm
It is responsible for creating (running) / deleting (stopping) / monitoring virtual machines¶

It watches the /v1/vm/ prefix for any changes like (Virtual Machines with status like SCHEDULED_DEPLOY or REQUEST_DELETE.

On, SCHEDULED_DEPLOY it creates the VM and run it. On, REQUEST_DELETE it stops the VM.

It is also responsible for monitoring VM statues i.e whether they are RUNNING or KILLED (in action). If a VM is killed we put that in log file, notify system administrator and restart it.