Task #7555


Setup uncloud at server11 and server12

Added by Ahmed Bilal almost 4 years ago. Updated almost 4 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
PM Check date:


Ensure that both server11 and server12 are running with uncloud today and can be used in production? And please note all steps that you took in a redmine ticket. All uncloud scripts should run as user uncloud

Actions #1

Updated by Ahmed Bilal almost 4 years ago

Allow etcd prefix for developer role

ungleich-etcdctl-root role grant-permission --prefix=true developer readwrite /server11/
ungleich-etcdctl-root role grant-permission --prefix=true developer readwrite /server12/


Replace server12 with server11 when setting up server11

# The following commands with ~# prompt are run as root
~# tmux
~# userdel -r uncloud -f
~# useradd -m uncloud

~# echo "uncloud ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
~# apt install python3-pip chrony qemu sudo
~# update-rc.d chrony enable

# Check whether time synchronization is successful
~# chronyc tracking

~# su - uncloud

# correcting prompt
exec bash

# Setting up uncloud
git clone
cd uncloud
pip3 install .
echo "alias uncloud='/home/uncloud/.local/bin/uncloud'" >> ~/.bashrc
source ~/.bashrc

# Setting up etcd connection

# Creating directory for etcd certificates. Manually upload etcd certs into it
mkdir ~/certs

# Put your OTP credentials in following command where it says replace me
cat > /home/uncloud/uncloud/uncloud.conf <<EOF
url =
port = 2379
base_prefix = /server12/
ca_cert = /home/uncloud/certs/ca.pem
cert_cert = /home/uncloud/certs/developer.pem
cert_key = /home/uncloud/certs/developer-key.pem

name = replace_me
realm = replace_me
seed = replace_me
api_server = http://localhost:5000

# Configuring uncloud (OTP, Netbox, Storage, Network)
uncloud configure otp --verification-controller-url --auth-name replace_me --auth-realm ungleich-auth --auth-seed replace_me

uncloud configure netbox --url --token replace_me

uncloud configure storage --file-dir /home/uncloud/backend/files/ filesystem --vm-dir /home/uncloud/backend/vms/ --image-dir /home/uncloud/backend/images/

uncloud configure network --prefix-length 64 --prefix 2a0a:e5c0:50::/48 --vxlan-phy-dev enp4s0f0

# Creating backend directories
mkdir -p /home/uncloud/backend/files/
mkdir -p /home/uncloud/backend/vms/
mkdir -p /home/uncloud/backend/images/

# Downloading alpine qcow2 file. We would make an image out of it.
mkdir -p /home/uncloud/backend/files/ahmedbilal-admin
wget -O /home/uncloud/backend/files/ahmedbilal-admin/alpine-ucloud.qcow2

# Run filescanner to track our files
uncloud filescanner

# Create image store
python3 ~/uncloud/uncloud/api/

# Run uncloud api
uncloud api

# Create image (replace uuid with uuid of your file)
uncloud cli image create --name alpine --uuid c3919922-3ff8-4557-a579-b6a7cdf5b654 --image-store images

# Run uncloud imagescanner
uncloud imagescanner

# Verify that the image is created successfully
uncloud cli image list

# Create host
uncloud cli host create --hostname --cpu 64 --ram '64GB' --os-ssd '128GB'

# Start uncloud host
uncloud host --hostname

# Create VM
uncloud cli vm create --vm-name meow --cpu 1 --ram '1gb' --os-ssd '4gb' --image images:alpine

# Create Network (non-global fd00:/8)
uncloud cli network create --network-name simple --network-type vxlan

# Create VM with non-global network
uncloud cli vm create --vm-name meow2 --cpu 1 --ram '1gb' --os-ssd '4gb' --image images:alpine --network simple

Actions #2

Updated by Ahmed Bilal almost 4 years ago

Devuan ascii has too old QEMU i.e 2.8 while the latest is 4.2.

It is a problem because it is showing error messages which are not helpful at all and are replaced by good/sensible error messages in newer versions.

Device needs media, but drive is empty
qemu-system-x86_64: Initialization of device ide-hd failed: Device Initialization failed.

The problem was unrelated. An extra space was added to QEMU command that was causing all the fuss.

Why it appears now?
This affects VMs with no network. All the VMs I tested recently have networks. But, it is fixed now.

Actions #3

Updated by Ahmed Bilal almost 4 years ago

uncloud filescanner has to be modified as it was using xattrs to track files which does not work on rootfs/tmpfs which is filesystem of netbooted server e.g server{11,12}.

Actions #4

Updated by Ahmed Bilal almost 4 years ago

Remaining things
[ ] IPv6 Prefix on Server 12
[ ] VM with Global IPv6 (2a0a:e5c0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx)
[ ] Setup on Server 11

Actions #5

Updated by Ahmed Bilal almost 4 years ago

  • Assignee set to Ahmed Bilal
Actions #6

Updated by Ahmed Bilal almost 4 years ago

uncloud deployed at server{11, 12}. There are still some issues that would be problem for reliably running uncloud for longer period of time. e.g
1. (As, soon as etcd leader changes or etcd become temporarily unavailable our uncloud components start dying ;)

Although, if a VM is running it wouldn't be hurt. Thanks, to our shiny new vmm.

Actions #7

Updated by Ahmed Bilal almost 4 years ago

  • Status changed from Seen to Closed

Also available in: Atom PDF