Ucloud » History » Version 15
Ahmed Bilal, 07/30/2019 01:45 PM
1 | 13 | Nico Schottelius | h1. ucloud (pre v1) |
---|---|---|---|
2 | 1 | Ahmed Bilal | |
3 | 7 | Nico Schottelius | {{toc}} |
4 | |||
5 | 11 | Nico Schottelius | h2. Preface |
6 | 1 | Ahmed Bilal | |
7 | The main development work is described in ticked #6869. |
||
8 | 13 | Nico Schottelius | This manual is in an *early development status*. |
9 | 1 | Ahmed Bilal | |
10 | 11 | Nico Schottelius | Notations: |
11 | |||
12 | * $DOMAIN: refers to a dns domain that you will use for running ucloud |
||
13 | |||
14 | h2. Administrator Manual |
||
15 | |||
16 | (being written: how to setup the ucloud infrastructure) |
||
17 | |||
18 | h3. Requirements |
||
19 | |||
20 | ucloud is an opinionated cloud management framework. It is made to solve the problem of "virtual machine hosting" including migrations, scale out, etc. While there are many different ways on how to run, store and distribute VMs, there are not that many ways that work reliable and scale out. For this reason ucloud is based on existing software that has proven to be working for virtual machine hosting. |
||
21 | |||
22 | The following components are required for running ucloud: |
||
23 | |||
24 | 1. Python 3.7, pip, pipenv (the programming language of choice) |
||
25 | 2. etcd (for data storage) |
||
26 | 3. ceph (for storage of VMs) |
||
27 | 4. Guacamole (for console access) |
||
28 | |||
29 | Setting up these components is out of scope of this manual. |
||
30 | |||
31 | h3. Base installation |
||
32 | |||
33 | 6 | Ahmed Bilal | **Note:** This installation guide assumes that the ~ is /root |
34 | |||
35 | * **Install QEMU** |
||
36 | <pre><code class="shell"> |
||
37 | sudo apt install qemu qemu-kvm |
||
38 | </code></pre> |
||
39 | |||
40 | |||
41 | * **Create Directory Structure** |
||
42 | <pre><code class="shell"> |
||
43 | mkdir /var/vm/ |
||
44 | mkdir /var/www/ |
||
45 | </code></pre> |
||
46 | |||
47 | |||
48 | * **Create User Directories** |
||
49 | <pre><code class="shell"> |
||
50 | mkdir /var/www/ahmedbilal-admin |
||
51 | mkdir /var/www/nico |
||
52 | mkdir /var/www/kjg |
||
53 | cd ~ |
||
54 | </code></pre> |
||
55 | |||
56 | * **Download and place Alpine Linux image in _/var/www/ahmedbilal-admin_** |
||
57 | <pre><code class="shell"> |
||
58 | wget https://www.dropbox.com/s/5eyryhxun6847hx/alpine.zip?dl=1 -O alpine.zip |
||
59 | unzip alpine.zip |
||
60 | mv alpine.qcow2 /var/www/ahmedbilal-admin |
||
61 | </code></pre> |
||
62 | |||
63 | * **Clone repos** |
||
64 | <pre><code class="shell"> |
||
65 | git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-api.git |
||
66 | git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-scheduler.git |
||
67 | git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-vm.git |
||
68 | git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-file-scan.git |
||
69 | git clone --recurse-submodules git@code.ungleich.ch:ungleich-public/ucloud-image-scanner.git |
||
70 | </code></pre> |
||
71 | 1 | Ahmed Bilal | |
72 | 6 | Ahmed Bilal | |
73 | 11 | Nico Schottelius | h3. Env file installation |
74 | 6 | Ahmed Bilal | |
75 | * **Run "ucloud-api" as daemon** |
||
76 | <pre><code class="shell"> |
||
77 | cd ucloud-api |
||
78 | pipenv install |
||
79 | pipenv run gunicorn --bind [::]:80 main:app --daemon |
||
80 | wget https://www.dropbox.com/s/lyu3atymxyw3846/setup.py?dl=1 -O setup.py |
||
81 | pipenv run python setup.py |
||
82 | cd ~ |
||
83 | </code></pre> |
||
84 | |||
85 | * **Run "ucloud-scheduler" as daemon** |
||
86 | <pre><code class="shell"> |
||
87 | cd ucloud-scheduler |
||
88 | pipenv install |
||
89 | pipenv run python main.py &>/dev/null & |
||
90 | cd ~ |
||
91 | </code></pre> |
||
92 | |||
93 | * **Run "ucloud-vm" as daemon** |
||
94 | <pre><code class="shell"> |
||
95 | cd ucloud-vm |
||
96 | pipenv install |
||
97 | pipenv run python main.py /v1/host/1 --vm True &>/dev/null & |
||
98 | cd ~ |
||
99 | </code></pre> |
||
100 | |||
101 | * **Install ucloud-file-scan** |
||
102 | <pre><code class="shell"> |
||
103 | cd ucloud-file-scan |
||
104 | pipenv install |
||
105 | cd ~ |
||
106 | </code></pre> |
||
107 | |||
108 | * **Install ucloud-image-scanner** |
||
109 | <pre><code class="shell"> |
||
110 | cd ucloud-image-scanner |
||
111 | pipenv install |
||
112 | cd ~ |
||
113 | </code></pre> |
||
114 | |||
115 | * **Run @crontab -e@ and put these entries there and save by pressing @Ctrl+x@ then @y@ then @enter@** |
||
116 | <pre> |
||
117 | */1 * * * * (cd /root/ucloud-file-scan && /usr/local/bin/pipenv run python main.py) |
||
118 | */1 * * * * (cd /root/ucloud-image-scanner/ && /usr/local/bin/pipenv run python main.py) |
||
119 | 1 | Ahmed Bilal | </pre> |
120 | |||
121 | |||
122 | 11 | Nico Schottelius | h2. User Manual |
123 | |||
124 | (being written: how to use ucloud as a user) |
||
125 | |||
126 | h3. Creating a VM |
||
127 | |||
128 | (to be filled in by ahmed) |
||
129 | |||
130 | h3. Viewing the console of the VM |
||
131 | |||
132 | * Go to console.$DOMAIN |
||
133 | * Login with your username password |
||
134 | * Select the VM that you want to view |
||
135 | |||
136 | h3. Deleting a VM |
||
137 | |||
138 | |||
139 | h2. API reference |
||
140 | |||
141 | (to be moved here by Ahmed) |
||
142 | |||
143 | |||
144 | h3. ucloud-api |
||
145 | |||
146 | 1 | Ahmed Bilal | Outside world (indirect) communication with the internal systems. |
147 | |||
148 | Its responsibilities are |
||
149 | 14 | Ahmed Bilal | * Create VM *(Done)* |
150 | * Delete VM *(Done)* |
||
151 | * Status VM *(Done)* |
||
152 | 1 | Ahmed Bilal | * Create new network |
153 | * Attach network to VM |
||
154 | * Detach network from VM |
||
155 | * Delete network |
||
156 | |||
157 | 11 | Nico Schottelius | h3. ucloud-scheduler |
158 | 1 | Ahmed Bilal | |
159 | It schedules/reschedules VM that are created using ucloud-api. How does it schedules? It does by watching any changes under the _/v1/vm/_ prefix. Basically, it deals with two ETCD entries |
||
160 | |||
161 | 1. Virtual Machines Entries that looks like |
||
162 | <pre><code class="javascript"> |
||
163 | /v1/vm/1 |
||
164 | { |
||
165 | "owner": "ahmedbilal-admin", |
||
166 | "specs": { |
||
167 | "cpu": 20, |
||
168 | "ram": 2, |
||
169 | 8 | Nico Schottelius | "hdd": 10, |
170 | 9 | Nico Schottelius | "ssd": 10 |
171 | }, |
||
172 | 10 | Nico Schottelius | "hostname": "", |
173 | "status": "REQUESTED_NEW" |
||
174 | 9 | Nico Schottelius | } |
175 | </code></pre> |
||
176 | 2. Hosts Entries that look like |
||
177 | 1 | Ahmed Bilal | <pre><code class="javascript"> |
178 | 9 | Nico Schottelius | /v1/host/1 |
179 | 1 | Ahmed Bilal | { |
180 | 8 | Nico Schottelius | "cpu": 32, |
181 | 1 | Ahmed Bilal | "ram": 128, |
182 | 8 | Nico Schottelius | "hdd": 1024, |
183 | 1 | Ahmed Bilal | "ssd": 0 |
184 | "status": "UP" |
||
185 | } |
||
186 | </code></pre> |
||
187 | |||
188 | It loop through list of host entries found under _/v1/host/_ and check whether we can run the incoming VM on that host. if we can, then it (ucloud-scheduler) sets the hostname key of incoming VM to that host. Suppose, our scheduler decides that we can run the earlier shown VM _/v1/vm/1_ on _/v1/host/1_. Then, our updated entry would look like |
||
189 | |||
190 | <pre><code class="javascript"> |
||
191 | /v1/vm/1 |
||
192 | { |
||
193 | 8 | Nico Schottelius | "owner": "ahmedbilal-admin", |
194 | "specs": { |
||
195 | "cpu": 20, |
||
196 | "ram": 2, |
||
197 | "hdd": 10, |
||
198 | 1 | Ahmed Bilal | "ssd": 10 |
199 | }, |
||
200 | "hostname": "/v1/host/1", |
||
201 | 9 | Nico Schottelius | "status": "REQUESTED_NEW" |
202 | } |
||
203 | </code></pre> |
||
204 | |||
205 | 8 | Nico Schottelius | *Note* the hostname key/value pair. |
206 | |||
207 | h3. ucloud-vm |
||
208 | |||
209 | 15 | Ahmed Bilal | It is responsible for creating / deleting / starting / shutdown / suspending / resuming / migrating and monitoring virtual machines. It does that by watching the _/v1/vm/_ prefix for events and act upon that. |
210 | 9 | Nico Schottelius | |
211 | 14 | Ahmed Bilal | Currently, we have the following states that a VM can have |
212 | 9 | Nico Schottelius | |
213 | 14 | Ahmed Bilal | # *REQUESTED_NEW* (Assigned to brand new VM) |
214 | # *SCHEDULED_DEPLOY* ( _ucloud-scheduler_ assign some host to the VM. _ucloud-vm_ look for such VM and create its image file in CEPH and set its status to *REQUESTED_START* ) |
||
215 | # *REQUESTED_START* ( _ucloud-vm_ starts the VM) |
||
216 | # *RUNNING* (VM is running now) |
||
217 | # *REQUESTED_SUSPEND* (User requested to suspend execution of VM. If suspension requests is accepted the VM would go to *SUSPENDED* state otherwise it would remain in the *REQUESTED_SUSPEND* ) |
||
218 | # *SUSPENDED* (VM is suspended i.e No CPU is being used, but the memory is still owned/maintained, not sure about the network resources yet but assume them still owned/maintained) |
||
219 | # *REQUESTED_RESUME* (User requested to resume execution of VM. If resumption request is accepted the VM would go to *RUNNING* state otherwise it would remain in the *REQUESTED_RESUME* ) |
||
220 | 1 | Ahmed Bilal | # *REQUESTED_SHUTDOWN* (User requested to shutdown the VM. If shutdown request is accepted the VM would go to *STOPPED* state otherwise it would remain in the *REQUESTED_SHUTDOWN* ) |
221 | # *STOPPED* (VM is stopped) |
||
222 | # *KILLED* (VM is killed i.e it is detected to be not running when it should be running by _ucloud-vm_ or _ucloud-scheduler_ detects that the host on which the VM is hosted is dead so it set all VM running on that hosts to *KILLED* ) |
||
223 | 15 | Ahmed Bilal | |
224 | There are some problems with current design. For Example, we can't track previous state when user requested some action i.e Suppose, our VM is in KILLED state and user issued REQUESTED_SUSPEND. _ucloud-vm_ would check that the VM isn't running so it would reject the request. But, our VM would stay in REQUESTED_SUSPEND. It would also create another problem which is _ucloud-scheduler_ schedules VMs that are either brand new (having status == "REQUESTED_NEW") or status == KILLED. But, as status was changed to REQUESTED_SUSPEND because of user's request _ucloud-scheduler won't schedule it on any host. |
||
225 | |||
226 | h4. Solution |
||
227 | |||
228 | # Have Two Statuses (So, we would check actual state before doing any kind of action) |
||
229 | |||
230 | * One For Actual State (in our previous case KILLED) |
||
231 | |||
232 | * The other one is requested state (in our previous case REQUESTED_SUSPEND) |
||
233 | |||
234 | # Make _ucloud-api_ more intelligent (For Example, if the VM is not running, _ucloud_api_ should not accept the REQUESTED_SUSPEND i.e we won't change the status of VM from KILLED to REQUESTED_SUSPEND) |
||
235 | |||
236 | * Check before changing the status of VM whether the request is valid or not. |