Activity
From 10/04/2019 to 11/02/2019
11/02/2019
- AB 04:56 PM Task #7191: Setup time servers and monitor time differences
- Please verify whether the alerts are correct according to above mentioned criteria https://monitoring.place6.ungleich.ch/d/D4dhTzgik/ceph-cluster?orgId=1&panelId=59&fullscreen&edit&tab=alert
- AB 04:38 PM Task #7283: Cleanups for ucloud 0.0.2
- Now, using process instead of threads.
- AB 01:22 PM Task #7283: Cleanups for ucloud 0.0.2
- We can create new ones, but suppose we want to change some parameters i.e passing certificates path when opening connection to etcd key/value store than we have to change it in two places :). Still, if you say I would do it with processe...
- NS 11:43 AM Task #7283: Cleanups for ucloud 0.0.2
- Not sure if I agree with you in terms of sharing - why would we want to
share the etcd_client handle and not reallocate/take a new one?
redmine@ungleich.ch writes: - AB 10:12 AM Task #7283: Cleanups for ucloud 0.0.2
- Nico Schottelius wrote:
> * host/main.py: do not hard code /var/vm -> read this from a key in etcd
> ...
* I already removed hard coded thingies but didn't pushed them yet.
* Threading should be used in our case because we want to sha... - NS 01:00 AM Task #7295 (Rejected): Flow 10: as a sysadmin, I want to be able to see statistics
- * how many vms running
* how many vms on which server running
* how much ram used (/per server)
* how much money is paid in certain time frame - NS 12:59 AM Task #7294 (Rejected): Flow 9: as a user I want to have an overview of my spendings and bills
- NS 12:58 AM Task #7293 (Rejected): Flow 8: As a user I want to be able to order a new network and use it
- * variant1: layer 2 only
* variant2: ipam
* variant3: ipam+routing (default)
A user can assign/add it to any amount of VMs - NS 12:54 AM Task #7292 (Rejected): Flow 7: as a sysadmin I want to be able to migrate VMs
- * for whatever reason
* live migration
* non-live migration - NS 12:53 AM Task #7291 (Rejected): Flow 6: as a sysadmin I want to know / install all requirements
- * router
** for networks, firewall, etc.
** for nat64
** at least 1
** n at maximum
* server
** at least 1
** for running VMs - NS 12:51 AM Task #7290 (Rejected): Flow 5: as a user I want to map an IPv4 address to another VM
- * Change the assignment / mapping
* Will require change in jool - NS 12:50 AM Task #7289 (Rejected): Flow 4: As a user I want to have a dualstack reachable VM
- * Order a dual stack VM
** Creates NAT64 mapping additional
** Need to expose the information to the user
* Ensure that the mapping is deleted when VM is deleted - NS 12:49 AM Task #7288 (Rejected): Flow 3: As a user I want to order a VM with my own OS (IPv6 only)
- * Image format
* How to upload
* How to create an image/bootable from it
* What to change in the image (ssh, metadata) - NS 12:48 AM Task #7287 (Rejected): Flow 2: As a user I want to order an IPv6 only VM (via cli) (existing OS)
- * create account on uotp enabled server
* Add credit card (?)
* go to website (?)
* Order VM via cli
* User can login to VM via ssh - NS 12:46 AM Task #7286 (Closed): Flow 1: as a sysadmin, I want to setup ucloud w/o ceph and run a VM
- Write doc/check it's there
* How to get all components
* What to prepare outside of ucloud
* all steps until 1 VM is usable including networking - NS 12:43 AM Task #7285 (Rejected): Flows to implement and do document for go live (parent)
11/01/2019
- NS 10:05 PM Task #7284 (Closed): router1.place6 phase back in
- h2. Step 1: routing setup
* Accept all networks from upstreams and ungleich peers
* don't announce any network
* full routing table in kernel
* no ip hacks (dummy0, lo)
* do we still need to solve the "next hop self ebgp;" problem... - NS 09:14 PM Task #7283 (Closed): Cleanups for ucloud 0.0.2
- * host/main.py: do not hard code /var/vm -> read this from a key in etcd
** Not sure if we want to use threading or multiprocess - I'd say the latter is probably easier
* don't just call main(), protect it by if __name__ == '__main__'!
10/31/2019
- AB 07:42 PM Task #7191: Setup time servers and monitor time differences
- -I think that the prometheus query @node_ntp_offset@ would be the one we needed. For more details, please look https://github.com/prometheus/node_exporter/blob/master/docs/TIME.md.-
- AB 06:56 PM Task #7191: Setup time servers and monitor time differences
- -Install ntp-
-apt install ntp-
-service ntp start-
-To enable, ntp collector run the following command on targets only if the file is not all commented out.-
-cat > /etc/default/prometheus-node-exporter << EOF-
-ARGS="--col... - AB 09:19 AM Task #7278: Define how networking works in ucloud
- Currently, I setup ucloud with bridged networking (radvd + rdnssd).
10/30/2019
- NS 07:13 PM Task #7278 (Rejected): Define how networking works in ucloud
- Objective: this ticket defines the details
h2. Assumptions
* we want to support empty L2 networks
* We realise L2 networks with vxlans (based on IPv6 underlay), however we should basically abstract this away
** i.e. there could p... - NS 12:08 PM Task #7172: Setup new VM/service: search.ungleich.ch
- The idea is to have all our sites indexed by search.ungleich.ch and that
if someone wants to search on any of our pages, we will use the search
form of our yaci to provide the results.
redmine@ungleich.ch writes: - NS 12:02 PM Task #7172: Setup new VM/service: search.ungleich.ch
- ping for sanghee
- AB 05:53 AM Task #7172: Setup new VM/service: search.ungleich.ch
- For what purpose, do we need yacy at search.ungleich.ch? (https://redmine.ungleich.ch/issues/7172).
*username:* admin
*password:* goanaiG5caijeiliiko9fi5Uyohreith
It offers three use cases.
* Community based web search
* Search po...
10/26/2019
- AB 01:45 PM Task #7172: Setup new VM/service: search.ungleich.ch
- https://wiki.yacy.net/index.php/En:Debian_High_Availability (Setup nginx + Create init.rc script)
10/25/2019
- LN 07:28 PM Task #7261 (Rejected): Create ipv6-spoofing nebula n-interface for place5 and understand how and why it exists
- its needed to route subnets to vm-s --> ABK wireguard setup
- LN 06:45 PM Task #7260: Add rng pci-e device to vms
- we should test/release it next week
- LN 06:44 PM Task #7260 (Rejected): Add rng pci-e device to vms
- A customer requested, because our vm-s start slow because of entropy starvation.
A fix is to add the pcie device, however easily we can only fix this for new vms:
http://docs.opennebula.org/5.8/deployment/open_cloud_host_setup/pci_pa... - AB 01:00 PM Task #7228 (Closed): Make ucloud-{api,cli} to take specs from command line instead of json file
- ucloud-cli now takes specs from command line instead of json file.
https://asciinema.org/a/Fgubl6N0Zco1yRv6BydMzLPef - AB 12:58 PM Task #6909 (Closed): Accept units in ucloud-api, but store in non-unit format
- ??nico: Hmm. I am sure the client and probably the API both should understand units, because people will post weird json to it and don't want / should not need to talk in bytes / cents
It might even be easier for us, if we store units i... - AB 12:50 PM Task #7249 (Seen): Merge ucloud components into one repository
- ucloud-{api,scheduler,host,filescanner,imagescanner,metadata} merged into single repo called ucloud. It can be found at https://code.ungleich.ch/ucloud/ucloud.
I haven't merged ucloud-pay into it as it needs some work to integrate wit...
10/23/2019
- LN 08:00 PM Task #7246 (Seen): create Ubuntu 19.10 for dcl and IPv6onlyhosting
- LN 08:00 PM Task #7245 (Seen): create Debian Bullseye image for dcl and ipv6onlyhosting
- LN 08:00 PM Task #7244 (Seen): create CentOS8 image for dcl and ipv6onlyhosting
- LN 08:00 PM Task #7239 (Seen): Explain to Nico how a data block is distributed / saved in ceph and relate it to our missing space problem
- LN 04:17 PM Task #7254 (Closed): Zfs root vm image proposal - alpine
- We had a request for zfs root: https://support.ungleich.ch/Ticket/Display.html?id=1557
Why shouldnt we have it as a choice for everyone?
We can make a promo of its features. The main benefits for me: snapshotting, automatic compres... - AB 08:23 AM Task #7249 (In Progress): Merge ucloud components into one repository
10/22/2019
- AB 05:32 PM Task #7249 (Rejected): Merge ucloud components into one repository
- Merge the different ucloud components into one repository and create a ucloud script that can be called like ucloud api
- NS 05:18 PM Task #7191: Setup time servers and monitor time differences
- Ah - the time difference should probably better monitored on
monitoring1.place{5,6}, not on monitoring.place11.
monitoring.place11 should be mainly configured as "monitoring us from
outside".
redmine@ungleich.ch writes: - AB 04:51 PM Task #7191: Setup time servers and monitor time differences
- What is red* and black*?
Can I add their time difference monitoring at https://monitoring.place11.ungleich.ch
10/21/2019
- NS 10:35 PM Task #7248 (Rejected): reinstall server1.place4
- It just took much longer for rebooting.
- NS 08:35 PM Task #7248 (Rejected): reinstall server1.place4
- ... it does not come up anymore -- use https://robot.your-server.de/server
- NS 06:16 PM Task #7247 (Closed): Our switches in place8 should be moved to a different location / rack
- * Coordinate with Kilian Bäbler
* Visit place8 once before doing the actual move
** Checkout the UPS situation
** Verify that both switches are connected to the UPS
** Verify that the UPS can hold power for both switches
h2. Step
... - SK 03:31 PM Task #7246 (Closed): create Ubuntu 19.10 for dcl and IPv6onlyhosting
- SK 03:30 PM Task #7245 (Closed): create Debian Bullseye image for dcl and ipv6onlyhosting
- SK 03:30 PM Task #7244 (Closed): create CentOS8 image for dcl and ipv6onlyhosting
- AB 12:08 PM Task #7122 (Feedback): Setup production etcd cluster in place6
- AB 12:04 PM Task #7122: Setup production etcd cluster in place6
- Handing it over to rouxdo for review and future maintaining.
- AB 10:15 AM Task #7122: Setup production etcd cluster in place6
- @nico Yes, it is in etcd-cluster branch
10/20/2019
- NS 09:53 PM Task #7122: Setup production etcd cluster in place6
- Is it already in cdist?
redmine@ungleich.ch writes: - NS 05:10 PM Task #7242: Test active-active IPv6/IPv4 configurations on routers
- v6 based test w/ dad ignorance on:
- NS 04:51 PM Task #7242 (In Progress): Test active-active IPv6/IPv4 configurations on routers
- * Set IP "185.203.114.50" to hold in opennebula -> can use it for testing
- NS 04:39 PM Task #7242 (Closed): Test active-active IPv6/IPv4 configurations on routers
- * To be able to run w/o keepalived
- NS 12:41 PM Task #7239 (Rejected): Explain to Nico how a data block is distributed / saved in ceph and relate it to our missing space problem
- Given a 4MB block, how does ceph distribute it?
* Reliance on number of servers
* Order of algorithm
* Find out why a new host was used intensively
-> 95% sure everything is in the crushmap
h2. Expected outcome
* Explanat...
10/19/2019
- LN 05:54 PM Task #7238 (Closed): some hosts (or vms) cannot migrate to server13
a case: from server8 to server13 (server2 is included because it works from there)
vm id:24592 log:
Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/migrate 'one-24592' 'server13.place6.ungleich.ch' 'server8.place6.ungle...- AB 01:13 PM Task #7122: Setup production etcd cluster in place6
- * Deployed at place6
* Authentication enabled. Only clients with valid certificate issued by ungleich's private CA authority can access the etcd.
* Even finer control is employed by setting permissions for individual user to access spe...
10/18/2019
- AB 08:16 PM Task #7122: Setup production etcd cluster in place6
- cdist type is ready. Testing underway.
- AB 07:26 PM Task #7122: Setup production etcd cluster in place6
- AB 09:44 AM Task #7122: Setup production etcd cluster in place6
- Issue fixed.
- AB 08:16 PM Task #7184: Create a mailing list for maintenance notifications
- Please close the issue when you are satisfied with its working.
Thanks,
Ahmad Bilal Khalid (ABK) - AB 02:58 PM Task #7184 (Feedback): Create a mailing list for maintenance notifications
- Sanghee pinged.
- AB 02:28 PM Task #7184 (In Progress): Create a mailing list for maintenance notifications
- NS 04:14 PM Task #7185 (New): Setup network monitoring system on new off-site VPS
- Well. done. Balazs, can you confirm/ensure that you can
* reproduce the setup
* understand how monitoring is done there
* can make changes to prometheus and grafana
Additionally, please create a new ticket for creating an email a... - AB 01:13 PM Task #7185: Setup network monitoring system on new off-site VPS
- Monitoring system is accessible at https://monitoring.place11.ungleich.ch/
10/16/2019
- AB 12:32 PM Task #7122: Setup production etcd cluster in place6
- ETCD is having some issues.
Specifically, it is saying **Cluster ID mismatch** I file an issue in ETCD's Github repository. https://github.com/etcd-io/etcd/issues/11263
10/15/2019
- AB 06:03 PM Task #7221: Pre 0.0.1 release fixes
- Nico, it is possible although may not be that much of value. Sample command as follow
- AB 04:10 PM Task #7228: Make ucloud-{api,cli} to take specs from command line instead of json file
- Done.
- AB 04:08 PM Task #7228: Make ucloud-{api,cli} to take specs from command line instead of json file
- ucloud-{file-scanner, image-scanner} Show Graceful messages if some required component is missing.
For Example,
* @qemu-img@ must be installed.
* @setfattr@ must be present.
* @getfattr@ must be present.
10/14/2019
- AB 02:20 PM Task #7083 (Closed): Survey Django App Installation and Test
- Backup to place6.
- LN 01:44 PM Task #7129 (Closed): Turn the netbox installation into a production setup
- LN 12:46 PM Task #7129 (Feedback): Turn the netbox installation into a production setup
- LN 12:45 PM Task #7129 (Closed): Turn the netbox installation into a production setup
- We reviewed it.
- LN 12:45 PM Task #6998 (Closed): Install & Setup netbox on an IPv6 only VM and find out how to get / reserve the next free ip address
- AB 12:22 PM Task #7229 (Rejected): Make __letsencrypt_nginx_cert to work on alpine
- LN 11:53 AM Task #7084 (Closed): Survey Django App Installation v2
- LN 11:52 AM Task #7084: Survey Django App Installation v2
- Looks good from my side.
- AB 09:18 AM Task #7228 (Closed): Make ucloud-{api,cli} to take specs from command line instead of json file
- This behavior occurs in
* Creation of VM
* Creation of Host
10/13/2019
- MR 06:02 AM Task #7221: Pre 0.0.1 release fixes
- I added graceful handling of errors when ucloud-pay is not configured.
Now, it shows just the message.
10/12/2019
- NS 06:15 PM Task #7221 (Rejected): Pre 0.0.1 release fixes
- I'll add things I find on the way of installing ucloud 0.0.1 in this ticket and they should be fixed prior to release.
strike -through- what has been fixed
h2. Small / doc Fixes
* ucloud-api
** README: include curl example on ...
10/11/2019
- AB 07:18 PM Task #7185: Setup network monitoring system on new off-site VPS
- How to change admin password? https://grafana.com/docs/administration/cli/
- AB 07:16 PM Task #7185: Setup network monitoring system on new off-site VPS
- /etc/prometheus/blackbox.yml
- NS 07:09 PM Task #7185: Setup network monitoring system on new off-site VPS
- Ah, sorry, thought it was assigned to me!
- NS 07:09 PM Task #7185: Setup network monitoring system on new off-site VPS
- Seems not to be solved:
monitoring.place11.ungleich.ch’s server IP address could not be found.
redmine@ungleich.ch writes: - AB 12:47 PM Task #7122: Setup production etcd cluster in place6
- h2. Correct, permissions
10/10/2019
- AB 07:37 PM Task #7205: Try 1: Installing ucloud on Arch Linux
- I agree with this. It is so much pain to handle installing all these different repositories. But, need to think more.
- NS 07:09 PM Task #7205: Try 1: Installing ucloud on Arch Linux
- Note: while separating repos is cool, installing from so many code repos is pain. Maybe merge all the source code repos again? Not sure though.
CLI wise this could be quite nice though: - AB 07:22 PM Task #7122 (In Progress): Setup production etcd cluster in place6
- AB 06:07 PM Task #7122: Setup production etcd cluster in place6
- **To Start Fresh**
- AB 02:03 PM Task #7122: Setup production etcd cluster in place6
- ca-config.json
- NS 06:40 PM Task #7206 (In Progress): Create ucloud page under ungleich.ch/ucloud
- using lektor
10/09/2019
- NS 06:16 PM Task #7184: Create a mailing list for maintenance notifications
- For tech tasks like this, please ping Balazs.
Ping Balazs.
I think we only whitelist the v6 address -> that one can use us our MX
OR: we setup a standard postfix on the VPS -- you two discuss and decide - I'd be a bit in favor f... - AB 12:12 PM Task #7184 (Waiting): Create a mailing list for maintenance notifications
- Please whitelist post.ungleich.ch in ungleich's smtp.
- NS 05:25 PM Task #7084: Survey Django App Installation v2
- Question: is your code merged back into cdist? If yes, close the issue, if no, coordinate with Balazs for merging.
- AB 02:04 PM Task #7084: Survey Django App Installation v2
- It is done.
- NS 12:22 PM Task #7215 (Closed): Move Notes for upgrading / managing nextcloud instances to the open infstructure wiki
- * Name of the page: "How to manage the Glarnercloud"
* Find out when customer does not use the cloud in general
* Create a wiki page for this
* OS package updates
** PHP upgrade
** 7.x to 7.y should not be a problem
* next cloud up... - AB 12:17 PM Task #7207 (Feedback): Change/add etcd backend to ungleich-otp and rename it to uotp
- It is done AFAIK. Please check it https://code.ungleich.ch/ungleich-public/uotp
10/06/2019
- NS 02:23 PM Task #7184: Create a mailing list for maintenance notifications
- Please ping @sanghee when it's done for announcement
redmine@ungleich.ch writes: - NS 02:19 PM Task #7185: Setup network monitoring system on new off-site VPS
- you can also add another name to the VPS like
monitoring.place11.ungleich.ch ;-)
redmine@ungleich.ch writes: - AB 12:47 PM Task #7185: Setup network monitoring system on new off-site VPS
- I have setup (prometheus + grafana + blackbox exporter) on debian.sandbox.ahmedbilal.ungleich.cloud
You can see the Grafana at http://debian.sandbox.ahmedbilal.ungleich.cloud:3000
* **username**: admin
* **password**: admin
You c... - NS 02:00 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
- go ahead w/ flask
redmine@ungleich.ch writes: - AB 12:57 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
- I think, we should also convert it to flask app (as Django for such thing looks overkill or over complicated). It wouldn't take long as most of the code is Django boilerplate.
- AB 12:51 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
- Please explain little more. I am little out of context. Should we use etcd instead of Postgresql in ungleich-otp?
- NS 11:22 AM Task #7207 (Rejected): Change/add etcd backend to ungleich-otp and rename it to uotp
- AB 08:55 AM Task #7206 (Rejected): Create ucloud page under ungleich.ch/ucloud
- * Little Introduction
* Installation Instruction
* Documentation/References about how to use it. - NS 07:28 AM Task #7205: Try 1: Installing ucloud on Arch Linux
- How to install? Going to https://code.ungleich.ch/ucloud I find https://code.ungleich.ch/ucloud/ucloud-setup and https://code.ungleich.ch/ucloud/ucloud-init - both names seem to be promising.
- NS 07:22 AM Task #7205 (Rejected): Try 1: Installing ucloud on Arch Linux
- In theory it should "just" work, let's see what is necessary.
We need:
* ungleich-otp for authentication
* etcd for storing configurations and run time data
h2. The story / handbook
* My objective is to use ucloud for schedu...
10/04/2019
- JK 03:37 PM Task #7135 (Seen): Monitor nginx status codes in prometheus
- JK 03:37 PM Task #7136 (Seen): Create Git submodules
- JK 03:36 PM Task #7190 (Seen): Add mdadm monitoring via prometheus
- AB 11:09 AM Task #7184: Create a mailing list for maintenance notifications
- Done. Temporarily, hosted at https://debian.sandbox.ahmedbilal.ungleich.cloud/listinfo. Will soon deploy it at post.ungleich.ch
- AB 06:49 AM Task #7185 (Seen): Setup network monitoring system on new off-site VPS
- AB 06:48 AM Task #7191 (Seen): Setup time servers and monitor time differences