Project

General

Profile

Activity

From 09/24/2019 to 10/23/2019

10/23/2019

08:00 PM Task #7246 (Seen): create Ubuntu 19.10 for dcl and IPv6onlyhosting
ll nu
08:00 PM Task #7245 (Seen): create Debian Bullseye image for dcl and ipv6onlyhosting
ll nu
08:00 PM Task #7244 (Seen): create CentOS8 image for dcl and ipv6onlyhosting
ll nu
08:00 PM Task #7239 (Seen): Explain to Nico how a data block is distributed / saved in ceph and relate it to our missing space problem
ll nu
04:17 PM Task #7254 (Closed): Zfs root vm image proposal - alpine
We had a request for zfs root: https://support.ungleich.ch/Ticket/Display.html?id=1557
Why shouldnt we have it as ...
ll nu
08:23 AM Task #7249 (In Progress): Merge ucloud components into one repository
Ahmed Bilal

10/22/2019

05:32 PM Task #7249 (Rejected): Merge ucloud components into one repository
Merge the different ucloud components into one repository and create a ucloud script that can be called like ucloud api Ahmed Bilal
05:18 PM Task #7191: Setup time servers and monitor time differences
Ah - the time difference should probably better monitored on
monitoring1.place{5,6}, not on monitoring.place11.
moni...
Nico Schottelius
04:51 PM Task #7191: Setup time servers and monitor time differences
What is red* and black*?
Can I add their time difference monitoring at https://monitoring.place11.ungleich.ch
Ahmed Bilal

10/21/2019

10:35 PM Task #7248 (Rejected): reinstall server1.place4
It just took much longer for rebooting. Nico Schottelius
08:35 PM Task #7248 (Rejected): reinstall server1.place4
... it does not come up anymore -- use https://robot.your-server.de/server Nico Schottelius
06:16 PM Task #7247 (Closed): Our switches in place8 should be moved to a different location / rack
* Coordinate with Kilian Bäbler
* Visit place8 once before doing the actual move
** Checkout the UPS situation
** ...
Nico Schottelius
03:31 PM Task #7246 (Closed): create Ubuntu 19.10 for dcl and IPv6onlyhosting
Sanghee Kim
03:30 PM Task #7245 (Closed): create Debian Bullseye image for dcl and ipv6onlyhosting
Sanghee Kim
03:30 PM Task #7244 (Closed): create CentOS8 image for dcl and ipv6onlyhosting
Sanghee Kim
12:08 PM Task #7122 (Feedback): Setup production etcd cluster in place6
Ahmed Bilal
12:04 PM Task #7122: Setup production etcd cluster in place6
Handing it over to rouxdo for review and future maintaining. Ahmed Bilal
10:15 AM Task #7122: Setup production etcd cluster in place6
@nico Yes, it is in etcd-cluster branch Ahmed Bilal

10/20/2019

09:53 PM Task #7122: Setup production etcd cluster in place6
Is it already in cdist?
redmine@ungleich.ch writes:
Nico Schottelius
05:10 PM Task #7242: Test active-active IPv6/IPv4 configurations on routers
v6 based test w/ dad ignorance on:... Nico Schottelius
04:51 PM Task #7242 (In Progress): Test active-active IPv6/IPv4 configurations on routers
* Set IP "185.203.114.50" to hold in opennebula -> can use it for testing... Nico Schottelius
04:39 PM Task #7242 (Closed): Test active-active IPv6/IPv4 configurations on routers
* To be able to run w/o keepalived Nico Schottelius
12:41 PM Task #7239 (Rejected): Explain to Nico how a data block is distributed / saved in ceph and relate it to our missing space problem
Given a 4MB block, how does ceph distribute it?
* Reliance on number of servers
* Order of algorithm
* Find out ...
Nico Schottelius

10/19/2019

05:54 PM Task #7238 (Closed): some hosts (or vms) cannot migrate to server13

a case: from server8 to server13 (server2 is included because it works from there)
vm id:24592 log:
Command e...
ll nu
01:13 PM Task #7122: Setup production etcd cluster in place6
* Deployed at place6
* Authentication enabled. Only clients with valid certificate issued by ungleich's private CA a...
Ahmed Bilal

10/18/2019

08:16 PM Task #7122: Setup production etcd cluster in place6
cdist type is ready. Testing underway. Ahmed Bilal
07:26 PM Task #7122: Setup production etcd cluster in place6
... Ahmed Bilal
09:44 AM Task #7122: Setup production etcd cluster in place6
Issue fixed. Ahmed Bilal
08:16 PM Task #7184: Create a mailing list for maintenance notifications
Please close the issue when you are satisfied with its working.
Thanks,
Ahmad Bilal Khalid (ABK)
Ahmed Bilal
02:58 PM Task #7184 (Feedback): Create a mailing list for maintenance notifications
Sanghee pinged. Ahmed Bilal
02:28 PM Task #7184 (In Progress): Create a mailing list for maintenance notifications
Ahmed Bilal
04:14 PM Task #7185 (New): Setup network monitoring system on new off-site VPS
Well. done. Balazs, can you confirm/ensure that you can
* reproduce the setup
* understand how monitoring is done...
Nico Schottelius
01:13 PM Task #7185: Setup network monitoring system on new off-site VPS
Monitoring system is accessible at https://monitoring.place11.ungleich.ch/ Ahmed Bilal

10/16/2019

12:32 PM Task #7122: Setup production etcd cluster in place6
ETCD is having some issues.
Specifically, it is saying **Cluster ID mismatch** I file an issue in ETCD's Github r...
Ahmed Bilal

10/15/2019

06:03 PM Task #7221: Pre 0.0.1 release fixes
Nico, it is possible although may not be that much of value. Sample command as follow... Ahmed Bilal
04:10 PM Task #7228: Make ucloud-{api,cli} to take specs from command line instead of json file
Done.... Ahmed Bilal
04:08 PM Task #7228: Make ucloud-{api,cli} to take specs from command line instead of json file
ucloud-{file-scanner, image-scanner} Show Graceful messages if some required component is missing.
For Example,
...
Ahmed Bilal

10/14/2019

02:20 PM Task #7083 (Closed): Survey Django App Installation and Test
Backup to place6. Ahmed Bilal
01:44 PM Task #7129 (Closed): Turn the netbox installation into a production setup
ll nu
12:46 PM Task #7129 (Feedback): Turn the netbox installation into a production setup
ll nu
12:45 PM Task #7129 (Closed): Turn the netbox installation into a production setup
We reviewed it. ll nu
12:45 PM Task #6998 (Closed): Install & Setup netbox on an IPv6 only VM and find out how to get / reserve the next free ip address
ll nu
12:22 PM Task #7229 (Rejected): Make __letsencrypt_nginx_cert to work on alpine
Ahmed Bilal
11:53 AM Task #7084 (Closed): Survey Django App Installation v2
ll nu
11:52 AM Task #7084: Survey Django App Installation v2
Looks good from my side.
ll nu
09:18 AM Task #7228 (Closed): Make ucloud-{api,cli} to take specs from command line instead of json file
This behavior occurs in
* Creation of VM
* Creation of Host
Ahmed Bilal

10/13/2019

06:02 AM Task #7221: Pre 0.0.1 release fixes
I added graceful handling of errors when ucloud-pay is not configured.
Now, it shows just the message....
Mondi Ravi

10/12/2019

06:15 PM Task #7221 (Rejected): Pre 0.0.1 release fixes
I'll add things I find on the way of installing ucloud 0.0.1 in this ticket and they should be fixed prior to release... Nico Schottelius

10/11/2019

07:18 PM Task #7185: Setup network monitoring system on new off-site VPS
How to change admin password? https://grafana.com/docs/administration/cli/ Ahmed Bilal
07:16 PM Task #7185: Setup network monitoring system on new off-site VPS
/etc/prometheus/blackbox.yml... Ahmed Bilal
07:09 PM Task #7185: Setup network monitoring system on new off-site VPS
Ah, sorry, thought it was assigned to me! Nico Schottelius
07:09 PM Task #7185: Setup network monitoring system on new off-site VPS
Seems not to be solved:
monitoring.place11.ungleich.ch’s server IP address could not be found.
redmine@unglei...
Nico Schottelius
12:47 PM Task #7122: Setup production etcd cluster in place6
h2. Correct, permissions... Ahmed Bilal

10/10/2019

07:37 PM Task #7205: Try 1: Installing ucloud on Arch Linux
I agree with this. It is so much pain to handle installing all these different repositories. But, need to think more. Ahmed Bilal
07:09 PM Task #7205: Try 1: Installing ucloud on Arch Linux
Note: while separating repos is cool, installing from so many code repos is pain. Maybe merge all the source code rep... Nico Schottelius
07:22 PM Task #7122 (In Progress): Setup production etcd cluster in place6
Ahmed Bilal
06:07 PM Task #7122: Setup production etcd cluster in place6
**To Start Fresh**... Ahmed Bilal
02:03 PM Task #7122: Setup production etcd cluster in place6
ca-config.json... Ahmed Bilal
06:40 PM Task #7206 (In Progress): Create ucloud page under ungleich.ch/ucloud
using lektor Nico Schottelius

10/09/2019

06:16 PM Task #7184: Create a mailing list for maintenance notifications
For tech tasks like this, please ping Balazs.
Ping Balazs.
I think we only whitelist the v6 address -> that one...
Nico Schottelius
12:12 PM Task #7184 (Waiting): Create a mailing list for maintenance notifications
Please whitelist post.ungleich.ch in ungleich's smtp. Ahmed Bilal
05:25 PM Task #7084: Survey Django App Installation v2
Question: is your code merged back into cdist? If yes, close the issue, if no, coordinate with Balazs for merging. Nico Schottelius
02:04 PM Task #7084: Survey Django App Installation v2
It is done. Ahmed Bilal
12:22 PM Task #7215 (Closed): Move Notes for upgrading / managing nextcloud instances to the open infstructure wiki
* Name of the page: "How to manage the Glarnercloud"
* Find out when customer does not use the cloud in general
* C...
Nico Schottelius
12:17 PM Task #7207 (Feedback): Change/add etcd backend to ungleich-otp and rename it to uotp
It is done AFAIK. Please check it https://code.ungleich.ch/ungleich-public/uotp Ahmed Bilal

10/06/2019

02:23 PM Task #7184: Create a mailing list for maintenance notifications
Please ping @sanghee when it's done for announcement
redmine@ungleich.ch writes:
Nico Schottelius
02:19 PM Task #7185: Setup network monitoring system on new off-site VPS
you can also add another name to the VPS like
monitoring.place11.ungleich.ch ;-)
redmine@ungleich.ch writes:
Nico Schottelius
12:47 PM Task #7185: Setup network monitoring system on new off-site VPS
I have setup (prometheus + grafana + blackbox exporter) on debian.sandbox.ahmedbilal.ungleich.cloud
You can see th...
Ahmed Bilal
02:00 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
go ahead w/ flask
redmine@ungleich.ch writes:
Nico Schottelius
12:57 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
I think, we should also convert it to flask app (as Django for such thing looks overkill or over complicated). It wou... Ahmed Bilal
12:51 PM Task #7207: Change/add etcd backend to ungleich-otp and rename it to uotp
Please explain little more. I am little out of context. Should we use etcd instead of Postgresql in ungleich-otp?
Ahmed Bilal
11:22 AM Task #7207 (Rejected): Change/add etcd backend to ungleich-otp and rename it to uotp
Nico Schottelius
08:55 AM Task #7206 (Rejected): Create ucloud page under ungleich.ch/ucloud
* Little Introduction
* Installation Instruction
* Documentation/References about how to use it.
Ahmed Bilal
07:28 AM Task #7205: Try 1: Installing ucloud on Arch Linux
How to install? Going to https://code.ungleich.ch/ucloud I find https://code.ungleich.ch/ucloud/ucloud-setup and http... Nico Schottelius
07:22 AM Task #7205 (Rejected): Try 1: Installing ucloud on Arch Linux
In theory it should "just" work, let's see what is necessary.
We need:
* ungleich-otp for authentication
* etc...
Nico Schottelius

10/04/2019

03:37 PM Task #7135 (Seen): Monitor nginx status codes in prometheus
Jin-Guk Kwon
03:37 PM Task #7136 (Seen): Create Git submodules
Jin-Guk Kwon
03:36 PM Task #7190 (Seen): Add mdadm monitoring via prometheus
Jin-Guk Kwon
11:09 AM Task #7184: Create a mailing list for maintenance notifications
Done. Temporarily, hosted at https://debian.sandbox.ahmedbilal.ungleich.cloud/listinfo. Will soon deploy it at post.u... Ahmed Bilal
06:49 AM Task #7185 (Seen): Setup network monitoring system on new off-site VPS
Ahmed Bilal
06:48 AM Task #7191 (Seen): Setup time servers and monitor time differences
Ahmed Bilal

10/03/2019

04:30 PM Task #7201: Test BGP based virtual ip
The commands in the previous comment are enough to enable the IP address *globally*. However, it is not enough for en... Nico Schottelius
01:20 PM Task #7201: Test BGP based virtual ip
... Nico Schottelius
03:18 AM Task #7174: Port the current blog's design to lektor (staticcms)
I intend to some page improvements suggested here in google pageinsights
https://developers.google.com/speed/pag...
Mondi Ravi

10/02/2019

07:15 PM Task #7174: Port the current blog's design to lektor (staticcms)
Here's how the modified design looks like
https://dev.ungleich.ch/u/blog/
Code:
https://code.ungleich.ch/mravi/u...
Mondi Ravi
06:39 PM Task #7201 (Closed): Test BGP based virtual ip
* IP on dummy/loopback
* Announced /128 via bgp
Nico Schottelius
12:18 PM Task #7119 (Closed): Alternative Blog Software Research

going w/ lektor for the moment
redmine@ungleich.ch writes:
Nico Schottelius
09:28 AM Task #7119 (Feedback): Alternative Blog Software Research
Ahmed Bilal
08:39 AM Task #7122: Setup production etcd cluster in place6
* -To Dump ETCD https://www.npmjs.com/package/etcd-dump (Not Working Correctly)-
* To Create Snapshot https://github...
Ahmed Bilal

10/01/2019

06:50 PM Task #7118 (Seen): Test ucloud
Ahmed Bilal
12:28 PM Task #7118 (Waiting): Test ucloud
Ahmed Bilal
05:34 PM Task #7194: Test delta switches
Ease of use / access
* cumulus something like 95% debian ++ tools
** https://docs.cumulusnetworks.com/cumulus-lin...
Nico Schottelius
05:29 PM Task #7194: Test delta switches
IPv6 only updates work:... Nico Schottelius
05:26 PM Task #7194 (Closed): Test delta switches
* reachable at root@cumulus.place9.ungleich.ch
* Need to verified whether they are suitable for our infra
* Main fo...
Nico Schottelius
05:04 PM Task #7193 (Closed): Move switches in place8 to a different rack
* in the week from the 14th
* Get in touch with Kilian Bäbler (tbgs) regarding appointment latest at 11th
* Will di...
Nico Schottelius
04:20 PM Task #7172 (Seen): Setup new VM/service: search.ungleich.ch
Ahmed Bilal
11:25 AM Task #7172 (Waiting): Setup new VM/service: search.ungleich.ch
It is little hairy. I will try to work on this in few days. Ahmed Bilal

09/30/2019

09:15 PM Task #7187: Configure prometheus blackbox exporter to monitor heise.de via IPv4 and IPv6
adding heise additionally to devuan is probably a good idea - heise is
certainly reachable by v6 and v4, so we can te...
Nico Schottelius
09:05 PM Task #7187 (Seen): Configure prometheus blackbox exporter to monitor heise.de via IPv4 and IPv6
Current setting (since the beginning of blackbox-exporter ;-) )... Dominique Roux
01:51 PM Task #7187 (Closed): Configure prometheus blackbox exporter to monitor heise.de via IPv4 and IPv6
So that we can quickly check whether v4/v6 traffic outgoing works
Please document steps in here so that the others...
Nico Schottelius
04:25 PM Task #7191 (Closed): Setup time servers and monitor time differences
* Setup ntpd on black* and red*
* Configure hosts to synchronise clock with the local time servers
* Raise an alert...
Nico Schottelius
04:23 PM Task #7190 (Rejected): Add mdadm monitoring via prometheus
Metrics
* node_md_disks
* node_md_disks_active
Alert if node_md_disks_active < node_md_disks
Please ping Domi...
Nico Schottelius
03:53 PM Task #7180: Phase in new router1.place6
* router1.place6 is hacked back into prometheus -> monitoring is active. We (=Balazs) need to make it clean Nico Schottelius
02:24 PM Task #7180: Phase in new router1.place6
* added full routing table to router1.place6
** Just "conf" was not enough to add it to the kernel, needed a full re...
Nico Schottelius
02:17 PM Task #7180: Phase in new router1.place6
* Restarted bird with limitations:
** no peering with iBGP routers
** no announcements of routes to upstream
Nico Schottelius
01:55 PM Task #7172: Setup new VM/service: search.ungleich.ch
I tried creating alpine package but it is little difficult because the YaCy is written in Java which is a little mess... Ahmed Bilal
11:51 AM Task #7179 (Waiting): Add Slowdown/Cooldown in TOTP verification/serializer
Successful logins never need to be cooled down, only if unsuccessful are there.
The services that use otp for veri...
Nico Schottelius
11:40 AM Task #7186 (Closed): Add support for general VPN including IPv4
* Need some NAT'ing IPv4 addresses
** "burned IPs"
* We will support NAT64 as an addition
** Create specific name ...
Nico Schottelius
11:17 AM Task #7184: Create a mailing list for maintenance notifications
VPS is ordered https://ctrlpanel.mythic-beasts.com/customer/servers/virtual/ungleich Sanghee Kim
11:02 AM Task #7184: Create a mailing list for maintenance notifications
Nico changed his mind again, going with mythic beasts. Will buy a VPS now. Sanghee Kim
10:48 AM Task #7184: Create a mailing list for maintenance notifications
@ahmedbilal Please go ahead with server1.place4, @llnu can help you with access Sanghee Kim
10:47 AM Task #7184: Create a mailing list for maintenance notifications
Nico wanted to go with our existing Hetzner server.
Sanghee Kim
10:38 AM Task #7184 (In Progress): Create a mailing list for maintenance notifications
created an account as info@ungleich.ch, now waiting for their manual review of order Sanghee Kim
10:11 AM Task #7184 (Rejected): Create a mailing list for maintenance notifications
* On-Site or off-site?
** Off-site will be reachable if we have network problems
** if off-site, which provider?
...
Nico Schottelius
11:12 AM Task #7185 (Rejected): Setup network monitoring system on new off-site VPS
h2. Objective
* We want to have a live update similar to https://status.ungleich.ch/ running on the VPS.
* We wan...
Nico Schottelius

09/29/2019

06:54 PM Task #7084: Survey Django App Installation v2
To change "Welcome to Django Survey app" message to "Hack4Glarus application"... Ahmed Bilal

09/28/2019

08:09 PM Task #7179 (Feedback): Add Slowdown/Cooldown in TOTP verification/serializer
Ahmed Bilal
09:49 AM Task #7179: Add Slowdown/Cooldown in TOTP verification/serializer
While, implementing cool down, we need to keep in mind the applications (ucloud-api, etc) that are verifying OTP cred... Ahmed Bilal
09:12 AM Task #7179 (Rejected): Add Slowdown/Cooldown in TOTP verification/serializer
Ahmed Bilal
07:54 PM Task #7118: Test ucloud
I think may be it would be good or little more cleaner if we setup the ucloud using cdist. Ahmed Bilal
07:52 PM Task #7163 (Seen): Setup wireguard endpoints in place5
Ahmed Bilal
07:51 PM Task #7172 (Seen): Setup new VM/service: search.ungleich.ch
Ahmed Bilal
07:43 PM Task #7112 (In Progress): Userguide for ucloud
Ahmed Bilal
07:42 PM Task #7084: Survey Django App Installation v2
Can be found at https://survey.hack4glarus.ungleich.ch and cdist type can be seen at https://code.ungleich.ch/ungleic... Ahmed Bilal
07:24 PM Task #7084 (Feedback): Survey Django App Installation v2
Ahmed Bilal
07:24 PM Task #7083 (Feedback): Survey Django App Installation and Test
Ahmed Bilal
07:23 PM Task #7083: Survey Django App Installation and Test
backup5 is still not resized. But, I created a cron job to do local backup every hour + there would be remote backup ... Ahmed Bilal
04:36 PM Task #7125: Implement "ordering" and "products" with ucloud
I am stuck at some issues with integration of ucloud-pay with other ucloud projects. See https://code.ungleich.ch/ucl... Ahmed Bilal
04:35 PM Task #6931 (In Progress): ucloud-host
Ahmed Bilal
04:34 PM Task #6998: Install & Setup netbox on an IPv6 only VM and find out how to get / reserve the next free ip address
Related task is also complete. Ahmed Bilal
04:33 PM Task #7129 (Feedback): Turn the netbox installation into a production setup
Everything is done. Just, I think llnu haven't completed the testing yet. Merge Request can be found at https://code.... Ahmed Bilal
02:54 PM Task #7181 (Closed): Setup 2 new routers for replacement in place5
its done :)
router1-new.intern.place6.ungleich.ch
router2-new.intern.place6.ungleich.ch
is up!
ll nu
10:50 AM Task #7181 (Closed): Setup 2 new routers for replacement in place5
* 64gb ram
* fiber card
* mdadm
* ssh keys
* ip address on internal interface
Nico Schottelius
11:46 AM Task #7182 (Closed): Update router configuration for place5
Pending changes that were already applied in place6:
* Comment out / remove keepalived configuartion in manifest/d...
Nico Schottelius
10:55 AM Task #7180: Phase in new router1.place6
Removed all static routes from new router temporarily: ... Nico Schottelius
10:48 AM Task #7180 (Closed): Phase in new router1.place6
* router1-new is up and running
h2. Steps
* Verify / update cdist configuration
* Checkout IP configuration
*...
Nico Schottelius
08:37 AM Task #7178 (Closed): Replace routers: router1.place5, router2.place5, router1.place6, router2.place6
To be able to support the full routing table and to mitigate the current routing problems, we wil replace all four ro... Nico Schottelius

09/26/2019

09:13 PM Task #7129: Turn the netbox installation into a production setup
* -Create a new VM-
* -Configure it only using cdist-
** -No manual package installations/configs/etc.!-
* -Ensure...
Ahmed Bilal

09/25/2019

02:42 PM Task #7163: Setup wireguard endpoints in place5
how far are we with production netbox?
Working on it. Backup + LDAP is left.
Ahmed Bilal
01:42 PM Task #7163: Setup wireguard endpoints in place5
Exactly. Each server should have a /40 IPv6 network assigned to. We will
need to update our bgp routing configuration...
Nico Schottelius
02:38 PM Task #7084: Survey Django App Installation v2
I will. Thanks :thumbup: Ahmed Bilal
12:31 PM Task #7084: Survey Django App Installation v2
Sounds about right -- can you coordinate with Sanghee to launch the
survey by Sunday?
redmine@ungleich.ch writes:
Nico Schottelius
06:21 AM Task #7084: Survey Django App Installation v2
Just make it public and ask them their emails. Ahmed Bilal
04:55 AM Task #7084: Survey Django App Installation v2
If we skip LDAP, how would people be authenticated to fill in the
survey?
redmine@ungleich.ch writes:
Nico Schottelius
05:59 AM Task #7174 (In Progress): Port the current blog's design to lektor (staticcms)
Mondi Ravi
04:29 AM Task #7174 (Rejected): Port the current blog's design to lektor (staticcms)
Mondi Ravi

09/24/2019

09:23 PM Task #7084: Survey Django App Installation v2
(Django-survey-and-report + sqlite3 + gunicorn + supervisord + nginx + letsencrypt (https:// working) ) with cdist. S... Ahmed Bilal
09:10 PM Task #7138: Nico's open / next points for ucloud
@nico
* To assign an IPv6 to a VM, we need its mac address. What strategy should we adopt to get a unique mac add...
Ahmed Bilal
07:28 PM Task #7163: Setup wireguard endpoints in place5
* By endpoint you mean wireguard server?
* When, you say place4/5 what does it mean? Does it mean setting up a VM in...
Ahmed Bilal
06:57 PM Task #7172: Setup new VM/service: search.ungleich.ch
genau!
(yes)
redmine@ungleich.ch writes:
Nico Schottelius
06:53 PM Task #7172: Setup new VM/service: search.ungleich.ch
yacy? Ahmed Bilal
03:36 PM Task #7172 (Closed): Setup new VM/service: search.ungleich.ch
* use yaci.net
* Do full setup using cdist
* Use IPv6 only alpine VM
* Setup https server
h2. Yaci installation...
Nico Schottelius
 

Also available in: Atom