Activity
From 01/01/2020 to 01/30/2020
01/28/2020
- 02:38 PM Task #7650: Synchronise opennebula VMs with etcd
- Nico its done. https://code.ungleich.ch/ungleich-public/ungleich-tools/tree/master/opennebula-vm-etcd
1. Putting V... - 10:36 AM Task #7654: Get VMs info from Opennebula and save it in etcd
- ping sre team
- 10:35 AM Task #7654: Get VMs info from Opennebula and save it in etcd
- clearly a cron job.
Let's create a new (IPv6 only) service VM based on Alpine, which is
dedicated for running statel... - 09:31 AM Task #7654: Get VMs info from Opennebula and save it in etcd
- We can run it either as cron job or put a sleep in the code....
01/27/2020
- 09:29 PM Task #7654: Get VMs info from Opennebula and save it in etcd
- Where does it run / update every 10 minutes?
- 05:37 PM Task #7654: Get VMs info from Opennebula and save it in etcd
- Done
- 10:59 AM Task #7654 (Rejected): Get VMs info from Opennebula and save it in etcd
- Create a service that gets VMs' information from OpenNebula and save it in etcd.
It should also update/sync this inf... - 04:58 PM Task #7632: Setup rados / s3 storage on ceph
- ...
- 01:25 PM Task #7650: Synchronise opennebula VMs with etcd
- I actually created this ticket some time ago
01/26/2020
- 05:56 PM Task #7641: create images for uncloud
- * The image definition scripts are defined in: https://code.ungleich.ch/uncloud/images
* The images have been upload... - 10:02 AM Task #7653 (In Progress): Move VMs with routed /64 into their own /64 [was: Instructions for adding /64 to VM]
- Good point.
Timothee, can you create a new network for VMs with routed networks so that we can again decide on DNS...
01/25/2020
- 05:01 PM Task #7653 (Closed): Move VMs with routed /64 into their own /64 [was: Instructions for adding /64 to VM]
- Document process for moving VM to /64 rather than /128.
Importantly: customer should modify their name server conf...
01/24/2020
- 11:18 AM Task #7649: Sketch a VM backup & restore
- h2. OpenNebula testing...
- 11:16 AM Task #7649: Sketch a VM backup & restore
- Snapshotting:...
- 10:49 AM Task #7649: Sketch a VM backup & restore
- h2. Ceph
It seems ceph even has a limit support:... - 10:16 AM Task #7649 (Rejected): Sketch a VM backup & restore
- * User wants to have possibility to roll back in time
h2. Implementation
* We can use ceph snapshots
* We migh... - 11:08 AM Task #7650 (Rejected): Synchronise opennebula VMs with etcd
- * etcd prefix: /opennebula
*
h2. What to synchronise
* /opennebula/vm/<VM id>
** all information about the V... - 10:03 AM Task #7636: Find out current retention period for monitoring servers and ensure that data is kept for 5 years
- Go with it
redmine@ungleich.ch writes:
01/23/2020
- 10:57 PM Task #7636: Find out current retention period for monitoring servers and ensure that data is kept for 5 years
- it's configured in __dcl_monitoring_server type...
- 05:53 PM Task #7632: Setup rados / s3 storage on ceph
- Note for production checking: what happens/is required to do if the
certificate is replaced with a new one?
redmine... - 05:46 PM Task #7632: Setup rados / s3 storage on ceph
- updated conf with ssl
[client.rgw.rgw]
host = rgw
#rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.... - 05:30 PM Task #7632: Setup rados / s3 storage on ceph
- Radosgw is running. The correct way to start is1:
/usr/bin/radosgw --cluster ceph --name client.rgw.`hostname ... - 01:41 PM Task #7632: Setup rados / s3 storage on ceph
- using this manual the radosgw is configured at rgw.llnu.at
https://access.redhat.com/documentation/en-us/red_hat_c...
01/22/2020
- 03:33 PM Task #7630: Cleanup the DNS64 situation
- No. If somebody does not want that, they should remove the appropriate
script below /etc/one-contex.d
redmine@u... - 02:00 PM Task #7630: Cleanup the DNS64 situation
- What can we do about that the /etc/resolv.conf is rewritten at every reboot?
Would the best be to advise:
chattr +i... - 09:11 AM Task #7630 (Closed): Cleanup the DNS64 situation
Thanks!
redmine@ungleich.ch writes:
01/21/2020
- 09:42 PM Task #7632: Setup rados / s3 storage on ceph
redmine@ungleich.ch writes:- 07:55 PM Task #7632 (Seen): Setup rados / s3 storage on ceph
- 07:55 PM Task #7631 (Seen): Report details about pleroma problems to upstream
- 02:16 PM Task #7641 (In Progress): create images for uncloud
- 12:38 PM Task #7641 (Closed): create images for uncloud
- First images should be the latest alpine, fedora, ubuntu, debian.
How should they be configured:
they should ge... - 10:57 AM Task #7545 (In Progress): Switch production LDAPs to cdist-managed alpine
- 10:32 AM Task #7483 (In Progress): Update the __consul cdist type for alpine
- I got the same issue with `__consul_agent` on Debian, which I am currently fixing upstream.
https://code.ungleich....
01/20/2020
- 05:11 PM Task #7630 (Feedback): Cleanup the DNS64 situation
- See https://redmine.ungleich.ch/projects/open-infrastructure/wiki/The_ungleich_DNS_infrastructure.
- 12:06 PM Task #7630 (Seen): Cleanup the DNS64 situation
- 11:03 AM Task #7630 (Closed): Cleanup the DNS64 situation
- h2. Old situation
* bind nameservers on routers decide based on source IPv6 address whether to give out NAT64 or n... - 03:19 PM Task #7636 (Closed): Find out current retention period for monitoring servers and ensure that data is kept for 5 years
- * Looking at monitoring.place6 I see data for less than 90 days.
* My expectation is to be able to zoom out to 5 ye... - 03:06 PM Task #7635 (Closed): Create a simple page explaining DNS64/NAT64 for customers
- * So that we can reference it in support tickets.
* Include exapmles, how to reach github, show the AAAA record reso... - 12:06 PM Task #6694 (Closed): Setup matrix server and bridge matermost into it
- Relevant channels have been bridged. Closing.
- 12:05 PM Task #7560 (In Progress): Document DNS64 setup for VMs
- 12:05 PM Task #7496 (Closed): Create 2 new IPv6 only unbound based resolving DNS servers providing DNS64
- Unbound DNS(64) servers are now monitored by the prometheus blackbox exporter. Closing.
- 11:35 AM Task #7632 (Closed): Setup rados / s3 storage on ceph
- * Including permissions
* Document the setup
* Document how to use it - 11:04 AM Task #7631 (Closed): Report details about pleroma problems to upstream
- Follow up with https://git.pleroma.social/pleroma/pleroma-support/issues/10#note_49605
* Create an IPv6 only VM
*... - 09:27 AM Task #7629 (Rejected): Add referral link system in dynamicweb (DCL, IPv6OnlyHosting etc)
- Referral links are used to reward user/(reviewing website) whenever someone purchase VM (or other service) using thei...
01/19/2020
- 02:48 PM Task #7543 (Closed): Write image definition script for ubuntu 19.10
- The image has been deployed in ONE and configured for the `public-Ubuntu 19.10` and `ipv6only-Ubuntu 19.10` templates...
01/18/2020
- 07:39 PM Task #7496: Create 2 new IPv6 only unbound based resolving DNS servers providing DNS64
- It's deployed: there's just monitoring to setup before it can be closed.
- 01:35 PM Task #7625: Manually fix consul+node_exporter on new router1.place6
- ...
- 01:27 PM Task #7625: Manually fix consul+node_exporter on new router1.place6
- ...
- 01:26 PM Task #7625: Manually fix consul+node_exporter on new router1.place6
- Use alpine's init script:...
- 01:02 PM Task #7625 (Rejected): Manually fix consul+node_exporter on new router1.place6
- ...
01/15/2020
- 07:28 PM Task #6671: Setup mastodon/pleroma for ungleich
- could we have a 13373r name?
01/13/2020
- 07:30 PM Task #7604 (In Progress): Find out why ciara2 was not automatically detected to be offline
- * ciara2 is half correctly outside of the consul cluster
** It should actually still be inside the cluster, but mark... - 07:28 PM Task #7604 (Rejected): Find out why ciara2 was not automatically detected to be offline
- * Consul status / prometheus / alert manager should have noticed
- 06:37 PM Task #7186: Add support for general VPN including IPv4
- Errrr what is it with your VPN pricing? Did you go skiing and get altitude sickness?
Market price for VPN services i... - 05:52 PM Task #7544: Write "beginner's guide" for datacenterlight customers
- -IPv6 and IPv4: making the services on my IPv6 VM visible to the IPv4 world
-Guido to VM Management tools: dashboard... - 12:01 PM Task #7602 (Rejected): Align dynamicweb / opennebula with uncloud
- Stuff that we can & should export from our current setup to etcd in an uncloud alike format:
Prefix for everything... - 11:57 AM Task #7601: Setup an SSH jump host
- Proxycommand w/ windows exists in putty and usually uses plink - more details soon.
- 11:54 AM Task #7601 (Seen): Setup an SSH jump host
- * I am familiar with LDAP-backed auth with nslcd.
* ProxyCommand is standard for SSH bastions, it is even available ... - 11:12 AM Task #7601 (Rejected): Setup an SSH jump host
- * Authenticated against our ldap
* Allows user to connect to our IPv6 networks
The ways for users to use it:
*... - 08:37 AM Task #7555 (Closed): Setup uncloud at server11 and server12
- 03:14 AM Task #7582 (Closed): Add hostname in uncloud file scanning
- 03:14 AM Task #7519 (Closed): uncloud test run 2019-12-21
- The above mentioned things were fixed.
01/12/2020
- 09:18 PM Task #7580: Preparing for matrix-as-a-service
- channels that can be exported to IRC or matrix:
* ipv6
* foss
* hacking-and-learning
* Town Square
* datacente... - 09:11 PM Task #7580 (In Progress): Preparing for matrix-as-a-service
- 09:10 PM Task #7580: Preparing for matrix-as-a-service
- * Synapse and Matrix Cdist types are (almost) OK.
- I missed one small thing in my __postgres upstream cdist patch... - 09:12 PM Task #6694: Setup matrix server and bridge matermost into it
- Everything's in place, we just have to choose the channels to be bridged.
01/09/2020
- 08:56 PM Task #7596: uncloud-api refactoring & make schemas less horrible
- To see merge request https://code.ungleich.ch/uncloud/uncloud/merge_requests/1
- 08:54 PM Task #7596: uncloud-api refactoring & make schemas less horrible
- * Done `uncloud api` refactoring which was due for a long time and is the last refactoring beside https://redmine.ung...
- 08:54 PM Task #7596 (Closed): uncloud-api refactoring & make schemas less horrible
- 08:55 PM Task #7585 (Closed): Check whether uncloud-api break if some field is missing
- The behavior is verified and corrected.
- 09:32 AM Task #7591: uncloud production checklist 2020-01
- Also, please note
uncloud deployed at server{11, 12}. There are still some issues that would be problem for re... - 09:26 AM Task #7591: uncloud production checklist 2020-01
- *Can all required components be deployed (checking on server11) -- document the installation procedures*
Documente... - 09:19 AM Task #7591: uncloud production checklist 2020-01
- h1. Installation/Setup
Allow etcd prefix for developer role... - 09:17 AM Task #7591 (In Progress): uncloud production checklist 2020-01
- 09:16 AM Task #7591 (Rejected): uncloud production checklist 2020-01
- h2. Objective
* Migrate internal VMs to uncloud
h2. Checklist
* Can all required components be deployed (che... - 08:55 AM Task #7590: Expect everything to fail (uncloud)
- I am little uncertain how to handle failures in etcd. For Example, put every etcd function call in try/except block o...
- 08:48 AM Task #7590: Expect everything to fail (uncloud)
- h2. How do we plan to handle failures in etcd
Failures can be temporarily (leadership change) or permanent (etcd c... - 08:48 AM Task #7590 (Rejected): Expect everything to fail (uncloud)
- Especially external components, such as
1. etcd
2. netbox
3. otp.ungleich.ch
01/08/2020
- 08:21 PM Task #7583: Handle etcd leader change or temporary unavailability gracefully in uncloud
- We have to re-evaluate/re-check all the usage of etcd in uncloud to make sure we handle these events correctly/gracef...
- 07:34 AM Task #7583: Handle etcd leader change or temporary unavailability gracefully in uncloud
- The later unavailability is due to election for leader.
- 12:21 PM Task #7580: Preparing for matrix-as-a-service
- We can do-it in a second-stage, witout a TURN server VoIP might or might not work depending on the situation.
Note... - 11:27 AM Task #7580: Preparing for matrix-as-a-service
- Can we do voip in a second stage or will things "look weird" without it?
redmine@ungleich.ch writes: - 11:17 AM Task #7580: Preparing for matrix-as-a-service
- We'll also need a TURN server for VoIP.
01/07/2020
- 06:38 PM Task #7555: Setup uncloud at server11 and server12
- uncloud deployed at server{11, 12}. There are still some issues that would be problem for reliably running uncloud fo...
- 06:33 PM Task #7582: Add hostname in uncloud file scanning
- Nico Schottelius wrote:
> For the example above: the host would probably often more something like "files1.datacente... - 04:19 PM Task #7582: Add hostname in uncloud file scanning
- For the example above: the host would probably often more something like "files1.datacenterlight.ch" or "username.dat...
- 04:18 PM Task #7582: Add hostname in uncloud file scanning
- We should have a "created_at" and "deleted_at" for every object.
- 02:44 PM Task #7582: Add hostname in uncloud file scanning
- Done. Sample
Before... - 01:41 PM Task #7582 (Closed): Add hostname in uncloud file scanning
- It is required as nico said that there would be multiple file hosts and not necessarily all files are available on so...
- 04:45 PM Task #7585 (Closed): Check whether uncloud-api break if some field is missing
- The behavior is seen in the past that uncloud-api breaks if we don't pass some fields like *name*, *realm* or *token*...
- 02:07 PM Task #7583 (Rejected): Handle etcd leader change or temporary unavailability gracefully in uncloud
- Here is leader change....
- 12:53 PM Task #7580 (Closed): Preparing for matrix-as-a-service
- Once matrix is deployed at ungleich:
* Build & document MaaS deployment and maintenance pipeline.
- Wiki page.
...
01/06/2020
- 12:21 PM Task #7543 (In Progress): Write image definition script for ubuntu 19.10
- From Nico:...
- 12:09 PM Task #7543 (Waiting): Write image definition script for ubuntu 19.10
- There's already a 19.10 image deployed... ???
- 11:57 AM Task #7543 (In Progress): Write image definition script for ubuntu 19.10
01/05/2020
- 07:09 PM Task #7555: Setup uncloud at server11 and server12
- Remaining things
[ ] IPv6 Prefix on Server 12
[ ] VM with Global IPv6 (2a0a:e5c0:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx)
[ ... - 07:02 PM Task #7555: Setup uncloud at server11 and server12
- uncloud filescanner has to be modified as it was using xattrs to track files which does not work on rootfs/tmpfs whic...
- 04:58 PM Task #7555: Setup uncloud at server11 and server12
- Devuan ascii has too old QEMU i.e 2.8 while the latest is 4.2.
It is a problem because it is showing error message... - 11:03 AM Task #7565 (Rejected): uncloud run 2020-01-05
- h2. Objective
A test ride to get more nearby prod use
h2. What Nico wants to do us a customer
* register a...
01/03/2020
- 05:35 PM Task #7561: Update mystrom switches to support IPv6 only networks
- ...
- 05:35 PM Task #7561 (Rejected): Update mystrom switches to support IPv6 only networks
- Using an experimental firmware from mystrom directly:...
- 04:08 PM Task #7560 (Closed): Document DNS64 setup for VMs
- * After #7496
* Document on how to use it in the [[The_ungleich_DNS_infrastructure]]
* Reference it in [[The_unglei... - 11:38 AM Task #7555: Setup uncloud at server11 and server12
- Allow etcd prefix for developer role...
- 10:58 AM Task #7555 (Closed): Setup uncloud at server11 and server12
- Ensure that both server11 and server12 are running with uncloud today and can be used in production? And please note ...
01/02/2020
- 05:09 PM Task #7496: Create 2 new IPv6 only unbound based resolving DNS servers providing DNS64
- Merge request opened against dot-cdist: https://code.ungleich.ch/ungleich-intern/dot-cdist/merge_requests/65
- 03:30 PM Task #7436 (Closed): Hack-a-ucloud-weekend (2019-12-07)
- 03:30 PM Task #7438 (Closed): Explore local ucloud setup
- 03:30 PM Task #7437 (Closed): Run ucloud with a single authentication token
- 03:30 PM Task #7439 (Rejected): Add support for different authentication methods
- Postponed.
- 01:35 PM Task #7553: Setup conntrackd to allow active active firewalls
- And config looks like this:...
- 01:33 PM Task #7553: Setup conntrackd to allow active active firewalls
- Seems like the code is in read_config.yy.c:...
- 01:08 PM Task #7553: Setup conntrackd to allow active active firewalls
- Added sync section, now getting an IPv6 exception:...
- 12:59 PM Task #7553: Setup conntrackd to allow active active firewalls
- router2.place6:...
- 12:50 PM Task #7553 (Rejected): Setup conntrackd to allow active active firewalls
- * So that firewall rules still work with state tracking
Change of objective: get this running on two IPv6 only Alp... - 12:47 PM Task #7552 (Closed): Add some non-critical traffic to router1.place6
- * might required conntrackd
Networks first stage:
* internal network
* server network
- 12:46 PM Task #7306 (Rejected): Phase in new routers
- Closing in favor of #7284
- 12:44 PM Task #7307 (Closed): Update __ungleich_bgp_router for IPv6 based multip bgp
- Not doing this atm, as routers still need IPv4 addresses for VMs
- 12:21 PM Task #7520 (Closed): Checkout whether OSPF can be helpful for DCL
- It works!
But not for eBGP routes. - 12:07 PM Task #6930: cdist configuration for etcd
- Current state I know of:
* there is etcd{1,2,3}.ungleich.ch
* They have an unknown configuration
* And they have...
01/01/2020
- 05:49 PM Task #7546: VM Security based on LDAP accounts
- We don't have any rate limiting to any of the apis that we have so far.
I think rate limiting would primarily be n... - 05:05 PM Task #7546: VM Security based on LDAP accounts
- * Moris, thanks for reporting.
* Mondi, can you start with your tasks and handover to Timothee when done?
- 05:05 PM Task #7546: VM Security based on LDAP accounts
- h2. Clarification 1: "shared login"
We use LDAP servers as a backend to redmine and django (the dashboard). Both s... - 04:53 PM Task #7546 (In Progress): VM Security based on LDAP accounts
Also available in: Atom