ungleich redmine: Issueshttp://localhost:3000/http://localhost:3000/favicon.ico?16699092332021-02-11T15:50:32Zungleich redmine
Redmine Open Infrastructure - Task #8888 (Closed): Meta-Issue for Matrix late-winter 2021 cleanuphttp://localhost:3000/issues/88882021-02-11T15:50:32ZTimothée Floure
<p>Hello there,</p>
<p>We have some performance issues and infrastructure rot on our matrix deployments: I'll work on it here and there over the next few weeks. This meta-issue will make following easier to follow what's going on. I might add things on the fly as I encounter them / link to other issues later on.</p>
<ul>
<li>[x] Cleaning up and upstreaming the __matrix-synapse cdist type. <a class="issue tracker-5 status-5 priority-1 priority-lowest closed" title="Task: Cleanup & upstream matrix-related types (Closed)" href="http://localhost:3000/issues/7345">#7345</a><br /> - [x] Clean-up.<br /> - [x] Bring configuration template up-to-date.<br /> - [x] Add more performance-related flags.<br /> - [x] Add support for multi-workers (a new __matrix_synapse_worker type might be needed)<br /> - [x] Upstream to cdist-contrib See <a class="external" href="https://code.ungleich.ch/ungleich-public/cdist-contrib/-/merge_requests/9">https://code.ungleich.ch/ungleich-public/cdist-contrib/-/merge_requests/9</a></li>
<li>[ ] Cleanup and simplify the __ungleich_matrix type<br /> - [ ] Allow PGSQL tuning / auto-tune from explorer if not provided.<br /> - [x] Adapt to updated __matrix_synapse type</li>
<li>[ ] Revamp matrix monitoring: we need something simpler and more robust.<br /> - [x] Get back missing instances in monitoring.<br /> - [ ] Add alerts.<br /> - [x] Add PGSQL performance monitoring.</li>
<li>[x] Update admin UI</li>
<li>[ ] Investigate performance issues.<br /> - [~] Checking out database bottlenecks.<br /> - [~] Checking out synapse bottlenecks.<br /> - [ ] Possibily add periodic database cleanup.</li>
<li>[ ] Check out the state of the Jitsi integration.<br /> - [x] Rebuilt with CDIST (small issue with watermark - see <a class="external" href="https://code.ungleich.ch/ungleich-public/cdist-contrib/-/issues/4">https://code.ungleich.ch/ungleich-public/cdist-contrib/-/issues/4</a>)<br /> - [x] Wire Prometheus to the new Jitsi Exporter<br /> - [ ] Add simple blackbox monitoring</li>
<li>[x] Check state of ext.ungleich.ch homeserver</li>
<li>[ ] LOW_PRIO check out if it is useful to deploy our own integration server</li>
<li>[ ] Don't forget to document!</li>
</ul> Open Infrastructure - Task #8887 (Closed): Update synapse-adminhttp://localhost:3000/issues/88872021-02-11T15:14:53ZTimothée Floure
<p>The synapse-admin instance running at admin.matrix.ungleich.cloud is outdated - we need to updated it.</p> Open Infrastructure - Task #8877 (Rejected): Checkout ext.ungleich.ch matrix instance on server1....http://localhost:3000/issues/88772021-02-10T08:38:57ZTimothée Floure
<blockquote>
<p>we used to have an external/smaller/simpler matrix server.<br />I have asked Timothee to have a look at this to revive it on</p>
</blockquote>
<p>server1.place4 as ext.ungleich.ch again.</p> Open Infrastructure - Task #8852 (Closed): Investigate matrix.ungleich.ch slownesshttp://localhost:3000/issues/88522021-02-05T08:32:39ZTimothée Floure
<p>matrix.ungleich.ch is so slow it becomes unusable, I'm currently investigating and:</p>
<ul>
<li>Disabling Presence</li>
<li>Increase cache sizes</li>
<li>Configure Synapse with multiple workers</li>
</ul> Open Infrastructure - Task #8123 (Rejected): Document how to update the BGP configurationhttp://localhost:3000/issues/81232020-06-05T11:01:36ZNico Schotteliusnico.schottelius@ungleich.ch
<ul>
<li>checkout latest cdist commit</li>
<li>Upstream contacts are in netbox</li>
<li>We need to update OUR ripe objects to say that we peer with the specific ASN
<ul>
<li>Modify the object <strong>as-set "as-ungleich-downstream"</strong></li>
<li>on ripe.net</li>
</ul></li>
</ul> Open Infrastructure - Task #8110 (Closed): Investigate unbound{1,2}.place6.ungleich.ch crasheshttp://localhost:3000/issues/81102020-06-03T08:02:11ZTimothée Floure
<p>I increased log verbosity on unbound1.p6, and will try to see if there's anything amiss.</p> Open Infrastructure - Task #8091 (Closed): Alpine-based Opennebula workershttp://localhost:3000/issues/80912020-05-30T08:57:26ZTimothée Floure
<p>Plan: move our ONE workers from devuan to alpine.</p>
<ul>
<li>I managed to get an alpine node to join my test ONE cluster.
<ul>
<li>Now waiting for llnu to set me up a pet CEPH cluster.</li>
</ul>
</li>
<li>TODO: package/cdistify/upstream alpine node configuration.
<ul>
<li>Related (where are ONE package definitions tracked?): <a class="external" href="https://github.com/OpenNebula/one/issues/4844">https://github.com/OpenNebula/one/issues/4844</a></li>
</ul></li>
</ul> Open Infrastructure - Task #8048 (Closed): Unable to log in to matrix/riothttp://localhost:3000/issues/80482020-05-25T17:07:06ZStefan Baur
<p>Hi Nico, hi folks,</p>
<p>It seems I can't sign in to matrix/riot at <a class="external" href="https://matrix.ungleich.ch">https://matrix.ungleich.ch</a></p>
<p>What I tried:<br /><a class="email" href="mailto:X2Go-ML-1@baur-itcs.de">X2Go-ML-1@baur-itcs.de</a> as User Name<br /><a class="email" href="mailto:X2Go-ML-1@baur-itcs.de">X2Go-ML-1@baur-itcs.de</a> as E-Mail Address<br />same with all lowercase.</p>
<p>I have two different passwords for mattermost and accounts/redmine; the one for accounts/redmine works for logging in to redmine (obviously, as I would be unable to open this issue if it wouldn't work), but not for matrix/riot, even though the page says it should be the same. The mattermost password only works for mattermost.</p>
<p>Attempting to reset the password on matrix/riot doesn't work, either, it claims it doesn't know my E-Mail address.</p>
<p>Would be cool if someone could fix this or tell me what I'm doing wrong.</p>
<p>-Stefan</p> Open Infrastructure - Task #7768 (Closed): Add monitoring to Matrix-as-a-Servicehttp://localhost:3000/issues/77682020-02-24T08:27:57ZTimothée FloureOpen Infrastructure - Task #7641 (Closed): create images for uncloudhttp://localhost:3000/issues/76412020-01-21T12:38:42ZSanghee Kimsanghee.kim@ungleich.ch
<p>First images should be the latest alpine, fedora, ubuntu, debian.</p>
<p>How should they be configured:</p>
<p>they should get an ipv6 address from the first network interface, <br />they should automatically increase the root file system, if the disk is increased in size <br />they should get public ssh key from <a class="external" href="http://metadata/ssh_keys">http://metadata/ssh_keys</a><br />VMs need to get the DNS servers from the rooter advertisements, this is usually done with RDNSSD. (that’s a program only needs to be installed and running.)</p>
<p>Please prepare VMs with just this settings.</p> Open Infrastructure - Task #7580 (Closed): Preparing for matrix-as-a-servicehttp://localhost:3000/issues/75802020-01-07T12:53:44ZTimothée Floure
<p>Once matrix is deployed at ungleich:</p>
<ul>
<li>Build & document MaaS deployment and maintenance pipeline.<br /> - Wiki page.<br /> - A staging environment will be required to test upgrades.</li>
<li>1 or 2 blog entries about it? First one maybe a bit more as introduction, why we want to support matrix and second one more about the technical details? (quoting Nico here)</li>
<li>Be mentionned in "This Week In Matrix" (Weekly matrix news) and on <a class="external" href="https://matrix.org/docs/projects/hosting/">https://matrix.org/docs/projects/hosting/</a><br /> - We should emphasize on the decent/green (hydro/old building/second-hand servers/...) factor, as I expect it will interest some (sub-)communities.</li>
<li>Upstream `__matrix_*` cdist types.</li>
<li>Investigate the application services we could offer.</li>
</ul>
<p>Feel free to put this task in another project if it doesn't fit here.</p> Open Infrastructure - Task #7545 (Closed): Switch production LDAPs to cdist-managed alpinehttp://localhost:3000/issues/75452019-12-31T15:20:43ZTimothée Floure
<p>Our production LDAP nodes do not seem to be managed by cdist (anymore?):
* No relevant mention in `grep -R __ungleich_ldap dot-cdist/` or `grep -R ldap1 dot-cdist/`
* Deployed configuration do not exactly match `__ungleich_ldap` type.</p>
<p>=> Investigate and update dot-cdist to handle production ldap{1,2}.ungleich.ch</p> Open Infrastructure - Task #7496 (Closed): Create 2 new IPv6 only unbound based resolving DNS ser...http://localhost:3000/issues/74962019-12-17T12:00:09ZNico Schotteliusnico.schottelius@ungleich.ch
<a name="Background"></a>
<h2 >Background<a href="#Background" class="wiki-anchor">¶</a></h2>
<p>We want VPN users or anyone to be able to select if they get NAT64 or not. Currently our bind decides based on the query source IP, whether to provide NAT64 or not and which prefix to use.</p>
<p>We want to have an alternative to that: DNS servers that <strong>always</strong> respond with NAT64 replies. So users can decide on their own if they want NAT64 or not.</p>
<a name="Details"></a>
<h2 >Details<a href="#Details" class="wiki-anchor">¶</a></h2>
<ul>
<li>2 VMs in place6
<ul>
<li>names: unbound1.place6.ungleich.ch, unbound2.place6.ungleich.ch</li>
</ul>
</li>
<li>OS: Alpine</li>
<li>fully cdist configured
<ul>
<li>create a new type __ungleich_unbound that accepts as parameter:
<ul>
<li>upstream dns servers</li>
<li>dns64 prefix</li>
</ul>
</li>
</ul>
</li>
<li>Use the DNS64 prefix from place6</li>
<li>Configure unbound to lookup names via 2a0a:e5c0:2:1::5 and 2a0a:e5c0:2:1::6
<ul>
<li>Because unbound is IPv6 only, it is not able to query IPv4 only domains</li>
</ul></li>
</ul>
<a name="documentation"></a>
<h2 >documentation<a href="#documentation" class="wiki-anchor">¶</a></h2>
<ul>
<li>After it has been implemented and verified by Balazs, please update the documentation on <a class="wiki-page" href="http://localhost:3000/projects/open-infrastructure/wiki/The_ungleich_DNS_infrastructure">The_ungleich_DNS_infrastructure</a></li>
</ul> Open Infrastructure - Task #7478 (Closed): Create script to create centos8 image suitable for ope...http://localhost:3000/issues/74782019-12-15T17:31:20ZNico Schotteliusnico.schottelius@ungleich.ch
<ul>
<li>centos8-build-opennebula.sh or similar</li>
<li>goes into ungleich-tools</li>
</ul> Open Infrastructure - Task #6694 (Closed): Setup matrix server and bridge matermost into ithttp://localhost:3000/issues/66942019-05-16T17:23:02ZNico Schotteliusnico.schottelius@ungleich.ch
<ul>
<li>... and maybe on the way switch to matrix by default?
<ul>
<li>unclear, needs testing before</li>
</ul>
</li>
<li>Requested by at least two users, last one on <a class="external" href="https://twitter.com/qryoxis/status/1129034888924016644">https://twitter.com/qryoxis/status/1129034888924016644</a></li>
<li>As usual everything via cdist</li>
</ul>