Project

General

Profile

Actions

Task #8852

closed

Investigate matrix.ungleich.ch slowness

Added by Timothée Floure over 1 year ago. Updated 9 months ago.

Status:
Closed
Priority:
Normal
Target version:
-
Start date:
02/05/2021
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

matrix.ungleich.ch is so slow it becomes unusable, I'm currently investigating and:

  • Disabling Presence
  • Increase cache sizes
  • Configure Synapse with multiple workers

Related issues 1 (0 open1 closed)

Blocks Open Infrastructure - Task #8888: Meta-Issue for Matrix late-winter 2021 cleanupClosedTimothée Floure02/11/2021

Actions
Actions #1

Updated by Timothée Floure over 1 year ago

  • Status changed from New to In Progress
Actions #2

Updated by Timothée Floure over 1 year ago

I spend most of my morning playing around with this synapse issues. I moved some processing out of the main process and increased a bit the cache but there's still high latency on registering new events: in-depth investigation including a deeper look at what's going with synapse's internals and potential slowness with PGSQL are required.

I'd also like to:

  • Revamp the whole monitoring pipeline which is utterly broken right now: autodiscovery with consul and a monitoring LAN sounds nice in theory, but implies too many flying components in practice. I want to drop it in favor of good old cdist-managed static configuration files. We also need some alerts (= instead of ranting customers..).
  • Clean up the synapse type (written in my early cdist days ~ 1 year ago...) and merge it into cdist-contrib.
  • Add the multi-worker setup to cdist. It doesn't matter if we add more core with a single synapse process...
  • Make our internal cdist manifest more concise, it's somewhat messy right now.

This means a few work-days. Your thoughts @Nico Schottelius?

Actions #3

Updated by Timothée Floure over 1 year ago

Note: I commented out the cdist config for our matrix HS, since the changes I applied this morning were all done by hand.

Actions #4

Updated by Timothée Floure over 1 year ago

+ write/deploy and run some DB cleanup tools, since it's growing exponentially...

Actions #5

Updated by Timothée Floure over 1 year ago

  • Blocks Task #8888: Meta-Issue for Matrix late-winter 2021 cleanup added
Actions #6

Updated by Nico Schottelius 9 months ago

  • Status changed from In Progress to Closed
Actions

Also available in: Atom PDF