Project

General

Profile

The ungleich Matrix infrastructure » History » Revision 21

Revision 20 (Jin-Guk Kwon, 12/10/2020 10:21 AM) → Revision 21/22 (Timothée Floure, 02/19/2021 01:32 PM)

h1. The ungleich Matrix infrastructure 

 {{toc}} 

 *%{color:red}This document concerns the infrastructure side of our MaaS offer and is intended for ungleich staff. See [[Ungleich Matrix-as-a-Service (MaaS)]] page for end-user/customer documentation. Common operations can be found on: [[https://redmine.ungleich.ch/projects/open-infrastructure/wiki/Common_operations_on_X-as-a-Service]].%* 

 h2. Status 

 This document is *A DRAFT*. This service is not in production. Ask @fnux for details. 

 h2. Environment 

 Our Matrix deployments make use of the "Synapse (reference) Matrix homeserver":https://github.com/matrix-org/synapse/ and "Riot web client":https://about.riot.im/. We use Debian buster as base Operating system, leveraging the "matrix-synapse package from the buster-backports repository":https://packages.debian.org/buster-backports/matrix-synapse. The riot client (= static files) is directly fetched from "upstream releases on github":https://github.com/vector-im/riot-web/releases. 

 The matrix deployments run on ipv6only VMs, HTTP(S) traffic - including federation - being proxy by the [[How to use the IPv4-to-IPv6-Proxy|ungleich v4-to-v6 proxy]]. Federation is delegated using a @/.well-known/@ URI as described in [[Ungleich Matrix-as-a-Service (MaaS)|the customer FAQ]]. 

 !maas.jpg! 

 h2. Tooling 

 The whole MaaS setup is defined in the *manifest/matrix-as-a-service* of *dot-cdist* file, which wraps the *__ungleich_matrix* type. This type leverages: 

 * *__matrix_synapse* 
 * *__matrix_riot* 
 * *__ungleich_nginx_static_type* 
 * *__postgres_role* and *__postgres_database* from upstream cdist. 

 The "matterbridge":https://github.com/42wim/matterbridge application service can be deployed with the *__matterbridge* type. 

 *Matrix Federation Tester: https://federationtester.matrix.org/* 

 h2. Synapse Admin UI 

 * Available on: https://admin.matrix.ungleich.cloud 
 * Leverages https://github.com/Awesome-Technologies/synapse-admin/ 
 * Static deployment hence hosted on staticweb. 

 * Allows manual admin management of users and rooms. 
 * Use direct address of homeserver for login: @https://XXXX.matrix.ungleich.cloud@ 

 h2. Monitoring 

 Autodiscovered from consul in monitoring LAN. 

 h2. Ungleich Deployments 

 We maintain our own deployments alongside the customer MaaS, both for our own usage and for testing. 

 h3. matrix-staging.ungleich.ch 

 Staging instance used to tests the deployment pipeline and Matrix updates. 

 h3. matrix.ungleich.ch 

 Production instance for ungleich. Some rooms are bridged to the "chat.ungleich.ch":https://chat.ungleich.ch mattermost instance. The bridge makes use of the @#matterbridge:ungleich.ch@ local matrix user (i.e. not from LDAP) and @matterbridge@ mattermost user (linked to matterbridge AT ungleich ch). 

 h2. Customer Deployments 

 That's too sensitive to be public: "this way":https://redmine.ungleich.ch/projects/datacenterlight/wiki/Matrix_customer_deployments ! 
  
 h2. Shared TURN server 

 Coturn is running at matrix-turn.ungleich.ch. See @matrix-as-a-service@ manifest in dot-cdist for details. 

 

 h2. create mass users on matrix host 

 - info.txt 

 <pre> 
 FirstName LastName Email user_id1 pw1 
 FirstName LastName Email user_id2 pw2 
 FirstName LastName Email user_id3 pw3 
 </pre> 

 - creating user shall script 

 <pre> 
 while read A B C D E 
 do 
   pw=$E 
   name=$A" "$B 
   ad=$C 
   id="@"$D 
   data='{"password":"'${pw}'", "displayname": "'${name}'", "threepids": [ { "medium": "email", "address": "'${ad}'" }], "admin": false, "deactivated": false, "avatar_url": null }' 
   h='Authorization: Bearer <AccessToken>' 
   curl -v -X PUT -H "$h" -d "$data" http://localhost:8008/_synapse/admin/v2/users/$id:politikergespraeche.germanzero.org 
   sleep 2 
 done < info.txt 
 </pre> 

 

 h2. invite mass users on matrix host 

 - info.txt 

 <pre> 
 FirstName LastName Email user_id1 pw1 
 FirstName LastName Email user_id2 pw2 
 FirstName LastName Email user_id3 pw3 
 </pre> 

 - get user token 

 --> user : admin room user : to invite user, admin must be in room. 

 <pre> 
 curl -XPOST -d '{"type":"m.login.password", "user":"[user_id]", "password":"[user_password]"}' "http://localhost:8008/_matrix/client/r0/login" 
 </pre> 

 - inviting user script 

 <pre> 
 matrixserver="testbr.lab.ungleich.ch" 

 generate_post_data() 
 { 
   cat <<EOF 
 { 
   "user_id":"@$D:$matrixserver"  
 } 
 EOF 
 } 

 roomID=( 'roomID1 wihtout !' 'roomID2 wihtout !'    ) 
 errcode="M_LIMIT_EXCEEDED"  
 for rm_id in ${roomID[*]} 
 do 
    while read A B C D E 
    do 
       echo $rm_id 
       res2=""  
       res=$(curl -XPOST -d "$(generate_post_data)" "http://localhost:8008/_matrix/client/r0/rooms/%21$rm_id%3Apolitikergespraeche.germanzero.org/invite?access_token=<ACCESS_TOKEN>") 
       echo $res 
       #avoid error { "errcode": "M_LIMIT_EXCEEDED", "error": "Too Many Requests", "retry_after_ms": 2895 }  
       if [[ $res =~ $errcode ]];then 
         sleep 5 
         res2=$(curl -XPOST -d "$(generate_post_data)" "http://localhost:8008/_matrix/client/r0/rooms/%21$rm_id%3Apolitikergespraeche.germanzero.org/invite?access_token=<ACCESS_TOKEN>") 
       fi 

       if [[ $res2 =~ $errcode ]];then 
         echo "===error==="  
         echo $res2 
         exit 1 
       fi 
       sleep 1 
     done < info.txt 
 done 
 </pre> 

 .h2 Admin interface at admin.matrix.ungleich.cloud 

 It's based on synapse-admin:"https://github.com/Awesome-Technologies/synapse-admin" and is just a static JS app. It's deployed on staticweb.ungleich.ch. 

 h3. Update procedure 

 <pre> 
 # Clone sources 
 git clone https://github.com/Awesome-Technologies/synapse-admin.git 

 # Checkout latest tag 
 git checkout tags/x.y.z 

 # Fetch dependencies 
 yarn install 

 # Generate static files 
 yarn build 

 # Then: copy over content of build/ to staticweb 
 </pre>