The ungleich kubernetes infrastructure » History » Revision 3
Revision 2 (Nico Schottelius, 07/01/2021 03:31 PM) → Revision 3/222 (Nico Schottelius, 07/01/2021 03:48 PM)
h1. The ungleich kubernetes infrastructure {{toc}} h2. Status This document is **pre-production** h2. General architecture and components overview * All k8s clusters are IPv6 only * We use BGP peering to propagate podcidr and serviceCidr networks to our infrastructure * The main / public repository is "ungleich-k8s":https://code.ungleich.ch/ungleich-public/ungleich-k8s h2. ungleich kubernetes infrastructure v1 We are using the following components: * Calico as a CNI + BGP * "kubernetes-secret-generator":https://github.com/mittwald/kubernetes-secret-generator for creating secrets * "ungleich-certbot":https://hub.docker.com/repository/docker/ungleich/ungleich-certbot to get certificates * "rook with ceph rbd + cephfs":https://rook.io/ for storage ** rbd for almost everything, *ReadWriteOnce* ** cephfs for smaller things, multi access *ReadWriteMany* h2. Open Issues / To be discussed * "Maybe add Autoscaling support?":https://github.com/kubernetes-sigs/metrics-server ** https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ ** https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/ * Certainly deploy in-cluster monitoring ** "prometheus-operator":https://github.com/prometheus-operator/prometheus-operator CR ** "kube-prometheus":https://github.com/prometheus-operator/kube-prometheus complete example, based on prometheus-operator ** "kubernetes dashboard":https://github.com/kubernetes/dashboard