Task #8447
Updated by Nico Schottelius about 4 years ago
h2. Setup * -Create 3 Alpine nodes upgraded to edge, set hostnames- ** -Resize to at least 2 cores per node- * -Reserve networks- ** 2a0a:e5c0:2:12::/64 = pods ** 2a0a:e5c0:2:13::/64 = service * -Configure routers to accept BGP session (done: in cdist)- * -Deploy kubernetes on first node- * *-Deploy kube-router-*: fail, not IPv6 ready * Deploy calico: fail * Deploy cilium: fail kube-router * Create BGP peering * Verify BGP peering * Setup access to CEPH for persistent storage h2. OS commands <pre> echo node2 > /etc/hostname cat > /etc/resolv.conf << EOF nameserver 2a0a:e5c0:2:12:0:f0ff:fea9:c451 nameserver 2a0a:e5c0:2:12:0:f0ff:fea9:c45d search k8s.ungleich.ch EOF chattr +i /etc/resolv.conf cat > /etc/apk/repositories << EOF https://mirror.ungleich.ch/mirror/packages/alpine/edge/main https://mirror.ungleich.ch/mirror/packages/alpine/edge/community https://mirror.ungleich.ch/mirror/packages/alpine/edge/testing EOF apk upgrade apk add kubeadm kubelet kubectl docker rc-update add kubelet default rc-update add docker default echo 'net.ipv6.conf.default.forwarding=1' > /etc/sysctl.d/k8s.conf </pre> kubeadm.conf: <pre> localAPIEndpoint: advertiseAddress: 2a0a:e5c0:2:2:0:84ff:fe41:f263 --- apiVersion: kubeadm.k8s.io/v1beta2 kind: ClusterConfiguration networking: serviceSubnet: 2a0a:e5c0:2:13::/110 podSubnet: 2a0a:e5c0:2:12::/64 </pre> Init cluster: <pre> kubeadm init --config kubeadm.conf useradd -m k8s -s /bin/bash mkdir ~k8s/.kube cp /etc/kubernetes/admin.conf ~k8s/.kube/config chown -R k8s ~k8s </pre> h2. Take aways * docker sets ipv4 forwarding, but not ipv6 (needs manual sysctl entry) * Reachability by name (node1) w/o fqdn seems to be important h2. Current results * kube-router does not work out-of-the-box "too many colons" * calico does not work out of the box "calico-kube-controllers stays in pending" / no network provided