Task #8447
Updated by Nico Schottelius about 4 years ago
h2. Setup
* -Create 3 Alpine nodes upgraded to edge, set hostnames-
** -Resize to at least 2 cores per node-
* -Reserve networks-
** 2a0a:e5c0:2:12::/64 = pods
** 2a0a:e5c0:2:13::/64 = service
* -Configure routers to accept BGP session (done: in cdist)-
* -Deploy kubernetes on first node-
* Deploy kube-router
* Create BGP peering
* Verify BGP peering
* Setup access to CEPH for persistent storage
h2. OS commands
<pre>
echo node2 > /etc/hostname
cat > /etc/resolv.conf << EOF
nameserver 2a0a:e5c0:2:12:0:f0ff:fea9:c451
nameserver 2a0a:e5c0:2:12:0:f0ff:fea9:c45d
search k8s.ungleich.ch
EOF
chattr +i /etc/resolv.conf
cat > /etc/apk/repositories << EOF
https://mirror.ungleich.ch/mirror/packages/alpine/edge/main
https://mirror.ungleich.ch/mirror/packages/alpine/edge/community
https://mirror.ungleich.ch/mirror/packages/alpine/edge/testing
EOF
apk upgrade
apk add kubeadm kubelet kubectl docker
rc-update add kubelet default
rc-update add docker default
echo 'net.ipv6.conf.default.forwarding=1' > /etc/sysctl.d/k8s.conf
</pre>
kubeadm.conf:
<pre>
localAPIEndpoint:
advertiseAddress: 2a0a:e5c0:2:2:0:84ff:fe41:f263
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
networking:
serviceSubnet: 2a0a:e5c0:2:13::/110
podSubnet: 2a0a:e5c0:2:12::/64
</pre>
Init cluster:
<pre>
kubeadm init --config kubeadm.conf
useradd -m k8s -s /bin/bash
mkdir ~k8s/.kube
cp /etc/kubernetes/admin.conf ~k8s/.kube/config
chown -R k8s ~k8s
</pre>
h2. Take aways
* docker sets ipv4 forwarding, but not ipv6 (needs manual sysctl entry)
* Reachability by name (node1) w/o fqdn seems to be important