Task #6255
Updated by Nico Schottelius almost 6 years ago
Testing on
* 2a0a:e5c0:2:12:400:f0ff:fea9:c401
** --pod-cidr 2a0a:e5c0:102:3::/64
** --service-cidr 2a0a:e5c0:102:6::/64
* 2a0a:e5c0:2:12:400:f0ff:fea9:c402
** --pod-cidr 2a0a:e5c0:102:4::/64
* 2a0a:e5c0:2:12:400:f0ff:fea9:c403
** --pod-cidr 2a0a:e5c0:102:5::/64
Current findings below.
h2. Follow up reading on:
* "calico/ipv6":https://docs.projectcalico.org/v3.4/usage/ipv6
* "docker/ipv6 issue":https://github.com/containernetworking/cni/issues/531
* "k8s/apt repo":https://packages.cloud.google.com/apt/
h2. Possible options / next steps
* trying older docker version (< 17)
** working around the docker/ipv6 issue
* using calico instead of bridge+host-local
** working around the docker/ipv6 issue
* trying rkt instead of docker
** working around the docker/ipv6 issue
h2. 401 / with most options
<pre>
root@kube-master:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --apiserver-advertise-address=2a0a:e5c0:2:12:400:f0ff:fea9:c401 --service-cidr 2a0a:e5c0:102:6::/64
</pre>
Result: api server not starting
h2. 402 / plain kubeadm init
* apiserver starts
h2. 403 / with --pod-network-cidr and bridge cni
<pre>
root@kube-node2:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:5::/64
root@kube-node2:~# cat /etc/cni/net.d/10-bridge_v6.conf
{
"cniVersion": "0.3.0",
"name": "mynet",
"type": "bridge",
"bridge": "cbr0",
"isDefaultGateway": true,
"ipMasq": true,
"hairpinMode": true,
"ipam": {
"type": "host-local",
"ranges": [
[
{
"subnet": "2a0a:e5c0:102:5::/64",
"gateway": "2a0a:e5c0:102:5::1"
}
]
]
}
}
</pre>
* apiserver starts
* permission denied when trying to assign an IPv6 address
** known bug in newer docker versions, which DISABLE ipv6 with a systctl!
* should be fixed in 0.7.x release of kubernetes-cni
h2. k8s1 / service + pod cidr + calico
<pre>
root@k8s1:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --service-cidr 2a0a:e5c0:102:6::/64
</pre>
Result: failure at kubeadm init
h2. k8s2 / podcidr + calico
Not yet changing the yaml files of calico, but finding out how far the setup goes w/o tuning/changing
<pre>
kubeadm init --pod-network-cidr 2a0a:e5c0:102:4::/64
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml
</pre>
Result:
* kubeadm init works
* Applying config for calico works
* calico pod is started
* Errors not finding nodename
h2. k8s3 / podcidr + calico + calico guide
* following https://docs.projectcalico.org/v3.4/getting-started/kubernetes/ instead of https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
** different versions, shows etcd for calico
After kubeadm init w/ pod cidr:
<pre>
kubectl apply -f \
https://docs.projectcalico.org/v3.4/getting-started/kubernetes/installation/hosted/etcd.yaml
kubectl apply -f \
https://docs.projectcalico.org/v3.4/getting-started/kubernetes/installation/hosted/calico.yaml
</pre>
Result: getting same error of missing nodename
h2. k8s1 / podcidr + calico + calico ipv6 guide
* based on https://docs.projectcalico.org/v3.4/usage/ipv6
* calico.yaml from https://docs.projectcalico.org/v3.4/getting-started/kubernetes/installation/hosted/calico.yaml