Open Infrastructure

Task #6255

Updated by Nico Schottelius almost 6 years ago

Testing on  

 * 2a0a:e5c0:2:12:400:f0ff:fea9:c401 
 * 2a0a:e5c0:2:12:400:f0ff:fea9:c402  
 * 2a0a:e5c0:2:12:400:f0ff:fea9:c403  

 Current findings below. 

 h2. Follow up reading on: 

 * "calico/ipv6":https://docs.projectcalico.org/v3.4/usage/ipv6 
 * "docker/ipv6 issue":https://github.com/containernetworking/cni/issues/531 
 * "k8s/apt repo":https://packages.cloud.google.com/apt/ 

 h2. Possible options / next steps 

 * trying older docker version (< 17) 
 ** working around the docker/ipv6 issue 
 * using calico instead of bridge+host-local 
 ** working around the docker/ipv6 issue 
 * trying rkt instead of docker 
 ** working around the docker/ipv6 issue 



 h2. 401 / with most options 

 <pre> 
 root@kube-master:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --apiserver-advertise-address=2a0a:e5c0:2:12:400:f0ff:fea9:c401 --service-cidr 2a0a:e5c0:102:6::/64 
 </pre> 


 Result: api server not starting 

 h2. 402 / plain kubeadm init  

 * apiserver starts 

 h2. 403 / with --pod-network-cidr and bridge cni 

 <pre> 
 root@kube-node2:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:5::/64  
 root@kube-node2:~# cat /etc/cni/net.d/10-bridge_v6.conf  
 { 
   "cniVersion": "0.3.0", 
   "name": "mynet", 
   "type": "bridge", 
   "bridge": "cbr0", 
   "isDefaultGateway": true, 
   "ipMasq": true, 
   "hairpinMode": true, 
   "ipam": { 
     "type": "host-local", 
     "ranges": [ 
       [ 
         { 
           "subnet": "2a0a:e5c0:102:5::/64", 
           "gateway": "2a0a:e5c0:102:5::1" 
         } 
       ] 
     ] 
   } 
 } 

 </pre> 

 * apiserver starts 
 * permission denied when trying to assign an IPv6 address 
 ** known bug in newer docker versions, which DISABLE ipv6 with a systctl! 
 * should be fixed in 0.7.x release of kubernetes-cni 

 h2. k8s1 / service + pod cidr + calico 

 <pre> 
 root@k8s1:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --service-cidr 2a0a:e5c0:102:6::/64 
 </pre> 

 Result: failure at kubeadm init 

 h2. k8s2 / podcidr + calico 

 Not yet changing the yaml files of calico, but finding out how far the setup goes w/o tuning/changing 

 <pre> 
 kubeadm init --pod-network-cidr 2a0a:e5c0:102:4::/64 

 kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml 
 kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml 
 </pre> 

 Result: 

 * kubeadm init works testing in progress 
 * Applying config for calico works 
 * calico pod is started 
 * Errors not finding nodename 

Back