Project

General

Profile

Actions
Copy link

Task #6255

closed

Find the right settings for kubernetes in ipv6 only settings

Added by Nico Schottelius over 5 years ago. Updated 4 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
Nico Schottelius
Target version:
-
Start date:
12/23/2018
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

Testing on

  • 2a0a:e5c0:2:12:400:f0ff:fea9:c401
    • --pod-cidr 2a0a:e5c0:102:3::/64
    • --service-cidr 2a0a:e5c0:102:6::/64
  • 2a0a:e5c0:2:12:400:f0ff:fea9:c402
    • --pod-cidr 2a0a:e5c0:102:4::/64
    • --service-cidr 2a0a:e5c0:102:7::/64
  • 2a0a:e5c0:2:12:400:f0ff:fea9:c403
    • --pod-cidr 2a0a:e5c0:102:5::/64

Current findings below.

Follow up reading on:

Possible options / next steps

  • trying older docker version (< 17)
    • working around the docker/ipv6 issue
  • using calico instead of bridge+host-local
    • working around the docker/ipv6 issue
  • trying rkt instead of docker
    • working around the docker/ipv6 issue

401 / with most options

root@kube-master:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --apiserver-advertise-address=2a0a:e5c0:2:12:400:f0ff:fea9:c401 --service-cidr 2a0a:e5c0:102:6::/64

Result: api server not starting

402 / plain kubeadm init

  • apiserver starts

403 / with --pod-network-cidr and bridge cni

root@kube-node2:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:5::/64 
root@kube-node2:~# cat /etc/cni/net.d/10-bridge_v6.conf 
{
  "cniVersion": "0.3.0",
  "name": "mynet",
  "type": "bridge",
  "bridge": "cbr0",
  "isDefaultGateway": true,
  "ipMasq": true,
  "hairpinMode": true,
  "ipam": {
    "type": "host-local",
    "ranges": [
      [
        {
          "subnet": "2a0a:e5c0:102:5::/64",
          "gateway": "2a0a:e5c0:102:5::1" 
        }
      ]
    ]
  }
}
  • apiserver starts
  • permission denied when trying to assign an IPv6 address
    • known bug in newer docker versions, which DISABLE ipv6 with a systctl!
  • should be fixed in 0.7.x release of kubernetes-cni

k8s1 / service + pod cidr + calico

root@k8s1:~# kubeadm init --pod-network-cidr 2a0a:e5c0:102:3::/64 --service-cidr 2a0a:e5c0:102:6::/64

Result: failure at kubeadm init

k8s2 / podcidr + calico

Not yet changing the yaml files of calico, but finding out how far the setup goes w/o tuning/changing

kubeadm init --pod-network-cidr 2a0a:e5c0:102:4::/64

kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml
kubectl apply -f https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

Result:

  • kubeadm init works
  • Applying config for calico works
  • calico pod is started
  • Errors not finding nodename

k8s3 / podcidr + calico + calico guide

After kubeadm init w/ pod cidr:

kubectl apply -f \
https://docs.projectcalico.org/v3.4/getting-started/kubernetes/installation/hosted/etcd.yaml
kubectl apply -f \
https://docs.projectcalico.org/v3.4/getting-started/kubernetes/installation/hosted/calico.yaml

Result: getting same error of missing nodename

k8s1+k8s2 / podcidr + calico + calico ipv6 guide

result:

  • etcd need to have a service ip
  • specifying --service-cidr makes kubeadm init fail
Actions
Copy link

Also available in: Atom PDF