Task #9565
closed
Select a CI/CD for deploying helm charts/docker containers etc.
Added by Nico Schottelius over 2 years ago.
Updated over 2 years ago.
Description
- Basically: git push && pipeline that does the rest
- Input from your experiences is appreciated
Choices¶
Flux v2¶
- Overall nice
- Does not cover the build phase
- Has nice multi cluster support
- Unclear on how / where to store the output
- K8S support seems to be fragile
Jenkins¶
- The "standard"
- Very heavy (4GB+ memory)
- Old
- Static workers (easy to configure via k8s)
- Seems to be fast and easy to setup
- Python based
- Website and documentation down as of 2021-08-08
TL;DR: Does not even start in an IPv6 k8s cluster
Non-working installation:
helm upgrade --install --set server.service.type=ClusterIP,server.ingress.enabled=false gocd gocd/gocd
Gitlab¶
TL;DR: has a lot included, maybe too much
- Is heavy to maintain without containers.
- Highly integrated
- Can use k8s workers, can use docker
- Widely deployed
- Huge and tricky to maintain
- Docker:
- Gitlab/k8s seems to be strongly tied to terraform
- Not suitable for bare metal
- Rather complicated / big ecosystem
- Design to be cloud native
- Dependencies nicely solved
Argo flow¶
- Output is very S3 centered
- We could use this, even though it seems overkill
- This might be a practical requirement
- Might be able to ignore this feature
- Argo flow tries to access /var/run/docker.sock directly - which does not exist for crio based environments
- MountVolume.SetUp failed for volume "docker-sock" : hostPath type check failed: /var/run/docker.sock is not a socket file
Flows¶
DNS Update¶
Questions:
- Should we create a stand-alone zone repository?
- Would be very small
- Can only clone head/last commit
- If using git pull inside the container, we need to pass along credentials
Flow v1¶
- We change a zone file in git and push it somewhere
- A new helm chart is being created
- (maybe in between) bump the chartversion field?
- only if knot was able to run it?
- The new helm chart is uploaded to the chartmuseum
- The pods/services are notified about a new version
- How?
- Configmap change?
- git pull?
Flow v2: pull from git repo¶
- The helm chart is given a git repo (+possible secret)
- The pod tries reloading every minute
- if checkconf works: restart
- else: reject
- A webhook in gitea might be used to trigger the DNS server instances
- Faster deploy
- Question is where to, whether we have 1 hook per cluster, etc.
Disadvantage: need to build our own container (?)
- In theory a custom container could do that in a pod
Flow v3: push pipeline¶
- In theory we want every zone change to create a new version number
- Actually we have this already with the git commit
Nothing to be done here.
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
I prefer using travisCI or circleCI, they are light and easy to maintained.
I think too that they works good with Kubernetes
I do not have any preference -- I would probably go with Jenkins just because it is widely adopted.
GoCD looks promising also, though I've not used it personally.
- Project changed from 45 to Open Infrastructure
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
Argoflow notes¶
loop:
dag:
tasks:
- name: print-message
template: whalesay
arguments:
parameters:
- name: message
value: "{{item}}"
withItems:
- "hello world"
- "goodbye world"
Sequence
dag:
tasks:
- name: print-message
template: whalesay
arguments:
parameters:
- name: message
value: "{{item}}"
withSequence:
count: 5
- A steps template allows you to run a series of steps in sequence.
- A suspend template allows you to automatically suspend a workflow, e.g. while waiting on manual approval, or while an external system does some work.
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: exit-handler-
spec:
entrypoint: main
templates:
- name: main
dag:
tasks:
- name: a
template: whalesay
onExit: tidy-up
- name: whalesay
container:
image: docker/whalesay
- name: tidy-up
container:
image: docker/whalesay
command: [ cowsay ]
args: [ "tidy up!" ]
Parameters
- name: main
inputs:
parameters:
- name: message
container:
image: docker/whalesay
command: [ cowsay ]
args: [ "{{inputs.parameters.message}}" ]
Chaining in & out via a file:
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: parameters-
spec:
entrypoint: main
templates:
- name: main
dag:
tasks:
- name: generate-parameter
template: whalesay
- name: consume-parameter
template: print-message
dependencies:
- generate-parameter
arguments:
parameters:
- name: message
value: "{{tasks.generate-parameter.outputs.parameters.hello-param}}"
- name: whalesay
container:
image: docker/whalesay
command: [ sh, -c ]
args: [ "echo -n hello world > /tmp/hello_world.txt" ]
outputs:
parameters:
- name: hello-param
valueFrom:
path: /tmp/hello_world.txt
- name: print-message
inputs:
parameters:
- name: message
container:
image: docker/whalesay
command: [ cowsay ]
args: [ "{{inputs.parameters.message}}" ]
Workflowtemplate
- Basically a workflow stored in k8s that can be reused
CronWorkflow
Webhooks
- Very easy to create w/ input
- Using workflowtemplate
- Description updated (diff)
GoCD test¶
helm upgrade --install --set server.service.type=ClusterIP,server.ingress.enabled=false gocd gocd/gocd
Hangs in creating
[16:00] nb3:generic% kubectl describe pods gocd-server-5b8fb6b58f-54qc8
Name: gocd-server-5b8fb6b58f-54qc8
Namespace: default
Priority: 0
Node: server60/2a0a:e5c0:13:0:225:b3ff:fe20:3736
Start Time: Sun, 08 Aug 2021 15:58:49 +0200
Labels: app=gocd
component=server
pod-template-hash=5b8fb6b58f
release=gocd
Annotations: cni.projectcalico.org/podIP: 2a0a:e5c0:13:e1:ddc1:7d11:9a1f:95a5/128
cni.projectcalico.org/podIPs: 2a0a:e5c0:13:e1:ddc1:7d11:9a1f:95a5/128
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/gocd-server-5b8fb6b58f
Containers:
gocd-server:
Container ID:
Image: gocd/gocd-server:v21.2.0
Image ID:
Port: 8153/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Liveness: http-get http://:8153/go/api/v1/health delay=90s timeout=1s period=15s #success=1 #failure=10
Readiness: http-get http://:8153/go/api/v1/health delay=90s timeout=1s period=15s #success=1 #failure=10
Environment:
GOCD_PLUGIN_INSTALL_kubernetes-elastic-agents: https://github.com/gocd/kubernetes-elastic-agents/releases/download/v3.7.1-230/kubernetes-elastic-agent-3.7.1-230.jar
GOCD_PLUGIN_INSTALL_docker-registry-artifact-plugin: https://github.com/gocd/docker-registry-artifact-plugin/releases/download/v1.1.0-104/docker-registry-artifact-plugin-1.1.0-104.jar
Mounts:
/docker-entrypoint.d from goserver-vol (rw,path="scripts")
/godata from goserver-vol (rw,path="godata")
/home/go from goserver-vol (rw,path="homego")
/preconfigure_server.sh from config-vol (rw,path="preconfigure_server.sh")
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-2m9th (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
config-vol:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: gocd
Optional: false
goserver-vol:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: gocd-server
ReadOnly: false
kube-api-access-2m9th:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 100s default-scheduler Successfully assigned default/gocd-server-5b8fb6b58f-54qc8 to server60
Normal Pulled 97s kubelet Container image "gocd/gocd-server:v21.2.0" already present on machine
Normal Created 97s kubelet Created container gocd-server
Normal Started 97s kubelet Started container gocd-server
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Description updated (diff)
- Status changed from In Progress to Closed
We use argocd + argo workflow.
Also available in: Atom
PDF