Open Infrastructure

{{toc}}

h1. The ungleich hardware maintenance guide

This guide describes common operations on hardware we use.

h2. Using the ungleich-hardware container in kubernetes and docker

To manage hardware on server1 in kubernetes, you can use:

<pre>

apiVersion: v1

kind: Pod

metadata:

  name: ungleich-hardware

spec:

  containers:

  - name: ungleich-hardware

    image: harbor.ungleich.svc.p10.k8s.ooo/ungleich-public/ungleich-hardware:0.0.3

    args:

    - sleep

    - "1000000"

    volumeMounts:

      - mountPath: /dev

        name: dev

    securityContext:

      privileged: true

  nodeSelector:

    kubernetes.io/hostname: "server1"

  volumes:

    - name: dev

      hostPath:

        path: /dev

</pre>

To use it wit docker:

<pre>

docker run -v /dev:/dev --privileged -ti harbor.ungleich.svc.p10.k8s.ooo/ungleich-public/ungleich-hardware:0.0.3

</pre>

h2. APU Bios Update

* Download the correct bios from https://pcengines.github.io/

** Check whether it's apu1/2/3/4 before downloading

* Install flashrom

* "Flash bios using flashrom":https://github.com/pcengines/apu2-documentation/blob/master/docs/firmware_flashing.md

** @flashrom -w THEROMFILE -p internal@

h2. APU Serial and bootloader configuration

* Ensure that the bootloader has "console=ttyS0,115200" configured

* Ensure that there is a getty running on serial

* Use grub-bios as the bootloader

** Install using @grub-install /dev/sda@

h2. Updating the Perc H800 SAS controller

* @wget 'https://dl.dell.com/FOLDER03292738M/3/SAS-RAID_Firmware_XKF5X_LN_12.10.7-0001_A13.BIN?uid=4b8a2506-f4d4-46a9-ab19-3c2a5008a782&fn=SAS-RAID_Firmware_XKF5X_LN_12.10.7-0001_A13.BIN' -O  SAS-RAID_Firmware_XKF5X_LN_12.10.7-0001_A13.BIN@

* chmod u+x SAS-RAID_Firmware_XKF5X_LN_12.10.7-0001_A13.BIN

* ./SAS-RAID_Firmware_XKF5X_LN_12.10.7-0001_A13.BIN

h2. HP servers disk management

* See also http://www.datadisk.co.uk/html_docs/redhat/hpacucli.htm

Required kernel modules:

<pre>

sg

cciss

</pre>

Show all drives/controller overview:

<pre>

hpacucli ctrl all show config

hpacucli ctrl slot=0 pd all show

</pre>

Add a disk as raid0:

<pre>

hpacucli ctrl slot=0 create type=ld drives=1I:1:1 raid=0

</pre>

h2. Dell servers disk management

Listing all disks:

<pre>

megacli -PDList -aALL

</pre>

Adding disks:

<pre>

megacli -CfgLdAdd -r0 [Enclosure Device ID:slot] -aX (X : host is 0. md-array is 1)

# Sample call, if enclosure and slot are KNOWN (aka not N/A)

megacli -CfgLdAdd -r0 [32:0] -a0

# Sample call, if enclosure is N/A

megacli -CfgLdAdd -r0 [:0] -a0

</pre>

Remove cache of disks that are not in the server anymore:

<pre>

megacli -DiscardPreservedCache -Lall -aAll

</pre>

Remove foreign configurations on foreign disks

<pre>

megacli -CfgForeign -Clear -aAll

</pre>

Do both in many cases:

<pre>

megacli -DiscardPreservedCache -Lall -aAll

megacli -CfgForeign -Clear -aAll

</pre>

h2. SEE ALSO

* [[Managing OpenWRT]]

* [[The_ungleich_ceph_handbook]]