Project

General

Profile

Task #7026

place9 hacking for getting ceph + netboot running

Added by Nico Schottelius 4 months ago. Updated 3 months ago.

Status:
In Progress
Priority:
High
Assignee:
Target version:
-
Start date:
08/02/2019
Due date:
% Done:

0%

Estimated time:
PM Check date:

Description

  • configure switches to put copper ports into internal vlan 22
  • Setup 2nd router
  • Configure vlan tagging
  • Install dhcpv4 for netboot
  • Create netboot image
  • Setup http server for loading netboot image
  • Boot servers with ceph
  • Add servers into dns
  • Setup and connect IPMI interfaces
  • Verify BIOS settings
  • Setup all switch ports for all servers:
    • Create channel groups / mlags on ports 1-16

Files

kernelpanicVFS.png (12.3 KB) kernelpanicVFS.png ll nu, 08/12/2019 04:28 PM

History

#1

Updated by Nico Schottelius 4 months ago

  • Description updated (diff)
#2

Updated by Nico Schottelius 4 months ago

Configuring port of mikrotik switch:

switch2-place9(config)#interface ethernet 19
switch2-place9(config-if-Et19)#switchport access vlan 22
! Access VLAN does not exist. Creating vlan 22
switch2-place9(config-if-Et19)#vlan 22
switch2-place9(config-vlan-22)#name place9-internal
switch2-place9(config-vlan-22)#write 
Copy completed successfully.
switch2-place9(config-vlan-22)#
#3

Updated by Nico Schottelius 4 months ago

getting lldp to find out where we are:

0x06:/etc/network# apk add lldpd
0x06:/etc/network# /etc/init.d/lldpd start
0x06:/etc/network# lldpcli show neigh
-------------------------------------------------------------------------------
LLDP neighbors:
-------------------------------------------------------------------------------
Interface:    eth4, via: LLDP, RID: 2, Time: 0 day, 00:00:04
  Chassis:     
    ChassisID:    mac 00:1c:73:c8:ba:cb
    SysName:      switch1-place9
    SysDescr:     Arista Networks EOS version 4.18.6M running on an Arista Networks DCS-7050S-52
    MgmtIP:       10.3.4.27
    Capability:   Bridge, on
    Capability:   Router, on
  Port:        
    PortID:       ifname Ethernet15
    TTL:          120
-------------------------------------------------------------------------------
Interface:    eth5, via: LLDP, RID: 1, Time: 0 day, 00:00:04
  Chassis:     
    ChassisID:    mac 00:1c:73:92:39:a7
    SysName:      switch2-place9
    SysDescr:     Arista Networks EOS version 4.18.6M running on an Arista Networks DCS-7050S-52
    MgmtIP:       10.3.4.28
    Capability:   Bridge, on
    Capability:   Router, on
  Port:        
    PortID:       ifname Ethernet15
    TTL:          120
-------------------------------------------------------------------------------
0x06:/etc/network# 
#4

Updated by Nico Schottelius 4 months ago

  • Description updated (diff)
#5

Updated by Nico Schottelius 4 months ago

  • Description updated (diff)
#6

Updated by Nico Schottelius 4 months ago

  • Description updated (diff)
#7

Updated by Nico Schottelius 4 months ago

  • Description updated (diff)
#8

Updated by ll nu 4 months ago

dnsmasq:

switch2 port 19

switchport access vlan 22

ip addr add 192.168.0.1/24 dev bond0.22
#9

Updated by ll nu 3 months ago

using this
https://blog.skunkw0rks.io/blog/2018/09/03/pxe-boot-alpine-linux-pt.-1/

could be useful: https://blog.haschek.at/2019/build-your-own-datacenter-with-pxe-and-alpine.html

/etc/lighttpd/lighttpd.conf default plus the following modified to :

server.document-root = "/srv/pxe"

router1:/srv/pxe# cat /etc/dnsmasq.conf
interface=bond0.22
dhcp-range=192.168.0.50,192.168.0.150,12h
enable-tftp
tftp-root=/srv/pxe/

router1:/srv/pxe# tree
.
├── gpxe.kpxe
├── initramfs-vanilla
├── pxe
│   └── gpxe-script
└── vmlinuz-vanilla

alpine boots but kernel panic

#10

Updated by ll nu 3 months ago

I dont know how I instruct the system to load everyting to ramfs, because I havent seen extra parameters for it in our existing ipxescript configuration, fstab, rc.local, (maybe rcS. have relevant stuff)

Ill try to reach out today, first on irc.

#11

Updated by Nico Schottelius 3 months ago

fstab can be empty
rc.local can be empty

writes:

#12

Updated by ll nu 3 months ago

ill try to unpack an alpine iso to see how does it boot to ramfs

#13

Updated by ll nu 3 months ago

update:
dhcpv6 netbooting with ipxe usb sticks is the current approach

#14

Updated by Nico Schottelius 3 months ago

Please document it and transfer the knowledge of how to create the usb
sticks to Sami (i.e. document first, have Sami try it and use it)

Also port the dhcpv6 configuration into cdist

writes:

#15

Updated by ll nu 3 months ago

  • Priority changed from Normal to High
#16

Updated by ll nu 3 months ago

///unfinished

git clone git://git.ipxe.org/ipxe.git
cd ipxe/src/
vi config/general.h
should look like this:
#define NET_PROTO_IPV4 /* IPv4 protocol /
#define NET_PROTO_IPV6 /
IPv6 protocol /
#undef NET_PROTO_FCOE /
Fibre Channel over Ethernet protocol */ <--- maybe change for fiber booting

vi config/console.h
#define CONSOLE_SERIAL

make bin/ipxe.iso

#17

Updated by Nico Schottelius 3 months ago

ipxe related:

I'd even enable
  • #define DOWNLOAD_PROTO_HTTPS
  • #define REBOOT_CMD
  • #define POWEROFF_CMD
  • #define PING_CMD

and for usb boot:

make bin/ipxe.usb
sudo dd if=bin/ipxe.usb of=/dev/sdb

Setting up host config:


root@line:~# ip addr add 2a0a:e5c1:101:b00d::42/64 dev enx801f02d64c51
root@line:/etc# sysctl -w net.ipv6.conf.all.forwarding=1
root@line:/etc# cat radvd.conf 
interface enx801f02d64c51
{
  AdvSendAdvert on;
  MinRtrAdvInterval 3;
  MaxRtrAdvInterval 10;
  AdvDefaultLifetime 600;

  # clients should check dhcp
  AdvOtherConfigFlag on;

  prefix 2a0a:e5c1:101:b00d::/64 { };

  RDNSS 2a0a:e5c0:2:1::5 2a0a:e5c0:2:1::6  { AdvRDNSSLifetime 6000; };
  DNSSL nico.ungleich.cloud {  AdvDNSSLLifetime 6000; } ;
};

root@line:/etc/dhcp# cat dhcpd6.conf 
option dhcp6.user-class code 15 = string;
option dhcp6.bootfile-url code 59 = string;
option dhcp6.client-arch-type code 61 = array of unsigned integer 16;

option dhcp6.name-servers 2a0a:e5c0::3;

if exists dhcp6.client-arch-type and
   option dhcp6.client-arch-type = 00:07 {
    option dhcp6.bootfile-url "http://[2a0a:e5c0:2:1::5]/ipxescript";
} else if exists dhcp6.user-class and
          substring(option dhcp6.user-class, 2, 4) = "iPXE" {
    option dhcp6.bootfile-url "http://[2a0a:e5c0:2:1::5]/ipxescript";
}

subnet6 2a0a:e5c1:101:b00d::/64 {}

With above settings the notebook next to me boots usb->ipxe->ipv6only

#18

Updated by Nico Schottelius 3 months ago

Note: we can/should remove the if clause, the first block is not needed

Also available in: Atom PDF