Task #7026
closedplace9 hacking for getting ceph + netboot running
0%
Description
- configure switches to put copper ports into internal vlan 22
- Setup 2nd router
- Configure vlan tagging
- Install dhcpv4 for netboot
- Create netboot image
- Setup http server for loading netboot image
- Boot servers with ceph
- Add servers into dns
- Setup and connect IPMI interfaces
- Verify BIOS settings
- Setup all switch ports for all servers:
- Create channel groups / mlags on ports 1-16
Files
Updated by Nico Schottelius over 5 years ago
Configuring port of mikrotik switch:
switch2-place9(config)#interface ethernet 19 switch2-place9(config-if-Et19)#switchport access vlan 22 ! Access VLAN does not exist. Creating vlan 22 switch2-place9(config-if-Et19)#vlan 22 switch2-place9(config-vlan-22)#name place9-internal switch2-place9(config-vlan-22)#write Copy completed successfully. switch2-place9(config-vlan-22)#
Updated by Nico Schottelius over 5 years ago
getting lldp to find out where we are:
0x06:/etc/network# apk add lldpd 0x06:/etc/network# /etc/init.d/lldpd start 0x06:/etc/network# lldpcli show neigh ------------------------------------------------------------------------------- LLDP neighbors: ------------------------------------------------------------------------------- Interface: eth4, via: LLDP, RID: 2, Time: 0 day, 00:00:04 Chassis: ChassisID: mac 00:1c:73:c8:ba:cb SysName: switch1-place9 SysDescr: Arista Networks EOS version 4.18.6M running on an Arista Networks DCS-7050S-52 MgmtIP: 10.3.4.27 Capability: Bridge, on Capability: Router, on Port: PortID: ifname Ethernet15 TTL: 120 ------------------------------------------------------------------------------- Interface: eth5, via: LLDP, RID: 1, Time: 0 day, 00:00:04 Chassis: ChassisID: mac 00:1c:73:92:39:a7 SysName: switch2-place9 SysDescr: Arista Networks EOS version 4.18.6M running on an Arista Networks DCS-7050S-52 MgmtIP: 10.3.4.28 Capability: Bridge, on Capability: Router, on Port: PortID: ifname Ethernet15 TTL: 120 ------------------------------------------------------------------------------- 0x06:/etc/network#
Updated by ll nu over 5 years ago
dnsmasq:
switch2 port 19
switchport access vlan 22
ip addr add 192.168.0.1/24 dev bond0.22
Updated by ll nu over 5 years ago
using this
https://blog.skunkw0rks.io/blog/2018/09/03/pxe-boot-alpine-linux-pt.-1/
could be useful: https://blog.haschek.at/2019/build-your-own-datacenter-with-pxe-and-alpine.html
/etc/lighttpd/lighttpd.conf default plus the following modified to :
server.document-root = "/srv/pxe"
router1:/srv/pxe# cat /etc/dnsmasq.conf
interface=bond0.22
dhcp-range=192.168.0.50,192.168.0.150,12h
enable-tftp
tftp-root=/srv/pxe/
router1:/srv/pxe# tree
.
├── gpxe.kpxe
├── initramfs-vanilla
├── pxe
│ └── gpxe-script
└── vmlinuz-vanilla
alpine boots but kernel panic
Updated by ll nu over 5 years ago
I dont know how I instruct the system to load everyting to ramfs, because I havent seen extra parameters for it in our existing ipxescript configuration, fstab, rc.local, (maybe rcS. have relevant stuff)
Ill try to reach out today, first on irc.
Updated by Nico Schottelius over 5 years ago
fstab can be empty
rc.local can be empty
redmine@ungleich.ch writes:
Updated by ll nu over 5 years ago
- File kernelpanicVFS.png kernelpanicVFS.png added
ill try to unpack an alpine iso to see how does it boot to ramfs
Updated by ll nu over 5 years ago
update:
dhcpv6 netbooting with ipxe usb sticks is the current approach
Updated by Nico Schottelius over 5 years ago
Please document it and transfer the knowledge of how to create the usb
sticks to Sami (i.e. document first, have Sami try it and use it)
Also port the dhcpv6 configuration into cdist
redmine@ungleich.ch writes:
Updated by ll nu over 5 years ago
///unfinished
git clone git://git.ipxe.org/ipxe.git
cd ipxe/src/
vi config/general.h
should look like this:
#define NET_PROTO_IPV4 /* IPv4 protocol /
#define NET_PROTO_IPV6 / IPv6 protocol /
#undef NET_PROTO_FCOE / Fibre Channel over Ethernet protocol */ <--- maybe change for fiber booting
vi config/console.h
#define CONSOLE_SERIAL
make bin/ipxe.iso
Updated by Nico Schottelius over 5 years ago
ipxe related:
I'd even enable- #define DOWNLOAD_PROTO_HTTPS
- #define REBOOT_CMD
- #define POWEROFF_CMD
- #define PING_CMD
and for usb boot:
make bin/ipxe.usb
sudo dd if=bin/ipxe.usb of=/dev/sdb
Setting up host config:
root@line:~# ip addr add 2a0a:e5c1:101:b00d::42/64 dev enx801f02d64c51 root@line:/etc# sysctl -w net.ipv6.conf.all.forwarding=1 root@line:/etc# cat radvd.conf interface enx801f02d64c51 { AdvSendAdvert on; MinRtrAdvInterval 3; MaxRtrAdvInterval 10; AdvDefaultLifetime 600; # clients should check dhcp AdvOtherConfigFlag on; prefix 2a0a:e5c1:101:b00d::/64 { }; RDNSS 2a0a:e5c0:2:1::5 2a0a:e5c0:2:1::6 { AdvRDNSSLifetime 6000; }; DNSSL nico.ungleich.cloud { AdvDNSSLLifetime 6000; } ; }; root@line:/etc/dhcp# cat dhcpd6.conf option dhcp6.user-class code 15 = string; option dhcp6.bootfile-url code 59 = string; option dhcp6.client-arch-type code 61 = array of unsigned integer 16; option dhcp6.name-servers 2a0a:e5c0::3; if exists dhcp6.client-arch-type and option dhcp6.client-arch-type = 00:07 { option dhcp6.bootfile-url "http://[2a0a:e5c0:2:1::5]/ipxescript"; } else if exists dhcp6.user-class and substring(option dhcp6.user-class, 2, 4) = "iPXE" { option dhcp6.bootfile-url "http://[2a0a:e5c0:2:1::5]/ipxescript"; } subnet6 2a0a:e5c1:101:b00d::/64 {}
With above settings the notebook next to me boots usb->ipxe->ipv6only
Updated by Nico Schottelius over 5 years ago
Note: we can/should remove the if clause, the first block is not needed