This repository acts as the bootable setup and provisioning engine for Raspberry Pi nodes joining the Open Cream Cheese (OCC) cluster. It contains templates and configurations for Cloud-Init metadata and PXE Network Booting (DHCP Proxy + TFTP + NFS).

Architecture Overview

graph TD
    subgraph Host Network
        Router[Primary DHCP Router]
        PXEHost[Bagel-Shop Host]
    end

    subgraph "Raspberry Pi Nodes"
        Pi3[RPi Node 3]
        Pi2[RPi Node 2]
        Pi1[RPi Node 1]
        Pi3 --- Pi2 --- Pi1
    end

    subgraph "Traditional Servers"
        Srv3[Server 3]
        Srv2[Server 2]
        Srv1[Server 1]
        Srv3 --- Srv2 --- Srv1
    end

    PXEHost -->|Provisioning| Pi3
    PXEHost -->|Provisioning| Srv3
    
    Router -->|IP Assignment| Pi3
    Router -->|IP Assignment| Srv3

    Pi1 -->|Join| K8s[MicroK8s Cluster]
    Srv1 -->|Join| K8s

1. Cloud-Init Configuration

We pre-configure nodes using Cloud-Init. The templates are organized in the cloud-init/ directory:

user-data.yaml: Core system setup configuration.
network-config.yaml: Ethernet interface parameters.

Provisioning Features:

User Setup: Creates an administrator user steve with passwordless sudo rights and installs specified public SSH keys.
Container Networking: Installs iptables, load modules (overlay, br_netfilter), and sets required Kubernetes sysctl configurations.
iSCSI / NFS Support: Installs open-iscsi and nfs-common. Generates a unique iSCSI Initiator IQN based on the Ethernet MAC address of the Raspberry Pi (iqn.2026-08.net.fairfaxmedia.kube:node.<mac-address>) to ensure no volume mounting conflicts arise in Longhorn.
RPi cgroup configurations: Automates adding cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 to /boot/firmware/cmdline.txt to satisfy Kubernetes resource requirements.
MicroK8s Bootstrapping: Installs MicroK8s automatically via snap.

2. Flashing Bootable USB Drives / SD Cards

To flash a standard Ubuntu Server/Raspberry Pi OS image to local boot media and dynamically inject the Cloud-Init configs:

Locate your USB block device (e.g. /dev/sdb).
Download the Ubuntu Server arm64 image (e.g. ubuntu-24.04-preinstalled-server-arm64+raspi.img.xz).

Run the flash script:

sudo ./scripts/flash-usb.sh /dev/sdb ./ubuntu-24.04-preinstalled-server-arm64+raspi.img.xz

What it does:

Safely unmounts and flashes the base image to the target USB block device.
Mounts the FAT32 boot partition (labeled system-boot) and writes user-data and network-config to the root of the boot partition.
Syncs data and cleanly unmounts the device.

3. Setting up Network Boot (PXE Netboot)

To boot your Raspberry Pi cluster diskless (without individual SD cards or USB drives) by network-booting them from a central host computer:

Run the PXE setup script on your PXE Host VM/Server:
```
sudo ./scripts/setup-pxe-host.sh
```

What it does:

Installs dnsmasq, nfs-kernel-server, and dependencies.
Configures dnsmasq as a DHCP Proxy (meaning it will not interfere with your home router's DHCP IP assignment, but will provide network boot loaders to nodes requesting PXE boot).
Enables integrated TFTP server pointing to /srv/tftp.
Creates NFS root export path /srv/nfs/client-rpi and registers it in /etc/exports.

Manual Netboot Setup:

Flash files to TFTP: Extract the contents of the /boot partition of an Ubuntu Server arm64 image and copy them directly to /srv/tftp (specifically start4.elf, fixup4.dat, vmlinuz, initrd.img, and device trees).
Copy Root File System: Mount the main EXT4 partition of the Ubuntu Server image and rsync its contents (excluding boot files) directly into /srv/nfs/client-rpi.
Link Boot configurations:
- Copy netboot/config.txt to /srv/tftp/config.txt.
- Copy netboot/cmdline.txt to /srv/tftp/cmdline.txt. Edit cmdline.txt to ensure the nfsroot IP address matches your PXE Host machine's IP address.
Boot your Raspberry Pi: Ensure the Raspberry Pi is connected via Ethernet to the network. Power it on, and it will load its bootloader from TFTP and mount / via NFS dynamically!