A “small journey” into getting Google’s pixiecore to work with an API-server named waitron.

Intro

Ever since I started working in IT, I have an interest in setting up systems as quickly as possible. This must have find its origin way back in the late nineties where I had to do research at KPN (Dutch Telecom) on a HP-product named “Ignite” for HP-UX.
So when such a new tool shows up, I want to fiddle about with it, just as I did with tools like Cobbler, CoreOS Matchbox, etc.
It turned out waitron shines in the lack of documentation and is also not maintained since 2015, but I accepted the challenge to see if I could get this working. Here are some of the results.

Repositories

These GitHub-repositories are being used for the setup:

Hardware in use

Both pixiecore & waitron were installed on a small box running Ubuntu 16.04 Xenial, named dc2.internal. Ehr… DC2? Yes, see DC2 - Desktop Container Computer for Docker Containers.

I already have a DHCP-server (dnsmasq) running on a NAS and a FreeBSD-server acting as a TFTP/HTTP-server for a small set of distro’s, their kernels and ramdisks, etc.

On a MacBook Pro (Intel Core i7, 16GB RAM) I build the binary for waitron, with go version go1.11.2 darwin/amd64.

Testing with VMs was done on a Proxmox Virtual Environment (version 5.2-12), installed on a HP ProLiant MicroServer Gen8.

Install pixiecore

Go through the instructions at https://github.com/google/netboot/tree/master/pixiecore#debianubuntu on how to install pixiecore on Ubuntu / Debian.

The Ubuntu-box will run both pixiecore and waitron from the manually created directory /opt/waitron/.
pixiecore will run as an API-client and waitron as an API-server.

Build waitron

Make sure you have Go installed. I used to run brew install go on my Mac. You might check Getting Started.

I only need a Linux x86-64 executable file, and since my most powerful computer is a MacBook Pro, I used that building waitron:

$ go get github.com/jhaals/waitron.git
# github.com/jhaals/waitron
develop/gocode/src/github.com/jhaals/waitron/machine.go:84:40: multiple-value uuid.NewV4() in single-value context

FIX THIS ERROR
I made changes in two Go-files, but only the first one is really needed!

cd $GOPATH/src/github.com/haals/waitron
vi machine.go

In machine.go line 84 and in main.go line 254:

diff --git a/machine.go b/machine.go
index 7d2eeff..ddfdd52 100644
--- a/machine.go
+++ b/machine.go
@@ -81,7 +81,7 @@ func (m Machine) renderTemplate(template string, config Config) (string, error)
 // Posts machine macaddress to the forman proxy among with pxe configuration
 func (m Machine) setBuildMode(config Config) error {
        // Generate a random token used to authenticate requests
-       config.Tokens[m.Hostname]: uuid.NewV4().String()
+       config.Tokens[m.Hostname]: uuid.Must(uuid.NewV4()).String()
        log.Println(fmt.Sprintf("%s installation token: %s", m.Hostname, config.Tokens[m.Hostname]))
        // Add token to machine struct
        m.Token: config.Tokens[m.Hostname]

diff --git a/main.go b/main.go
index 35a23df..8a53094 100644
--- a/main.go
+++ b/main.go
@@ -251,5 +251,5 @@ func main() {
                })

        log.Println("Starting Server")
-       log.Fatal(http.ListenAndServe(":9090", handlers.LoggingHandler(os.Stdout, r)))
+       log.Fatal(http.ListenAndServe(":8090", handlers.LoggingHandler(os.Stdout, r)))
 }
}
  • The line reading “uuid” needs to be made compatible with the latest release of github.com/satori/go.uuid. If not you’ll get the error ./machine.go:84:40: multiple-value uuid.NewV4() in single-value context.
  • The Ubuntu-box also runs Prometheus, listening on port 9090, so I had to alter this (the port can not be overruled in the config…).

As stated I want a Linux binary, so I ran the build like this:

env GOOS=linux GOARCH=amd64 go build .

This left me with a 9.8M sized binary named waitron. I copied this binary to the Ubuntu-box: scp waitron dc2:/opt/waitron/

Configurations

File config.yaml. Altered, because the default_cmdline would offer a wrongly formatted URL:

templatepath: templates
machinepath: machines
baseurl: http://dc2.internal:8090
params:
    apt_hostname: "nl.archive.ubuntu.com"
    apt_path: "/ubuntu/"
    dns_nameservers: "192.168.178.3"
    ntp_server: "nl.pool.ntp.org"
default_cmdline: "interface=auto url={{ BaseURL }}/template/preseed/{{ Hostname }}/{{ Token }} ramdisk_size=10800 root=/dev/rd/0 rw auto hostname={{ Hostname }} console-setup/ask_detect=false console-setup/layout=USA console-setup/variant=USA keyboard-configuration/layoutcode=us localechooser/translation/warn-light=true localechooser/translation/warn-severe=true locale=en_US"
default_kernel: linux
default_image_url: http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/installer-amd64/current/images/netboot/ubuntu-installer/amd64/
default_initrd: initrd.gz

As you might have noticed, I have added some undocumented variables here under “params”, like apt_hostname and apt_path.
In the file templates/preseed.j2 we can see (Jinja2) entries like {{config.Params.apt_hostname}} and {{config.Params.apt_path}}, but if these are not defined anywhere, you’ll end up with a useless preseed-file!

OK. So the above defaults are Ubuntu-focused, where I want a Debian Stretch system.
Have a look at file machines/linux001.internal.yaml. NOTE: you need a unique per-system YAML-file!
Here the defaults from the above (global) configuration are overruled:

hostname: linux001.internal
operatingsystem: "Stretch"
preseed: debian9.j2
finish: finish.j2
#BaseURL is this waitron instance url, it is read from the config file
#Hostname is the hostname for this very host, it is read from this file
#Token is generated at runtime by waitron
cmdline: "interface=auto url={{ BaseURL }}/template/preseed/{{ Hostname }}/{{ Token }} locale=en_US.UTF-8 debian/priority=critical vga=normal debian-installer/keymap=us console-keymaps-at/keymap=us keyboard-configuration/xkb-keymap=us console-setup/layoutcode=en_US localechooser/translation/warn-light=true localechooser/translation/warn-severe=true console-setup/ask_detect=false netcfg/get_hostname={{ Hostname }} netcfg/get_domain=internal FRONTEND_BACKGROUND=original"
image_url: http://bsd002.internal:81/debian/stretch/amd64/
kernel: linux
initrd: initrd.gz
network:
  - name: eth0
    ipaddress: 192.168.178.141
    macaddress: 00:de:ad:90:74:60
    netmask: 255.255.255.0
    gateway: 192.168.178.1
params:
    foo: False
    bar: "Hello world"
    static: True

In this file I have added the parameter static: True, in case you need a static defined network on the system, as used in the preseed with {% if machine.Params.static == "True" %}.

In the end I decided to not bother with Jinja2-templating yet, so I added a Debian-specific preseed-file to templates/debian9.j2 and left it with the Jinja2-extension intact.

Start up

In seperate terminals on the Ubuntu-box:

  1. Start pixiecore: pixiecore api http://dc2.internal:8090/ -d --dhcp-no-bind
    In API-mode; pointing to the URL where waitron will be found; in debug-mode and to handle DHCP traffic without binding to the DHCP server port.

  2. Start waitron: env CONFIG_FILE=config.yaml ./waitron -d

Set build mode

First, a new system needs to be set in “build mode” before it is offered anything from pixiecore.
You’ll have to send waitron some specific API-calls. I prefer to use HTTPie for that (but curl might do as well):

$ http PUT http://dc2.internal:8090/build/linux001.internal
OK

Make sure you’re using the same name as used for the machine-definition, otherwise you’ll get an error like this:

$ http PUT http://dc2.internal:8090/build/linux001
Unable to find host definition for linux001

To check the status of that specific system I use :

$ http http://dc2.internal:8090/status
{
    "linux001.internal": "Installing"
}

Now it is time to start a system to be provisioned. Like a VM or bare-metal.

When the system is finished you’ll have to remove the server from build mode (even though I’m not completely certain about why doing that manually):

$ http http://dc2.internal:8090/done/linux001.internal/73f267c2-1ea3-40c2-8c5e-2f134a5e8ee7
OK

And finally check the system’s status:

$ http http://dc2.internal:8090/status
{
    "linux001.internal": "Installed"
}

You’re done!

<Rant/>: personally I think these parts don’t work as complete, quick and easy as using Cobbler and the Cobbler provider for Terraform I have setup.
Especially waitron feels incomplete and (indeed) unmaintained.

So the next stop will probably be trying out something like Shoelaces. :-)

Output

Some output that pixiecore could show. Notice the DHCP / TFTP / HTTP “headers”:

[Init] Starting Pixiecore goroutines
[DHCP] Ignoring packet from 1c:ab:a7:90:7f:c4: packet is DHCPREQUEST, not DHCPDISCOVER
[DHCP] Got valid request to boot 00:de:ad:90:74:60 (IA32)
[DHCP] Offering to boot 00:de:ad:90:74:60
[DHCP] Ignoring packet from 00:de:ad:90:74:60: packet is DHCPREQUEST, not DHCPDISCOVER
[TFTP] Sent "00:de:ad:90:74:60/4" to 192.168.178.52:59898
[DHCP] Got valid request to boot 00:de:ad:90:74:60 (IA32)
[DHCP] Offering to boot 00:de:ad:90:74:60
[DHCP] Got valid request to boot 00:de:ad:90:74:60 (IA32)
[DHCP] Offering to boot 00:de:ad:90:74:60
[DHCP] Ignoring packet from 00:de:ad:90:74:60: packet is DHCPREQUEST, not DHCPDISCOVER
[HTTP] Get bootspec for 00:de:ad:90:74:60 took 8.875419ms
[HTTP] Construct ipxe script for 00:de:ad:90:74:60 took 6.901628ms
[HTTP] Sending ipxe boot script to 192.168.178.52:26697
[HTTP] Writing ipxe script to 00:de:ad:90:74:60 took 52.11µs
[HTTP] handleIpxe for 00:de:ad:90:74:60 took 17.060643ms
[HTTP] Sent file "VRVYX8ribqXFhB4MxlMmE5kJMl8CpmNVg6O2z906baIMuGtqVml9z96G2HbRlMpxpFlsxN9YJ-z9RCTSpGYuJnMz4ufv-lhFtWLmx0ul6mVvVU2QriQY2gq9hRA=" to 192.168.178.52:26697
[HTTP] Sent file "4IT2hGZrET7LULh5a0FbAsrAf8JsGM9ilR3IQyeVxwP-7szfoxuqNnJOWVNNhYh3Bx9G_FsePTgvOqJP4NgoGXJtPMYeMrcxU1fZUc7wHpVSiWw6vrOgDfZfpzfuAwii" to 192.168.178.52:26697
[DHCP] Ignoring packet from 00:de:ad:90:74:60: not a PXE boot request (missing option 93)
[DHCP] Ignoring packet from 00:de:ad:90:74:60: packet is DHCPREQUEST, not DHCPDISCOVER

And the same for waitron. Notice the instalation token, being used in the manual API-calls:

2018/12/01 17:26:00 Starting Server
2018/12/01 17:26:04 linux001.internal installation token: 73f267c2-1ea3-40c2-8c5e-2f134a5e8ee7
192.168.178.21 - - [01/Dec/2018:17:26:04 +0100] "PUT /build/linux001.internal HTTP/1.1" 200 2
192.168.178.199 - - [01/Dec/2018:17:27:02 +0100] "GET /v1/boot/00:de:ad:90:74:60 HTTP/1.1" 200 697
192.168.178.199 - - [01/Dec/2018:17:27:03 +0100] "GET /v1/boot/00:de:ad:90:74:60 HTTP/1.1" 200 697
192.168.178.199 - - [01/Dec/2018:17:27:03 +0100] "GET /v1/boot/00:de:ad:90:74:60 HTTP/1.1" 200 697
192.168.178.52 - - [01/Dec/2018:17:27:44 +0100] "GET /template/preseed/linux001.internal/73f267c2-1ea3-40c2-8c5e-2f134a5e8ee7 HTTP/1.1" 200 2879
192.168.178.21 - - [01/Dec/2018:17:31:28 +0100] "GET /status HTTP/1.1" 200 34

Other resources