EVO-X3 from BIOS to Lemonade: a reproducible build

Release record: this guide describes evox3-llm-provisioner 2026.08.06.2. The repository keeps tested versions separate from the moving stable channel, and it does not claim that every Lemonade backend is a ROCm 7.14 userspace runtime.

The first seven articles in this series were about finding the limits of my EVO-X3: what the hardware exposes, how ROCm and Vulkan behave, which quantisations are useful and what survives a long context. I then had to turn that collection of working settings into something I could reinstall without retracing every mistake.

The result is the EVO X3 LLM provisioner repository. It starts with the firmware settings, builds a bootable Ubuntu image and leaves the installed machine on a tested update channel. This article explains the decisions in that automation, including the few things I deliberately refused to automate.

Start with the memory model, not a 96GB label

The most important firmware decision is counter-intuitive: select the smallest UMA frame-buffer or dedicated-graphics reservation the BIOS offers—typically Auto, 512 MiB or 1 GiB. Do not make a permanent 96 GiB GPU / 32 GiB CPU split.

Strix Halo’s CPU and Radeon 8060S physically share the same LPDDR5X memory. AMD’s current Strix Halo optimisation guide explains that GTT controls how much system RAM a user process may map into a GPU virtual address space. It is a dynamic mapping limit, not a second physical pool. A large firmware carve-out removes memory from Linux whether a model uses it or not.

This is the layout I kept on my 128 GiB EVO-X3:

The installer writes options ttm pages_limit=31457280 to its own managed modprobe file and regenerates initramfs. It refuses a conflicting pages_limit definition and never adds the deprecated amdgpu.gttsize option.

That layout was not chosen from theory alone. The DeepSeek V4 Flash target needs one 97,161 MiB managed allocation, and the target plus its optional draft model occupy more than 105 GiB before context and runtime allocations. The fixed 96/32 layout cannot run that monolithic profile and leaves the OS dangerously constrained.

I left the destructive choices outside the automation

The image builder follows Ubuntu’s Noble channel instead of baking one point-release filename into the project. It downloads Ubuntu’s SHA256SUMS and detached signature, verifies them with the Ubuntu archive keyring, selects the newest listed 24.04 live-server image, and verifies the ISO before modifying it.

The resulting USB contains the complete versioned provisioner and a NoCloud autoinstall seed. It handles the repeatable work but still asks for two decisions:

1. network configuration, when DHCP is not enough;
2. the target NVMe device that will be erased.

A reusable image has no business guessing the second one.

The builder also asks where SSH public keys should come from. It can read a GitHub username, an HTTPS key URL, or a local file. If no key is embedded, the installed machine opens a first-boot console wizard with the same choices plus pasted keys. The operator password is locked and password SSH remains disabled.

The build and write path is deliberately short:

sudo apt install curl gpgv python3 ubuntu-keyring xorriso
./scripts/build-autoinstall-iso.sh --github YOUR_GITHUB_USER
sudo ./scripts/write-usb.sh \
  dist/evox3-ubuntu-autoinstall.iso /dev/sdX

The USB writer displays device size, model, serial and transport, refuses the current system disk, and requires the exact whole-device path to be typed before it writes anything.

Ubuntu needs a property check, not merely a kernel number

After installation, the machine performs a full Ubuntu 24.04 upgrade and installs linux-oem-24.04. The kernel choice matters because ROCm backends on gfx1151 need KFD to export Context Wave Save/Restore sizes. Lemonade’s current gfx1151 kernel guidance names the two properties: cwsr_size and ctl_stack_size.

The provisioner tests those sysfs properties after reboot. A version string alone is weaker evidence because Ubuntu can backport the fix. It also rejects an installed amdgpu-dkms package: the separate DKMS module can shadow the corrected inbox amdgpu driver and break GPU discovery.

Only after the kernel and GTT checks pass does the second boot install Lemonade and its ROCm backend.

Lemonade supports gfx1151; that is not the same as “all ROCm 7.14”

This deserves a precise answer because my latest custom work has used a host ROCm 7.14 stack.

Lemonade does now support Strix Halo natively. Its llamacpp:rocm backend recognises gfx1151, and its experimental vllm:rocm path has a per-architecture gfx1151 bundle. Those managed assets nevertheless carry their own ROCm versions.

At this release boundary:

Lemonade deliberately reuses a system ROCm tree only when its expected major and minor runtime match. A host /opt/rocm 7.14 installation therefore does not magically turn a managed 7.13 or 7.12 asset into a 7.14 build.

The public installer uses Lemonade’s managed stable runtime by default. The repository separately records the exact custom 7.14 pins, checksums and promotion boundaries so the updater can advance them honestly when Lemonade publishes and validates an equivalent native bundle.

Two failed canaries changed the update rule

Two recent llama.cpp canaries are the reason the updater does more than launch a model and wait for a short answer.

Unmodified b10216 and b10290 could answer a short Qwen3-Coder request correctly, process a new uncached 8,191-token prompt, and then return the previous short answer. A fresh-process smoke had missed it. The root cause was a write-after-read race around device-owned pinned host input buffers on the integrated HIP path.

I therefore kept b10083 as the general custom backend. Qwen3-Coder alone may use b10290, and only with the narrow upstream PR 25863 host-buffer fix. That combination passed the sequential short-to-8K test, long decode, 4,775 focused ROCm backend operations and the throughput gate. A newer build does not qualify when it can return an answer from the wrong request.

Other evidence became operating policy:

• Lemonade is bound to localhost, LAN broadcast and telemetry are disabled, and remote access uses an SSH tunnel.
• max_loaded_models=1 is a safety control. A real DeepSeek/Phi overlap exhausted 128GiB and the kernel killed both runtimes.
• Qwen-specific KV cache, CPU masks, sparse routing and speculation settings are not applied globally.
• 32K and 64K context are comfortable; 128K is an exclusive, swap-active workload; 192K crossed the available-memory safety floor.
• DeepSeek’s accuracy profile keeps exact prefill, all six routed experts and target-only decode. Sparse/four-expert/DSpark configurations remain explicit speed-versus-quality experiments.

The update channel stays small enough to audit

Every installed machine checks one stable.env manifest in the GitHub repository. The updater parses it as restricted data rather than sourcing it as shell code. It verifies schema, channel, upgrade direction, minimum updater version, payload path, SHA-256 and the exact version inside the tagged release archive.

Releases live under /opt/evox3/releases/VERSION; the current symlink changes only after a complete copy. A successful refresh updates Ubuntu, reapplies the GTT/initramfs contract, upgrades Lemonade and its selected managed backends, and reapplies the performance service. Host-local choices remain in /etc/evox3/config.env.

The weekly timer runs in a maintenance window because a runtime update can restart Lemonade and unload a model. It can be disabled where another maintenance process owns that decision.

What the provisioner actually guarantees

The USB image is convenient. What matters more is the boundary the provisioner enforces:

• firmware leaves physical memory available to Linux;
• GTT makes that memory dynamically addressable by the GPU;
• the kernel proves the required KFD properties;
• Lemonade owns normal model and backend lifecycle;
• custom ROCm 7.14 work remains versioned rather than masquerading as a managed default;
• failed canaries remain visible and block unsafe promotion;
• updates are repeatable, checked and reversible at the release-directory level.

I can now rebuild the machine without relying on notes scattered across several tests, and another owner can see which settings are defaults and which remain experiments. The repository README starts at the BIOS checklist and follows the same path through to sudo evox3-verify.