liminix-rebuild: use `-f` flag to reboot effectively

My AP does not reboot upon `reboot` but `reboot -f`… why?

Signed-off-by: Raito Bezarius <masterancpp@gmail.com>

.gitignore

@@ -6,3 +6,4 @@ result-*
 _build
 *-secrets.nix
 examples/static-leases.nix
+/doc/hardware.rst

NEWS

@@ -31,5 +31,52 @@ Upstream changes that have led to incompatible Liminix changes are:
 * newer U-Boot version
 * util-linux can now be built (previously depended on systemd)
 
+2024-01-30
 
+New port! Thanks to Arnout Engelen <arnout@bzzt.net>, Liminix
+now runs on the TP-Link Archer AX23
+
+2024-02-12
+
+* We now build wifi drivers (mac80211) from the same kernel source as
+the running kernel, instead of using drivers from the linux-backports
+project.  This may be a regression on some devices that depend on
+OpenWrt patches for wireless functionality: if you have a device that
+used to work and now doesn't, refer to OpenWrt
+package/kernel/mac80211/patches/ to see if there's something in there
+that needs to be applied.
+
+* in general, we build kernel modules (e.g. for nftables) at the same
+time as the kernel itself instead of expecting to be able to build
+them afterwards as though they were "out of tree". Refer to commit
+b9c0d93670275e69df24902b05bf4aa4f0fcbe96 for a fuller explanation
+of how this simplifies things.
+
+2024-02-13
+
+So that we can be more consistent about services that would like their
+state to be preserved across boots (assuming a writable filesystem)
+these changes have been made
+
+* /run/service-state has been moved to /run/services/outputs
+ to better reflect what it's used for
+* /run/services/state is either a symlink to /persist/services/state
+ (if there's a writeable fs on /persist) or a directory (if there
+ isn't)
+
+The change will lose your ssh host key(s) unless you copy them from
+the old location to the new one before rebooting into the new system
+
+   mkdir -m 02751 -p /run/services/state/dropbear
+   cp /persist/secrets/dropbear/* /run/services/state/dropbear
+
+The `output`, `mkoutputs` functions defined by ${serviceFns}
+have been updated for the new location.
+
+2024-02-16
+
+New (or at least, previously unreported) port! Liminix now runs on the
+Turris Omnia and has been serving my family's internet needs for most
+of this week.  Thanks to NGI0 Entrust and the NLnet Foundation for
+sponsoring this development (and funding the hardware)
 

README.md

@@ -33,7 +33,7 @@ functioning version, see [the CI system](https://build.liminix.org/jobset/limini
 Documentation is in the [doc](doc/) directory. You can build it
 by running
 
-    nix-shell -p sphinx --run "make -C doc html"
+    nix-shell -p sphinx --run "make -C doc hardware.rst html"
 
 Rendered documentation corresponding to the latest commit on `main`
 is published to [https://www.liminix.org/doc/](https://www.liminix.org/doc/)

THOUGHTS.txt

@@ -3845,3 +3845,269 @@ store. Also probably it should mkdir $prefix/persist. Also it needs to
 create $prefix/boot: it's too late to do that with `activate`
 because u-boot will need it to exist in order to load the initramfs
 that runs activate
+
+Thu Jan 11 23:36:47 GMT 2024
+
+squashfs rootfsType doesn't rebuild when the kernel config is changed
+
+Mon Jan 22 19:04:45 GMT 2024
+
+setenv serverip 10.0.0.1
+setenv ipaddr 10.0.0.8
+compraddr=0x01000000
+tftpboot ${compraddr} recovery.img.lzma
+setexpr writeaddr ${filesize} + $compraddr
+lzmadec ${compraddr} $writeaddr
+usb start
+usb dev 0
+wdt dev watchdog@20300
+wdt stop
+usb write ${writeaddr} 0 ${filesize}
+
+
+Thu Jan 25 11:55:36 GMT 2024
+
+openwrt:
+CONFIG_BROADCOM_PHY=m
+CONFIG_FIXED_PHY=y
+CONFIG_GENERIC_PHY=y
+CONFIG_IP17XX_PHY=m    ?
+CONFIG_MARVELL_PHY=y
+CONFIG_MVSW61XX_PHY=y  ?
+CONFIG_RTL8366RB_PHY=m ?
+CONFIG_RTL8366S_PHY=m ?
+CONFIG_RTL8367B_PHY=m ?
+CONFIG_SWPHY=y
+CONFIG_USB_PHY=y
+
+CONFIG_FIXED_PHY=y
+CONFIG_GENERIC_PHY=y
+CONFIG_MARVELL_PHY=y
+CONFIG_PHY_MVEBU_A3700_COMPHY=y
+CONFIG_PHY_MVEBU_A38X_COMPHY=y
+CONFIG_SWPHY=y
+#
+
+Sat Jan 27 18:14:13 GMT 2024
+
+To make the recovery system (and tftpboot generally) more useful, it
+would be good to resize the root fs on boot. Need to do this before
+anything that writes to it
+
+Mon Jan 29 21:50:59 GMT 2024
+
+something is corrupted in the uncompressed rootfs
+
+
+$ head -c $(printf "%d" 0x2be0000) rootfs | sha1sum
+142571fe0436c18191727d1d4c2fd32163c1f2e1  -
+=> sha1sum 0x1000000 2be0000
+sha1 for 01000000 ... 03bdffff ==> 142571fe0436c18191727d1d4c2fd32163c1f2e1
+
+but!
+
+$  head -c $(printf "%d" 0x2bf0000) rootfs | sha1sum
+7aa004ba87c6772bade491fbade164e2dfe100f9  -
+=> sha1sum 0x1000000 2bf0000
+sha1 for 01000000 ... 03beffff ==> 1a0923a94784d0c0b86006c5e6fff1649770dad3
+
+something is trashing something in the range 03be0000 - 03beffff
+or else it's not being decompressed properly
+
+pxefile_addr_r=0x1900000
+ramdisk_addr_r=0x2200000
+    scriptaddr=0x1800000
+fdt_addr_r=0x2000000
+fdtcontroladdr=7fb19b30
+fdtfile=armada-385-turris-omnia.dtb
+     fdt_high=0x10000000
+  initrd_high=0x10000000
+kernel_addr_r=0x1000000
+              0x1700000;
+              0x10000000
+
+Sun Feb  4 11:55:00 GMT 2024
+
+restructuredtext headings:
+
+https://devguide.python.org/documentation/markup/#sections
+
+
+####### chapter (one per filename)
+*******
+=======
+-------
+
+Mon Feb  5 09:57:52 GMT 2024
+
+Before calling the Omnia "done" I'd like to get it to the point that
+I can actually use it as a CPE. This means
+
+- writing something down about how we handle static addresses
+  - hosts that need static ipv6 can configure it themselves as ::n
+    where n is a small number. this won't clash with slaac
+  - the `hosts` param to dnsmasq can specify static ipv4
+
+- dealing with port forwards and allowed incoming in the firewall
+
+- would be quite cool to run sniproxy instead of forwarding to
+  loaclhost (extra credit)
+
+Sat Feb 10 18:23:54 GMT 2024
+
+ARGH KERNEL
+
+You can't define CONFIG_NETFILTER=y in a monolithic kernel and expect
+later to separately build some modules that use it, because there are
+a bunch of symbols that only get defined if certain other CONFIG
+options are set at the time that the monolithic kernel is built.
+
+https://github.com/torvalds/linux/blob/master/net/netfilter/core.c#L689
+
+Another example is
+https://github.com/torvalds/linux/blob/master/include/linux/netdevice.h#L160
+- if you decide after building the kernel that you're going to build
+some wireless modules, you can't do that without rebuilding the kernel
+so that it knows to expect them
+
+The moral of the story seems to be: if you have a compiled Linux kernel source tree and you change some symbol from "is not set" to m and then run  make modules, you cannot in general expect that newly compiled  module to work.
+
+AP advertised VHT without HT, disabling HT/VHT/HE
+
+TODO
+
+- [done] support kernel version as parameter to builder pkgs/kernel/default.nix
+- [done] extract the change in how module loading works from omnia device config,
+  and fix the other thing that uses it
+- [axed] wlan module to take 'backported' as a parameter
+  half of the omnia conditionalConfig can go into the module
+- [done] upgrade omnia to kernel v6
+- figure out what mdns we need for local hostname resolution
+  (maybe bridging lan/wlan)?
+- [DONE] slow wifi because "AP advertised VHT without HT, disabling HT/VHT/HE"
+- [DONE] add local domain to secrets
+- run sniproxy instead of forwarding
+- [test] forward some port to loaclhost 22 for inbound ipv4 ssh
+
+
+Mon Feb 12 21:50:35 GMT 2024
+
+# find  /run/service-state/dhcp6c.wan.link.pppoe/address/
+/run/service-state/dhcp6c.wan.link.pppoe/address/
+/run/service-state/dhcp6c.wan.link.pppoe/address/2001-8b0-1111-1111-0-ffff-51bb-4cf2_LFoo015bSsM
+/run/service-state/dhcp6c.wan.link.pppoe/address/2001-8b0-1111-1111-0-ffff-51bb-4cf2_LFoo015bSsM/valid
+/run/service-state/dhcp6c.wan.link.pppoe/address/2001-8b0-1111-1111-0-ffff-51bb-4cf2_LFoo015bSsM/preferred
+/run/service-state/dhcp6c.wan.link.pppoe/address/2001-8b0-1111-1111-0-ffff-51bb-4cf2_LFoo015bSsM/len
+/run/service-state/dhcp6c.wan.link.pppoe/address/2001-8b0-1111-1111-0-ffff-51bb-4cf2_LFoo015bSsM/address
+#
+
+valid 7199 preferred 3599
+
+Tue Feb 13 19:44:57 GMT 2024
+
+Before we put this back live, would be good to
+
+[done] 1) move the leases file into /persist
+
+I think we'll do /persist/service/<name>/ and change ssh to use the same
+scheme.
+
+we could put mkpersist() in serviceFns which would check for /persist
+and return a directory in /persist/service/ or /run/service-state
+
+(will something bad happen if we use /run/service-state? it will also
+expose the thingy as an output, but whether it's accessible that way
+will depend on whether there's a writable fs or not, which is unexpected)
+
+: rename service-state to  /run/services/outputs
+: on boot
+:  if /persist
+:    create /persist/services/state and symlink /run/services/state to it
+:  else create /run/services/state
+
+
+[done] 2) maybe change the local domain back to .lan?  setting up
+  systemd-networkd with search domains is an awful faff
+
+[done] 3) work out what to do with incoming ssh from wan
+
+- For noetbook and thinkpad we have a vpn anyway so can expect to
+  reach loaclhost directly using ipv6
+
+- stop ssh from ever trying to get to our ipv4 address.
+  - we could get rid of A record for loaclhost.telent.net but
+    there are a bunch of CNAMES pointing at it for web servers.
+  - we could reject incoming connections to tcp4 port 22 in firewall
+    and then there is a clear signal to Dont Do That Then
+
+- for emergency use, dnat ipv4 2200 and 2201 to rotuer and loaclhost
+
+Tue Feb 13 22:31:03 GMT 2024
+
+* the reason we can't reboot is that there is a service to add each
+lan device to the bridge which does ifwait $dev running, which doesn't
+return until there's something plugged in. So s6-rc hangs indefinitely
+until the lan switch is fully populated. This is definitely a "next
+milestone" thing.
+
+* another example of "thing that depends on other thing but which it
+  is actually OK if neither of them happen" might be "mount a
+  filesystem if there is a usb mass storage device attached"
+
+* I don't know if failover also fits into the model we don't quite
+  have. LTE route depends on pppoe not being healthy
+
+we can have services (or bundles) that aren't part of the default target,
+and plumb them into events of some kind (netlink?) to bring them up/down?
+
+we can use s6-rc instanced services:
+https://skarnet.org/software/s6/instances.html
+
+"s6-instance-create and s6-instance-delete are relatively expensive operations, because they have to recursively copy or delete directories and use the synchronization mechanism with the instance supervisor, compared to s6-instance-control which only has to send commands to already existing supervisors. If you are going to turn instances on and off on a regular basis, it is more efficient to keep the instance existing and control it with s6-instance-control than it is to repeatedly create and delete it. "
+
+Probably we need something that reads netlink messages and converts
+them to a format that we can use to control services. Is there a
+benefit to using services here and not just running commands? it means
+the system state change we desire will stay changed.
+
+TODO items not to lose track of
+
+- speed testing (iperf)
+- make gl-ar750 tftpboot build again
+- finish belkin
+- install sniproxy
+- is there something simple we can do to make it reboot again?
+- turn rotuer,extneder examples into "profiles" that don't embed
+  hardware specifics
+
+Thu Feb 15 11:50:56 GMT 2024
+
+1) to make tftpboot work with old bootm implementations we need
+
+- compressed root
+- uncompressed root
+- kernel with dtb
+ dtb needs to know where uncompressed rootfs is and how big
+
+2) if the image is a zImage (arm32) or an Image (arm64) we have to stick
+with the three-arg bootz, and the dtb has to be lower in ram than the kernel
+
+Fri Feb 16 15:43:32 GMT 2024
+
+DHCP6c refresh is still wrong. We get updates for an address that
+hasn't changed prefix or length, when the expiry times have changed,
+and we can't action that by remove;add because remove will wipe out
+any routes through the interface but add won't put them back
+
+We can use "change" for both adds and changes, but we need to know that
+a change is not a delete
+
+The "identity" of an address is the address itself: kernel won't
+let you add the same address with two different prefixes.
+
+Keeping it simple, we could call "change" on every address in the
+new-addresses list and "del" on every address in old-addresses
+that is no longer in new-addresses
+
+If the upstream has changed length, "ip addr change" is ignored,
+so it needs to be in deleted as well as added/changed

ci.nix

@@ -11,6 +11,7 @@ let
   devices = [
     "gl-ar750" "gl-mt300n-v2" "gl-mt300a"
     "qemu" "qemu-aarch64" "qemu-armv7l"
+    "tp-archer-ax23"
   ];
   vanilla = ./vanilla-configuration.nix;
   for-device = name:

default.nix

@@ -67,6 +67,7 @@ in {
       go-l2tp
       min-copy-closure
       fennelrepl
+      lzma
     ];
   };
 }

devices/belkin-rt3200/default.nix

@@ -57,8 +57,8 @@
     kernel = {
       src = pkgs.pkgsBuildBuild.fetchurl {
         name = "linux.tar.gz";
-        url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.71.tar.gz";
-        hash = "sha256-yhO2cXIeIgUxkSZf/4aAsF11uxyh+UUZu6D1h92vCD8=";
+        url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+        hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
       };
       extraPatchPhase = ''
           ${pkgs.openwrt.applyPatches.mediatek}
@@ -144,6 +144,17 @@
         # SERIAL_8250_NR_UARTS="3";
         # SERIAL_8250_RUNTIME_UARTS="3";
         SERIAL_OF_PLATFORM="y";
+
+        # Must enble hardware watchdog drivers. Else the device reboots after several seconds
+        WATCHDOG = "y";
+        MEDIATEK_WATCHDOG = "y";
+      };
+      conditionalConfig = {
+        WLAN= {
+          MT7615E = "m";
+          MT7622_WMAC = "y";
+          MT7915E = "m";
+        };
       };
     };
     boot = {
@@ -165,12 +176,9 @@
     hardware =
       let
         openwrt = pkgs.openwrt;
-        mac80211 =  pkgs.mac80211.override {
-          drivers = [
-            "mt7615e"
-            "mt7915e"
-          ];
-          klibBuild = config.system.outputs.kernel.modulesupport;
+        mac80211 =  pkgs.kmodloader.override {
+          targets = ["mt7615e" "mt7915e"];
+          inherit (config.system.outputs) kernel;
         };
       in {
         ubi = {

devices/families/qemu.nix

@@ -7,8 +7,8 @@
     kernel = {
       src = pkgs.pkgsBuildBuild.fetchurl {
         name = "linux.tar.gz";
-        url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.71.tar.gz";
-        hash = "sha256-yhO2cXIeIgUxkSZf/4aAsF11uxyh+UUZu6D1h92vCD8=";
+        url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+        hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
       };
       config = {
         MTD = "y";

devices/gl-ar750/default.nix

@@ -71,9 +71,10 @@
           cp $blobdir/board.bin  $out/ath10k/QCA9887/hw1.0/
         '';
       };
-      mac80211 = pkgs.mac80211.override {
-        drivers = ["ath9k" "ath10k_pci"];
-        klibBuild = config.system.outputs.kernel.modulesupport;
+      mac80211 = pkgs.kmodloader.override {
+        targets = ["ath9k" "ath10k_pci"];
+        inherit (config.system.outputs) kernel;
+        dependencies = [ ath10k_cal_data ];
       };
       ath10k_cal_data =
         let
@@ -132,7 +133,7 @@
             };
             wlan5 = link.build {
               ifname = "wlan1";
-              dependencies = [ mac80211 ath10k_cal_data ];
+              dependencies = [ ath10k_cal_data mac80211  ];
             };
           };
       };
@@ -152,8 +153,8 @@
       kernel = {
         src = pkgs.pkgsBuildBuild.fetchurl {
           name = "linux.tar.gz";
-          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.71.tar.gz";
-          hash = "sha256-yhO2cXIeIgUxkSZf/4aAsF11uxyh+UUZu6D1h92vCD8=";
+          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+          hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
         };
 
         # Mainline linux 5.19 doesn't have device-tree support for
@@ -161,6 +162,8 @@
         # OpenWrt kernel patches
         extraPatchPhase = ''
           ${openwrt.applyPatches.ath79}
+          sed -i.bak -e '\,include <linux/hw_random.h>,a #include <linux/gpio/driver.h>'  drivers/net/wireless/ath/ath9k/ath9k.h # context reqd for next patch
+          patch -p1 <  ${openwrt.src}/package/kernel/mac80211/patches/ath9k/552-ath9k-ahb_of.patch
         '';
 
         config = {
@@ -211,14 +214,21 @@
           WATCHDOG = "y";
           ATH79_WDT = "y";  # watchdog timer
 
-          # this is all copied from nixwrt ath79 config. Clearly not all
-          # of it is device config, some of it is wifi config or
-          # installation method config or ...
-
           EARLY_PRINTK = "y";
 
           PRINTK_TIME = "y";
         };
+        conditionalConfig = {
+          WLAN = {
+            WLAN_VENDOR_ATH = "y";
+            ATH_COMMON = "m";
+            ATH9K = "m";
+            ATH9K_AHB = "y";
+            ATH10K = "m";
+            ATH10K_PCI = "m";
+            ATH10K_DEBUG = "y";
+          };
+        };
       };
     };
 }

devices/gl-mt300a/default.nix

@@ -47,9 +47,9 @@
     let
       inherit (pkgs.liminix.networking) interface;
       inherit (pkgs) openwrt;
-      mac80211 = pkgs.mac80211.override {
-        drivers = ["rt2800soc"];
-        klibBuild = config.system.outputs.kernel.modulesupport;
+      mac80211 = pkgs.kmodloader.override {
+        targets = ["rt2800soc"];
+        inherit (config.system.outputs) kernel;
       };
     in {
       imports = [
@@ -131,8 +131,8 @@
       kernel = {
         src = pkgs.fetchurl {
           name = "linux.tar.gz";
-          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.71.tar.gz";
-          hash = "sha256-yhO2cXIeIgUxkSZf/4aAsF11uxyh+UUZu6D1h92vCD8=";
+          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+          hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
         };
         extraPatchPhase = ''
           ${openwrt.applyPatches.ramips}
@@ -178,6 +178,14 @@
         } // lib.optionalAttrs (config.system.service ? vlan) {
           SWCONFIG = "y";
         };
+        conditionalConfig = {
+          WLAN = {
+            WLAN_VENDOR_RALINK = "y";
+            RT2800SOC = "m";
+            RT2X00 = "m";
+          };
+        };
+
       };
     };
 }

devices/gl-mt300n-v2/default.nix

@@ -43,9 +43,9 @@
       inherit (pkgs.pseudofile) dir symlink;
       inherit (pkgs) openwrt;
 
-      mac80211 = pkgs.mac80211.override {
-        drivers = ["mt7603e"];
-        klibBuild = config.system.outputs.kernel.modulesupport;
+      mac80211 = pkgs.kmodloader.override {
+        targets = ["mt7603e"];
+        inherit (config.system.outputs) kernel;
       };
       wlan_firmware = pkgs.fetchurl {
         url = "https://github.com/openwrt/mt76/raw/f24b56f935392ca1d35fae5fd6e56ef9deda4aad/firmware/mt7628_e2.bin";
@@ -127,8 +127,8 @@
       kernel = {
         src = pkgs.fetchurl {
           name = "linux.tar.gz";
-          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.71.tar.gz";
-          hash = "sha256-yhO2cXIeIgUxkSZf/4aAsF11uxyh+UUZu6D1h92vCD8=";
+          url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+          hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
         };
         extraPatchPhase = ''
           ${openwrt.applyPatches.ramips}
@@ -185,6 +185,15 @@
           RALINK_WDT = "y";  # watchdog
           MT7621_WDT = "y";  # or it might be this one
         };
+        conditionalConfig = {
+          WLAN = {
+            WLAN_VENDOR_RALINK = "y";
+            WLAN_VENDOR_MEDIATEK = "y";
+            MT7603E = "m";
+          };
+        };
+
       };
+
     };
 }

devices/tp-archer-ax23/default.nix

@@ -0,0 +1,442 @@
+{
+  description = ''
+    TP-Link Archer AX23 / AX1800 Dual Band Wi-Fi 6 Router
+    *****************************************************
+
+    Hardware summary
+    ================
+
+    - MediaTek MT7621 (880MHz)
+    - 16MB Flash
+    - 128MB RAM
+    - WLan hardware: Mediatek MT7905, MT7975
+
+    Limitations
+    ===========
+
+    Status LEDs do not work yet.
+
+    Uploading an image via tftp doesn't work yet, because the Archer uboot
+    version is so old it doesn't support overriding the DTB from the mboot
+    command. The tftpboot module doesn't support this yet, see
+    https://gti.telent.net/dan/liminix/pulls/5 for the WiP.
+  '';
+
+  system = {
+    crossSystem = {
+      config = "mipsel-unknown-linux-musl";
+      gcc = {
+        abi = "32";
+        # https://openwrt.org/docs/techref/instructionset/mipsel_24kc
+        arch = "24kc";
+      };
+    };
+  };
+
+  module = {pkgs, config, lib, lim, ... }:
+    let firmware = pkgs.stdenv.mkDerivation {
+          name = "wlan-firmware";
+          phases = ["installPhase"];
+          installPhase = ''
+            mkdir $out
+            cp ${pkgs.linux-firmware}/lib/firmware/mediatek/{mt7915,mt7615,mt7622}* $out
+          '';
+        };
+    in {
+      imports = [
+        ../../modules/arch/mipsel.nix
+        ../../modules/outputs/tftpboot.nix
+        ../../modules/outputs/tplink-safeloader.nix
+      ];
+      config = {
+        kernel = {
+          src = pkgs.pkgsBuildBuild.fetchurl {
+            name = "linux.tar.gz";
+            url = "https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.15.137.tar.gz";
+            hash = "sha256-PkdzUKZ0IpBiWe/RS70J76JKnBFzRblWcKlaIFNxnHQ=";
+          };
+          extraPatchPhase = ''
+            ${pkgs.openwrt.applyPatches.ramips}
+          '';
+          config = {
+            # Initially taken from openwrt's ./target/linux/ramips/mt7621/config-5.15,
+            # then tweaked here and there
+            ARCH_32BIT_OFF_T="y";
+            ARCH_HIBERNATION_POSSIBLE="y";
+            ARCH_KEEP_MEMBLOCK="y";
+            ARCH_MMAP_RND_BITS_MAX="15";
+            ARCH_MMAP_RND_COMPAT_BITS_MAX="15";
+            ARCH_SUSPEND_POSSIBLE="y";
+            AT803X_PHY="y";
+            BLK_MQ_PCI="y";
+            BOARD_SCACHE="y";
+            CEVT_R4K="y";
+            CLKSRC_MIPS_GIC="y";
+            CLK_MT7621="y";
+            CLOCKSOURCE_WATCHDOG="y";
+            CLONE_BACKWARDS="y";
+            CMDLINE_BOOL="y";
+            COMMON_CLK="y";
+            COMPAT_32BIT_TIME="y";
+            CPU_GENERIC_DUMP_TLB="y";
+            CPU_HAS_DIEI="y";
+            CPU_HAS_PREFETCH="y";
+            CPU_HAS_RIXI="y";
+            CPU_HAS_SYNC="y";
+            CPU_LITTLE_ENDIAN="y";
+            CPU_MIPS32="y";
+            CPU_MIPS32_R2="y";
+            CPU_MIPSR2="y";
+            CPU_MIPSR2_IRQ_EI="y";
+            CPU_MIPSR2_IRQ_VI="y";
+            CPU_NEEDS_NO_SMARTMIPS_OR_MICROMIPS="y";
+            CPU_R4K_CACHE_TLB="y";
+            CPU_RMAP="y";
+            CPU_SUPPORTS_32BIT_KERNEL="y";
+            CPU_SUPPORTS_HIGHMEM="y";
+            CPU_SUPPORTS_MSA="y";
+            CRC16="y";
+            CRYPTO_DEFLATE="y";
+            CRYPTO_HASH_INFO="y";
+            CRYPTO_LIB_BLAKE2S_GENERIC="y";
+            CRYPTO_LIB_POLY1305_RSIZE="2";
+            CRYPTO_LZO="y";
+            CRYPTO_ZSTD="y";
+            CSRC_R4K="y";
+            DIMLIB="y";
+            DMA_NONCOHERENT="y";
+            DTB_RT_NONE="y";
+            DTC="y";
+            EARLY_PRINTK="y";
+            FIXED_PHY="y";
+            FWNODE_MDIO="y";
+            FW_LOADER_PAGED_BUF="y";
+            GENERIC_ATOMIC64="y";
+            GENERIC_CLOCKEVENTS="y";
+            GENERIC_CMOS_UPDATE="y";
+            GENERIC_CPU_AUTOPROBE="y";
+            GENERIC_FIND_FIRST_BIT="y";
+            GENERIC_GETTIMEOFDAY="y";
+            GENERIC_IOMAP="y";
+            GENERIC_IRQ_CHIP="y";
+            GENERIC_IRQ_EFFECTIVE_AFF_MASK="y";
+            GENERIC_IRQ_SHOW="y";
+            GENERIC_LIB_ASHLDI3="y";
+            GENERIC_LIB_ASHRDI3="y";
+            GENERIC_LIB_CMPDI2="y";
+            GENERIC_LIB_LSHRDI3="y";
+            GENERIC_LIB_UCMPDI2="y";
+            GENERIC_PCI_IOMAP="y";
+            GENERIC_PHY="y";
+            GENERIC_PINCONF="y";
+            GENERIC_SCHED_CLOCK="y";
+            GENERIC_SMP_IDLE_THREAD="y";
+            GENERIC_TIME_VSYSCALL="y";
+            GLOB="y";
+            GPIOLIB_IRQCHIP="y";
+            GPIO_CDEV="y";
+            GPIO_GENERIC="y";
+            GPIO_MT7621="y";
+            GRO_CELLS="y";
+            HANDLE_DOMAIN_IRQ="y";
+            HARDWARE_WATCHPOINTS="y";
+            HAS_DMA="y";
+            HAS_IOMEM="y";
+            HAS_IOPORT_MAP="y";
+            I2C="y";
+            I2C_ALGOBIT="y";
+            I2C_BOARDINFO="y";
+            I2C_CHARDEV="y";
+            I2C_GPIO="y";
+            I2C_MT7621="y";
+            ICPLUS_PHY="y";
+            IRQCHIP="y";
+            IRQ_DOMAIN="y";
+            IRQ_DOMAIN_HIERARCHY="y";
+            IRQ_FORCED_THREADING="y";
+            IRQ_MIPS_CPU="y";
+            IRQ_WORK="y";
+            LIBFDT="y";
+            LOCK_DEBUGGING_SUPPORT="y";
+            LZO_COMPRESS="y";
+            LZO_DECOMPRESS="y";
+            MDIO_BUS="y";
+            MDIO_DEVICE="y";
+            MDIO_DEVRES="y";
+            MEDIATEK_GE_PHY="y";
+            MEMFD_CREATE="y";
+            MFD_SYSCON="y";
+            MIGRATION="y";
+            MIKROTIK="y";
+            MIKROTIK_RB_SYSFS="y";
+            MIPS="y";
+            MIPS_ASID_BITS="8";
+            MIPS_ASID_SHIFT="0";
+            MIPS_CLOCK_VSYSCALL="y";
+            MIPS_CM="y";
+            MIPS_CPC="y";
+            MIPS_CPS="y";
+            MIPS_CPU_SCACHE="y";
+            MIPS_GIC="y";
+            MIPS_L1_CACHE_SHIFT="5";
+            MIPS_LD_CAN_LINK_VDSO="y";
+            MIPS_MT="y";
+            MIPS_MT_FPAFF="y";
+            MIPS_MT_SMP="y";
+            MIPS_NR_CPU_NR_MAP="4";
+            MIPS_PERF_SHARED_TC_COUNTERS="y";
+            MIPS_SPRAM="y";
+            MODULES_USE_ELF_REL="y";
+            MTD_CMDLINE_PARTS="y";
+            MTD_NAND_CORE="y";
+            MTD_NAND_ECC="y";
+            MTD_NAND_ECC_SW_HAMMING="y";
+            MTD_NAND_MT7621="y";
+            MTD_NAND_MTK_BMT="y";
+            MTD_RAW_NAND="y";
+            MTD_ROUTERBOOT_PARTS="y";
+            MTD_SERCOMM_PARTS="y";
+            MTD_SPI_NOR="y";
+            MTD_SPLIT_FIT_FW="y";
+            MTD_SPLIT_MINOR_FW="y";
+            MTD_SPLIT_SEAMA_FW="y";
+            MTD_SPLIT_TPLINK_FW="y";
+            MTD_SPLIT_TRX_FW="y";
+            MTD_SPLIT_UIMAGE_FW="y";
+            MTD_UBI="y";
+            MTD_UBI_BEB_LIMIT="20";
+            MTD_UBI_BLOCK="y";
+            MTD_UBI_WL_THRESHOLD="4096";
+            MTD_VIRT_CONCAT="y";
+            NEED_DMA_MAP_STATE="y";
+            NET_DEVLINK="y";
+            NET_DSA="y";
+            NET_DSA_MT7530="y";
+            NET_DSA_MT7530_MDIO="y";
+            NET_DSA_TAG_MTK="y";
+            NET_FLOW_LIMIT="y";
+            NET_MEDIATEK_SOC="y";
+            NET_SELFTESTS="y";
+            NET_SWITCHDEV="y";
+            NET_VENDOR_MEDIATEK="y";
+            NO_HZ_COMMON="y";
+            NO_HZ_IDLE="y";
+            NR_CPUS="4";
+            NVMEM="y";
+            OF="y";
+            OF_ADDRESS="y";
+            OF_EARLY_FLATTREE="y";
+            OF_FLATTREE="y";
+            OF_GPIO="y";
+            OF_IRQ="y";
+            OF_KOBJ="y";
+            OF_MDIO="y";
+            PAGE_POOL="y";
+            PAGE_POOL_STATS="y";
+            PCI="y";
+            PCIE_MT7621="y";
+            PCI_DISABLE_COMMON_QUIRKS="y";
+            PCI_DOMAINS="y";
+            PCI_DOMAINS_GENERIC="y";
+            PCI_DRIVERS_GENERIC="y";
+            PCS_MTK_LYNXI="y";
+            PERF_USE_VMALLOC="y";
+            PGTABLE_LEVELS="2";
+            PHYLIB="y";
+            PHYLINK="y";
+            PHY_MT7621_PCI="y";
+            PINCTRL="y";
+            PINCTRL_AW9523="y";
+            PINCTRL_MT7621="y";
+            PINCTRL_RALINK="y";
+            PINCTRL_SX150X="y";
+            POWER_RESET="y";
+            POWER_RESET_GPIO="y";
+            POWER_SUPPLY="y";
+            PTP_1588_CLOCK_OPTIONAL="y";
+            QUEUED_RWLOCKS="y";
+            QUEUED_SPINLOCKS="y";
+            RALINK="y";
+            RATIONAL="y";
+            REGMAP="y";
+            REGMAP_I2C="y";
+            REGMAP_MMIO="y";
+            REGULATOR="y";
+            REGULATOR_FIXED_VOLTAGE="y";
+            RESET_CONTROLLER="y";
+            RFS_ACCEL="y";
+            RPS="y";
+            RTC_CLASS="y";
+            RTC_DRV_BQ32K="y";
+            RTC_DRV_PCF8563="y";
+            RTC_I2C_AND_SPI="y";
+            SCHED_SMT="y";
+            SERIAL_8250="y";
+            SERIAL_8250_CONSOLE="y";
+            SERIAL_8250_NR_UARTS="3";
+            SERIAL_8250_RUNTIME_UARTS="3";
+            SERIAL_MCTRL_GPIO="y";
+            SERIAL_OF_PLATFORM="y";
+            SGL_ALLOC="y";
+            SMP="y";
+            SMP_UP="y";
+            SOCK_RX_QUEUE_MAPPING="y";
+            SOC_BUS="y";
+            SOC_MT7621="y";
+            SPI="y";
+            SPI_MASTER="y";
+            SPI_MEM="y";
+            SPI_MT7621="y";
+            SRCU="y";
+            SWPHY="y";
+            SYNC_R4K="y";
+            SYSCTL_EXCEPTION_TRACE="y";
+            SYS_HAS_CPU_MIPS32_R1="y";
+            SYS_HAS_CPU_MIPS32_R2="y";
+            SYS_HAS_EARLY_PRINTK="y";
+            SYS_SUPPORTS_32BIT_KERNEL="y";
+            SYS_SUPPORTS_ARBIT_HZ="y";
+            SYS_SUPPORTS_HIGHMEM="y";
+            SYS_SUPPORTS_HOTPLUG_CPU="y";
+            SYS_SUPPORTS_LITTLE_ENDIAN="y";
+            SYS_SUPPORTS_MIPS16="y";
+            SYS_SUPPORTS_MIPS_CPS="y";
+            SYS_SUPPORTS_MULTITHREADING="y";
+            SYS_SUPPORTS_SCHED_SMT="y";
+            SYS_SUPPORTS_SMP="y";
+            SYS_SUPPORTS_ZBOOT="y";
+            TARGET_ISA_REV="2";
+            TICK_CPU_ACCOUNTING="y";
+            TIMER_OF="y";
+            TIMER_PROBE="y";
+            TREE_RCU="y";
+            TREE_SRCU="y";
+            UBIFS_FS="y";
+            USB_SUPPORT="y";
+            USE_OF="y";
+            WEAK_ORDERING="y";
+            XPS="y";
+            XXHASH="y";
+            ZLIB_DEFLATE="y";
+            ZLIB_INFLATE="y";
+            ZSTD_COMPRESS="y";
+            ZSTD_DECOMPRESS="y";
+          } // lib.optionalAttrs (config.system.service ? watchdog) {
+            RALINK_WDT = "y";  # watchdog
+            MT7621_WDT = "y";  # or it might be this one
+          };
+          conditionalConfig = {
+            WLAN = {
+              MT7915E = "m";
+            };
+          };
+        };
+        tplink-safeloader.board = "ARCHER-AX23-V1";
+        boot = {
+          commandLine = [ "console=ttyS0,115200" ];
+          tftp = {
+            # Should be a segment of free RAM, where the tftp artifact
+            # can be stored before unpacking it to the 'hardware.loadAddress'
+            # The 'hardware.loadAddress' is 0x80001000, which suggests the
+            # RAM would start at 0x8000000 and (being 128MB) go to
+            # to 0x8800000. Let's put it at the 100MB mark at
+            # 0x8000000+0x0640000=0x86400000
+            loadAddress = lim.parseInt "0x86400000";
+          };
+        };
+        filesystem =
+        let inherit (pkgs.pseudofile) dir symlink;
+             in
+               dir {
+                 lib = dir {
+                   firmware = dir {
+                     mediatek = symlink firmware;
+                   };
+                 };
+               };
+
+      hardware =
+        let
+          openwrt = pkgs.openwrt;
+          mac80211 =  pkgs.kmodloader.override {
+            targets = [
+              "mt7915e"
+            ];
+            inherit (config.system.outputs) kernel;
+          };
+        in {
+          # from OEM bootlog (openwrt wiki):
+          # 4 cmdlinepart partitions found on MTD device raspi
+          # Creating 4 MTD partitions on "raspi":
+          # 0x000000000000-0x000000040000 : "uboot"
+          # 0x000000040000-0x000000440000 : "uImage"
+          # 0x000000440000-0x000000ff0000 : "rootfs"
+          # 0x000000ff0000-0x000001000000 : "ART"
+          # from openwrt bootlog (openwrt wiki):
+          # 5 fixed-partitions partitions found on MTD device spi0.0
+          # OF: Bad cell count for /palmbus@1e000000/spi@b00/flash@0/partitions
+          # OF: Bad cell count for /palmbus@1e000000/spi@b00/flash@0/partitions
+          # OF: Bad cell count for /palmbus@1e000000/spi@b00/flash@0/partitions
+          # OF: Bad cell count for /palmbus@1e000000/spi@b00/flash@0/partitions
+          # Creating 5 MTD partitions on "spi0.0":
+          # 0x000000000000-0x000000040000 : "u-boot"
+          # 0x000000040000-0x000000fa0000 : "firmware"
+          # 2 uimage-fw partitions found on MTD device firmware
+          # Creating 2 MTD partitions on "firmware":
+          # 0x000000000000-0x0000002c0000 : "kernel"
+          # 0x0000002c0000-0x000000f60000 : "rootfs"
+          # mtd: setting mtd3 (rootfs) as root device
+          # 1 squashfs-split partitions found on MTD device rootfs
+          # 0x000000640000-0x000000f60000 : "rootfs_data"
+          # 0x000000fa0000-0x000000fb0000 : "config"
+          # 0x000000fb0000-0x000000ff0000 : "tplink"
+          # 0x000000ff0000-0x000001000000 : "radio"
+          flash = {
+            # from the OEM bootlog 'Booting image at bc040000'
+            # (0x40000 from 0xbc000000)
+            address = lim.parseInt "0xbc040000";
+            # 0x000000040000-0x000000fa0000
+            size = lim.parseInt "0xf60000";
+            # TODO: find in /proc/mtd on a running system
+            eraseBlockSize = 65536;
+          };
+
+          # since this is mentioned in the partition table as well?
+          defaultOutput = "tplink-safeloader";
+          # taken from openwrt sysupgrade image:
+          # openwrt-23.05.2-ramips-mt7621-tplink_archer-ax23-v1-squashfs-sysupgrade.bin: u-boot legacy uImage, MIPS OpenWrt Linux-5.15.137, Linux/MIPS, OS Kernel Image (lzma), 2797386 bytes, Tue Nov 14 13:38:11 2023, Load Address: 0X80001000, Entry Point: 0X80001000, Header CRC: 0X19F74C5B, Data CRC: 0XF685563C
+          loadAddress = lim.parseInt "0x80001000";
+          entryPoint = lim.parseInt "0x80001000";
+          rootDevice = "/dev/mtdblock3";
+          dts = {
+            src = "${openwrt.src}/target/linux/ramips/dts/mt7621_tplink_archer-ax23-v1.dts";
+            includes =  [
+              "${openwrt.src}/target/linux/ramips/dts"
+              "${config.system.outputs.kernel.modulesupport}/arch/arm64/boot/dts/mediatek/"
+            ];
+          };
+
+          networkInterfaces =
+            let
+              inherit (config.system.service.network) link;
+              inherit (config.system.service) bridge;
+            in rec {
+              lan1 = link.build { ifname = "lan1"; };
+              lan2 = link.build { ifname = "lan2"; };
+              lan3 = link.build { ifname = "lan3"; };
+              lan4 = link.build { ifname = "lan4"; };
+              wan = link.build { ifname = "wan"; };
+
+              wlan = link.build {
+                ifname = "wlan0";
+                dependencies = [ mac80211 ];
+              };
+              wlan5 = link.build {
+                ifname = "wlan1";
+                dependencies = [ mac80211 ];
+              };
+            };
+        };
+    };
+    };
+}

devices/turris-omnia/default.nix

@@ -10,19 +10,141 @@
     to work (and provides you an easy rollback if you decide you don't
     like Liminix after all).
 
-    The install process is designed so that you should not need to open
-    the device and add a serial console (although it may be handy
-    for visibility  and in case anything goes wrong). In outline
+    The install process has two stages, and is intended that you
+    should not need to open the device and add a serial console
+    (although it may be handy for visibility, and in case anything
+    goes wrong). First we build a minimal installation/recovery
+    system, then we reboot into that recovery image to prepare the
+    device for the full target install.
 
-      1. build a "recovery" system with useful btrfs tools
-      2. boot that system using TFTP or a USB stick
-      3. once booted, mount the real root filesystem on /mnt
-      4. take a snapshot using schnapps, and then delete everything
-      5. use min-copy-closure -d /mnt/@ to copy the real configuration
-         to the device
-      6. reboot into a fully operational system
+    Installation using a USB stick
+    ==============================
+
+    First, build the image for the USB stick. Review
+    :file:`examples/recovery.nix` in order to change the default
+    root password (which is ``secret``) and/or the SSH keys, then
+    build it with
+
+    .. code-block:: console
+
+        $ nix-build -I liminix-config=./examples/recovery.nix \
+          --arg device "import ./devices/turris-omnia" \
+          -A outputs.mbrimage -o mbrimage
+        $ file -L mbrimage
+        mbrimage: DOS/MBR boot sector; partition 1 : ID=0x83, active, start-CHS (0x0,0,5), end-CHS (0x6,130,26), startsector 4, 104602 sectors
+
+    Next, copy the image from your build machine to a USB storage
+    medium using :command:`dd` or your other most favoured file copying
+    tool, which might be a comand something like this:
+
+    .. code-block:: console
+
+        $ dd if=mbrimage of=/dev/path/to/the/usb/stick \
+          bs=1M conv=fdatasync status=progress
+
+    The Omnia's default boot order only checks USB after it has failed
+    to boot from eMMC, which is not ideal for our purpose.  Unless you
+    have a serial cable, the easiest way to change this is by booting
+    to TurrisOS and logging in with ssh:
+
+    .. code-block:: console
+
+        root@turris:/# fw_printenv boot_targets
+        boot_targets=mmc0 nvme0 scsi0 usb0 pxe dhcp
+        root@turris:/# fw_setenv boot_targets usb0 mmc0
+        root@turris:/# fw_printenv boot_targets
+        boot_targets=usb0 mmc0
+        root@turris:/# reboot -f
+
+    It should now boot into the recovery image. It expects a network
+    cable to be plugged into LAN2 with something on the other end of
+    it that serves DHCP requests.  Check your DHCP server logs for a
+    request from a ``liminix-recovery`` host and figure out what IP
+    address was assigned.
+
+    .. code-block:: console
+
+        $ ssh liminix-recovery.lan
+
+    You should get a "Busybox" banner and a root prompt. Now you can
+    start preparing the device to install Liminix on it. First we'll
+    mount the root filesystem and take a snapshot:
+
+    .. code-block:: console
+
+        # mkdir /dest && mount /dev/mmcblk0p1 /dest
+        # schnapps -d /dest create "pre liminix"
+        # schnapps -d /dest list
+        ERROR: not a valid btrfs filesystem: /
+            # | Type      | Size        | Date                      | Description
+        ------+-----------+-------------+---------------------------+------------------------------------
+            1 | single    |    16.00KiB | 1970-01-01 00:11:49 +0000 | pre liminix
+
+    (``not a valid btrfs filesystem: /`` is not a real error)
+
+    then we can remove all the files
+
+    .. code-block:: console
+
+        # rm -r /dest/@/*
+
+    and then it's ready to install the real Liminix system onto. On
+    your build system, create the Liminix configuration you wish to
+    install: here we'll use the ``rotuer`` example.
+
+    .. code-block:: console
+
+        build$ nix-build -I liminix-config=./examples/rotuer.nix \
+          --arg device "import ./devices/turris-omnia" \
+          -A outputs.systemConfiguration
+
+    and then use :command:`min-copy-closure` to copy it to the device.
+
+    .. code-block:: console
+
+        build$ nix-shell --run \
+          "min-copy-closure -r /dest/@  root@liminix-recovery.lan result"
+
+    and activate it
+
+    .. code-block:: console
+
+        build$ ssh root@liminix-recovery.lan \
+          "/dest/@/$(readlink result)/bin/install /dest/@"
+
+    The final steps are performed directly on the device again: add
+    a symlink so U-Boot can find :file:`/boot`, then restore the
+    default boot order and reboot into the new configuration.
+
+    .. code-block:: console
+
+        # cd /dest && ln -s @/boot .
+        # fw_setenv boot_targets "mmc0 nvme0 scsi0 usb0 pxe dhcp"
+        # cd / ; umount /dest
+        # reboot
+
+
+    Installation using a TFTP server and serial console
+    ===================================================
+
+    If you have a :ref:`serial` console connection and a TFTP server,
+    and would rather use them than fiddling with USB sticks, the
+    :file:`examples/recovery.nix` configuration also works
+    using the ``tftpboot`` output. So you can do
+
+    .. code-block:: console
+
+        build$ nix-build -I liminix-config=./examples/recovery.nix \
+          --arg device "import ./devices/turris-omnia" \
+          -A outputs.tftpboot
+
+    and then paste the generated :file:`result/boot.scr` into
+    U-Boot, and you will end up with the same system as you would