12 changed files with 405 additions and 1001 deletions
--- a/THOUGHTS.txt
+++ b/THOUGHTS.txt
@ -2337,102 +2337,4 @@ Here is a working shebang for write-fennel:
 #!/nix/store/5iwv3h2jjbk2vib2bpwx3g9knpb02x3y-lua-5.3.6/bin/lua -e dofile(arg[0]).run()
 Tue Sep 12 20:47:52 BST 2023
 We don't handle unbound or stopped states in odhcp consumers. I think
 probably we should do this in odhcp-script by deleting the outputs,
 rather than making each consumer do it.
 ... turns out that odhcp6c itself unsets ADDRESSES and PREFIXES before
 calling the script with "unbound", so maybe we don't need to do
 anything special.
 Wed Sep 13 17:55:33 BST 2023
@400000000000001f2723b3cb eth1.link.pppoe Script /nix/store/nyks8zl86dcp44k5sjcc76digrnfgm17-ip-up finished (pid 403), status = 0x0
@400000000000001f27b2db3b eth1.link.pppoe Script /nix/store/ds0lc4qd1zfiyxsva87rpplyr21awjh1-ip6-up finished (pid 404), status = 0x1
@400000000000001f30a7c5c5 /nix/store/v9ijgyywizqbbd9y73r2wifkxc0d1jjm-route-default-1a22c69d0e1f-up: line 4: input: not found
@400000000000001f31abf9b5 ip: command line is not complete, try "help"
@400000000000001f31ca1395 s6-rc: warning: unable to start service route-default-1a22c69d0e1f: command exited 1
@400000000000001f31f236b4 s6-rc: info: service route-default-d2586cf00da0 successfully started
@
 Wed Sep 13 18:05:38 BST 2023
 TODO
 - service for dhcp6 client
 - move acquire-{wan,lan} scripts out of examples/
 - service for resolvconf
 - nftables syntax error
 - tidy up the dependency handling in serviceDefn build
   (interface is fine, implementation is a bit brutal)
 - docs
 considerations:
 1) in some ways, we should be able to specify acquire-{wan,lan} as if
 they were just additional addresses on the respective
 interfaces. However, they're longruns so the implementation of
 "address" doesn't really fit.
 2) should they be bundled into a dhcp client service? I think the
 answer is "no" because which of the dhcp config we want to
 honour locally (and how) is policy not mechainmsm
 svc.dhcp6c.client.build { interface = wan; };
 svc.dhcp6c.address.build {
  inherit client;
  interface = lan;
 };
 svc.dhcp6c.address.build {
  inherit client;
  interface = wan;
 };
 svc.dhcp6c.prefix.build {
  inherit client;
  interface = lan;
  index = 1;			# default to first interface
 };
 svc.dhcp6c.prefix.build {
  inherit client;
  interface = vpn;
  index = 2;
 };
 Fri Sep 15 12:04:25 BST 2023
 Qemu worked example provides dhcp and ssh service
 Hardware worked example needs to be plugged into same lan as build
 machine if we are going to tftp the image onto it - so it might be
 awkward if we run dhcp on it
 The device I have lying around is the A
 How do we do the actual flash step? Assuming the device is running
 stock firmware, from a laptop we can wifi to it and use the web ui to
 upgrade
 we can't build the hellonet config because it requires tftp
 plug in mt300a
 put stock firmware on it
 Sun Sep 17 00:08:03 BST 2023
 I don't think the user manual needs a full justification of why we
 have the module/service split. Maybe we should have "decision records"
 in the git tree instead
 Sun Sep 17 16:44:31 BST 2023
 Can we figure out which bits of the old doc are missing from the new
 one and just transplant those? Then we can merge it sooner
 instead of blocking on writig all the new stuff
--- a/doc/admin.rst
+++ b/doc/admin.rst
@ -1,188 +0,0 @@
 System Administration
 #####################
 Services on a running system
 ****************************
 * add an s6-rc cheatsheet here
 Flashing and updating
 *********************
 Flashing from an existing Liminix system with :command:`flashcp`
 ================================================================
 The flash procedure from an existing Liminix-system has two steps.
 First we reboot the device (using "kexec") into an "ephemeral"
 RAM-based version of the new configuration, then when we're happy it
 works we can flash the image - and if it doesn't work we can reboot
 the device again and it will boot from the old image.
 Building the RAM-based image
 ----------------------------
 To create the ephemeral image, build ``outputs.kexecboot`` instead of
 ``outputs.default``.  This generates a directory containing the root
 filesystem image and kernel, along with an executable called `kexec`
 and a `boot.sh` script that runs it with appropriate arguments.
 For example
 .. code-block:: console
   nix-build --show-trace -I liminix-config=./examples/arhcive.nix \
     --arg device "import ./devices/gl-ar750"
     -A outputs.kexecboot && \
     (tar chf - result | ssh root@the-device tar -C /run -xvf -)
 and then login to the device and run
 .. code-block:: console
   cd /run/result
   sh ./boot.sh .
 This will load the new kernel and map the root filesystem into a RAM
 disk, then start executing the new kernel. *This is effectively a
 reboot - be sure to close all open files and finish anything else
 you were doing first.*
 If the new system crashes or is rebooted, then the device will revert
 to the old configuration it finds in flash.
 Building the second (permanent) image
 -------------------------------------
 While running in the kexecboot system, you can copy the permanent
 image to the device with :command:`ssh`
 .. code-block:: console
   build-machine$ tar chf - result/firmware.bin | \
    ssh root@the-device tar -C /run -xvf -
 Next you need to connect to the device and locate the "firmware"
 partition, which you can do with a combination of :command:`dmesg`
 output and the contents of :file:`/proc/mtd`
 .. code-block:: console
   <5>[    0.469841] Creating 4 MTD partitions on "spi0.0":
   <5>[    0.474837] 0x000000000000-0x000000040000 : "u-boot"
   <5>[    0.480796] 0x000000040000-0x000000050000 : "u-boot-env"
   <5>[    0.487056] 0x000000050000-0x000000060000 : "art"
   <5>[    0.492753] 0x000000060000-0x000001000000 : "firmware"
   # cat /proc/mtd
   dev:    size   erasesize  name
   mtd0: 00040000 00001000 "u-boot"
   mtd1: 00010000 00001000 "u-boot-env"
   mtd2: 00010000 00001000 "art"
   mtd3: 00fa0000 00001000 "firmware"
   mtd4: 002a0000 00001000 "kernel"
   mtd5: 00d00000 00001000 "rootfs"
 Now run (in this example)
 .. code-block:: console
   flashcp -v firmware.bin /dev/mtd3
 "I know my new image is good, can I skip the intermediate step?"
 ----------------------------------------------------------------
 In addition to giving you a chance to see if the new image works, this
 two-step process ensures that you're not copying the new image over
 the top of the active root filesystem. Sometimes it works, but you
 will at least need physical access to the device to power-cycle it
 because it will be effectively frozen afterwards.
 Flashing from the boot monitor
 ==============================
 If you are prepared to open the device and have a TTL serial adaptor
 of some kind to connect it to, you can probably flash it using U-Boot.
 This is quite hardware-specific, and sometimes involves soldering:
 please refer to the Developer Manual.
 Flashing from OpenWrt (not currently advised!)
 ==============================================
 .. CAUTION:: At your own risk! This will (at least in some
             circumstances) lead to bricking the device: we think this
             flash method is currently incompatible with use of a
             writeable (jffs2) filesystem.
 If your device is running OpenWrt then it probably has the
 :command:`mtd` command installed. After transferring the image onto the
 device using e.g. :command:`ssh`,  you can run it as follows:
 .. code-block:: console
   mtd -r write /tmp/firmware.bin firmware
 For more information, please see the `OpenWrt manual <https://openwrt.org/docs/guide-user/installation/sysupgrade.cli>`_ which may also contain (hardware-dependent) instructions on how to flash an image using the vendor firmware - perhaps even from a web interface.
 Updating an installed system (JFFS2)
 ************************************
 Adding packages
 ===============
 If your device is running a JFFS2 root filesystem, you can build
 extra packages for it on your build system and copy them to the
 device: any package in Nixpkgs or in the Liminix overlay is available
 with the ``pkgs`` prefix:
 .. code-block:: console
    nix-build -I liminix-config=./my-configuration.nix \
     --arg device "import ./devices/mydevice" -A pkgs.tcpdump
    nix-shell -p min-copy-closure root@the-device result/
 Note that this only copies the package to the device: it doesn't update
 any profile to add it to ``$PATH``
 Rebuilding the system
 =====================
 :command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example:
 .. code-block:: console
     liminix-rebuild root@the-device  -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750"
 This will
 * build anything that needs building
 * copy new or changed packages to the device
 * reboot the device
 It doesn't delete old packages automatically: to do that run
 :command:`min-collect-garbage`, which will delete any packages not in
 the current system closure. Note that Liminix does not have the NixOS
 concept of environments or generations, and there is no way back from
 this except for building the previous configuration again.
 Caveats
 ~~~~~~~
 * it needs there to be enough free space on the device for all the new
  packages in addition to all the packages already on it - which may be
  a problem if a lot of things have changed (e.g. a new version of
  nixpkgs).
 * it cannot upgrade the kernel, only userland
--- a/doc/adr/README
+++ b/doc/adr/README
@ -1,6 +0,0 @@
 Architecture Decision Records
 #############################
 In this directory you will find descriptions of Liminix architecture
 decisions.
--- a/doc/adr/module-system.rst
+++ b/doc/adr/module-system.rst
@ -1,201 +0,0 @@
 Module system
 #############
 **Status:** Adopted; implemented in July-September 2023
 Context
 *******
 Liminix users need a way to assemble a full system configuration by
 combining smaller, more isolated and reusable components, otherwise
 systems will be unwieldy and copy-and-paste will be rife.
 Alternatives
 ************
 NixOS module system
 ===================
 The NixOS module system addresses many of these concerns. A module is
 a Nix function which accepts a ``configuration`` attrset and some
 other parameters, and returns a new fragment of ``configuration``
 which is merged into it. It includes a DSL describing the permitted
 types of values for each key in the configuration, which is used for
 checking that the supplied parameters are valid and also governs what
 to do if two modules both specify a value for the same key. (Usually
 they are "merged", using some type-appropriate concept of merging.)
 Usually a NixOS module looks only (or mostly only) at a particular
 subtree of the overall configuration which is hardcoded in the module
 definition, but the configuration fragment it returns may touch any
 part of the schema. For example, the factorio module refers to
 ``config.services.factorio``, and it returns values for keys in
 ``systemd.services.factorio`` and ``networking.firewall``. There is no
 way to use this module to run **two** factorio services with different
 config (e.g. on different ports) - the only way to make that
 possible would be to extend the module definition so that it
 accepts a collection of game configurations and then create
 a systemd service for each.
 NixWRT module system
 ====================
 NixWRT, the (now defunct) predecessor of Liminix, used a homegrown
 module system modelled on the Nixpkgs overlay pattern.  Each module is
 a function that accepts  ``super`` and ``self`` parameters, and
 using <handwaves>that fixpoint magic thing</handwaves>
 is called in a chain with the configuration returned by the previous
 module and the final configuration.
 NixWRT modules mostly don't refer to the configuration object to
 decide how to configure themselves, but accept their parameters
 directly as function parameters. For example, the configuration
 file for "arhcive" (a backup server) includes this text:
 .. code-block:: nix
   (sshd {
     hostkey = secrets.sshHostKey;
     authkeys = { root = lib.splitString "\n" secrets.myKeys; };
   })
   busybox
   (usbdisk {
     label = "backup-disk";
     mountpoint = "/srv";
     fstype = "ext4";
     options = "rw";
   })
 This gives us flexibility that NixOS modules don't: for example, if we
 want to mount two USB disks, we can simply repeat that module twice
 with different parameters - and the module definition doesn't have to
 handle it specially.
 However, the downside of this system is that we didn't implement any
 concept of "types" - there is no type information, so there is no
 systematic checking that parameters are valid, and if two modules set
 the same config key then the rules for merging are entirely ad hoc.
 There is a further (arguable) downside, which is that the
 configuration is not just data - it's now part code. While it could be
 feasible (though I've never seen it done) to encode a NixOS
 configuration using Yaml or XML and then manipulate it as data, this
 is not even possible using the NixWRT system.
 Use services for everything
 ===========================
 The most common properties that a Liminix configuration needs to
 define are:
 * which services (processes) to run
 * what packages to install
 * permitted users and groups
 * Linux kernel configuration options
 * Busybox applets
 * filesystem layout
 Suppose we only had services?
 A Liminix service is (also) a derivation, so it is able to
 create any files it likes inside its own store path, and
 transitively require other packages simply by referring to them.
 If it needs particular kernel options it could define them
 as kernel modules to be loaded on demand when the service
 starts (see the nftables module for an example). However:
 * there is no way for a service to add busybox modules
 * it cannot create files outside of its store path, so
  wouldn't be able to make e.g. :file:`/etc/something.conf`
 * no way to create users/groups. We could steal the DynamicUsers idea
  from systemd and make them on demand, but this starts to get a bit
  more complicated.
 These limitations force us to reject this option as a general
 solution - though we should strive *where possible* to implement
 functionality as services and to minimise the proportion of Liminix
 that manipulates the global configuration.
 Decision
 ********
 "Why not both?"  None of these options is sufficient alone, so we are
 going to do a mixture.
 We will use the NixOS module system, but instead of expecting modules
 to create systemd services as instances, they will expose "service
 templates": functions that accept an attrset and return an
 appropriately configured service that can be assigned by the caller
 to a key in ``config.services``.
 We will typecheck the service template function parameters using the
 same type-checking code as NixOS uses for its modules.
 An example may make this clearer: to add an NTP
 service you first add :file:`modules/ntp` to your ``imports`` list,
 then you create a service by calling
 :code:`config.system.service.ntp.build { .... }` with the appropriate
 service-dependent configuration parameters.
 .. code-block:: nix
  let svc = config.system.service;
  in {
    # ...
    imports = [
      ./modules/ntp
      # ....
    ];
    config.services.ntp = svc.ntp.build {
      pools = { "pool.ntp.org" = ["iburst"]; };
      makestep = { threshold = 1.0; limit = 3; };
    };
 Merely including the module won't define the service on its own: it
 only creates the template in ``config.system.service.foo`` and you
 have to create the actual service using the template.
 Consequences
 ************
 This decision has both good and bad consequences
 Pro
 ===
 * We have a workable system for reusing configuration elements in
  Liminix.
 * We have type checking for most imortant things, reducing the risk of
  deploying an invalid configuration.
 * We have a simple mechanism for creating multiple services based on
  the same module, without buulding that logic into the module
  definition itself. For example, we could create two SSH daemons on
  different ports, or DHCP clients with different configurations on
  different network devices.
 * We expect to be able to automate the generation of module
  documentation.
 Con
 ===
 * By departing somewhat from the NixOS conventions we increase the
  amount of code we have to write/maintain ourselves - and the
  learning burden on users who are already familiar with that system.
 * Liminix configurations contain function calls and aren't just data,
  which means we can ony realistically interpret or introspect
  them with the Nix interpreter itself - we can't query them
  as data with other non-Nix tools.
--- a/doc/configuration.rst
+++ b/doc/configuration.rst
@ -1,156 +0,0 @@
 Configuration
 #############
 Liminix uses the Nix language to provide congruent configuration
 management.  This means that to change anything about the way in
 which a Liminix system works, you make that change in
 your :file:`configuration.nix` (or one of the other files it references),
 and rerun :command:`nix-build` or :command:`liminix-rebuild` to action
 the change. It is not possible (at least, without shenanigans) to make
 changes by logging into the device and running imperative commands
 whose effects may later be overridden: :file:`configuration.nix`
 always describes the entire system and can be used to recreate that
 system at any time.  You can usefully keep it under version control.
 If you are familiar with NixOS, you will notice some similarities
 between NixOS and Liminix configuration, and also some
 differences. Sometimes the differences are due to the
 resource-constrained devices we deploy onto, sometimes due to
 differences in the uses these devices are put to.
 Configuration taxonomy
 **********************
 There are many things you can specify in a configuration, but these
 are the ones you most commonly need to change:
 * which services (processes) to run
 * what packages to install
 * permitted users and groups
 * Linux kernel configuration options
 * Busybox applets
 * filesystem layout
 Modules
 *******
 **Modules** are a means of abstraction which allow "bundling"
 of configuration options related to a common purpose or theme. For
 example, the ``dnsmasq`` module defines a template for a dnsmasq
 service, ensures that the dnsmasq package is installed, and provides a
 dnsmasq user and group for the service to run as. The ``ppp`` module
 defines a service template and also enables various PPP-related kernel
 configuration.
 Not all modules are included in the configuration by default, because
 that would mean that the kernel (and the Busybox binary providing
 common CLI tools) was compiled with many unnecessary bells and whistles
 and therefore be bigger than needed. (This is not purely an academic concern
 if your device has little flash storage).  Therefore, specifying a
 service is usually a two-step process.  For example, to add an NTP
 service you first add :file:`modules/ntp` to your ``imports`` list,
 then you create a service by calling
 :code:`config.system.service.ntp.build { .... }` with the appropriate
 service-dependent configuration parameters.
 .. code-block:: nix
  let svc = config.system.service;
  in {
    # ...
    imports = [
      ./modules/ntp
      # ....
    ];
    config.services.ntp = svc.ntp.build {
      pools = { "pool.ntp.org" = ["iburst"]; };
      makestep = { threshold = 1.0; limit = 3; };
    };
 Merely including the module won't define the service on its own: it
 only creates the template in ``config.system.service.foo`` and you
 have to create an actual service using the template. This is an
 intentional choice to allow the creation of multiple
 differently-configured services based on the same template - perhaps
 e.g. when you have multiple networks (VPNs etc) in different trust
 domains, or you want to run two SSH daemons on different ports.
 (For the background to this, please refer to the :doc:`architecture decision record <adr/module-system>`)
 .. tip:: Liminix modules should be quite familiar (but also different)
 	 if you already know how to use NixOS modules. We use the
 	 NixOS module infrastructure code, meaning that you should
 	 recognise the syntax, the type system, the rules for
 	 combining configuration values from different sources. We
 	 don't use the NixOS modules themselves, because the
 	 underlying system is not similar enough for them to work.
 Services
 ********
 We use the `s6-rc service manager <https://www.skarnet.org/software/s6-rc/overview.html>`_  to start/stop/restart services and handle
 service dependencies. Any attribute in `config.services` will become
 part of the default set of services that s6-rc will try to bring up on
 boot.
 For the most part, for common use cases, hopefully the services you
 need will be defined by modules and you will only have to pass the
 right parameters to ``build``.
 Should you need to create a custom service of your own devising, use
 the `oneshot` or `longrun` functions:
 * a "longrun" service is the "normal" service concept: it has a
  ``run`` action which describes the process to start, and it watches
  that process to restart it if it exits. The process should not
  attempt to daemonize or "background" itself, otherwise s6-rc will think
  it died. Whatever it prints to standard output/standard error
  will be logged.
 .. code-block:: nix
    config.services.cowsayd = pkgs.liminix.services.longrun {
      name = "cowsayd";
      run = "${pkgs.cowsayd}/bin/cowsayd --port 3001 --breed hereford";
      # don't start this until the lan interface is ready
      dependencies = [ config.services.lan ];
    }
 * a "oneshot" service doesn't have a process attached. It consists of
  ``up`` and ``down`` actions which are bits of shell script that
  are run at the appropriate points in the service lifecycle
 .. code-block:: nix
    config.services.greenled = pkgs.liminix.services.oneshot {
      name = "greenled";
      up = ''
 	echo 17 > /sys/class/gpio/export
 	echo out > /sys/class/gpio/gpio17/direction
 	echo 0   > /sys/class/gpio/gpio17/value
      '';
      down = ''
 	echo 0   > /sys/class/gpio/gpio17/value
      '';
    }
 Services may have dependencies: as you see above in the ``cowsayd``
 example, it depends on some service called ``config.services.lan``,
 meaning that it won't be started until that other service is up.
 ..
 	TODO: explain service outputs
 ..
 	TODO: outputs that change, and services that poll other services
 Module implementation
 *********************
 TODO: make your own modules
 * how a module exposes services
 * defining types
--- a/doc/development.rst
+++ b/doc/development.rst
@ -1,14 +1,14 @@
-Development
+Developer Manual
-###########
+################
 As a developer working on Liminix, or implementing a service or
 module, you probably want to test your changes more conveniently
-than by building and flashing a new image every time. This section
+than by building and flashing a new image every time. This manual
 documents various affordances for iteration and experiments.
 In general, packages and tools that run on the "build" machine are
 available in the ``buildEnv`` derivation and can most easily
-be added to your environment by running :command:`nix-shell`.
+be added to your environment by running :command:`nix-shell`
--- a/doc/etc.rst
+++ b/doc/etc.rst
@ -0,0 +1,51 @@
 The Future
 ##########
 What about NixWRT?
 This is an in-progress rewrite of NixWRT, incorporating Lessons
 Learned. That said, as of today it is not yet at feature parity.
 Liminix will eventually provide these differentiators over NixWRT:
 * a writable filesystem so that software updates or reconfiguration
  (e.g. changing passwords) don't require taking the device offline to
  reflash it.
 * more flexible service management with dependencies, to allow
  configurations such as "route through PPPoE if it is healthy, with
  fallback to LTE"
 * a spec for valid configuration options (a la NixOS module options)
  to that we can detect errors at evaluation time instead of producing
  a bad image.
 * a network-based mechanism for secrets management so that changes can
  be pushed from a central location to several Liminix devices at once
 * send device metrics and logs to a monitoring/alerting/o11y
  infrastructure
 Today though, it does approximately none of these things and certainly
 not on real hardware.
 Articles of interest
 ####################
 * `Build Safety of Software in 28 Popular Home Routers <https://cyber-itl.org/assets/papers/2018/build_safety_of_software_in_28_popular_home_routers.pdf>`_: "of the access
   points and routers we reviewed, not a single one took full
   advantage of the basic application armoring features provided by
   the operating system. Indeed, only one or two models even came
   close, and no brand did well consistently across all models tested"
 * `A PPPoE Implementation for Linux <https://static.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/skoll/skoll_html/index.html>`_:
  "Many DSL service providers use PPPoE for residential broadband
  Internet access. This paper briefly describes the PPPoE protocol,
  presents strategies for implementing it under Linux and describes in
  detail a user-space implementation of a PPPoE client."
 * `PPP IPV6CP vs DHCPv6 at AAISP <https://www.revk.uk/2011/01/ppp-ipv6cp-vs-dhcpv6.html>`_
 * `Creating a Home IPv6 Network (James Bottomley) <https://blog.hansenpartnership.com/creating-a-home-ipv6-network/>`_
--- a/doc/index.rst
+++ b/doc/index.rst
@ -6,10 +6,9 @@ Liminix
   :caption: Contents:
   intro
-   tutorial
+   user
-   configuration
+   developer
-   admin
+   etc
   development
 Indices and tables
--- a/doc/tutorial.rst
+++ b/doc/tutorial.rst
@ -1,257 +0,0 @@
 Getting Started
 ###############
 Liminix is very configurable, which can make it initially quite
 daunting, especially if you're learning Nix or Linux or networking
 concepts at the same time. In this section we build some "worked
 example" Liminix images to introduce the concepts. If you follow the
 examples exactly, they should work. If you change things as you go
 along, they may work differently or not at all, but the experience
 should be educational either way.
 .. warning:: The first example we will look at runs under emulation,
 	     so there is no danger of bricking your hardware
 	     device. For the second example you may (if you have
 	     appropriate hardware and choose to do so) flash the
 	     configuration onto an actual router. There is always a
 	     risk of rendering the device unbootable when you do this,
 	     and various ways to recover depending on what went wrong.
 	     We'll write more about that at the appropriate point
 Requirements
 ************
 You will need a reasonably powerful computer running Nix.  Target
 devices for Liminix are unlikely to have the CPU power and disk space
 to be able to build it in situ, so the build process is based around
 "cross-compilation" from another computer. The build machine can be
 any reasonably powerful desktop/laptop/server PC running NixOS.
 Standalone Nixpkgs installations on other Linux distributions - or on
 MacOS, or even in a Docker container - also ought to work but are
 untested.
 Running in Qemu
 ***************
 You can try out Liminix without even having a router to play with.
 Clone the Liminix git repository and change into its directory
 .. code-block:: console
    git clone https://gti.telent.net/dan/liminix
    cd liminix
 Now build Liminix
 .. code-block:: console
    nix-build -I liminix-config=./examples/hello-from-qemu.nix \
     --arg device "import ./devices/qemu" -A outputs.default
 In this command ``liminix-config`` points to the desired software
 configuration (e.g. services, users, filesystem, secrets) and
 ``device`` describes the hardware (or emulated hardware) to run it on.
 ``outputs.default`` tells Liminix that we want the default image
 output for flashing to the device: for the Qemu "hardware" it's an
 alias for ``outputs.vmbuild``, which creates a directory containing a
 root filesystem image and a kernel.
 .. tip:: The first time you run this it may take several hours,
         because it builds all of the dependencies including a full
         MIPS gcc and library toolchain. Once those intermediate build
         products are in the nix store, subsequent builds will be much
         faster - practically instant, if nothing has changed.
 Now you can try it:
 .. code-block:: console
    nix-shell --run "mips-vm ./result/vmlinux ./result/rootfs"
 This starts the Qemu emulator with a bunch of useful options, to run
 the Liminix configuration you just built.  It connects the emulated
 device's serial console and the `QEMU monitor
 <https://www.qemu.org/docs/master/system/monitor.html>`_ to
 stdin/stdout.
 You should now see Linux boot messages and after a few seconds be
 presented with a login prompt. You can login on the console as
 ``root`` (password is "secret") and poke around to see what processes are
 running. To kill the emulator, press ^P (Control P) then c to enter the
 "QEMU Monitor", then type ``quit`` at the ``(qemu)`` prompt.
 To see that it's running network services we need to connect to its
 emulated network. Start the machine again, if you had stopped it, and
 open up a second terminal on your build machine. We're going to run
 another virtual machine attached to the virtual network, which will
 request an IP address from our Liminix system and give you a shell you
 can run ssh from.
 We use `System Rescue <https://www.system-rescue.org/>`_ in tty
 mode (no graphical output) for this example, but if you have some
 other favourite Linux Live CD ISO - or, for that matter, any other OS
 image that QEMU can boot - adjust the command to suit.
 Download the System Rescue ISO:
 .. code-block:: console
    curl https://fastly-cdn.system-rescue.org/releases/10.01/systemrescue-10.01-amd64.iso -O
 and run it
 .. code-block:: console
    nix-shell -p qemu --run " \
    qemu-system-x86_64 \
 	-echr 16 \
 	-m 1024 \
 	-cdrom systemrescue-10.01-amd64.iso \
 	-netdev socket,mcast=230.0.0.1:1235,localaddr=127.0.0.1,id=lan \
 	-device virtio-net,disable-legacy=on,disable-modern=off,netdev=lan,mac=ba:ad:3d:ea:21:01 \
 	-display none -serial mon:stdio"
 System Rescue displays a boot menu at which you should select the
 "serial console" option, then after a few moments it boots to a root
 prompt. You can now try things out:
 * run :command:`ip a` and see that it's been allocated an IP address in the range 10.3.0.0/16.
 * run :command:`ping 10.3.0.1` to see that the Liminix VM responds
 * run :command:`ssh root@10.3.0.1` to try logging into it.
 Congratulations! You have installed your first Liminix system - albeit
 it has no practical use and it's not even real. The next step is to try
 running it on hardware.
 Installing on hardware
 **********************
 For the next example, we're going to install onto an actual hardware
 device.  These steps have been tested using a GL-iNet GL-MT300A, which
 has been chosen for the purpose because it's cheap and easy to
 unbrick. Using some other Liminix-supported MIPS hardware device also
 *ought* to work here, but you accept the slightly greater bricking
 risk if it doesn't.
 You may want to acquire a `USB TTL serial cable
 <https://cpc.farnell.com/ftdi/ttl-232r-rpi/cable-debug-ttl-232-usb-rpi/dp/SC12825?st=usb%20to%20uart%20cable>`_
 when you start working with Liminix on real hardware. You
 won't *need* it for this example, assuming it works, but it
 allows you
 to see the boot monitor and kernel messages, and to login directly to
 the device if for some reason it doesn't bring its network up. You have options
 here: the FTDI-based cables are the Rolls Royce of serial cables,
 whereas the ones based on PL2303 and CP2102 chipsets are cheaper but
 also fussier - or you could even get creative and use e.g. a
 `Raspberry Pi <https://pinout.xyz/#>`_ or other SBC with a UART and
 TX/RX/GND header pins. Make sure that the voltages are compatible:
 this is a 3.3v device and you don't want to be sending it 5v or (even
 worse) 12v.
 Now we can build Liminix. Although we could use the same example
 configuration as we did for Qemu, you might not want to plug a DHCP
 server into your working LAN because it will compete with the real
 DHCP service. So we're going to use a different configuration with a
 DHCP client: this is :file:`examples/hello-from-mt300.nix`
 It's instructive to compare the two configurations:
 .. code-block:: console
    diff -u examples/hello-from-qemu.nix examples/hello-from-mt300.nix
 You'll see a new ``boot.tftp`` stanza which you can ignore,
 ``services.dns`` has been removed, and the static IP address allocation
 has been replaced by a ``dhcp.client`` service.
 .. code-block:: console
    nix-build -I liminix-config=./examples/hello-from-mt300.nix \
     --arg device "import ./devices/gl-mt300a" -A outputs.default
 .. tip:: The first time you run this it may take several hours.
         Again? Yes, even if you ran the previous example. Qemu is
         set up as a big-endian system whereas the MediaTek SoC
         on this device is little-endian - so it requires building
         all of the dependencies including an entirely different
         MIPS gcc and library toolchain to the other one.
 This time in :file:`result/` you will see a bunch of files. Most of
 them you can ignore for the moment, but :file:`result/firmware.bin` is
 the firmware image you can flash.
 Flashing
 ========
 Again, there are a number of different ways you could do this: using
 TFTP with a serial cable, through the stock firmware's web UI, or
 using the `vendor's "debrick" process
 <https://docs.gl-inet.com/router/en/3/tutorials/debrick/>`_. The last
 of these options has a lot to recommend it for a first attempt:
 * it works no matter what firmware is currently installed
 * it doesn't require plugging a router into the same network as your
  build system and potentially messing up your actual upstream
 * no need to open the device and add cables
 You can read detailed instructions on the vendor site, but the short version is:
 1. turn the device off
 2. connect it by ethernet cable to a computer
 3. configure the computer to have static ip address 192.168.1.10
 4. while holding down the Reset button, turn the device on
 5. after about five seconds you can release the Reset button
 6. visit http://192.168.1.1/ using a web browser on the connected computer
 7. click on "Browse" and choose :file:`result/firmware.bin`
 8. click on "Update firmware"
 9. wait a minute or so while it updates.
 There's no feedback from the web interface when the flashing is
 finished, but what should happen is that the router reboots and
 starts running Liminix. Now you need to figure out what address it got
 from DHCP - e.g. by checking the DHCP server logs, or maybe by pinging
 ``hello.lan`` or something. Once you've found it on the
 network you can ping it and ssh to it just like you did the Qemu
 example, but this time for real.
 .. warning:: Do not leave the default root password in place on any
             device exposed to the internet!  Although it has no
             writable storage and no default route, a motivated attacker
 	     with some imagination could probably still do something
 	     awful using it.
 Congratulations Part II! You have installed your first Liminix system on
 actual hardware - albeit that it *still* has no practical use.
 Exercise for the reader: change the default password by editing
 :file:`examples/hello-from-mt300.nix`, and then create and upload a
 new image that has it set to something less hopeless.
 Final thoughts
 **************
 * These are demonstration configs for pedagogical purposes. If you'd
  like to see some more realistic uses of Liminix,
  :file:`examples/rotuer,arhcive,extneder.nix` are based on some
  actual real hosts in my home network.
 * These example images are not writable. Later we will explain how to
  generate an image that can be changed after installation, and
  even use :command:`liminix-rebuild` (analogous to :command:`nixos-rebuild`)
  to keep it up to date.
 * The technique used here for flashing was chosen mostly because it
  doesn't need much infrastructure/tooling, but it is a bit of a faff
  (requires physical access, vendor specific). There are slicker ways
  to do it that need a bit more setup - we'll talk about that later as
  well.
--- a/doc/user.rst
+++ b/doc/user.rst
@ -0,0 +1,347 @@
 User Manual
 ###########
 This manual is an early work in progress, not least because Liminix is
 not yet really ready for users who are not also developers. Your
 feedback to improve it is very welcome.
 Installation
 ************
 The Liminix installation process is not quite like installing NixOS on
 a real computer, but some NixOS experience will nevertheless be
 helpful in understanding it.  The steps are as follows:
 * Decide whether you want the device to be updatable in-place (there
  are advantages and disadvantages), or if you are happy to generate
  and flash a new image whenever changes are required.
 * Create a :file:`configuration.nix` describing the system you want
 * Build an image
 * Flash it to the device
 Supported devices
 =================
 For a list of devices that Liminix (present or previous versions)
 has run on, refer to `devices/ in the source repo <https://gti.telent.net/dan/liminix/src/branch/main/devices>`_. For devices that _currently_ build,
 cross-reference it with `the CI status <https://build.liminix.org/jobset/liminix/build#tabs-jobs>`_. Everything that builds is (usually) expected
 to run, so if you end up with an image that builds but doesn't
 boot, please report it as a bug.
 As of June 2023 the device list is a little thin. Adding devices based
 on the Atheros or Mediatek (Ralink) platform should be quite
 straightforward if you have some C/Linux kernel experience and are
 prepared to open it up and attach serial wires: please refer to the
 Developer Manual.
 Choosing a flavour (read-only or updatable)
 ===========================================
 Liminix installations come in two "flavours"- read-only or in-place
 updatable:
 * a read-only installation can't be updated once it is flashed to your
  device, and so must be reinstalled in its entirety every time you
  want to change it.  It uses the ``squashfs`` filesystem which has
  very good compression ratios and so you can pack quite a lot of
  useful stuff onto your device.  This is good if you don't expect
  to change it often.
 * an updatable installation has a writable filesystem so that you can
  update configuration, upgrade packages and install new packages over
  the network after installation. This uses the `jffs2
  <http://www.linux-mtd.infradead.org/doc/jffs2.html>`_ filesystem:
  although it does compress the data, the need to support writes means
  that it can't pack quite as small as squashfs, so you will not have
  as much space to play with.
 Updatability caveats
 ~~~~~~~~~~~~~~~~~~~~
 At the time of writing this manual the read-only squashfs support is
 much more mature. Consider also that it may not be possible to perform
 "larger" updates in-place even if you do opt for updatability.  If you
 have (for example) an 11MB system on a 16MB device, you won't be able
 to do an in-place update of something fundamental like the C library
 (libc), as this will temporarily require 22MB to install all the
 packages needing the new library before the packages using the old
 library can be removed. A writable system will be more useful for
 smaller updates such as installing a new package (perhaps you
 temporarily need tcpdump to diagnose a network problem) or for
 changing configuration files.
 Note also that the kernel is not part of the filesystem so cannot be
 updated this way. Kernel changes require a full reflash.
 Creating configuration.nix
 ==========================
 You need to create a :file:`configuration.nix` that describes your
 device and the services that you want to run on it. The best way to
 get started is by reading one of the examples such as
 :file:`examples/rotuer.nix` and modifying it to your needs.
 :file:`configuration.nix` conventionally describes the packages, services,
 user accounts etc of the device. It does not describe the hardware
 itself, which is specified separately in the build command (as you
 will see below).
 Most of the functionality of a Liminix system is driven by *services*
 which are declared by *modules*: thus, to add for example an NTP service
 you first add :file:`modules/ntp` to your ``imports`` list, then
 you create a service by calling :code:`config.system.service.ntp.build { .... }`
 with the appropriate service-dependent configuration parameters.
 .. code-block:: nix
  let svc = config.system.service;
  in {
    # ...
    imports = [
      ./modules/ntp
      # ....
    ];
    config.services.ntp = svc.ntp.build {
      pools = { "pool.ntp.org" = ["iburst"]; };
      makestep = { threshold = 1.0; limit = 3; };
    };
 A :ref:`full list of module options <module-options>` is provided
 later in this manual.
 You *most likely* want to include the ``standard`` module unless you
 have a quite unusual use case for a very minimal system, in which case
 you will understand what it does and what happens if you leave it out.
 .. code-block:: nix
    imports = [
      ./modules/standard.nix
    ]
    configuration.rootfsType = "jffs2"; # or "squashfs"
 Building
 ========
 Build Liminix using the :file:`default.nix` in the project toplevel
 directory, passing it arguments for configuration and hardware. For
 example:
 .. code-block:: console
    nix-build -I liminix-config=./tests/smoke/configuration.nix \
     --arg device "import ./devices/qemu" -A outputs.default
 In this command ``<liminix-config>`` points to your
 :file:`configuration.nix`, ``device`` is the file for your hardware device
 definition, and ``outputs.default`` will generate some kind of
 Liminix image output appropriate to that device.
 For the qemu device in this example, ``outputs.default`` is an alias
 for ``outputs.vmbuild``, which creates a directory containing a
 squashfs root image and a kernel. You can use the :command:`mips-vm` command to
 run this.
 For the currently supported hardware devices, ``outputs.default``
 creates a directory containing a file called ``firmware.bin``.  This
 is a raw image file that can be written directly to the firmware flash
 partition.
 Flashing
 ========
 Flashing from the boot monitor
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 If you are prepared to open the device and have a TTL serial adaptor
 of some kind to connect it to, you can probably flash it using U-Boot.
 This is quite hardware-specific, and sometimes involves soldering:
 please refer to the Developer Manual.
 Flashing from an existing Liminix system with :command:`flashcp`
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The flash procedure from an existing Liminix-system is two-step.
 First we reboot the device (using "kexec") into an "ephemeral"
 RAM-based version of the new configuration, then when we're happy it
 works we can flash the image - and if it doesn't work we can reboot
 the device again and it will boot from the old image.
 Building the RAM-based image
 ............................
 To create the ephemeral image, build ``outputs.kexecboot`` instead of
 ``outputs.default``.  This generates a directory containing the root
 filesystem image and kernel, along with an executable called `kexec`
 and a `boot.sh` script that runs it with appropriate arguments.
 For example
 .. code-block:: console
   nix-build --show-trace -I liminix-config=./examples/arhcive.nix \
     --arg device "import ./devices/gl-ar750"
     -A outputs.kexecboot && \
     (tar chf - result | ssh root@the-device tar -C /run -xvf -)
 and then login to the device and run
 .. code-block:: console
   cd /run/result
   sh ./boot.sh .
 This will load the new kernel and map the root filesystem into a RAM
 disk, then start executing the new kernel. *This is effectively a
 reboot - be sure to close all open files and finish anything else
 you were doing first.*
 If the new system crashes or is rebooted, then the device will revert
 to the old configuration it finds in flash.
 Building the second (permanent) image
 .....................................
 While running in the kexecboot system, you can copy the permanent
 image to the device with :command:`ssh`
 .. code-block:: console
   build-machine$ tar chf - result/firmware.bin | \
    ssh root@the-device tar -C /run -xvf -
 Next you need to connect to the device and locate the "firmware"
 partition, which you can do with a combination of :command:`dmesg`
 output and the contents of :file:`/proc/mtd`
 .. code-block:: console
   <5>[    0.469841] Creating 4 MTD partitions on "spi0.0":
   <5>[    0.474837] 0x000000000000-0x000000040000 : "u-boot"
   <5>[    0.480796] 0x000000040000-0x000000050000 : "u-boot-env"
   <5>[    0.487056] 0x000000050000-0x000000060000 : "art"
   <5>[    0.492753] 0x000000060000-0x000001000000 : "firmware"
   # cat /proc/mtd
   dev:    size   erasesize  name
   mtd0: 00040000 00001000 "u-boot"
   mtd1: 00010000 00001000 "u-boot-env"
   mtd2: 00010000 00001000 "art"
   mtd3: 00fa0000 00001000 "firmware"
   mtd4: 002a0000 00001000 "kernel"
   mtd5: 00d00000 00001000 "rootfs"
 Now run (in this example)
 .. code-block:: console
   flashcp -v firmware.bin /dev/mtd3
 "I know my new image is good, can I skip the intemediate step?"
 ```````````````````````````````````````````````````````````````
 In addition to giving you a chance to see if the new image works, this
 two-step process ensures that you're not copying the new image over
 the top of the active root filesystem. It might work, or it might
 crash in surprising ways.
 Flashing from OpenWrt (not currently advised!)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 .. CAUTION:: At your own risk! This will (at least in some
             circumstances) lead to bricking the device: we think this
             flash method is currently incompatible with use of a
             writeable (jffs2) filesystem.
 If your device is running OpenWrt then it probably has the
 :command:`mtd` command installed. After transferring the image onto the
 device using e.g. :command:`ssh`,  you can run it as follows:
 .. code-block:: console
   mtd -r write /tmp/firmware.bin firmware
 For more information, please see the `OpenWrt manual <https://openwrt.org/docs/guide-user/installation/sysupgrade.cli>`_ which may also contain (hardware-dependent) instructions on how to flash an image using the vendor firmware - perhaps even from a web interface.
 Updating an installed system (JFFS2)
 ************************************
 Adding packages
 ===============
 If your device is running a JFFS2 root filesystem, you can build
 extra packages for it on your build system and copy them to the
 device: any package in Nixpkgs or in the Liminix overlay is available
 with the ``pkgs`` prefix:
 .. code-block:: console
    nix-build -I liminix-config=./my-configuration.nix \
     --arg device "import ./devices/mydevice" -A pkgs.tcpdump
    nix-shell -p min-copy-closure root@the-device result/
 Note that this only copies the package to the device: it doesn't update
 any profile to add it to ``$PATH``
 Rebuilding the system
 =====================
 :command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example:
 .. code-block:: console
     liminix-rebuild root@the-device  -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750"
 This will
 * build anything that needs building
 * copy new or changed packages to the device
 * reboot the device
 It doesn't delete old packages automatically: to do that run
 :command:`min-collect-garbage`, which will delete any packages not in
 the current system closure. Note that Liminix does not have the NixOS
 concept of environments or generations, and there is no way back from
 this except for building the previous configuration again.
 Caveats
 ~~~~~~~
 * it needs there to be enough free space on the device for all the new
  packages in addition to all the packages already on it - which may be
  a problem if a lot of things have changed (e.g. a new version of
  nixpkgs).
 * it cannot upgrade the kernel, only userland
 Configuration options
 *********************
 .. _module-options:
 .. include:: modules.rst
--- a/examples/hello-from-mt300.nix
+++ b/examples/hello-from-mt300.nix
@ -1,43 +0,0 @@
 { config, pkgs, lib, ... } :
 let
  inherit (pkgs) serviceFns;
  svc = config.system.service;
 in rec {
  imports = [
    ../modules/network
    ../modules/ssh
    ../modules/vlan
    ../modules/flashimage.nix
  ];
  boot.tftp = {
    # IP addresses to use in the boot monitor when flashing/ booting
    # over TFTP. If you are flashing using the stock firmware's Web UI
    # then these dummy values are fine
    ipaddr = "192.0.2.115"; # my address
    serverip = "192.0.2.5"; # build machine or other tftp server
  };
  hostname = "hello";
  services.dhcpc = svc.network.dhcp.client.build {
    interface = config.hardware.networkInterfaces.lan;
    # don't start DHCP until the hostname is configured,
    # so it can identify itself to the DHCP server
    dependencies = [ config.services.hostname ];
  };
  services.sshd = svc.ssh.build { };
  users.root = {
    # the password is "secret". Use mkpasswd -m sha512crypt to
    # create this hashed password string
    passwd = "$6$y7WZ5hM6l5nriLmo$5AJlmzQZ6WA.7uBC7S8L4o19ESR28Dg25v64/vDvvCN01Ms9QoHeGByj8lGlJ4/b.dbwR9Hq2KXurSnLigt1W1";
  };
  defaultProfile.packages = with pkgs; [
    figlet
  ];
 }
--- a/examples/hello-from-qemu.nix
+++ b/examples/hello-from-qemu.nix
@ -1,44 +0,0 @@
 { config, pkgs, lib, ... } :
 let
  inherit (pkgs) serviceFns;
  svc = config.system.service;
 in rec {
  imports = [
    ../modules/network
    ../modules/dnsmasq
    ../modules/ssh
  ];
  hostname = "hello";
  # configure the internal network (LAN) with an address
  services.int = svc.network.address.build {
    interface = config.hardware.networkInterfaces.lan;
    family = "inet"; address ="10.3.0.1"; prefixLength = 16;
  };
  services.sshd = svc.ssh.build { };
  users.root = {
    # the password is "secret". Use mkpasswd -m sha512crypt to
    # create this hashed password string
    passwd = "$6$y7WZ5hM6l5nriLmo$5AJlmzQZ6WA.7uBC7S8L4o19ESR28Dg25v64/vDvvCN01Ms9QoHeGByj8lGlJ4/b.dbwR9Hq2KXurSnLigt1W1";
  };
  services.dns =
    let interface = services.int;
    in svc.dnsmasq.build {
      inherit interface;
      ranges = [
        "10.3.0.10,10.3.0.240"
        "::,constructor:$(output ${interface} ifname),ra-stateless"
      ];
      domain = "example.org";
    };
  defaultProfile.packages = with pkgs; [
    figlet
  ];
 }