commit 648ac2eb7fd3baaf7a5ecdb1ff34343276294ebe
parent b1f4db00a07cce0cdd7d7d62eb52328a1dbef98a
Author: Daniel Barlow <dan@telent.net>
Date: Wed, 17 May 2023 15:04:46 +0100
Document jffs2, min-copy-closure, liminix-rebuild
Some of the code is now out of date w.r.t. some of the text
Diffstat:
2 files changed, 196 insertions(+), 82 deletions(-)
diff --git a/doc/developer.rst b/doc/developer.rst
@@ -149,6 +149,7 @@ module(s) it uses. I have this segment in configuration.nix which you
may be able to adapt:
.. code-block:: nix
+
boot = {
kernelParams = [ "intel_iommu=on" ];
kernelModules = [
diff --git a/doc/user.rst b/doc/user.rst
@@ -2,32 +2,101 @@ User Manual
###########
This manual is an early work in progress, not least because Liminix is
-not yet ready for users who are not also developers.
+not yet really ready for users who are not also developers. Your
+feedback to improve it is very welcome.
-Configuring for your use case
-*****************************
+Installation
+************
-You need to create a ``configuration.nix`` that describes your router
+The Liminix installation process is not quite like installing NixOS on
+a real computer, but some NixOS experience will nevertheless be
+helpful in understanding it. The steps are as follows:
+
+* Decide whether you want the device to be updatable in-place (there
+ are advantages and disadvantages), or if you are happy to generate
+ and flash a new image whenever changes are required.
+
+* Create a :file:`configuration.nix` describing the system you want
+
+* Build an image
+
+* Flash it to the device
+
+
+Choosing a flavour (read-only or updatable)
+===========================================
+
+Liminix installations come in two "flavours"- read-only or in-place
+updatable:
+
+* a read-only installation can't be updated once it is flashed to your
+ device, and so must be reinstalled in its entirety every time you
+ want to change it. It uses the ``squashfs`` filesystem which has
+ very good compression ratios and so you can pack quite a lot of
+ useful stuff onto your device. This is good if you don't expect
+ to change it often.
+
+* an updatable installation has a writable filesystem so that you can
+ update configuration, upgrade packages and install new packages over
+ the network after installation. This uses the `jffs2
+ <http://www.linux-mtd.infradead.org/doc/jffs2.html>`_ filesystem:
+ although it does compress the data, the need to support writes means
+ that it can't pack quite as small as squashfs, so you will not have
+ as much space to play with.
+
+Updatability caveats
+~~~~~~~~~~~~~~~~~~~~
+
+At the time of writing this manual the read-only squashfs support is
+much more mature. Consider also that it may not be possible to perform
+"larger" updates in-place even if you do opt for updatability. If you
+have (for example) an 11MB system on a 16MB device, you won't be able
+to do an in-place update of something fundamental like the C library
+(libc), as this will temporarily require 22MB to install all the
+packages needing the new library before the packages using the old
+library can be removed. A writable system will be more useful for
+smaller updates such as installing a new package (perhaps you
+temporarily need tcpdump to diagnose a network problem) or for
+changing configuration files.
+
+Note also that the kernel is not part of the filesystem so cannot be
+updated this way. Kernel changes require a full reflash.
+
+
+
+Creating configuration.nix
+==========================
+
+
+You need to create a ``configuration.nix`` that describes your device
and the services that you want to run on it. Start by copying
``vanilla-configuration.nix`` and adjusting it, or look in the `examples`
directory for some pre-written configurations.
-Your configuration may include modules and probably _should_
+``configuration.nix`` conventionally describes the packages, services,
+user accounts etc of the device. It does not describe the hardware
+itself, which is specified separately in the build command (as you
+will see below).
+
+Your configuration may include modules: it probably *should*
include the ``standard`` module unless you understand what it
does and what happens if you leave it out.
-.. code-block: nix
+.. code-block:: nix
- imports = [
- ./modules/standard.nix
- ]
+ imports = [
+ ./modules/standard.nix
+ ]
+ configuration.rootfsType = "jffs2"; # or "squashfs"
-Building and flashing
-*********************
+Building
+========
-An example command to build Liminix might look like this:
+Build Liminix using the :file:`default.nix` in the project toplevel
+directory, passing it arguments for configuration and hardware. For
+example:
.. code-block:: console
@@ -50,52 +119,8 @@ is a raw image file that can be written directly to the firmware flash
partition.
-Flashing with :command:`flashcp`
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This requires an existing Liminix system, or perhaps some other
-operating system on the device which provides the :command:`flashcp`
-command. You need to locate the "firmware" partition, which you can do
-with a combination of :command:`dmesg` output and the contents of
-:file:`/proc/mtd`
-
-**Don't do this on a device that's running on the same flash partition
-as you're about to overwrite, otherwise you're likely to crash it. Use
-kexecboot (see "Updates to running devices" below) first to reboot
-into a RAM-based system.**
-
-
-.. code-block:: console
-
- <5>[ 0.469841] Creating 4 MTD partitions on "spi0.0":
- <5>[ 0.474837] 0x000000000000-0x000000040000 : "u-boot"
- <5>[ 0.480796] 0x000000040000-0x000000050000 : "u-boot-env"
- <5>[ 0.487056] 0x000000050000-0x000000060000 : "art"
- <5>[ 0.492753] 0x000000060000-0x000001000000 : "firmware"
-
- # cat /proc/mtd
- dev: size erasesize name
- mtd0: 00040000 00001000 "u-boot"
- mtd1: 00010000 00001000 "u-boot-env"
- mtd2: 00010000 00001000 "art"
- mtd3: 00fa0000 00001000 "firmware"
- mtd4: 002a0000 00001000 "kernel"
- mtd5: 00d00000 00001000 "rootfs"
-
-Then you can copy the image to the device with :command:`ssh`
-
-.. code-block:: console
-
- build-machine$ tar chf - result/firmware.bin | \
- ssh root@the-device tar -C /run -xvf -
-
-and then connect to the device and run
-
-
-.. code-block:: console
-
- flashcp -v firmware.bin /dev/mtd3
-
+Flashing
+========
Flashing from OpenWrt (untested)
@@ -108,7 +133,7 @@ If your device is running OpenWrt then it probably has the
mtd -r write /tmp/firmware_image.bin firmware
-For more information, please see the `OpenWrt manual <https://openwrt.org/docs/guide-user/installation/sysupgrade.cli>`_
+For more information, please see the `OpenWrt manual <https://openwrt.org/docs/guide-user/installation/sysupgrade.cli>`_ which may also contain (hardware-dependent) instructions on how to flash an image using the vendor firmware - perhaps even from a web interface.
Flashing from the boot monitor
@@ -116,20 +141,26 @@ Flashing from the boot monitor
If you are prepared to open the device and have a TTL serial adaptor
of some kind to connect it to, you can probably flash it using U-Boot.
-This is quite hardware-specific: please refer to the Developer Manual.
+This is quite hardware-specific, and sometimes involves soldering:
+please refer to the Developer Manual.
+
+Flashing from an existing Liminix system with :command:`flashcp`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Updates to running devices
-**************************
+The flash procedure from an existing Liminix-system is two-step.
+First we reboot the device (using "kexec") into an "ephemeral"
+RAM-based version of the new configuration, then when we're happy it
+works we can flash the image - and if it doesn't work we can reboot
+the device again and it will boot from the old image.
-To mitigate the risk of flashing a new configuration and potentially
-render the device unresponsive if the configuration is unbootable or
-doesn't bring up a network device, Liminix has a
-"try before write" mode.
-To test a configuration without writing it to flash, import the
-``kexecboot`` module and build ``outputs.kexecboot`` instead of
+Building the RAM-based image
+............................
+
+
+To creatr the ephemeral image, build ``outputs.kexecboot`` instead of
``outputs.default``. This generates a directory containing the root
filesystem image and kernel, along with an executable called `kexec`
and a `boot.sh` script that runs it with appropriate arguments.
@@ -157,29 +188,111 @@ reboot - be sure to close all open files and finish anything else
you were doing first.*
If the new system crashes or is rebooted, then the device will revert
-to the old configuration it finds in flash. Thus, by combining kexec
-boot with a hardware watchdog you can try new images with very little
-chance of bricking anything. When you are happy that the new
-configuration is correct, build and flash a flashable image of it.
+to the old configuration it finds in flash.
+Building the second (permanent) image
+.....................................
-Module options (tbd)
-**************
+While running in the kexecboot system, you can copy the permanent
+image to the device with :command:`ssh`
+.. code-block:: console
-Foo module
-==========
+ build-machine$ tar chf - result/firmware.bin | \
+ ssh root@the-device tar -C /run -xvf -
-Module docs will go here. This part of the doc should be autogenerated.
+Next you need to connect to the device and locate the "firmware"
+partition, which you can do with a combination of :command:`dmesg`
+output and the contents of :file:`/proc/mtd`
+
+
+.. code-block:: console
+
+ <5>[ 0.469841] Creating 4 MTD partitions on "spi0.0":
+ <5>[ 0.474837] 0x000000000000-0x000000040000 : "u-boot"
+ <5>[ 0.480796] 0x000000040000-0x000000050000 : "u-boot-env"
+ <5>[ 0.487056] 0x000000050000-0x000000060000 : "art"
+ <5>[ 0.492753] 0x000000060000-0x000001000000 : "firmware"
+
+ # cat /proc/mtd
+ dev: size erasesize name
+ mtd0: 00040000 00001000 "u-boot"
+ mtd1: 00010000 00001000 "u-boot-env"
+ mtd2: 00010000 00001000 "art"
+ mtd3: 00fa0000 00001000 "firmware"
+ mtd4: 002a0000 00001000 "kernel"
+ mtd5: 00d00000 00001000 "rootfs"
+
+Now run (in this example)
+
+.. code-block:: console
+
+ flashcp -v firmware.bin /dev/mtd3
+
+
+"I know my new image is good, can I skip the intemediate step?"
+```````````````````````````````````````````````````````````````
+In addition to giving you a chance to see if the new image works, this
+two-step process ensures that you're not copying the new image over
+the top of the active root filesystem. It might work, or it might
+crash in surprising ways.
-Bar module
-==========
-Baz module
-==========
-Quuz net device
+Updating an installed system (JFFS2)
+************************************
+
+
+Adding packages
===============
+
+
+If your device is running a JFFS2 root filesystem, you can build
+extra packages for it on your build system and copy them to the
+device: any package in Nixpkgs or in the Liminix overlay is available
+with the ``pkgs`` prefix:
+
+.. code-block:: console
+
+ nix-build -I liminix-config=./my-configuration.nix \
+ --arg device "import ./devices/mydevice" -A pkgs.tcpdump
+
+ nix-shell -p min-copy-closure root@the-device result/
+
+Note that this only copies the package to the device: it doesn't update
+any profile to add it to ``$PATH``
+
+
+Rebuilding the system
+=====================
+
+:command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example:
+
+.. code-block:: console
+
+ liminix-rebuild root@the-device -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750"
+
+
+Caveats
+~~~~~~~
+
+* it needs there to be enough free space on the device for all the new
+ packages in addition to all the packages already on it - which may be
+ a problem if a lot of things have changed (e.g. a new version of
+ nixpkgs).
+
+* it cannot upgrade the kernel, only userland
+
+* it reboots the device!
+
+
+
+Configuration Options
+*********************
+
+
+
+Module docs will go here. This part of the doc should be autogenerated.
