provide stdout to ppp callback scripts

pppd runs them with 0,1,2 => /dev/null but we actually quite like
seeing errors in the logs

NEWS

@@ -115,3 +115,21 @@ the value of profile.gateway.wan. For the pppoe case this is now only
 very slightly more verbose, and it allows using the gateway profile
 with other kinds of upstream.
 
+2024-8-16
+
+As part of implementing log shipping, the default directory for system
+logs has beenchanged from /run/uncaught-logs to /run/log
+
+2024-10-09
+
+liminix-rebuild is being deprecated. From hereon in, the preferred way
+to do an incremental update on an installed device with a writable
+filesystem is to build the systemConfiguration output
+
+    nix-build  -I liminix-config=hosts/myhost.nix --argstr deviceName turris-omnia -A outputs.systemConfiguration 
+
+and then run the generated `install.sh` script
+
+    result/install.sh root@192.168.8.1
+
+

THOUGHTS.txt

@@ -6513,14 +6513,29 @@ We had it working in a VM, and the service is installed on loaclhost
 
 TODO
 
-1) make a module-based service for client-cert
+[done] 1) make a module-based service for client-cert
 caCertificateFile
 secretFile
 subject
 url
 
-2) make the shipping service a consumer-for
+[done] 2) make the shipping service a consumer-for
 
-3) can we reduce the verbiage in the shipping service somehow?
+[not by much] 3) can we reduce the verbiage in the shipping service somehow?
 
 4) rebuild an actual device with all this stuff
+
+Tue Oct  8 23:50:00 BST 2024
+
+idea: create outputs.update which builds a systemConfiguration and
+also a result/install.sh which does  min-copy-closure and
+restart-services as per liminix-rebuild, then we don't have
+to nix-shell liminix-rebuild
+
+nix-build ../liminix/ -I liminix-config=hosts/rotuer.nix --argstr deviceName turris-omnia -A outputs.update-system  -o rotuer && ./rotuer/install.sh
+
+this would make it much more straightforward to build a bunch of hosts
+using a Makefile
+
+idea 2: when a configuration contains levitate, something similar
+but necessarily more "manual" to do the analogous thing

default.nix

@@ -33,6 +33,7 @@ let
       ./modules/busybox.nix
       ./modules/hostname.nix
       ./modules/kernel
+      ./modules/klogd.nix
       device.module
       liminix-config
       ./modules/s6

doc/admin.rst

@@ -161,17 +161,32 @@ any profile to add it to ``$PATH``
 Rebuilding the system
 =====================
 
-:command:`liminix-rebuild` is the Liminix analogue of :command:`nixos-rebuild`, although its operation is a bit different because it expects to run on a build machine and then copy to the host device. Run it with the same ``liminix-config`` and ``device`` parameters as you would run :command:`nix-build`, and it will build any new/changed packages and then copy them to the device using SSH. For example:
+Liminix has a mechanism for in-place updates of a running system which
+is analogous to :command:`nixos-rebuild`, but its operation is a
+bit different because it expects to run on a build machine and then
+copy to the host device. To use this, build the `outputs.systemConfiguration`
+target and then run the :command:`result/install.sh` script it generates.
 
 .. code-block:: console
 
-     liminix-rebuild root@the-device  -I liminix-config=./examples/rotuer.nix --arg device "import ./devices/gl-ar750"
+    nix-build -I liminix-config=./my-configuration.nix \
+       --arg device "import ./devices/mydevice" \
+       -A outputs.systemConfiguration
+    ./result/install.sh root@the-device 
 
-This will
+The install script uses min-copy-closure to copy new or changed
+packages to the device, then (perhaps) reboots it. The reboot
+behaviour can be affected by flags:
+
+* `--no-reboot` will cause it not to reboot at all, if you would
+  rather do that yourself. Note that none of the newly-installed or
+  updated services will be running until you do.
+
+* `--fast` causes it tn not do a full reboot, but instead to restart
+  only the services that have been changed. This will restart all of
+  the services that have updated store paths (and anything that
+  depends on them), but will not affect services that haven't changed.
 
-* build anything that needs building
-* copy new or changed packages to the device
-* reboot the device
 
 It doesn't delete old packages automatically: to do that run
 :command:`min-collect-garbage`, which will delete any packages not in

doc/installation.rst

@@ -15,7 +15,7 @@ Liminix uses the Nix language to provide congruent configuration
 management.  This means that to change anything about the way in
 which a Liminix system works, you make that change in
 your :file:`configuration.nix` (or one of the other files it references),
-and rerun :command:`nix-build` or :command:`liminix-rebuild` to action
+and rerun :command:`nix-build` to action
 the change. It is not possible (at least, without shenanigans) to make
 changes by logging into the device and running imperative commands
 whose effects may later be overridden: :file:`configuration.nix`

doc/tutorial.rst

@@ -293,14 +293,17 @@ the hostname) and then run
 
 .. code-block:: console
 
-    nix-shell --run "liminix-rebuild root@address-of-the-device  -I liminix-config=./my-router.nix --arg device "import ./devices/gl-ar750""
+    nix-build  -I liminix-config=./my-router.nix \
+      --arg device "import ./devices/gl-ar750" \
+      -A outputs.systemConfiguration && \
+    result/install.sh root@address-of-the-device 
 
 (This requires the device to be network-accessible from your build
 machine, which for a test/demo system might involve a second network
 device in your build system - USB ethernet adapters are cheap - or
 a bit of messing around unplugging cables.)
 
-For more information about :code:`liminix-rebuild`, see the manual section :ref:`Rebuilding the system`.
+For more information about in-place-updates, see the manual section :ref:`Rebuilding the system`.
 
 
 Final thoughts

modules/firewall/default-rules.nix

@@ -90,7 +90,7 @@ in {
       # accept inbound from the WAN
       (if allow-incoming
        then accept "oifname \"int\" iifname \"ppp0\""
-       else "oifname \"int\" iifname \"ppp0\" jump incoming-allowed-ip6"
+       else "iifname \"ppp0\" jump incoming-allowed-ip6"
       )
       # allow all outbound and any inbound that's part of a
       # recognised (outbound-initiated) flow
@@ -210,7 +210,7 @@ in {
       "icmp type { echo-request, echo-reply } accept"
       "iifname int jump input-ip4-lan"
       "iifname ppp0 jump input-ip4-wan"
-      "oifname \"int\" iifname \"ppp0\" jump incoming-allowed-ip4"
+      "iifname ppp0 jump incoming-allowed-ip4"
       "ct state established,related accept"
       "log prefix \"DENIED CHAIN=input-ip4 \""
     ];

modules/kernel/default.nix

@@ -91,9 +91,8 @@ in {
         IKCONFIG_PROC = "y";
         PROC_FS = "y";
 
-        KEXEC = "y";
         MODULES = if modular then "y" else "n";
-        MODULE_SIG = if modular then "y" else "n";
+        MODULE_UNLOAD = if modular then "y" else "n";
         DEBUG_FS = "y";
 
         # basic networking protocols

modules/klogd.nix

@@ -0,0 +1,15 @@
+{ config, lib, pkgs, ... }:
+let
+  inherit (pkgs.liminix.services) longrun;
+in {
+  config.services.klogd = longrun {
+    name = "klogd";
+    run = ''
+      echo "1 2 1 8"  > /proc/sys/kernel/printk
+      cat /proc/kmsg
+    '';
+    finish = ''
+      echo "8 4 1 8"  > /proc/sys/kernel/printk
+    '';
+  };
+}

modules/ntp/service.nix

@@ -6,6 +6,7 @@
 }:
 params:
 let
+  name = "ntp"; # bad name, needs to be unique
   inherit (liminix.services) longrun;
   inherit (lib) concatStringsSep mapAttrsToList;
   configFile = p:
@@ -23,11 +24,15 @@ let
     ++ (lib.optional (p.bindaddress != null) "bindaddress ${p.bindaddress}")
     ++ (lib.optional (p.binddevice != null) "binddevice ${p.binddevice}")
     ++ (lib.optional (p.dumpdir != null) "dumpdir ${p.dumpdir}")
+    ++ [
+      "bindcmdaddress /" # disable unix socket
+      "pidfile /run/${name}.pid"
+    ]
     ++ [p.extraConfig];
 
   config = writeText "chrony.conf"
     (concatStringsSep "\n" (configFile params));
 in longrun {
-  name = "ntp"; # bad name, needs to be unique
+  inherit name;
   run = "${chrony}/bin/chronyd -f ${config} -d";
 }

modules/ppp/common.nix

@@ -15,25 +15,26 @@ let
   inherit (builtins) toJSON toString typeOf;
 
   ip-up = writeAshScript "ip-up" {} ''
+    exec >&5 2>&5
     . ${serviceFns} 
-    (in_outputs ${name}
-     echo $1 > ifname
-     echo $2 > tty
-     echo $3 > speed
-     echo $4 > address
-     echo $5 > peer-address
-     echo $DNS1 > ns1
-     echo $DNS2 > ns2
-    )
-    echo >/proc/self/fd/10
+    in_outputs ${name}
+    echo $1 > ifname
+    echo $2 > tty
+    echo $3 > speed
+    echo $4 > address
+    echo $5 > peer-address
+    set +o nounset
+    if test -n "''${DNS1}" ;then echo ''${DNS1} > ns1 ; fi
+    if test -n "''${DNS2}" ;then echo ''${DNS2} > ns2 ; fi
+    test -e ipv6-address && echo >/proc/self/fd/10
   '';
   ip6-up = writeAshScript "ip6-up" {} ''
+    exec >&5 2>&5
     . ${serviceFns} 
-    (in_outputs ${name}
-     echo $4 > ipv6-address
-     echo $5 > ipv6-peer-address
-    )
-    echo >/proc/self/fd/10
+    in_outputs ${name}
+    echo $4 > ipv6-address
+    echo $5 > ipv6-peer-address
+    test -e ifname && echo >/proc/self/fd/10
   '';
   literal_or_output =
     let v = o: ({
@@ -58,6 +59,9 @@ let
         "ipv6-up-script" ip6-up
         "ipparam" name
         "nodetach"
+        # usepeerdns requests DNS servers from peer (which is good),
+        # then attempts to write them to /nix/store/xxxx/ppp/resolv.conf
+        # which causes an unsightly but inconsequential error message
         "usepeerdns"
         "nodefaultroute"
         "logfd" "2"
@@ -69,6 +73,7 @@ let
       chmod 0700 /run/${name}
       in_outputs ${name}
       echo ${escapeShellArgs ppp-options'} | ${output-template}/bin/output-template '{{' '}}' > /run/${name}/ppp-options
+      fdmove -c 5 2 \
       ${command}
     '';
     notification-fd = 10;

pkgs/liminix-tools/services/default.nix

@@ -14,8 +14,9 @@ let
     . ${serviceFns}
     ${commands}
   '';
-  cleanupScript = name : ''
+  cleanupScript = name : cmds : ''
     #!/bin/sh
+    ${if cmds != null then cmds else ""}
     if test -d ${prefix}/${name} ; then rm -rf ${prefix}/${name} ; fi
   '';
   service = {
@@ -51,6 +52,7 @@ let
   longrun = {
     name
     , run
+    , finish ? null
     , notification-fd ? null
     , buildInputs ? []
     , producer-for ? null
@@ -68,7 +70,7 @@ let
       buildInputs = buildInputs ++ lib.optional (producer-for == null) logger;
       serviceType = "longrun";
       run = serviceScript run;
-      finish = cleanupScript name;
+      finish = cleanupScript name finish;
       producer-for = if producer-for != null then producer-for else "${name}-log";
     });
 
@@ -82,7 +84,7 @@ let
     up = writeScript "${name}-up" (serviceScript up);
     down = writeScript
       "${name}-down"
-      "${serviceScript down}\n${cleanupScript name}";
+      "${serviceScript down}\n${cleanupScript name null}";
   });
   bundle = { contents ? []
     , dependencies ? []

pkgs/ppp/default.nix

@@ -3,12 +3,10 @@
   stdenv,
   autoreconfHook,
   substituteAll,
-# , openssl
 }: stdenv.mkDerivation {
   pname  = "ppp";
   version = "2.5.0";
   nativeBuildInputs = [ autoreconfHook ];
-#  buildInputs= [ openssl ];
 
   src = fetchFromGitHub {
     repo = "ppp";
@@ -22,6 +20,7 @@
     "--disable-peap"
     "--disable-openssl-engine"
     "--without-openssl"
+    "--with-runtime-dir=/run/"
   ];
 
   postPatch = ''

pkgs/pppoe/default.nix

@@ -22,10 +22,6 @@ stdenv.mkDerivation rec {
     export PPPD=${ppp}/sbin/pppd
   '';
 
-  configureFlags = lib.optionals (stdenv.buildPlatform != stdenv.hostPlatform) [
-    "rpppoe_cv_pack_bitfields=rev"
-  ];
-
   postConfigure = ''
     sed -i Makefile -e 's@DESTDIR)/etc/ppp@out)/etc/ppp@'
     sed -i Makefile -e 's@PPPOESERVER_PPPD_OPTIONS=@&$(out)@'

pkgs/systemconfig/build-system-install.sh

@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+# this shell script can be run on the build system to
+# min-copy-closure the system configuration onto the device
+# and reboot/restart services as requested
+
+die() {
+    echo "$@"
+    exit 1
+}
+
+# add min-copy-closure to the path. removing junk characters
+# inserted by default.nix (q.v.)
+min_copy_closure=@min-copy-closure@; PATH=${min_copy_closure//_/}/bin:$PATH
+
+ssh_command=${SSH_COMMAND-ssh}
+
+reboot="reboot"
+
+case "$1" in
+    "--no-reboot")
+	unset reboot
+	shift
+	;;
+    "--fast")
+	reboot="soft"
+	shift
+	;;
+esac
+
+target_host=$1
+shift
+
+test -n "$target_host" || \
+    die "Usage: $0 [--no-reboot] [--fast] target-host"
+
+toplevel=$(cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && readlink `pwd` )
+test -e $toplevel/etc/nix-store-paths || die "missing etc/nix-store-paths, is this really a system configuration?"
+echo installing from systemConfiguration $toplevel to host $target_host
+
+min-copy-closure $target_host $toplevel
+set -x
+$ssh_command $target_host $toplevel/bin/install
+case "$reboot" in
+    reboot)
+	$ssh_command $target_host "sync; source /etc/profile; reboot"
+	;;
+    soft)
+	$ssh_command $target_host $toplevel/bin/restart-services
+	;;
+    *)
+	;;
+esac

pkgs/systemconfig/default.nix

@@ -7,6 +7,7 @@
 {
   writeText,
   writeFennel,
+  buildPackages,
   lib,
   s6-init-bin,
   closureInfo,
@@ -82,6 +83,11 @@ in attrset:
       $STRIP --remove-section=.note  --remove-section=.comment --strip-all makedevs -o $out/bin/activate
       ln -s ${s6-init-bin}/bin/init $out/bin/init
       cp -p ${writeFennel "restart-services" {} ./restart-services.fnl} $out/bin/restart-services
+      # obfuscate the store path of min-copy-closure so that the output
+      # closure doesn't include a bunch of build system stuff
+      f=${buildPackages.min-copy-closure}; f=$(echo $f | sed 's/\(.....\)/\1_/g')
+      substitute ${./build-system-install.sh} $out/install.sh --subst-var-by min-copy-closure $f
+      chmod +x $out/install.sh
       cat > $out/bin/install <<EOF
       #!/bin/sh -e
       prefix=\''${1-/}