add pppoe

README.md

@@ -32,11 +32,53 @@ you plan to install onto it. For example:
 
 `outputs.default` is intended to do something appropriate for the
 device, whatever that is. For the qemu device, it creates a directory
-containing a squashfs root image and a kernel, with which you could
+containing a squashfs root image and a kernel.
-then run
 
-    ./run-qemu.sh result/vmlinux result/squashfs
 
+## QEMU
+
+QEMU is useful for developing userland without needing to keep
+flashing or messing with U-Boot: it also enables testing against
+emulated network peers using [QEMU socket networking](https://wiki.qemu.org/Documentation/Networking#Socket),
+which may be preferable to letting Liminix loose on your actual LAN.
+
+We have some tooling to make this easier.
+
+### Networks
+
+We observe these conventions for QEMU network sockets, so that we can
+run multiple emulated instances and have them wired up to each other
+in the right way
+
+* multicast 230.0.0.1:1234  : access (interconnect between router and "isp")
+* multicast 230.0.0.1:1235  : lan
+* multicast 230.0.0.1:1236  : world (the internet)
+
+### Running instances
+
+`./scripts/run-qemu.sh` accepts a kernel vmlinux image and a squashfs
+and runs qemu with appropriate config for two ethernet interfaces
+hooked up to "lan" and "access" respectively. It connects the Liminix serial console
+and the [QEMU monitor](https://www.qemu.org/docs/master/system/monitor.html) to
+stdin/stdout. Use ^P (not ^A) to switch to the monitor.
+
+If you run with `--background /path/to/unix/socket` it will fork into
+the background and open a Unix socket at that pathname to communicate
+on. Use `./scripts/connect-qemu.sh` to connect to it, and ^O to
+disconnect.
+
+### Emulated upstream connection
+
+In the tests/support/ppp-server directory there are instructions and a script
+to configure [Mikrotik RouterOS](https://mikrotik.com/software) as
+a PPPoE access concentrator connected to the `access` and `world`
+networks, so that Liminix PPPoE client support can be tested.
+_Liminix does not provide RouterOS licences and it is your own
+responsibility if you use this to ensure you're compliant with
+the terms of Mikrotik's licencing._
+
+This may be supplemented or replaced in time with configuurations for
+RP-PPPoE and/or Accel PPP.
 
 ## Running tests
 
@@ -53,3 +95,5 @@ took full advantage of the basic application armoring features
 provided by the operating system. Indeed, only one or two models even
 came close, and no brand did well consistently across all models
 tested"
+
+* [A PPPoE Implementation for Linux](https://static.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/skoll/skoll_html/index.html): "Many DSL service providers use PPPoE for residential broadband Internet access. This paper briefly describes the PPPoE protocol, presents strategies for implementing it under Linux and describes in detail a user-space implementation of a PPPoE client."

STYLE.md

@@ -0,0 +1,22 @@
+# Some notes on Nix language style
+
+In an attempt to keep this more consistent than NixWRT ended up being,
+here is a Nix language style guide for this repo.
+
+* favour `callPackage` over raw `import` for calling derivations
+or any function that may generate one - any code that might need
+`pkgs` or parts of it.
+
+* prefer `let inherit (quark) up down strange charm` over `with
+quark`, in any context where the scope is more than a single
+expression or there is more than one reference to `up`, `down` etc.
+`with pkgs; [ foo bar baz]` is OK,
+`with lib; stdenv.mkDerivation { ... } ` is usually not.
+
+* <liminix> is defined only when running tests, so don't refer to it in
+"application" code
+
+* the parameters to a derivation are sorted alphabetically, except for
+`lib`, `stdenv` and maybe other non-package "special cases"
+
+* indentation is whatever emacs nix-mode says it is

THOUGHTS.txt

@@ -0,0 +1,71 @@
+Thu Sep 22 00:12:42 BST 2022
+
+Making quite reasonable progress, though only running under emulation.
+Since almost everything so far has been a recap of nixwrt, that's to
+be expected.
+
+The example config starts some services at boot, or at least attempts
+to. Next we shoud
+
+ - add some network config to run-qemu
+ - implement udhcp and odhcp properly to write outputs
+  and create resolv.conf and all that
+ - write some kind of test so we can refactor the crap
+ - not let the tests write random junk everywhere
+
+Thu Sep 22 12:46:36 BST 2022
+
+We can store outputs in the s6 scan directory, it seems:
+
+> There is, however, a guarantee that s6-supervise will never touch subdirectories named data or env. So if you need to store user information in the service directory with the guarantee that it will never be mistaken for a configuration file, no matter the version of s6, you should store that information in the data or env subdirectories of the service directory.
+
+https://skarnet.org/software/s6/servicedir.html
+
+> process 'store/pj0b27l5728cypa5mmagz0q8ibzpik0h-execline-mips-unknown-linux-musl-2.9.0.1-bin/bin/execlineb' started with executable stack
+
+https://skarnet.org/lists/skaware/1550.html
+
+
+Thu Sep 22 16:14:49 BST 2022
+
+what network peers do we want to model for testing?
+
+- wan: pppoe
+- wan: ip over ethernet, w/ dhcp service provided
+- wan: l2tp over (ip over ethernet, w/ dhcp service provided)
+- lan: something with a dhcp client
+
+https://accel-ppp.readthedocs.io/en/latest/ could use this for testing
+pppoe and l2tp?
+
+
+Thu Sep 22 22:57:47 BST 2022
+
+To build a nixos vm with accel-ppp installed (not yet configured)
+
+  nix-build '<nixpkgs/nixos>' -A vm -I nixos-config=./tests/ppp-server-configuration.nix -o ppp-server
+  QEMU_OPTS="-display none -serial mon:stdio -nographic" ./ppp-server/bin/run-nixos-vm
+
+To test it's configured I thought I'd run it against an OpenWrt qemu
+install, so, fun with qemu networking ensues. This config in ../openwrt-qemu
+is using two multicast socket networks -
+
+nix-shell -p qemu --run "./run.sh ./openwrt-22.03.0-x86-64-generic-kernel.bin openwrt-22.03.0-x86-64-generic-ext4-rootfs.img "
+
+so hopefully we can spin up other VMs connected either to its lan or
+its wan: *however* we do first need to configure its wan to use pppoe
+
+uci set network.wan=interface
+uci set network.wan.device='eth1'
+uci set network.wan.proto='pppoe'
+uci set network.wan.username='db123@a.1'
+uci set network.wan.password='NotReallyTheSecret'
+
+(it's ext4 so this will probably stick)
+
+
+Fri Sep 23 10:27:22 BST 2022
+
+* mcast=230.0.0.1:1234  : access (interconnect between router and isp)
+* mcast=230.0.0.1:1235  : lan
+* mcast=230.0.0.1:1236  : world (the internet)

default.nix

@@ -4,15 +4,10 @@
 let
   overlay = import ./overlay.nix;
   nixpkgs = import <nixpkgs> ( device.system // {overlays = [overlay]; });
-  baseConfig = {
+  config = (import ./merge-modules.nix) [
-    systemPackages = [];
+    (import ./modules/base.nix { inherit device; })
-    services = {};
+    <liminix-config>
-    kernel = device.kernel;
+  ] nixpkgs.pkgs;
-  };
-  config = baseConfig // (import <liminix-config>) {
-    config = baseConfig;
-    inherit (nixpkgs) pkgs;
-  };
   finalConfig = config // {
     packages = (with nixpkgs.pkgs; [ s6-rc ]) ++
                config.systemPackages ++

merge-modules.nix

@@ -0,0 +1,8 @@
+modules : pkgs :
+let evalModules = (import <nixpkgs/lib>).evalModules;
+in (evalModules {
+  modules =
+    [
+      { _module.args = { inherit pkgs; lib = pkgs.lib; }; }
+    ] ++ modules;
+}).config

modules/base.nix

@@ -0,0 +1,17 @@
+{ device } :
+{ lib, ...}:
+let inherit (lib) mkEnableOption mkOption types;
+in {
+  options = {
+    systemPackages = mkOption {
+      type = types.listOf types.package;
+    };
+    services = mkOption {
+      type = types.anything;
+    };
+    kernel = mkOption {
+      type = types.anything;
+      default = { inherit (device.kernel) config checkedConfig; };
+    };
+  };
+}

overlay.nix

@@ -3,4 +3,50 @@ final: prev: {
   s6-init-files = final.callPackage ./pkgs/s6-init-files {};
   strace = prev.strace.override { libunwind = null; };
   liminix = final.callPackage ./pkgs/liminix-tools {};
+  pppoe = prev.rpPPPoE.overrideAttrs (o: {
+    # use newer rp-pppoe, it builds cleanly
+    src = final.fetchFromGitHub {
+      owner = "dfskoll";
+      repo = "rp-pppoe";
+      rev = "7cfd8c0405d14cf1c8d799d41d8207fd707979c1";
+      hash = "sha256-MFdCwNj8c52blxEuXH5ltT2yYDmKMH5MLUgtddZV25E=";
+    };
+  });
+  ppp =
+    (prev.ppp.override {
+      libpcap = null;
+    }).overrideAttrs (o : {
+      stripAllList = [ "bin" ];
+      buildInputs = [];
+
+      # patches =
+      #   o.patches ++
+      #   [(final.fetchpatch {
+      #     name = "ipv6-script-options.patch";
+      #     url = "https://github.com/ppp-project/ppp/commit/874c2a4a9684bf6938643c7fa5ff1dd1cf80aea4.patch";
+      #     sha256 = "sha256-K46CKpDpm1ouj6jFtDs9IUMHzlRMRP+rMPbMovLy3o4=";
+      #   })];
+
+      postPatch = ''
+        sed -i -e  's@_PATH_VARRUN@"/run/"@'  pppd/main.c
+        sed -i -e  's@^FILTER=y@# FILTER unset@'  pppd/Makefile.linux
+        sed -i -e  's/-DIPX_CHANGE/-UIPX_CHANGE/g'  pppd/Makefile.linux
+      '';
+      buildPhase = ''
+        runHook preBuild
+        make -C pppd CC=$CC USE_TDB= HAVE_MULTILINK= USE_EAPTLS= USE_CRYPT=y
+        make -C pppd/plugins/pppoe CC=$CC
+        make -C pppd/plugins/pppol2tp CC=$CC
+        runHook postBuild;
+      '';
+      installPhase = ''
+        runHook preInstall
+        mkdir -p $out/bin $out/lib/pppd/2.4.9
+        cp pppd/pppd pppd/plugins/pppoe/pppoe-discovery $out/bin
+        cp pppd/plugins/pppoe/pppoe.so $out/lib/pppd/2.4.9
+        cp pppd/plugins/pppol2tp/{open,pppo}l2tp.so $out/lib/pppd/2.4.9
+        runHook postInstall
+      '';
+      postFixup = "";
+    });
 }

pkgs/liminix-tools/default.nix

@@ -77,6 +77,7 @@ in {
       name = "${interface.device}.odhcp";
       run = "odhcpcd ${interface.device}";
     };
+    pppoe = callPackage ./networking/pppoe.nix {};
   };
   services = {
     inherit longrun oneshot bundle target;

pkgs/liminix-tools/networking/pppoe.nix

@@ -0,0 +1,48 @@
+{
+  liminix
+, lib
+, busybox
+, ppp
+, pppoe
+, writeShellScript
+} :
+let
+  inherit (liminix.services) longrun;
+  ip-up = writeShellScript "ip-up" ''
+action=$1
+env > /run/udhcp.values
+
+set_address() {
+    ip address replace $ip/$mask dev $interface
+    mkdir -p data/outputs
+    for i in lease mask ip router siaddr dns serverid subnet opt53 interface ; do
+        echo ''${!i} > data/outputs/$i
+    done
+}
+case $action in
+  deconfig)
+    ip address flush $interface
+    ip link set up dev $interface
+    ;;
+  bound)
+    # this doesn't actually replace, it adds a new address.
+    set_address
+    ;;
+  renew)
+    set_address
+    ;;
+  nak)
+    echo "received NAK on $interface"
+    ;;
+esac
+'';
+
+in
+interface: {
+  synchronous ? false
+, ppp-options ? []
+, ...
+} @ args: longrun {
+  name = "${interface.device}.ppppoe";
+  run = "${ppp}/bin/pppd pty '${pppoe}/bin/pppoe -I ${interface.device}' ${lib.concatStringsSep " " ppp-options}" ;
+}

run-qemu.sh

@@ -1,8 +0,0 @@
-#!/usr/bin/env nix-shell
-#! nix-shell -i bash -p qemu
-
-qemu-system-mips \
-    -M malta -m 256 \
-    -append "default console=ttyS0,38400n8 panic=10 oops=panic init=/bin/init  loglevel=8 root=/dev/vda" \
-    -drive file=$2,format=raw,readonly,if=virtio \
-    -kernel $1  -nographic -display none -serial mon:stdio

scripts/connect-qemu.sh

@@ -0,0 +1,2 @@
+#!/usr/bin/env sh
+nix-shell -p socat --run "socat -,raw,echo=0,icanon=0,isig=0,icrnl=0,escape=0x0f    unix-connect:$1"

scripts/run-qemu.sh

@@ -0,0 +1,22 @@
+#!/usr/bin/env nix-shell
+#! nix-shell -i bash -p qemu
+
+if test "$1" = "--background" ; then
+    socket=$2
+    echo "running in background, socket is $socket"
+    flags="--daemonize -chardev socket,id=sock,path=$2,server=on,wait=off,mux=on -mon chardev=sock,mode=readline -serial chardev:sock  "
+    shift;shift
+else
+    flags="-serial mon:stdio"
+fi
+
+qemu-system-mips \
+    -M malta -m 256 \
+    -echr 16 \
+    -append "default console=ttyS0,38400n8 panic=10 oops=panic init=/bin/init  loglevel=8 root=/dev/vda" \
+    -drive file=$2,format=raw,readonly=on,if=virtio \
+    -netdev socket,id=access,mcast=230.0.0.1:1234 \
+    -device virtio-net-pci,disable-legacy=on,disable-modern=off,netdev=access,mac=ba:ad:1d:ea:21:02 \
+    -netdev socket,id=lan,mcast=230.0.0.1:1235 \
+    -device virtio-net-pci,disable-legacy=on,disable-modern=off,netdev=lan,mac=ba:ad:1d:ea:21:01 \
+    -kernel $1 -display none $flags

tests/module/configuration.nix

@@ -0,0 +1,10 @@
+{ config, pkgs, ... } :
+{
+  imports = [ ./defs.nix ./module.nix ];
+  config = {
+    services.a.enable = true;
+    services.b.enable = true;
+
+    systemPackages = [ pkgs.hello ] ;
+  };
+}

tests/module/defs.nix

@@ -0,0 +1,16 @@
+{ lib, ...}:
+let inherit (lib) mkEnableOption mkOption types;
+in {
+  options = {
+    services.a = {
+      enable = mkEnableOption "hello service";
+    };
+    services.b = {
+      enable = mkEnableOption "other service";
+    };
+    services.z = mkOption {   };
+    systemPackages = mkOption {
+      type = types.listOf types.package;
+    };
+  };
+}

tests/module/module.nix

@@ -0,0 +1,6 @@
+{ config, pkgs, ... } :
+{
+  services.z = pkgs.figlet;
+
+  systemPackages = [ pkgs.units ] ;
+}

tests/module/run.sh

@@ -0,0 +1,2 @@
+set -e
+NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 nix-build --arg device "import <liminix/devices/qemu.nix>" test.nix

tests/module/test.nix

@@ -0,0 +1,17 @@
+{ device }:
+let
+  overlay = import <liminix/overlay.nix> ;
+  nixpkgs = import <nixpkgs> ( device.system // {overlays = [overlay]; });
+  inherit (nixpkgs) lib pkgs;
+  inherit (lib.asserts) assertMsg;
+  config =
+    (import <liminix/merge-modules.nix>) [./configuration.nix] pkgs;
+  res1 = assertMsg
+    # check we have packages from both modules
+    (config.systemPackages == ( with pkgs; [ units hello ])) "failed";
+  res2 = let s = config.services;
+         in assertMsg (s.a.enable && s.b.enable && (s.z != null) ) "failed";
+in pkgs.writeText "foo" ''
+   ${if res1 then "OK" else "not OK"}
+   ${if res2 then "OK" else "not OK"}
+''

tests/pppoe/configuration.nix

@@ -0,0 +1,47 @@
+{ config, pkgs, ... } :
+let
+  inherit (pkgs.liminix.networking) interface address pppoe;
+  inherit (pkgs.liminix.services) oneshot longrun bundle target output;
+in rec {
+  services.loopback =
+    let iface = interface { type = "loopback"; device = "lo";};
+    in bundle {
+      name = "loopback";
+      contents = [
+        (address iface { family = "inet4"; addr ="127.0.0.1";})
+        (address iface { family = "inet6"; addr ="::1";})
+      ];
+    };
+
+  kernel.config = {
+    "PPP" = "y";
+    "PPPOE" = "y";
+    "PPPOL2TP" = "y";
+  };
+
+  services.pppoe =
+    let iface = interface { type = "hardware"; device = "eth0"; };
+    in pppoe iface {};
+
+  services.defaultroute4 =
+    let iface = services.pppoe;
+    in oneshot {
+      name = "defaultroute4";
+      up = ''
+        ip route add default gw $(cat ${output iface "address"})
+        echo "1" > /sys/net/ipv4/$(cat ${output iface "ifname"})
+      '';
+      down = ''
+        ip route del default gw $(cat ${output iface "address"})
+        echo "0" > /sys/net/ipv4/$(cat ${output iface "ifname"})
+      '';
+      dependencies = [iface];
+    };
+
+  services.default = target {
+    name = "default";
+    contents = with services; [ loopback defaultroute4 ];
+  };
+
+  systemPackages = [ pkgs.hello ] ;
+}

tests/pppoe/run.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env sh
+
+
+set -e
+
+cleanup(){
+    echo "do cleanup";
+}
+trap cleanup EXIT
+trap 'echo "command $(eval echo $BASH_COMMAND) failed with exit code $?"; exit $?' ERR
+
+NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 nix-build '<liminix>' -I liminix-config=./configuration.nix --arg device "import <liminix/devices/qemu.nix>" -A outputs.default $*
+
+
+if ! ( echo "cont" | socat -   unix-connect:../support/ppp-server/qemu-monitor); then
+    echo "need pppoe server running"
+    exit 1
+fi
+
+../../scripts/run-qemu.sh result/vmlinux result/squashfs

tests/smoke/configuration.nix

@@ -1,4 +1,4 @@
-{ config, pkgs } :
+{ config, pkgs, ... } :
 let
   inherit (pkgs.liminix.networking) interface address udhcpc odhcpc;
   inherit (pkgs.liminix.services) oneshot longrun bundle target output;

tests/smoke/run.sh

@@ -2,10 +2,12 @@ set -e
 NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1 nix-build '<liminix>' -I liminix-config=./configuration.nix --arg device "import <liminix/devices/$DEVICE.nix>" -A outputs.squashfs -o smoke.img $*
 
 TESTS=$(cat <<"EOF"
-
+test -n "${TMPDIR}"
-trap 'echo "command $(eval echo $BASH_COMMAND) failed with exit code $?"; exit $?' ERR
 dest_path=${TMPDIR}/smoke.img-$$
 echo $dest_path
+cleanup(){  test -n ${dest_path} && test -d ${dest_path} && chmod -R +w ${dest_path} && rm -rf ${dest_path};  }
+trap cleanup EXIT
+trap 'echo "command $(eval echo $BASH_COMMAND) failed with exit code $?"; exit $?' ERR
 unsquashfs -q -d $dest_path -excludes smoke.img  /dev
 cd $dest_path;
 db=nix/store/*-s6-rc-db/compiled/