liminix/THOUGHTS.txt

Thu Sep 22 00:12:42 BST 2022

Making quite reasonable progress, though only running under emulation.
Since almost everything so far has been a recap of nixwrt, that's to
be expected.

The example config starts some services at boot, or at least attempts
to. Next we shoud

 - add some network config to run-qemu
 - implement udhcp and odhcp properly to write outputs
  and create resolv.conf and all that
 - write some kind of test so we can refactor the crap
 - not let the tests write random junk everywhere

Thu Sep 22 12:46:36 BST 2022

We can store outputs in the s6 scan directory, it seems:

> There is, however, a guarantee that s6-supervise will never touch subdirectories named data or env. So if you need to store user information in the service directory with the guarantee that it will never be mistaken for a configuration file, no matter the version of s6, you should store that information in the data or env subdirectories of the service directory.

https://skarnet.org/software/s6/servicedir.html

> process 'store/pj0b27l5728cypa5mmagz0q8ibzpik0h-execline-mips-unknown-linux-musl-2.9.0.1-bin/bin/execlineb' started with executable stack

https://skarnet.org/lists/skaware/1550.html


Thu Sep 22 16:14:49 BST 2022

what network peers do we want to model for testing?

- wan: pppoe
- wan: ip over ethernet, w/ dhcp service provided
- wan: l2tp over (ip over ethernet, w/ dhcp service provided)
- lan: something with a dhcp client

https://accel-ppp.readthedocs.io/en/latest/ could use this for testing
pppoe and l2tp?


Thu Sep 22 22:57:47 BST 2022

To build a nixos vm with accel-ppp installed (not yet configured)

  nix-build '<nixpkgs/nixos>' -A vm -I nixos-config=./tests/ppp-server-configuration.nix -o ppp-server
  QEMU_OPTS="-display none -serial mon:stdio -nographic" ./ppp-server/bin/run-nixos-vm

To test it's configured I thought I'd run it against an OpenWrt qemu
install, so, fun with qemu networking ensues. This config in ../openwrt-qemu
is using two multicast socket networks -

nix-shell -p qemu --run "./run.sh ./openwrt-22.03.0-x86-64-generic-kernel.bin openwrt-22.03.0-x86-64-generic-ext4-rootfs.img "

so hopefully we can spin up other VMs connected either to its lan or
its wan: *however* we do first need to configure its wan to use pppoe

uci set network.wan=interface
uci set network.wan.device='eth1'
uci set network.wan.proto='pppoe'
uci set network.wan.username='db123@a.1'
uci set network.wan.password='NotReallyTheSecret'

(it's ext4 so this will probably stick)


Fri Sep 23 10:27:22 BST 2022

* mcast=230.0.0.1:1234  : access (interconnect between router and isp)
* mcast=230.0.0.1:1235  : lan
* mcast=230.0.0.1:1236  : world (the internet)


Sun Sep 25 20:56:28 BST 2022

TODO - bugs, missing bits, other infelicities as they occur to me:

DONE 1) shutdown doesn't work as its using the busybox one not s6.

2) perhaps we shouldn't have process-based services like dhcp, ppp
implement "address provider interface" - instead have a separate
service for interface address that depends on the service and uses its
output

* ppp is not like dhcp because dhcp finds addresses for an existing
  interface but ppp makes a new one

3) when I killed ppp it restarted, but I don't think it reran
defaultroute which is supposed to depend on it. (Might be important
e.g. if we'd been assigned a different IP address). Investigate
semantics of s6-rc service dependencies

DONE 4) make the pppoe test run unattended

5) write a test for udhcp

6) squashfs size is ~ 14MB for a configuration with not much in it,
look for obvious wastes of space

7) some of the pppoe config should be moved into a ppp service

8) some of configuration.nix (e.g. defining routes) should be moved into
tools

DONE 9) split tools up instead of having it all one file

10) is it OK to depend on squashfs pseudofiles if we might want to
switch to ubifs? will there always be a squashfs underneath? might
we want to change the pseudofiles in an overlay?

11) haven't done (overlayfs) overlays at all

12) overlay.nix needs splitting up

13) upgrade ppp to something with an ipv6-up-script option

14) add ipv6 support generally

15) "ip address add" seems to magically recognise v4 vs v6 but
is that specified or fluke?

16) tighten up the module specs. (DONE) services.foo should be a s6-rc
service, (DONE) kernel config should be checked in some way

DONE 17) rename nixwrt references in kernel builder

18) maybe stop suffixing all the service names with .service

19) syslogd - use busybox or s6?

chat -s -S ogin:--ogin: root / "ip address show dev ppp0 | grep ppp0" 192.168.100.1  "/nix/store/*-s6-linux-init-*/bin/s6-linux-init-hpr -p"


Working towards a general goal of having a derivation we can
usefully run `nix path-info` on - or some other tool that will
tell us what's making the images big. The squashfs doesn't
have this information.

Towards that end (really? can't remember how ...) what would be a
way for packages to declare "I want to add files to /etc"? Is that
even a good idea?

Thinking we should turn s6-init-files back into a real derivation.

Tue Sep 27 00:31:45 BST 2022

> Thinking we should turn s6-init-files back into a real derivation.

This turns out to be Not That Simple, because it contains weird shit
(sticky bits and fifos).

Tue Sep 27 09:50:44 BST 2022

* allow modules to register activation scripts that are run on the
root filesystem once all packages are installed

  - do they run on build or on host? if we're upgrading in place
  how do we ship filesystem changes to the host?

or:

* allow modules to declare environment.*, use pseudofile on build and
  create real files on host. will need to keep the implementation on
  host faily simple because restricted environment