dan
/
liminix
1
0
Fork 5
Code Issues 3 Pull Requests 2 Projects Releases Wiki Activity

convert docs to restructuredText

module-based-network
Daniel Barlow 2023-02-16 22:06:50 +00:00
parent b60ce985b7
commit 40739d780b
7 changed files with 371 additions and 192 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@ -1,4 +1,6 @@
*.img
*.bin
result
result-*
result-*
*.qcow2
_build

192
README.md
View File

@ -1,12 +1,8 @@
# Liminix
A Nix-based system for configuring consumer wifi routers.
## What is this?
This is a Nix-based collection of software tailored for domestic wifi
router or IoT device devices, of the kind that OpenWrt or DD-WRT or
Gargoyle or Tomato run on. It's a reboot/restart/rewrite of NixWRT.
A Nix-based system for configuring consumer wifi routers or IoT device
devices, of the kind that OpenWrt or DD-WRT or Gargoyle or Tomato run
on. It's a reboot/restart/rewrite of NixWRT.
This is not NixOS-on-your-router: it's aimed at devices that are
underpowered for the full NixOS experience. It uses busybox tools,
@ -17,184 +13,14 @@ of "limen", or "of the threshold". Your router stands at the threshold
of your (online) home and everything you send to/receive from the
outside word goes across it.
### What about NixWRT?
## What about NixWRT?
This is an in-progress rewrite of NixWRT, incorporating Lessons
Learned. That said, as of today (September 2022) it is not yet
anywhere _near_ feature parity.
Learned.
Liminix will eventually provide these differentiators over NixWRT:
## Documentation
* a writable filesystem so that software updates or reconfiguration
(e.g. changing passwords) don't require taking the device offline to
reflash it.
Documentation is in the [doc](doc/) directory. You can build it
by running
* more flexible service management with dependencies, to allow
configurations such as "route through PPPoE if it is healthy, with
fallback to LTE"
* a spec for valid configuration options (a la NixOS module options)
to that we can detect errors at evaluation time instead of producing
a bad image.
* a network-based mechanism for secrets management so that changes can
be pushed from a central location to several Liminix devices at once
* send device metrics and logs to a monitoring/alerting/o11y
infrastructure
Today though, it does approximately none of these things and certainly
not on real hardware.
## Building
### For the device
These instructions assume you have nixpkgs checked out in a peer
directory of this one.
You need a `configuration.nix` file pointed to by `<liminix-config>`, a
hardware device definition as argument `device`, and to choose an
appropriate output attribute depending on what your device is and how
you plan to install onto it. For example:
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=./tests/smoke/configuration.nix --arg device "import ./devices/qemu" -A outputs.default
`outputs.default` is intended to do something appropriate for the
device, whatever that is. For the qemu device, it creates a directory
containing a squashfs root image and a kernel.
### For the build machine
Liminix also includes some tools intended for the build machine. You can
run
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
to get a shell environment with (currently) a tftp server and
a script to start a PPPoE server in QEMU for testing against.
#### QEMU
QEMU is useful for developing userland without needing to keep
flashing or messing with U-Boot: it also enables testing against
emulated network peers using [QEMU socket networking](https://wiki.qemu.org/Documentation/Networking#Socket),
which may be preferable to letting Liminix loose on your actual LAN.
We observe these conventions for QEMU network sockets, so that we can
run multiple emulated instances and have them wired up to each other
in the right way
* multicast 230.0.0.1:1234 : access (interconnect between router and "isp")
* multicast 230.0.0.1:1235 : lan
* multicast 230.0.0.1:1236 : world (the internet)
### Running Liminix in Qemu
In a `buildEnv` nix-shell, you can use the `mips-vm` command
to run Qemu with appropriate config for two ethernet interfaces
hooked up to "lan" and "access" respectively. It connects the Liminix
serial console and the [QEMU monitor](https://www.qemu.org/docs/master/system/monitor.html) to stdin/stdout. Use ^P (not ^A) to switch to the monitor.
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' --run "mips-vm result/vmlinux result/squashfs"
If you run with `--background /path/to/some/directory` as the first
parameter, it will fork into the background and open Unix sockets in
that directory for console and monitor. Use `connect-vm` (also in the
`buildEnv` environment) to connect to either of these sockets, and ^O
to disconnect.
### Emulated upstream connection
In pkgs/routeros there is a derivation to install and configure
[Mikrotik RouterOS](https://mikrotik.com/software) as a PPPoE access
concentrator connected to the `access` and `world` networks, so that
Liminix PPPoE client support can be tested.
This is made available in the `buildEnv`, so you can do something like
mkdir ros-sockets
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
nix-shell$ routeros ros-sockets
nix-shell$ connect-vm ./ros-sockets/console
to start it and connect to it.
_Liminix does not provide RouterOS licences and it is your own
responsibility if you use this to ensure you're compliant with the
terms of Mikrotik's licencing._It may be supplemented or replaced in
time with configurations for RP-PPPoE and/or Accel PPP.
## Running tests
Assuming you have nixpkgs checked out in a peer directory of this one,
you can run all of the tests by evaluating `ci.nix`:
nix-build --argstr liminix `pwd` --argstr nixpkgs `pwd`/../nixpkgs --argstr unstable `pwd`/../unstable-nixpkgs/ ci.nix
or to run a named test, use the `-A` flag. For example, `-A pppoe`
## Hardware
How you get the thing onto hardware will vary according to the device,
but is likely to involve U-Boot and TFTP.
There is a rudimentary TFTP server bundled with the system which runs
from the command line, has an allowlist for client connections, and
follows symlinks, so you can have your device download images direct
from the `./result` directory without exposing `/nix/store/` to the
internet or mucking about copying files to `/tftproot`. If the
permitted device is to be given the IP address 192.168.8.251 you might
do something like this:
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' \
--run "tufted -a 192.168.8.251 result"
## Troubleshooting
### Diagnosing unexpectedly large images
Sometimes you can add a package and it causes the image size to balloon
because it has dependencies on other things you didn't know about. Build the
`outputs.manifest` attribute, which is a json representation of the
filesystem, and you can run `nix-store --query` on it.
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=path/to/your/configuration.nix --arg device "import ./devices/qemu" -A outputs.manifest -o manifest
nix-store -q --tree manifest
## Contributing
Contributions are welcome, though in these early days there may be a
bit of back and forth involved before patches are merged. Have a read
of [CONTRIBUTING](CONTRIBUTING.md) and [STYLE](STYLE.md) and try to
intuit the unarticulated vision :-)
Liminix' primary repo is https://gti.telent.net/dan/liminix but that
doesn't help you much because registrations are closed:
* There's a [mirror on Github](https://github.com/telent/liminix) for
convenience and visibility: you can open PRs against that
* or, you can send me your patch by email using [`git send-email`](https://git-send-email.io/)
* or in the future, some day, we will have federated Gitea using
ActivityPub.
## Articles of interest
* [Build Safety of Software in 28 Popular Home Routers](https://cyber-itl.org/assets/papers/2018/build_safety_of_software_in_28_popular_home_routers.pdf):
"of the access points and routers we reviewed, not a single one
took full advantage of the basic application armoring features
provided by the operating system. Indeed, only one or two models even
came close, and no brand did well consistently across all models
tested"
* [A PPPoE Implementation for Linux](https://static.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/skoll/skoll_html/index.html): "Many DSL service providers use PPPoE for residential broadband Internet access. This paper briefly describes the PPPoE protocol, presents strategies for implementing it under Linux and describes in detail a user-space implementation of a PPPoE client."
* [PPP IPV6CP vs DHCPv6 at AAISP](https://www.revk.uk/2011/01/ppp-ipv6cp-vs-dhcpv6.html)
nix-shell -p sphinx --run "make -C doc html"

225
doc/developer.rst Normal file
View File

@ -0,0 +1,225 @@
Developer Manual
################
As a developer working on Liminix, or implementing a service or
module, you probably want to test your changes more conveniently
than by building and flashing a new image every time. This manual
documents various affordances for iteration and experiments.
In general, packages and tools that run on the "build" machine are
available in the ``buildEnv`` derivation.
.. code-block:: console
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
Emulated devices
****************
Liminix has a ``qemu`` device, which generates images suitable for
running on your build machine using the free `QEMU machine emulator <http://www.qemu.org>`_.
This is useful for developing userland without needing to keep
flashing or messing with U-Boot: it also enables testing against
emulated network peers using `QEMU socket networking <https://wiki.qemu.org/Documentation/Networking#Socket>`_,
which may be preferable to letting Liminix loose on your actual LAN.
To build it,
.. code-block:: console
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=path/to/your/configuration.nix --arg device "import ./devices/qemu" -A outputs.default
In a ``buildEnv`` nix-shell, you can use the ``mips-vm`` command
to run Qemu with appropriate options. It connects the Liminix
serial console and the `QEMU monitor <https://www.qemu.org/docs/master/system/monitor.html>`_ to stdin/stdout. Use ^P (not ^A) to switch to the monitor.
.. code-block:: console
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' --run "mips-vm result/vmlinux result/squashfs"
If you run with ``--background /path/to/some/directory`` as the first
parameter, it will fork into the background and open Unix sockets in
that directory for console and monitor. Use ``connect-vm`` (also in the
``buildEnv`` environment) to connect to either of these sockets, and ^O
to disconnect.
Networking
==========
VMs can network with each other using QEMU
socket networking. We observe these conventions, so that we can run
multiple emulated instances and have them wired up to each other in
the right way
* multicast 230.0.0.1:1234 : access (interconnect between router and "isp")
* multicast 230.0.0.1:1235 : lan
* multicast 230.0.0.1:1236 : world (the internet)
A VM started with ``mips-vm`` is connected to "lan" and "access", and
the emulated border network gateway (see below) runs PPPoE and is
connected to "access" and "world".
Border Network Gateway
----------------------
In pkgs/routeros there is a derivation to install and configure
`Mikrotik RouterOS <https://mikrotik.com/software>`_ as a PPPoE access
concentrator connected to the ``access`` and ``world`` networks, so that
Liminix PPPoE client support can be tested without actual hardware.
This is made available as the ``routeros`` command in ``buildEnv``, so you
can do something like::
mkdir ros-sockets
nix-shell -A buildEnv --arg device '(import ./devices/qemu)'
nix-shell$ routeros ros-sockets
nix-shell$ connect-vm ./ros-sockets/console
to start it and connect to it. Note that by default it runs in the
background. It is connected to "access" and "world" virtual networks
and runs a PPPoE service on "access" - so a Liminix VM with a
PPPOE client can connect to it and thus reach the virtual internet.
[ check, but pretty sure this is not the actual internet ]
`Liminix does not provide RouterOS licences and it is your own
responsibility if you use this to ensure you're compliant with the
terms of Mikrotik's licencing. It may be supplemented or replaced in
time with configurations for RP-PPPoE and/or Accel PPP.`
Hardware devices
****************
How you get your image onto hardware will vary according to the
device, but is likely to involve taking it apart to add wires to
serial console pads/headers, then using U-Boot to fetch images over
TFTP.
There is a rudimentary TFTP server bundled with the system which runs
from the command line, has an allowlist for client connections, and
follows symlinks, so you can have your device download images direct
from the ``./result`` directory without exposing ``/nix/store/`` to the
internet or mucking about copying files to ``/tftproot``. If the
permitted device is to be given the IP address 192.168.8.251 you might
do something like this:
.. code-block:: console
nix-shell -A buildEnv --arg device '(import ./devices/qemu)' \
--run "tufted -a 192.168.8.251 result"
and then issue appropriate U-boot commands to download and flash the
image.
For quicker development cycle, you can build a TFTP-bootable image
instead of flashing. [ .... add this bit ....]
Running tests
*************
You can run all of the tests by evaluating ``ci.nix``, which is the
input I use in Hydra. Note that it expects Nixpkgs stable `and` unstable
as inputs, because it builds the qemu device against both.
.. code-block:: console
nix-build --argstr liminix `pwd` --arg nixpkgs "<nixpkgs>" \
--argstr unstable `pwd`/../unstable-nixpkgs/ ci.nix
or to run a named test, use the ``-A`` flag. For example, ``-A pppoe``
Troubleshooting
***************
Diagnosing unexpectedly large images
====================================
Sometimes you can add a package and it causes the image size to balloon
because it has dependencies on other things you didn't know about. Build the
``outputs.manifest`` attribute, which is a JSON representation of the
filesystem, and you can run ``nix-store --query`` on it.::
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=path/to/your/configuration.nix --arg device "import ./devices/qemu" -A outputs.manifest -o manifest
nix-store -q --tree manifest
Contributing
************
Contributions are welcome, though in these early days there may be a
bit of back and forth involved before patches are merged:
Please get in touch somehow `before` you invest a lot of time into a
code contribution I haven't asked for. Just so I know it's expected
and you're not wasting time doing something I won't accept or have
already started on.
Nix language style
==================
In an attempt to keep this more consistent than NixWRT ended up being,
here is a Nix language style guide for this repo.
* favour ``callPackage`` over raw ``import`` for calling derivations
or any function that may generate one - any code that might need
``pkgs`` or parts of it.
* prefer ``let inherit (quark) up down strange charm`` over
``with quark``, in any context where the scope is more than a single
expression or there is more than one reference to ``up``, ``down``
etc. ``with pkgs; [ foo bar baz]`` is OK,
``with lib; stdenv.mkDerivation { ... }`` is usually not.
* ``<liminix>`` is defined only when running tests, so don't refer to it
in "application" code
* the parameters to a derivation are sorted alphabetically, except for
``lib``, ``stdenv`` and maybe other non-package "special cases"
* indentation is whatever emacs nix-mode says it is.
* where a ``let`` form defines multiple names, put a newline after the
token ``let``, and indent each name two characters
* to decide whether some code should be a package or a module?
Packages are self-contained - they live in ``/nix/store/eeeeeee-name``
and don't directly change system behaviour by their presence or
absense. modules can add to
``/etc`` or ``/bin`` or other global state, create services, all that
side-effecty stuff. Generally it should be a package unless it
can't be.
Copyright
=========
The Nix code in Liminix is MIT-licenced (same as Nixpkgs), but the
code it combines from other places (e.g. Linux, OpenWrt) may have a
variety of licences. I have no intention of asking for copyright
assignment: just like when submitting to the Linux kernel you retain
the copyright on the code you contribute.
Code of Conduct
===============
Please govern yourself in Liminix project venues according to the guidance in the `geekfeminism "Community Anti-harassment Policy" <https://geekfeminism.wikia.org/wiki/Community_anti-harassment/Policy>`_.
Where to send patches
=====================
Liminix' primary repo is https://gti.telent.net/dan/liminix but that
doesn't help you much, because it doesn't have open registrations.
* There's a `mirror on Github <https://github.com/telent/liminix>`_ for
convenience and visibility: you can open PRs against that
* or, you can send me your patch by email using `git send-email <https://git-send-email.io/>`_
* or in the future, some day, we will have federated Gitea using
ActivityPub.

48
doc/etc.rst Normal file
View File

@ -0,0 +1,48 @@
The Future
##########
What about NixWRT?
This is an in-progress rewrite of NixWRT, incorporating Lessons
Learned. That said, as of today it is not yet at feature parity.
Liminix will eventually provide these differentiators over NixWRT:
* a writable filesystem so that software updates or reconfiguration
(e.g. changing passwords) don't require taking the device offline to
reflash it.
* more flexible service management with dependencies, to allow
configurations such as "route through PPPoE if it is healthy, with
fallback to LTE"
* a spec for valid configuration options (a la NixOS module options)
to that we can detect errors at evaluation time instead of producing
a bad image.
* a network-based mechanism for secrets management so that changes can
be pushed from a central location to several Liminix devices at once
* send device metrics and logs to a monitoring/alerting/o11y
infrastructure
Today though, it does approximately none of these things and certainly
not on real hardware.
Articles of interest
####################
* [Build Safety of Software in 28 Popular Home Routers](https://cyber-itl.org/assets/papers/2018/build_safety_of_software_in_28_popular_home_routers.pdf): "of the access
points and routers we reviewed, not a single one took full
advantage of the basic application armoring features provided by
the operating system. Indeed, only one or two models even came
close, and no brand did well consistently across all models tested"
* [A PPPoE Implementation for Linux](https://static.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/skoll/skoll_html/index.html):
"Many DSL service providers use PPPoE for residential broadband
Internet access. This paper briefly describes the PPPoE protocol,
presents strategies for implementing it under Linux and describes in
detail a user-space implementation of a PPPoE client."
* [PPP IPV6CP vs DHCPv6 at AAISP](https://www.revk.uk/2011/01/ppp-ipv6cp-vs-dhcpv6.html)

15
doc/index.rst
View File

@ -1,15 +1,14 @@
.. Liminix documentation master file, created by
sphinx-quickstart on Wed Feb 15 23:24:26 2023.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Liminix's documentation!
===================================
Liminix
#######
.. toctree::
:maxdepth: 2
:maxdepth: 3
:caption: Contents:
intro
user
developer
etc
Indices and tables

15
doc/intro.rst Normal file
View File

@ -0,0 +1,15 @@
Introduction
############
Liminix is a Nix-based collection of software tailored for domestic
wifi router or IoT device devices, of the kind that OpenWrt or DD-WRT
or Gargoyle or Tomato run on.
This is not NixOS-on-your-router: it's aimed at devices that are
underpowered for the full NixOS experience. It uses busybox tools,
musl instead of GNU libc, and s6-rc instead of systemd.
The Liminix name comes from Liminis, in Latin the genitive declension
of "limen", or "of the threshold". Your router stands at the threshold
of your (online) home and everything you send to/receive from the
outside word goes across it.

64
doc/user.rst Normal file
View File

@ -0,0 +1,64 @@
User Manual
###########
This manual is an early work in progress, not least because Liminix is
not yet ready for users who are not also developers.
Configuring for your use case
*****************************
You need to create a ``configuration.nix`` that describes your router
and the services that you want to run on it. Start by copying
``vanilla-configuration.nix`` and adjusting it.
Building and flashing
*********************
You need to set ``<liminix-config>`` to point to your
``configuration.nix``, the file for your hardware device definition as
argument ``device``, and to choose an appropriate output attribute
depending on what your device is and how you plan to install onto
it. For example:
.. code-block:: console
NIX_PATH=nixpkgs=../nixpkgs:$NIX_PATH nix-build -I liminix-config=./tests/smoke/configuration.nix --arg device "import ./devices/qemu" -A outputs.default
``outputs.default`` is intended to do something appropriate for the
device, whatever that is. For the qemu device, it creates a directory
containing a squashfs root image and a kernel.
As of Feb 2023, flashing devices is not implemented other than by
taking the covers off and connecting wires to the serial console
pads - so, check in the Developer Manual.
Future versions of this manual will at this point refer to
device-specific instructions for installing Liminix using the router's
Web UI or other non-invasive method.
Updating a running device
*************************
Feature forthcoming.
Module options
**************
Foo module
==========
Module docs will go here. This part of the doc should be autogenerated.
Bar module
==========
Baz module
==========
Quuz net device
===============