module-system.rst (6964B)
1 Module system 2 ############# 3 4 **Status:** Adopted; implemented in July-September 2023 5 6 7 Context 8 ******* 9 10 Liminix users need a way to assemble a full system configuration by 11 combining smaller, more isolated and reusable components, otherwise 12 systems will be unwieldy and copy-and-paste will be rife. 13 14 15 Alternatives 16 ************ 17 18 NixOS module system 19 =================== 20 21 The NixOS module system addresses many of these concerns. A module is 22 a Nix function which accepts a ``configuration`` attrset and some 23 other parameters, and returns a new fragment of ``configuration`` 24 which is merged into it. It includes a DSL describing the permitted 25 types of values for each key in the configuration, which is used for 26 checking that the supplied parameters are valid and also governs what 27 to do if two modules both specify a value for the same key. (Usually 28 they are "merged", using some type-appropriate concept of merging.) 29 30 Usually a NixOS module looks only (or mostly only) at a particular 31 subtree of the overall configuration which is hardcoded in the module 32 definition, but the configuration fragment it returns may touch any 33 part of the schema. For example, the factorio module refers to 34 ``config.services.factorio``, and it returns values for keys in 35 ``systemd.services.factorio`` and ``networking.firewall``. There is no 36 way to use this module to run **two** factorio services with different 37 config (e.g. on different ports) - the only way to make that 38 possible would be to extend the module definition so that it 39 accepts a collection of game configurations and then create 40 a systemd service for each. 41 42 43 NixWRT module system 44 ==================== 45 46 NixWRT, the (now defunct) predecessor of Liminix, used a homegrown 47 module system modelled on the Nixpkgs overlay pattern. Each module is 48 a function that accepts ``super`` and ``self`` parameters, and 49 using <handwaves>that fixpoint magic thing</handwaves> 50 is called in a chain with the configuration returned by the previous 51 module and the final configuration. 52 53 NixWRT modules mostly don't refer to the configuration object to 54 decide how to configure themselves, but accept their parameters 55 directly as function parameters. For example, the configuration 56 file for "arhcive" (a backup server) includes this text: 57 58 .. code-block:: nix 59 60 (sshd { 61 hostkey = secrets.sshHostKey; 62 authkeys = { root = lib.splitString "\n" secrets.myKeys; }; 63 }) 64 busybox 65 (usbdisk { 66 label = "backup-disk"; 67 mountpoint = "/srv"; 68 fstype = "ext4"; 69 options = "rw"; 70 }) 71 72 This gives us flexibility that NixOS modules don't: for example, if we 73 want to mount two USB disks, we can simply repeat that module twice 74 with different parameters - and the module definition doesn't have to 75 handle it specially. 76 77 However, the downside of this system is that we didn't implement any 78 concept of "types" - there is no type information, so there is no 79 systematic checking that parameters are valid, and if two modules set 80 the same config key then the rules for merging are entirely ad hoc. 81 82 There is a further (arguable) downside, which is that the 83 configuration is not just data - it's now part code. While it could be 84 feasible (though I've never seen it done) to encode a NixOS 85 configuration using Yaml or XML and then manipulate it as data, this 86 is not even possible using the NixWRT system. 87 88 89 Use services for everything 90 =========================== 91 92 The most common properties that a Liminix configuration needs to 93 define are: 94 95 * which services (processes) to run 96 * what packages to install 97 * permitted users and groups 98 * Linux kernel configuration options 99 * Busybox applets 100 * filesystem layout 101 102 Suppose we only had services? 103 104 A Liminix service is (also) a derivation, so it is able to 105 create any files it likes inside its own store path, and 106 transitively require other packages simply by referring to them. 107 If it needs particular kernel options it could define them 108 as kernel modules to be loaded on demand when the service 109 starts (see the nftables module for an example). However: 110 111 * there is no way for a service to add busybox modules 112 113 * it cannot create files outside of its store path, so 114 wouldn't be able to make e.g. :file:`/etc/something.conf` 115 116 * no way to create users/groups. We could steal the DynamicUsers idea 117 from systemd and make them on demand, but this starts to get a bit 118 more complicated. 119 120 These limitations force us to reject this option as a general 121 solution - though we should strive *where possible* to implement 122 functionality as services and to minimise the proportion of Liminix 123 that manipulates the global configuration. 124 125 126 Decision 127 ******** 128 129 "Why not both?" None of these options is sufficient alone, so we are 130 going to do a mixture. 131 132 We will use the NixOS module system, but instead of expecting modules 133 to create systemd services as instances, they will expose "service 134 templates": functions that accept an attrset and return an 135 appropriately configured service that can be assigned by the caller 136 to a key in ``config.services``. 137 138 We will typecheck the service template function parameters using the 139 same type-checking code as NixOS uses for its modules. 140 141 An example may make this clearer: to add an NTP 142 service you first add :file:`modules/ntp` to your ``imports`` list, 143 then you create a service by calling 144 :code:`config.system.service.ntp.build { .... }` with the appropriate 145 service-dependent configuration parameters. 146 147 .. code-block:: nix 148 149 let svc = config.system.service; 150 in { 151 # ... 152 imports = [ 153 ./modules/ntp 154 # .... 155 ]; 156 config.services.ntp = svc.ntp.build { 157 pools = { "pool.ntp.org" = ["iburst"]; }; 158 makestep = { threshold = 1.0; limit = 3; }; 159 }; 160 161 Merely including the module won't define the service on its own: it 162 only creates the template in ``config.system.service.foo`` and you 163 have to create the actual service using the template. 164 165 166 167 Consequences 168 ************ 169 170 This decision has both good and bad consequences 171 172 Pro 173 === 174 175 * We have a workable system for reusing configuration elements in 176 Liminix. 177 178 * We have type checking for most imortant things, reducing the risk of 179 deploying an invalid configuration. 180 181 * We have a simple mechanism for creating multiple services based on 182 the same module, without buulding that logic into the module 183 definition itself. For example, we could create two SSH daemons on 184 different ports, or DHCP clients with different configurations on 185 different network devices. 186 187 * We expect to be able to automate the generation of module 188 documentation. 189 190 Con 191 === 192 193 194 * By departing somewhat from the NixOS conventions we increase the 195 amount of code we have to write/maintain ourselves - and the 196 learning burden on users who are already familiar with that system. 197 198 * Liminix configurations contain function calls and aren't just data, 199 which means we can ony realistically interpret or introspect 200 them with the Nix interpreter itself - we can't query them 201 as data with other non-Nix tools.