NixOS flake - heimdall, thor, loki, vidar
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-09-04 08:11:30 +03:00
.forgejo remove lint in devshell 2026-08-30 15:12:04 +03:00
docs improve secrets handling 2026-09-02 03:27:27 +03:00
flake improve secrets handling 2026-09-02 03:27:27 +03:00
hosts fix tests 2 2026-09-04 08:11:30 +03:00
lib migrate to a different vm 2026-09-04 04:52:13 +03:00
modules make ntfy work, migrate to 443 2026-09-04 06:09:25 +03:00
overlays refactor fleet flake and validation 2026-08-24 20:56:38 +03:00
packages migrate to a different vm 2026-09-04 04:52:13 +03:00
roles fix tests 2026-09-04 06:30:15 +03:00
secrets fix tests 2 2026-09-04 08:11:30 +03:00
services fix tests 2 2026-09-04 08:11:30 +03:00
sites migrate to a different vm 2026-09-04 04:52:13 +03:00
tests fix tests 2 2026-09-04 08:11:30 +03:00
.betterleaks.toml improve secrets handling 2026-09-02 03:27:27 +03:00
.gitignore remove lint in devshell 2026-08-30 15:12:04 +03:00
.pre-commit-config.yaml improve secrets handling 2026-09-02 03:27:27 +03:00
.sops.yaml migrate to a different vm 2026-09-04 04:52:13 +03:00
flake.lock migrate to a different vm 2026-09-04 04:52:13 +03:00
flake.nix run hermes natively instead of as a container 2026-09-02 22:29:04 +03:00
README.md remove lint in devshell 2026-08-30 15:12:04 +03:00
statix.toml Initial commit 2026-07-06 15:11:32 +03:00

nix-desktop

A single NixOS flake that builds a five-machine fleet — a workstation, laptop, home server, VPS, and handheld — including ~20 services, a WireGuard mesh, a self-hosted recursive DNS stack, secrets split by trust boundary, and guarded deployments that roll themselves back when a host comes up unhealthy.

A host names the modules it composes in one place, and its identity in another. hosts/heimdall/composition.nix:

[
  ../../roles/server
  ../../services/jellyfin
  ../../services/forgejo
  ../../services/matrix-tuwunel
  ../../services/observability
  ../../services/backups
]

and hosts/inventory.nix carries only mesh membership and deployability:

heimdall = {
  deployment.enable = true;
  mesh = {
    address = "10.90.100.1/24";
    publicKey = "aHOI2skr4Ei3U96KpRONeR/gHRa8qLudvvmTXyzx+HI=";
    endpoint = "…:51820";
  };
};

Everything else — module set, dependency closure, mesh peer lists, mesh subnet, internal DNS records, /etc/hosts entries — is derived from that entry.

The Fleet

Host Kind Runs
thor Workstation Hyprland desktop, work VPN + tooling, gaming; Lix base system, pinned 7.1 kernel with the in-kernel AmneziaWG module
loki Laptop Hyprland desktop, work VPN + tooling
heimdall Home server, mesh hub Jellyfin, Navidrome, Forgejo, Matrix (Tuwunel + LiveKit + Element), Nix binary cache, Transmission, Grafana/VictoriaMetrics/VictoriaLogs, nightly restic backups
vidar VPS, mesh spoke AmneziaWG gateway, Knot Resolver + Blocky DNS, public DoH endpoint, coturn, append-only restic backup target, observability agent
magni Anbernic RG353V handheld AArch64 SD image, RetroArch, fast local boot, bootstrap Wi-Fi, USB gadget recovery

Design Notes

Four things in here are worth more than the rest:

Guarded deployment with automatic rollback. deploy <host> switch runs the old generation's own health probes as a preflight, pre-builds and dry-activates the entire selection, then switches hosts one at a time. Before each activation it arms a dead-man's-switch on the target that restores the exact captured generation unless it is explicitly confirmed. Confirmation requires both that /run/current-system points at the expected closure and that that closure's health probes pass. A failed switch requests rollback immediately; losing SSH mid-deploy leaves the timer armed, so a host that becomes unreachable recovers on its own. Implementation in the compiled command suite under packages/command-suite, and the failure paths — preflight failure, unhealthy activation, failed switch — are tested against mocked colmena and ssh in tests/deploy-safe-switch.nix.

Topology derived from a single source of truth. lib/topology.nix computes the mesh hub, subnet, listen port, per-host peer lists, and internal DNS from the inventory. Hub identity is inferred from which host declares an endpoint, and the build fails loudly if that yields zero or more than one candidate rather than silently picking one — first as an assertion in modules/options/fleet.nix, which names both candidates, and as a backstop in lib/topology.nix itself.

Secrets split by trust boundary. Each host decrypts with its own SSH host key via sops-nix; there is no shared age key. Files are scoped so that a workstation cannot read cluster credentials and a server cannot read Wi-Fi or work secrets. See Secrets.

Systemd hardening as reusable profiles. modules/options/hardening.nix defines graduated confinement profiles rather than per-service copy-paste, with the exclusions documented and justified — ProcSubset=pid is left out because JVM and Go runtimes read /proc/meminfo at startup, and JIT runtimes get a profile without MemoryDenyWriteExecute.

Architecture

The repo is organized around four layers:

  • modules/: shared option definitions and genuinely universal core behavior
  • roles/: reusable host capabilities such as desktop, server, or work
  • services/: single-purpose service definitions (Jellyfin, Navidrome, Tuwunel, etc.) opt-in via the inventory's services list
  • hosts/: machine-specific composition, hardware, storage, and local overrides

The important design constraint is that the shared baseline should stay portable. Personal or environment-specific defaults belong in host-layer modules, not in modules/base.nix.

Layout

.
├── flake.nix
├── flake/                  # Colmena, app/package, check, and installer outputs
├── hosts/
│   ├── inventory.nix
│   ├── site-defaults.nix
│   └── <hostname>/
├── modules/
│   ├── base.nix
│   ├── options/
│   └── core/
├── roles/
│   ├── workstation/
│   ├── desktop/
│   ├── work/
│   ├── gaming/
│   └── server/
├── services/
│   └── <service>/
├── lib/
├── overlays/
├── packages/
└── secrets/

Composition Model

Each declared host is built from:

  1. modules/base.nix
  2. the modules listed in hosts/<hostname>/composition.nix
  3. the host module in hosts/<hostname>/

composition.nix is a flat list of module paths, not a module with imports:

[
  ../../roles/desktop
  ../../roles/work
]

lib/fleet.nix splices that list directly into the host's module list. The shape matters. Nesting these one level deeper — inside a module's imports — reorders every list- and string-valued option that more than one module contributes to: environment.systemPackages (and therefore PATH precedence via system-path), Caddy's globalConfig, tmpfiles rules, Home Manager files. The content stays the same and the derivations change.

Roles import the roles they build on, so the dependency graph is stated in the tree rather than computed: gaming imports desktop, and desktop and work import workstation. The module system closes that graph transitively and deduplicates by path, so listing a role twice costs nothing. Services do the same where they have a hard dependency — blocky imports knot-resolver.

Fleet discovery and NixOS host construction live in lib/fleet.nix. flake.nix composes that result, while flake/ contains the Colmena, installer, app/package, and check output builders.

hosts/inventory.nix describes mesh membership and deployability only:

{
  my-host = {
    mesh = {
      address = "10.90.100.9/24";
      publicKey = "...";
    };
  };
}

Its shape is enforced by typed options in modules/options/fleet.nix, so a malformed entry reports the option path rather than a hand-written message. Setting deployment.enable = true adds the host to both the Colmena hive and the guarded deploy command; hosts without it remain available through nixosConfigurations.

That module also carries the fleet-wide invariants as assertions: exactly one endpoint-bearing mesh hub, no duplicate mesh addresses or public keys, and no name shared between a managed host and a mesh-only peer.

Shared vs Local Config

Portable defaults live in modules/.

Local defaults for your current machines live in hosts/site-defaults.nix. That file currently holds values that should not be assumed for every user of the flake:

  • primary user
  • SSH authorized keys
  • Git identity
  • private cache trust
  • server domain
  • fleet DNS-over-HTTPS endpoint
  • desktop Wi-Fi profiles
  • local repo path for nh

Existing machines import that file explicitly from their host modules. New machines do not need to use it. They can define only the settings they need in their own host module or in a smaller local helper module.

Important Options

The shared option surface for local customization is mostly under nd.*:

  • nd.meta.primaryUser: main interactive user for the host
  • nd.meta.bootstrap: temporary mode for installs and recovery
  • nd.meta.stateVersion.system: NixOS compatibility version from initial installation
  • nd.meta.stateVersion.home: Home Manager compatibility version from initial user setup
  • nd.user.openssh.authorizedKeys: SSH keys for the primary user
  • nd.user.git.name / nd.user.git.email: Git identity
  • nd.nix.substituters / nd.nix.trustedPublicKeys: extra cache settings
  • nd.security.privEsc: run0 for interactive hosts or doas for deployment targets
  • nd.desktop.iwd.countryCode: IWD regulatory domain
  • nd.desktop.iwd.profiles: Wi-Fi profiles rendered from SOPS secrets
  • nd.paths.repoRoot: checkout path used by nh

See modules/options/ for the authoritative option definitions — one file per concern (meta, security, user, network, nix, desktop, paths, theme), plus fleet.nix which types hosts/inventory.nix and asserts the fleet-wide invariants. Per-service knobs, such as nd.services.observability.backendHost, live in that service's directory.

Roles

Roles are self-contained directories under roles/. Each role owns both its composition and its implementation modules.

Notable role boundaries:

  • workstation: fish, Git, direnv, helix language servers, personal CLI packages, AI agents, credentials, and TPM-backed SSH agent
  • desktop: graphical environment, kmscon, and the session variables naming a terminal and browser; imports workstation
  • server: bash login shell, headless diagnostics, Caddy, and automatic upgrades
  • work: mail, employer-specific certificates, VPN, Ansible/YAML tooling, and work integrations; imports workstation
  • gaming: Steam, emulators, and gamemode; imports desktop

modules/base.nix contains shared options, core system behavior, and the two pieces of user tooling every host gets: helix, themed and configured but without language servers (modules/core/user/helix.nix), and Waypipe in the system profile so either end of an SSH-forwarded graphical application can find it (modules/core/network/waypipe.nix). Everything else user-facing belongs to a role so headless machines do not inherit a workstation environment accidentally.

CLI profiles

server and workstation are the two CLI profiles, and nothing imports both:

server workstation
login shell bashInteractive fish
tools btop, dust, jq, ripgrep, dnsutils, smartmontools the packages.nix set, git/delta/gitui, direnv, fzf, zoxide
delivery environment.systemPackages home.packages
helix baseline only baseline + nil, rust-analyzer, lua-language-server, pyright

The shell is set by each profile at normal priority rather than in modules/core/user/user.nix, so a host that ever imports both fails loudly instead of silently picking one. Server tools go in environment.systemPackages because smartctl and dig are run as root, and a root shell has no Home Manager profile on PATH.

Session variables live with what they name — EDITOR with helix, TERMINAL with foot, BROWSER with firefox, the portal variables with xdg-portal.nix — rather than in one fleet-wide file, so headless hosts never advertise a terminal or browser they do not have.

Services

Services live under services/ and are opt-in per host: add the module's path to hosts/<hostname>/composition.nix. Every service assumes the server role, which those hosts list alongside them.

A service module is imported only where it is wanted, so it does not gate itself behind an enable flag. Modules that need to know whether another service is present ask that service's real upstream option — config.services.forgejo.enable, config.services.jellyfin.enable. The one exception is observability, which is an assembly of VictoriaMetrics, VictoriaLogs, Grafana, vmagent, vector and node-exporter with no single upstream option, so it declares nd.services.observability.enable itself.

Adding a web service

A service fronted by Caddy declares itself once, and the Caddy vhost, the systemd hardening floor, the deployment health check and the Grafana dashboard all follow:

nd.services.web.jellyfin = {
  port = 8096;              # loopback port Caddy proxies to
  unit = "jellyfin";        # systemd unit, without .service
  job = "jellyfin";         # Prometheus job, if it ships an exporter
  probed = true;            # blackbox-probed, so the dashboard gets TLS expiry
  health = "/health";       # loopback health path, or omit for none
  required = false;         # true = failing it fails the deploy and rolls back
  hardening = "runtime";    # nd.hardening.floor tier, or omit for none
  blockMetrics = true;      # 404 /metrics on the public vhost
};

required is the deployment-safety tier. Reserve true for services whose absence costs access to the host, its communications, or the ability to deploy at all — Forgejo is true, Jellyfin is not. Set healthUnit = false for a service whose unit goes active well before it actually serves, so it is judged by its endpoint instead; harmonia is the case that exists for.

The served host is <vhost>.<serverDomain>, where vhost defaults to the attribute name — set it when they differ (harmonia serves cache, ntfy serves push). Set host instead when the domain is configurable in its own right and need not sit under the fleet domain, as Grafana's does.

Then add the module to hosts/<hostname>/composition.nix. Nothing needs touching in services/observability.

This deliberately does not cover every vhost. One that differs in kind rather than in name — Matrix delegation, LiveKit's split JWT path, the DoH endpoint, a static site — keeps writing its own extraConfig, and its dashboard entry stays hand-written in services/observability/dashboards.nix. An escape hatch wide enough to express those would defeat the point.

If the service holds state worth keeping, it also declares its own backup contract; see Secrets's neighbour nd.backups.state:

nd.backups.state.jellyfin.paths = [ "/var/lib/jellyfin/data" ];

Currently defined services:

  • awgGateway: AmneziaWG VPN gateway
  • matrixTuwunel: Matrix server (Tuwunel + LiveKit)
  • elementWeb: Element Web client
  • elementCall: self-hosted Element Call UI
  • nixCache: Nix binary cache (Harmonia + cache builder)
  • dufs: web file manager
  • forgejo: Git hosting
  • minecraftServer: PaperMC server with RCON
  • jellyfin: Jellyfin media server
  • navidrome: Navidrome music server
  • transmission: Transmission BitTorrent daemon
  • wgDashboard: WGDashboard web UI
  • turn: coturn relay for Matrix and Element Call (UDP fast path with TLS fallback)
  • backupTarget: append-only restic REST server (vidar)
  • backups: nightly restic backups of heimdall service state to vidar
  • observability: VictoriaMetrics, VictoriaLogs, Grafana, and host agents

The directories under services/ are the authoritative list.

Music library workflow

Heimdall manages Navidrome's files with Beets. New music is staged in /media/downloads/music; run music-library-import over SSH to review MusicBrainz/AcoustID matches. Every match defaults to skip until explicitly accepted. Accepted imports are copied into /media/music as 160-kbps VBR Opus with the verified metadata and cover art embedded. Each source audio file is deleted only after its Opus conversion succeeds.

The existing library is repaired in small batches with music-library-retag '/media/music/Artist/Album'. This writes tags in place but never re-encodes existing Opus files. music-library-audit produces a TSV report of missing tags and non-Opus formats; it also counts cue sheets, which must be split or converted album-by-album instead of having their referenced audio blindly replaced. music-library-beet exposes read-only Beets queries such as music-library-beet stats and music-library-beet list.

Grafana provisions its alert rules, Matrix contact point, and notification policy from Nix. Firing and resolved notifications go to the static matrix-alertmanager-receiver Go binary from nixpkgs, bound only to Heimdall's loopback interface. It converts Grafana's webhook payload to a Matrix message and sends it to the private #alerts:matrix.nestoris.online room through local Tuwunel. The dedicated @grafana-alerts:matrix.nestoris.online bot runs in a hardened systemd service. Its token and recovery password live only in secrets/hosts/heimdall.yaml; SOPS renders the token into the service's private environment file at activation time.

Host Modules

Within hosts/<hostname>/, keep modules named by concern:

  • hardware.nix
  • storage.nix / disks.nix
  • network.nix
  • ssh.nix
  • kernel.nix
  • nix.nix
  • hyprland.nix
  • vpn.nix

Avoid catch-all custom.nix files when the configuration can be named more precisely.

Common Commands

Switch a named host:

nh os switch . -H <hostname>
# or
sudo nixos-rebuild switch --flake .#<hostname>

Deploy a server through Colmena:

deploy vidar
deploy heimdall build
deploy all dry-activate
deploy all health
deploy all switch

The deployment command accepts only heimdall, vidar, or the explicit all selector. Its default goal is the guarded switch, so deploy all and deploy all switch are equivalent. unsafe-switch exposes a direct Colmena switch for emergency use without health checks or automatic rollback.

Each deployed generation carries its own deployment-guard, including the systemd, HTTP, and DNS probes declared by that generation. health executes the guard belonging to the currently active generation without changing the host. Generations created before this mechanism fall back to checking the systemd system state.

switch checks the old generation with its own guard, pre-builds and dry-activates the whole selection, then switches hosts one at a time. It obtains the expected system and guard paths from the same Colmena configuration used for those operations. Immediately before switching, the uploaded target guard runs interruption-safety checks; the LiveKit host refuses to switch while it has active rooms or participants. --allow-busy overrides only that gate when an interruption is intentional. The currently deployed guard then arms a timer that restores the exact generation captured before activation. Its duration is the configured five-minute activation allowance plus the configured health-retry delays, so health checks do not consume the switch budget. The target guard cancels the timer only after /run/current-system points at the expected new closure and that closure's own health checks pass. A failed switch or exhausted health retries requests the rollback through the captured guard immediately; loss of SSH leaves the timer armed. A globally degraded system remains deployable when every explicitly required unit and application probe is healthy.

The safety windows can be tuned for an unusually slow host:

DEPLOY_ROLLBACK_TIMEOUT_SEC=600 \
DEPLOY_HEALTH_ATTEMPTS=36 \
DEPLOY_HEALTH_INTERVAL_SEC=5 \
  deploy heimdall switch

Update flake inputs:

nix flake update

Apply the repository's safe formatters to Nix and Go sources:

nix fmt

Run the read-only quality gate over formatting, dead Nix bindings, Statix diagnostics, Go tests and vet, and Forgejo workflows:

nix run .#lint

The command builds the same sandboxed checks.<system>.lint derivation used by CI and never rewrites the working copy. Jujutsu pushes remain the native, transport-only jj git push; jj fix is still available as an explicit Nix-formatting convenience.

Evaluate every host and package output, then build every check for the current system:

nix run .#fleet-check

The driver starts a fresh Nix evaluator for every host and package output, then builds each check separately. Package evaluation covers every system exposed by the flake even when checks are built only for the runner's native system. This keeps peak evaluator memory near one host or package graph; a monolithic nix flake check retains the full fleet graph and exceeds the memory limit on Heimdall.

Forgejo builds the dedicated lint gate first, then runs the same fleet command on every push. The second pass over lint is already present in the Nix store. The check runner lives in a Nix-built NixOS systemd-nspawn container on Heimdall and accepts jobs only from npilosov/nix-desktop. The container has a private network and user namespace, blocks forwarded access to private networks, and exposes no inbound ports. The runner executes one job at a time and its container is limited to two CPUs, 5 GiB of memory (with reclaim pressure above 4 GiB), 2 GiB of swap, and 1024 tasks. Nix, Git, and the remaining runner tools are part of the container closure; Podman, Docker, and Node.js are absent.

The workflow uses Git directly for its exact-SHA checkout and the runner's native backend inside that boundary. fleet-check evaluates each NixOS host and package output and builds each native check in a separate process, including the Thor and Loki toplevels, which releases evaluator memory between targets and keeps the job within the container limit. As with standard NixOS containers, it receives the host's read-only Nix store and Nix daemon socket; the host daemon still enforces Heimdall's two-job/two-core limit. The private-network rule applies to direct container traffic: a source-specific route sends public egress through Heimdall's LAN while the forward firewall rejects private destinations. Nix builds are performed by the host daemon. DNS uses a stub bound only to the container's private host address and follows Heimdall's monitored, cached DoH path. Job homes and workspaces live under /run and are discarded whenever the runner service or container restarts. The host starts the runner only after container@forgejo-ci has configured the host end of the private veth. Its health monitor also verifies and repairs the source-specific egress rule if a network reactivation flushes it.

After the check job succeeds on the main branch, a separate nix-deploy host runner checks out the same commit and runs deploy all switch. This is the normal guarded fleet deployment: both targets are pre-built and dry-activated, switched serially, health-checked against the exact generation, and automatically rolled back on failure. Keeping the deployment executor outside forgejo-ci lets Heimdall restart the check container without terminating its own deployment. The deployment runner stays alive across a switch and picks up runner-definition changes on reboot.

The deployment runner receives its SSH identity as a systemd credential and copies it at mode 0600 into its ephemeral /run home before invoking OpenSSH; the check container cannot access it. Both server host keys are pinned declaratively. The restricted public key is authorized only for the system account deploy, only on hosts opted into automatic deployment. That account can enter a small validated activation/rollback protocol through doas; it does not share the human account's SSH authorization or general doas rule.

After changing runner configuration, verify both runners:

deploy all switch
ssh heimdall systemctl status \
  container@forgejo-ci \
  forgejo-runner-health \
  'gitea-runner-heimdall\x2ddeploy'
ssh heimdall systemctl --machine=forgejo-ci status 'gitea-runner-*'

The encrypted forgejo-runner-token in secrets/hosts/heimdall.yaml is a repository-scoped registration token. If it is ever rotated, generate it for the npilosov/nix-desktop scope and replace that SOPS value before deploying. The same file contains the dedicated forgejo-deploy-ssh-key; rotate its restricted public half in hosts/deploy-authorized-key.nix at the same time.

Format the repository with the official Nix formatter:

nix fmt

To evaluate and build the AArch64 checks, run the driver where an AArch64 builder or emulation is available:

nix run .#fleet-check -- --system aarch64-linux

Show what a host composes by reading its composition.nix, or ask the evaluated configuration what it actually runs:

cat hosts/thor/composition.nix

nix eval --json '.#nixosConfigurations' --apply \
  'cs: builtins.mapAttrs (_: c: c.config.services.jellyfin.enable) cs'

The second form reports what each host really runs rather than what was declared, and stays correct as services gain their own options.

Local Host Resolution

The flake output schema stays pure and always exposes only named hosts. The rebuild-current app reads /proc/sys/kernel/hostname at runtime, verifies that the name exists in nixosConfigurations, and invokes nh for that host.

Switch the current host, or select another supported nh os action:

nix run .#rebuild-current
nix run .#rebuild-current -- boot

Adding a New Host

  1. Create hosts/<hostname>/default.nix.
  2. Import ./hardware.nix and any host-local modules from that file.
  3. Pin nd.meta.stateVersion.system and nd.meta.stateVersion.home in that host module to the versions used for its initial installation and Home Manager setup. Do not bump them during ordinary input updates.
  4. Optionally import ../site-defaults.nix if the machine should inherit your current local defaults.
  5. Generate hardware config on the target machine:
nixos-generate-config --show-hardware-config > hosts/<hostname>/hardware.nix
  1. Add the host to hosts/inventory.nix under its hostname, with its mesh keys.
  2. Create hosts/<hostname>/composition.nix listing the roles and services it should run.
  3. Set deployment.enable = true in the inventory if Colmena and the guarded deploy command should manage the host.
  4. If needed, add disks.nix and a small storage.nix wrapper for Disko.
  5. Install with:
nixos-install --flake .#<hostname>

If you use Disko:

nix run github:nix-community/disko -- --mode disko ./hosts/<hostname>/disks.nix

Bootstrap and Recovery

Two separate mechanisms cover first-boot and recovery:

Installer ISO

The flake exposes a minimal installer target:

  • installer (under packages.<system>)

Build it with:

nix build .#installer

The installer ISO is a standalone bootable image with SSH, iwd, disko, and git — just enough to partition disks and run nixos-install from this flake. It is not a NixOS host configuration and is not managed via inventory.nix.

When built with the bootstrap mesh key available, the installer also joins the AmneziaWG mesh as the bootstrap peer (hostname bootstrap, 10.90.100.8), so installs and recovery sessions can be run over SSH from anywhere the mesh reaches. With the office Wi-Fi password available it also provisions an iwd profile for RUTUBE and connects on its own. Both secrets are decrypted on thor into fixed paths the installer module expects before building (the --impure is required — the build reads paths outside the flake):

sops decrypt --extract '["mesh-private-key"]' secrets/hosts/bootstrap.yaml > /tmp/nix-bootstrap-mesh.key
sops decrypt --extract '["eap-password"]' secrets/work.yaml > /tmp/nix-bootstrap-wifi-eap.key
nix build .#installer --impure

Without the key files the same target builds a plain, mesh-less installer with manual Wi-Fi. Because the private key and the Wi-Fi password are embedded in the ISO, treat the image like any other secret-bearing artifact: delete it after the install is done.

Bootstrap mode for managed hosts

Any managed host can opt into nd.meta.bootstrap = true to disable secret-backed features (SOPS, mesh, media and other services, etc.) until the machine has its normal SOPS material available again.

Typical bootstrap flow:

  1. Install or boot the target in bootstrap mode.
  2. Reach the machine using SSH or manual Wi-Fi setup.
  3. Restore the machine's existing /etc/ssh/ssh_host_ed25519_key, or replace its recipient in .sops.yaml with one derived from the new public host key.
  4. If the recipient changed, run sops updatekeys on every encrypted file the host consumes.
  5. Return the host to normal mode and rebuild.

Laptop Wi-Fi can be brought up manually with iwctl during bootstrap:

iwctl
device list
station wlan0 scan
station wlan0 get-networks
station wlan0 connect "<ssid>"

Secrets

This repo uses sops-nix with encrypted files split by trust boundary. Normal hosts decrypt with /etc/ssh/ssh_host_ed25519_key; they do not share an age private key. Every file also includes the administrator recipient so it can be edited with sops.

File Host recipients Purpose
secrets/common.yaml all user password hash and fleet DoH path token
secrets/desktop.yaml thor, loki Wi-Fi
secrets/work.yaml thor, loki shared work/mail credential plus GitLab, Forgejo, and Jira tokens
secrets/cluster.yaml heimdall, vidar backup and TURN credentials shared by the servers
secrets/hosts/<host>.yaml that host only host VPN/mesh keys and host-local service credentials

vpn-password is a runtime alias for the eap-password key in secrets/work.yaml. To edit a file, use sops, for example:

sops secrets/hosts/thor.yaml

Recipient policy lives in .sops.yaml. After replacing a host SSH key, convert its new public key with ssh-to-age, update the matching recipient, and run sops updatekeys on each file that host can read.

The former shared key at /var/lib/sops-nix/key.txt is no longer referenced by new configurations, but old installed generations may still require it during manual recovery. Keep that file until those generations have been removed from the host. The configuration preserves an existing copy and enforces mode 0600; it never creates or deletes the key.

DNS

Normal DNS has one client path across the fleet:

systemd-resolved -> local dnsproxy -> Caddy DoH on vidar -> Blocky -> Knot Resolver

Vidar uses its local Blocky directly so recovery of the DNS server does not depend on its own public Caddy endpoint. Blocky and Knot listen only on loopback; Caddy is the only network-facing DNS service. DHCP and AmneziaWG do not install DNS servers or catch-all routing domains.

The work VPN remains independent: snx-rs installs Check Point DNS servers with specific routing domains on its physical link. Those more-specific routes beat the global ~. DoH route for office names without changing normal DNS.

Notes

  • Home Manager manual pages and programs.man integration are disabled in the shared baseline to avoid man-cache build noise.
  • nix-ld is enabled from its own core system module, not from a role.
  • Generic workstation utilities live in roles/workstation/packages.nix; keep role package sets focused on the roles actual purpose.