Architecture

One wayland-server compositor per session, running in its own process against a virtual output. Never your real display.

Overview

waymux is a local-first headless Wayland session manager. A per-user daemon (waymuxd) supervises any number of isolated Wayland sessions, each backed by its own in-process compositor that renders to a virtual output. A single waymux CLI drives the daemon over a msgpack-RPC Unix socket to create sessions, spawn applications into them, inject input, capture screenshots, record video, and serve a low-latency WebRTC viewer to a browser.

Sessions can also be attached to an outer compositor (for example niri) so the virtual output appears as a normal window on the host. The canonical, fully open surface is local; the same control surface is reachable remotely over HTTPS for a hosted deployment, but local is the default and the only mode documented here.

Component diagram

                         +-------------------------------+
                         |          waymux CLI           |
                         |  (23 subcommands, clap)       |
                         +---------------+---------------+
                                         |
              Transport trait: LocalTransport | RemoteTransport
                                         |
         msgpack-RPC over Unix socket    |   HTTPS + Bearer (hosted)
         $XDG_RUNTIME_DIR/waymux.sock    |
                                         v
  +----------------------------------------------------------------------+
  |                          waymuxd (per-user daemon)                   |
  |                                                                      |
  |   Server (accept loop, SO_PEERCRED uid gate, Hello negotiation,      |
  |           reader/writer tasks, dispatch router, error mapping)       |
  |                              |                                       |
  |                              v                                       |
  |   Registry (core engine: session map, lifecycle, broadcast,         |
  |             supervisors, log history, session_control RPC)          |
  |                              |                                       |
  |   SessionBackend trait  --> LocalBackend (subprocess)               |
  |                                                                      |
  |   cgroup / tmpfs quota (best-effort)   usage_events (feature-gated)  |
  +--------------------------------+-------------------------------------+
                                   | spawn + per-session control socket
                                   v
  +----------------------------------------------------------------------+
  |             waymux-session (one process per session)                 |
  |                                                                      |
  |   Compositor thread          Control socket (tokio, msgpack-RPC)     |
  |   - inner Wayland server      - Info/ListWindows/Resize              |
  |   - xdg-shell, wl-shm,        - Inject{Key,Pointer,Touch,Batch}     |
  |     dmabuf, layer-shell       - Screenshot/ScreenshotDesktop         |
  |   - virtual output            - Record{Start,Stop} Viewer{Start..}   |
  |   - SurfaceData / windows     Events socket --> daemon broadcast     |
  |        |            |              |                                 |
  |        v            v              v                                 |
  |   Attach server   Recording    Viewer (neko-bridge child, Go)        |
  |   (waymux_attach  encoders:     - encoder thread -> Annex-B NALUs   |
  |    _v1, fd-pass)  ffv1/nvenc/    - Unix socket -> Pion WebRTC       |
  |        |          vaapi/vulkan)  - WS signaling + data channel       |
  |        v                              |                              |
  |   Outer compositor (niri)        Browser viewer (video + input)      |
  +----------------------------------------------------------------------+

Control plane

The control plane is the daemon plus its wire protocol. It is the path through which every CLI verb flows: session creation, spawning, input injection, capture, recording, and viewer control.

Transport

The daemon binds a per-user UnixListener (default $XDG_RUNTIME_DIR/waymux.sock, chmod 0600). The CLI connects through a Transport trait with two implementations: LocalTransport (Unix socket, msgpack-RPC) and RemoteTransport (HTTPS with a Bearer token). Eight of the 23 CLI verbs are transport-routable; the rest are local-only.

Every frame is a 4-byte big-endian length prefix followed by a msgpack payload, capped at a 20 MiB MAX_FRAME_SIZE. The first request on a connection must be Hello. The daemon accepts any client protocol from version 1 through the current version (4) and replies with its version and capabilities. A non-Hello first request, protocol version 0, or a version newer than the daemon's own returns E_PROTO_VERSION.

Registry

The Registry is the core engine. It holds a HashMap<String, SessionEntry> of session metadata, supervisor kill channels, child-PID tracking, rolling per-session log history (1024 lines), and a broadcast channel that fans events to all subscribers. Its public methods (create, destroy, spawn_child, session_control, list_windows, resize, screenshot, inject_*, record_*, viewer_*, tag_window, wait_for_idle, attach, detach, shutdown_all) are protocol-agnostic.

The Server's dispatch() is a match over RequestMethod (26 variants) that translates each wire request into a Registry call and maps typed engine errors into stable ErrorCode values (E_NOT_FOUND, E_ALREADY_EXISTS, E_NOT_IMPLEMENTED, E_BACKPRESSURE, E_INTERNAL, and others).

SessionBackend

A SessionBackend async trait (create / destroy / info) abstracts the session-lifecycle path. LocalBackend is the only shipped implementation: a thin wrapper over the Registry that manages subprocess sessions. The trait is the seam a future provisioning target would plug into (see Extension points).

Data plane

Each session is a separate waymux-session process: a full headless Wayland compositor for one virtual output. It holds the inner Wayland server, the capture logic, the recording thread, the attach server, and the viewer.

Inner compositor

The session advertises: xdg-shell, wl-shm, layer-shell, zwp_linux_dmabuf_v1 (GPU buffer import with modifier negotiation), viewporter, pointer/keyboard/touch, data-device (clipboard), presentation-time, pointer-constraints and relative-pointer, keyboard-shortcuts-inhibit, and KDE-specific protocols.

The compositor is observer-only: it tracks surfaces, subsurface trees, toplevels, and damage timestamps without rendering. Composition happens lazily at capture time via a recursive subsurface tree walk that blits into a single ARGB8888 buffer. This means no GPU is required for the compositor itself.

Capture and screenshots

Screenshot RPCs run on the control thread, look up the surface by window id, composite the subsurface tree on the CPU, and encode PNG with the image crate. The protocol prefers a fd-passed Dmabuf format with a PNG-over-SHM fallback. A buffer-hold ref-count mechanism keeps GPU buffers pinned while a capture or encode is in flight.

Recording

Four backends are available. The choice of codec determines whether a GPU is needed:

A LatestTaskSlot lets newer frames evict older ones so a slow encoder never back-pressures the compositor. Dual recording (primary plus --secondary-codec) writes two output paths from the same frame tap. Output is always Matroska (.mkv); recording paths are validated to be absolute and free of ...

WebRTC viewer

On ViewerStart the session spawns a Go waymux-neko-bridge child and an encoder thread that produces Annex-B NALUs tuned for low latency (baseline profile, no B-frames, periodic IDR at 60 fps; Vulkan emits every-frame IDR). NALUs cross a private per-session Unix socket using a typed 5-byte-header protocol (NALU, cursor image, cursor position, force-keyframe, inject-op, set-bitrate, shutdown). The bridge wraps frames into WebRTC with Pion (ICE, DTLS, RTP), signals over WebSocket, and exposes a data channel.

Browser input arrives as JSON, is translated to waymux InjectOp, and is written back over the socket into the session control loop. Multi-viewer is last-wins: only the primary viewer's input and GCC bandwidth estimate drive the shared encoder; other viewers receive video fan-out only.

Attach

A second Wayland server within the session advertises waymux_attach_v1. An attach client passes the outer compositor's display fd via SCM_RIGHTS; the session creates a proxy wl_surface / xdg_toplevel on the outer compositor and ferries the inner focused window's frames into an outer SHM buffer on each commit. The ferry path validates same-format ARGB8888 at 1:1 size and falls back to a placeholder otherwise.

Process and socket model

Every session gets its own set of Unix sockets. The daemon spawns the session process and communicates with it through these sockets throughout its lifetime.

Same-uid gating. Both the daemon accept loop and the per-session control socket check SO_PEERCRED and reject any connection whose uid differs from the owner. The daemon socket is chmod 0600. This is the primary local trust boundary.

Lifecycle. create spawns the session subprocess, waits for the ready handshake, sets up best-effort cgroup and tmpfs quota handles, and starts a session_supervisor task that owns the Child, drains stdout/stderr into the log ring, and emits SessionCreated. destroy removes the session, SIGTERMs tracked child PIDs, signals the supervisor, lazy-unmounts the tmpfs, and cleans up the cgroup. The supervisor also handles natural exit, emitting SessionDestroyed.

Spawning clients. spawn_child requires an absolute argv[0], clears the environment and re-adds only safe variables, optionally applies an fd-limit rlimit, joins the cgroup, and tracks the PID for crash detection.

Resource capping. SessionCgroup (cgroup v2) and SessionTmpfs are best-effort: if CAP_SYS_ADMIN is absent or a write fails, the daemon logs a warning and the session runs uncapped rather than failing.

Headless no-GPU path

The compositor does no rendering of its own. Frames are captured from CPU memory (SHM or CPU-mapped Dmabuf), screenshots are encoded on the CPU, and FFV1 is a CPU encoder. The whole loop runs green on a stock shared CI runner, making waymux the Wayland equivalent of Xvfb.

# Force Mesa software rendering (llvmpipe) and disable the DRM syncobj path
# (no /dev/dri node = implicit sync only).
export LIBGL_ALWAYS_SOFTWARE=1
export GALLIUM_DRIVER=llvmpipe
export WAYMUX_DISABLE_SYNCOBJ=1
waymux serve &                        # or let the first command auto-spawn the daemon

waymux new app --size 1280x800
WAYLAND_DISPLAY="$XDG_RUNTIME_DIR/waymux/app/wayland.sock" kwrite notes.txt &
waymux wait app --timeout-ms 15000
waymux screenshot-desktop app -o shot.png
waymux record start app --codec ffv1   # CPU-encoded lossless
waymux record stop app

The first five capabilities are everything needed for functional, layout, input, and visual-regression testing. CI does not need the viewer at all: it screenshots and records instead.

The repository ships a ready-made end-to-end test at tests/e2e/run-e2e-embedded.sh that launches Chromium and a KDE app as direct Wayland clients under llvmpipe, asserts the captured frames have real content (a center-contrast check), injects keystrokes and verifies they changed the capture, records FFV1, and counts genuinely unique frames (duplicate min-fps padding is stripped with mpdecimate first).

A GPU-free Docker image at tests/e2e/Dockerfile runs the same harness with no --gpus and no /dev/dri:

docker build -f tests/e2e/Dockerfile -t waymux-e2e .
docker run --rm waymux-e2e

Crate layout

waymux is a single Cargo workspace plus one Go module (the web viewer bridge).

Key data flows

Create a session

1. CLI new sends Hello then CreateSession over the local socket.
2. Server gates on Hello, then dispatch() calls registry.create().
3. Registry spawns waymux-session with the socket set and waits for the ready handshake (5 s timeout).
4. Registry installs cgroup/tmpfs handles and the supervisor, then emits SessionCreated.
5. CLI prints name (WxH).

Spawn a client

1. CLI spawn sends Spawn {argv, env, compositor}.
2. dispatch() calls registry.spawn_child(), which validates argv[0], sanitizes the environment, joins the cgroup, and starts the process with WAYLAND_DISPLAY pointing at the session's inner socket.
3. The Registry records the PID, drains its logs, and on exit emits ChildExited or SessionCrashed.
4. CLI prints pid N.

Screenshot

1. CLI screenshot sends Screenshot {window_id, format}.
2. The daemon forwards via session_control() to the session control socket.
3. The session composites the subsurface tree on the CPU, encodes PNG, and returns width/height plus the PNG bytes.
4. CLI writes the raw PNG to a file or stdout; metadata goes to stderr.

Record

1. CLI record start sends RecordStart {path, codec, secondary_codec, mode, min_fps}.
2. The session selects an encoder, validates the output path (absolute, no ..), and starts a recording thread fed by the compositor frame tap.
3. CLI prints the primary path (and the secondary path if a secondary codec was set).
4. record stop finalizes the MKV container.

Live view

1. CLI viewer start sends ViewerStart {bind, port}.
2. The session probes a viewer codec (NVENC, then Vulkan), spawns the neko-bridge child, and starts an encoder thread.
3. The browser opens the bridge URL, completes WebSocket signaling, and receives the H.264 WebRTC stream; input flows back over the data channel as InjectOp.
4. CLI prints the viewer URL.

Security model

waymux is designed to be safe to run locally without any network exposure or privileged configuration.

What is validated vs experimental. The local control plane, session lifecycle, input injection (key/pointer/touch), screenshots, recording, and the WebRTC viewer are implemented and exercised by the test suite. InjectSelector is the one reserved protocol slot that currently returns E_NOT_IMPLEMENTED: resolve the target with windows / wait and inject with an explicit window_id instead. Coordinate scaling for non-1x outputs is not yet implemented.

Extension points

waymux is designed for evolution. The primary extension seams are:

• SessionBackend trait. Implement create / destroy / info to target a new provisioning substrate. The local subprocess backend is the only shipped implementation, but the trait was designed from the start with a remote VM backend in mind.
• Recording and viewer encoders. The codec backends are pluggable behind the recording task interface. New encoders slot in alongside ffv1/nvenc/vaapi/vulkan without touching the rest of the session.
• Protocol evolution. New RequestMethod / SessionCtlMethod variants and struct fields are added with #[serde(default)] so older peers keep parsing. The version handshake accepts any client protocol from 1 through the daemon's current version.
• Event subscribers. Clients subscribe to topic-filtered events (sessions, windows, damage, logs, with :name scoping and log replay on subscribe). This is the integration point for external monitoring and automation.
• Attach protocol. waymux_attach_v1 is the seam for embedding a session's output into any outer Wayland compositor via display-fd passing.

Compositor support matrix

waymux hosts direct Wayland clients (any WAYLAND_DISPLAY-aware app) in every session. It also supports running a nested inner compositor as a client.

More broadly, the session advertises a limited set of dmabuf formats (ARGB/XRGB8888 only, no multiplanar/YUV) with LINEAR plus EGL-importable tiled modifiers, and a limited set of Wayland interface versions. A compositor that needs more globals or higher interface versions may fall back to software or fail to start. Broader validation and a published support matrix are planned for Phase 3.