Docs
- docs/ARCHITECTURE.md: how the pieces fit and why. The request path, the six
enforced layers, the data model, the scanner, the job queue, direct-play
playback, and the three protocol surfaces over one application.
- docs/decisions/: seven ADRs with the alternatives that were rejected —
SQLite-with-a-seam, direct-play-only, mark-missing-never-delete, the modular
monolith, the one item-access chokepoint, the table-backed queue, and
fixed-width timestamps.
- README: the layout section was stale (it predated catalog, home, scanner,
providers, engine, probe, playback, apitoken, arch and the dto packages) and
now lists what exists, plus links to the new docs.
Two real bugs, both found by verifying the docs instead of trusting them
1. The CLI could not run on a fresh install. `Config.Validate` created the data
and config directories, but the CLI never called it, so `vellum user add` —
the recovery path when you are locked out — failed with "unable to open
database file" until the server had booted at least once. Split the directory
creation into `Config.EnsureDirs`, call it from the CLI, and keep the media
root check out of it: an admin command must not fail because a library is
unmounted. The CLI tests now point DBPath inside DataDir, exactly as
LoadConfig derives it, so they exercise the fresh-install path; removing
EnsureDirs fails three of them.
2. A fresh clone could not build at all. `//go:embed all:dist` fails to compile
when web/dist is absent, and web/.gitignore expected a `dist/.gitkeep` that
was never created. Added it, so a clone builds and the server logs
"web UI not embedded; run the frontend build to include it" — the intended
behaviour for someone who only wants the API.
Also: the httpapi tests depended on a built frontend and FAILED rather than
skipping, so the suite was red on any fresh clone. findAsset and readAsset now
skip with a message naming the missing step. They still RUN when a bundle is
present (verified: zero skips here), so they have not been quietly disabled.
Verified from a simulated clone of tracked files only: go build exit 0, 25
packages pass, the server boots and applies all 12 migrations, and the CLI
creates a user and takes a backup. gofmt clean, vet exit 0, svelte-check 0/0.
Authors are searchable (internal/search, migrations/0009)
- The FTS index covered only title and path, so typing an artist or author
found nothing even though people/item_people already held the credits.
- Rather than a second index, the names are denormalised into a real
media_items.authors column, so the existing external-content FTS and its
triggers keep doing the work; item_people/people triggers recompute it, and
the media_items UPDATE trigger re-indexes automatically.
- Verified: searching "daft punk" now returns Random Access Memories, and so
does the partial "punk".
- The agent found and worked around a real FTS5 external-content bug: on an
UPGRADE the backfill UPDATE fires the delete trigger for rowids not yet in
the fresh index, raising "database disk image is malformed". A rebuild before
the backfill fixes it. This would have broken existing databases while a
fresh install looked fine.
gzip for the SPA (internal/httpapi/static.go)
- 456 KB of JS and 31 KB of CSS were served uncompressed on every cold load.
- Scoped to the embedded SPA assets only, never API or media: compressing a
Range response would break seeking, and wrapping every response would undo
Wave 2's Flusher/ReaderFrom forwarding.
- Compresses only when the client accepts gzip, the response is a compressible
type, there is no Range header, and nothing already set Content-Encoding.
Sets Vary: Accept-Encoding and drops the now-wrong Content-Length.
- Verified by sha256 round-trip (identity vs gzip-decompressed are identical),
and by confirming a Range request still returns 206 with no Content-Encoding.
JS 351,835 -> 108,794 bytes (69% smaller); CSS 80% smaller.
epub.js is code-split (Reader.svelte)
- It was a static import, so ~350 KB was downloaded by everyone on first load
even though most never open an ebook. Now loaded on demand.
- Main bundle 458 KB -> 112 KB (gzipped 149 KB -> 39.5 KB).
- Verified at runtime, which pnpm check cannot: only the main bundle loads
initially, the epubjs chunk is fetched when a book is opened, and the reader
still renders.
Home screen tests (internal/media)
- Coverage 23.4% -> 50.2%. (The 17.9% quoted in the brief was a stale reading;
the agent measured the real baseline by moving its own files aside.)
- Covers the five shelves, counts, Explore's invariants (visibility, no
duplicates, every kind represented, varies across calls, wrap-around),
the list offset/limit/total contract, MostViewed ordering, and — most
importantly — grant scoping, which is the security chokepoint.
Fixed a bug the tests found: Suggestions seeded from revoked libraries
- topEngagement selected the user's most recent progress row with no library
filter, so progress from a library whose grant had been revoked still steered
the home screen — a visible item was labelled "More books" based on content
the user could no longer read. Returned items were always correctly scoped, so
this was an information-flow gap rather than a content leak.
- Now scoped to visible libraries, and the test that pinned the old behaviour
flipped to assert the correct reason.
gofmt/vet clean, 14 packages pass, media tests run 3x for flakiness.
TouchSession throttle (internal/auth)
- Middleware wrote sessions.last_seen on EVERY authenticated request, taking
SQLite's single write lock each time. For media that is severe: one seek in
an audio file fires several Range requests, each taking the write lock and
competing with the scanner and progress saves.
- Now touched at most once per 5 minutes, decided from the LastSeen already
loaded by SessionByToken, so no extra query. A zero interval degrades to the
old per-request behaviour, so a hand-built Authenticator cannot silently stop
updating.
- Verified end to end with an AFTER UPDATE trigger: 20 requests now cause 0
session writes, down from 20.
Response writer forwarding (internal/httpapi)
- statusRecorder wrapped every ResponseWriter but implemented only Write and
WriteHeader, so it did not implement http.Flusher. That silently disabled
early flushing (progressive audio/video) and io.ReaderFrom, which is what
lets io.Copy use sendfile for multi-GB files. Now forwards Flush, Hijack and
ReadFrom, and WriteHeader is idempotent.
Security headers (internal/httpapi)
- Added nosniff, X-Frame-Options: DENY, Referrer-Policy and a CSP. Outermost in
the chain so a recovered panic's 500 still carries them.
- CORRECTION to an earlier claim: I said a malicious EPUB was stored XSS.
That was wrong. epub.js sets iframe sandbox to "allow-same-origin" and only
appends "allow-scripts" when settings.allowScriptedContent is true, which
defaults to false and Reader.svelte never sets. EPUB content cannot execute
script. The CSP is still worth having as defence in depth, but that specific
justification does not hold.
Final flush on teardown (AudiobookPlayer, Reader)
- The last playback/reading position was saved with a fire-and-forget fetch in
onDestroy, which the browser cancels on navigate-away — losing exactly the
thing worth persisting. Now persisted via fetch with keepalive: true, wired
to pagehide and visibilitychange(hidden), latched so the overlapping events
fire one request, and reset when the page becomes visible again so a bfcache
restore still flushes later. sendBeacon was not usable: it is POST-only and
progress is PUT-only.
- Also removed the queueMicrotask track-swap race in AudiobookPlayer: play() is
now driven by onLoadedMetadata once the new src has actually loaded.
Honest readers (Reader, ComicReader)
- registry.go advertises .mobi/.azw3/.fb2 under ebooks, but every ebooks item
was handed to epub.js, which cannot parse them. Now only .epub uses epub.js;
other containers show a message naming the format plus a download link. Items
stay catalogued and downloadable rather than silently vanishing.
- .cbr/.rar comics say the archive format is unsupported instead of surfacing
the server's raw 422 (the server reads zip only).
Verified in a real browser, not just compiled: all six reader surfaces render
(epub iframe, cbz images, audio element, photo, pdf canvas) with zero CSP
violations; .mobi shows the honest message with no epub.js iframe. gofmt/vet
clean, 14 packages pass, svelte-check 0/0.
Known issue, NOT introduced here and NOT fixed: the EPUB reader logs
"Error: No Element Provided" x3 and a TypeError in epub.js injectIdentifier
(doc.getElementsByTagName("head")[0] undefined). Proven pre-existing by A/B
against c50b3cf, and not caused by the test file (spec-correct XHTML) or by
caching (cold browser). Non-fatal: the book still renders. Needs its own
investigation.
The frontend the app needs to be usable at all.
- web/: Svelte 5 + Vite + TypeScript SPA. Login, app shell with per-type nav,
library list/create/scan/delete, item grid with covers, admin user
management, and an epub.js reader that syncs progress.
- Reader resumes by spine index — an integer — which is why the polymorphic
progress table needs no text column.
- Session state uses classic stores in a plain .ts module rather than runes in
a .svelte.ts module, avoiding special module resolution.
- Embedding: web/embed.go embeds web/dist. A committed .gitkeep keeps the
directive valid before the frontend is built, so `go build ./...` never
fails. Unknown non-API paths serve index.html for client routes; /api/* and
/healthz always return JSON and never fall back to HTML.
- Dockerfile builds the UI in a node stage and overlays dist into the Go stage.
- pnpm 12 needs allowBuilds in pnpm-workspace.yaml; esbuild approved, the two
unused transitive scripts explicitly denied so install exits 0.
svelte-check: 0 errors, 0 warnings. Verified served: SPA shell at /, route
fallback, hashed asset, and JSON 404 on /api/*.
Multi-user from the first migration, as planned.
- auth: argon2id password hashing (PHC encoded, parameters travel with the
hash), opaque session tokens stored only as SHA-256, login rate limiting,
and account enumeration resistance (unknown user hashes anyway)
- sessions: cookie or Bearer auth, HttpOnly + SameSite=Lax, revocation on
password change
- grants: library_grants is the single access-control chokepoint. Admins see
every library; everyone else sees exactly their can_read grants
- bootstrap: VELLUM_ADMIN_USER/PASSWORD create the first admin only on an
empty database. No default password is ever invented.
- httpapi: mounts modules on a shared mux, session middleware innermost
- httpx: shared JSON helpers, strict body decoding
Gate: two users see different libraries. Tests cover that, plus session
expiry, token-at-rest, grant upsert, and the full login/me/logout flow.
Bootstraps Vellum as a single Go binary with one SQLite database.
- core: env-driven config, slog logging, SQLite (WAL) with pooled pragmas,
embedded migration runner keyed on PRAGMA user_version
- migrations/0001_core.sql: identity, media, user state, infra. Media items
are global; all per-user state (progress, favorites, grants) is a join.
One polymorphic progress table covers every media kind.
- httpapi: request-ID, structured request logging, panic recovery, JSON
endpoints for /healthz and /api/v1/version
- multi-arch Dockerfile (pure-Go SQLite, CGO_ENABLED=0) with an entrypoint
that honours PUID/PGID like the rest of the self-hosted stack
- tests for migration idempotency, FK enforcement, WAL, config, middleware
Media does not exist yet. Nothing is exposed beyond health and version.
mithral