No description
Find a file
mithral 00c1c0d619 Wave 2: correctness and security
TouchSession throttle (internal/auth)
- Middleware wrote sessions.last_seen on EVERY authenticated request, taking
  SQLite's single write lock each time. For media that is severe: one seek in
  an audio file fires several Range requests, each taking the write lock and
  competing with the scanner and progress saves.
- Now touched at most once per 5 minutes, decided from the LastSeen already
  loaded by SessionByToken, so no extra query. A zero interval degrades to the
  old per-request behaviour, so a hand-built Authenticator cannot silently stop
  updating.
- Verified end to end with an AFTER UPDATE trigger: 20 requests now cause 0
  session writes, down from 20.

Response writer forwarding (internal/httpapi)
- statusRecorder wrapped every ResponseWriter but implemented only Write and
  WriteHeader, so it did not implement http.Flusher. That silently disabled
  early flushing (progressive audio/video) and io.ReaderFrom, which is what
  lets io.Copy use sendfile for multi-GB files. Now forwards Flush, Hijack and
  ReadFrom, and WriteHeader is idempotent.

Security headers (internal/httpapi)
- Added nosniff, X-Frame-Options: DENY, Referrer-Policy and a CSP. Outermost in
  the chain so a recovered panic's 500 still carries them.
- CORRECTION to an earlier claim: I said a malicious EPUB was stored XSS.
  That was wrong. epub.js sets iframe sandbox to "allow-same-origin" and only
  appends "allow-scripts" when settings.allowScriptedContent is true, which
  defaults to false and Reader.svelte never sets. EPUB content cannot execute
  script. The CSP is still worth having as defence in depth, but that specific
  justification does not hold.

Final flush on teardown (AudiobookPlayer, Reader)
- The last playback/reading position was saved with a fire-and-forget fetch in
  onDestroy, which the browser cancels on navigate-away — losing exactly the
  thing worth persisting. Now persisted via fetch with keepalive: true, wired
  to pagehide and visibilitychange(hidden), latched so the overlapping events
  fire one request, and reset when the page becomes visible again so a bfcache
  restore still flushes later. sendBeacon was not usable: it is POST-only and
  progress is PUT-only.
- Also removed the queueMicrotask track-swap race in AudiobookPlayer: play() is
  now driven by onLoadedMetadata once the new src has actually loaded.

Honest readers (Reader, ComicReader)
- registry.go advertises .mobi/.azw3/.fb2 under ebooks, but every ebooks item
  was handed to epub.js, which cannot parse them. Now only .epub uses epub.js;
  other containers show a message naming the format plus a download link. Items
  stay catalogued and downloadable rather than silently vanishing.
- .cbr/.rar comics say the archive format is unsupported instead of surfacing
  the server's raw 422 (the server reads zip only).

Verified in a real browser, not just compiled: all six reader surfaces render
(epub iframe, cbz images, audio element, photo, pdf canvas) with zero CSP
violations; .mobi shows the honest message with no epub.js iframe. gofmt/vet
clean, 14 packages pass, svelte-check 0/0.

Known issue, NOT introduced here and NOT fixed: the EPUB reader logs
"Error: No Element Provided" x3 and a TypeError in epub.js injectIdentifier
(doc.getElementsByTagName("head")[0] undefined). Proven pre-existing by A/B
against c50b3cf, and not caused by the test file (spec-correct XHTML) or by
caching (cold browser). Non-fatal: the book still renders. Needs its own
investigation.
2026-10-11 13:31:34 -04:00
cmd/vellum Wave 1: survive a real library 2026-10-11 13:10:34 -04:00
internal Wave 2: correctness and security 2026-10-11 13:31:34 -04:00
migrations Wave 1: survive a real library 2026-10-11 13:10:34 -04:00
scripts dev: fix the Vite proxy and add a one-command dev runner 2026-10-10 16:32:32 -04:00
web Wave 2: correctness and security 2026-10-11 13:31:34 -04:00
.dockerignore Phase 0: skeleton, config, migrations, health check, container 2026-10-09 16:26:08 -04:00
.gitignore Redesign the user-facing experience: home, browse, item pages 2026-10-09 21:38:11 -04:00
compose.yml Beta blockers: async scans, admin CLI, deploy docs, pagination, password UI 2026-10-10 18:00:35 -04:00
DEPLOY.md Beta blockers: async scans, admin CLI, deploy docs, pagination, password UI 2026-10-10 18:00:35 -04:00
Dockerfile Web UI: Svelte app, embedded into the binary 2026-10-09 16:40:58 -04:00
entrypoint.sh Phase 0: skeleton, config, migrations, health check, container 2026-10-09 16:26:08 -04:00
go.mod Phase 6c: photo thumbnails and viewer 2026-10-09 21:22:30 -04:00
go.sum Phase 6c: photo thumbnails and viewer 2026-10-09 21:22:30 -04:00
Makefile dev: fix the Vite proxy and add a one-command dev runner 2026-10-10 16:32:32 -04:00
README.md dev: fix the Vite proxy and add a one-command dev runner 2026-10-10 16:32:32 -04:00

Vellum

One application, one process, one database for every media type — ebooks, comics, audiobooks, music, photos, games, and video — behind a single account system with real per-user progress and per-library permissions.

A modular monolith. One Go binary, one SQLite file, one web frontend.

The full plan lives in ~/.opencode/plan/vellum.md.

Status

Phase Scope State
0 Repo, config, migrations, health, Docker done
1 Users, sessions, library grants done
2 Scanner, items, assets done
2b FTS5 search done
3 EPUB reader + progress done
4 CBZ comic reader done
PDF PDF reading (pdf.js) done
5 Job queue, metadata (Open Library, Google Books, MusicBrainz, ComicVine, IGDB) done
6a Audiobooks + music playback done
6c Photos + thumbnails done
7 Games: ROM scanning, platforms, cloud saves, EmulatorJS done
UI Svelte app: home, browse, item pages, readers done
6b Music smart playlists not started
— Video (stays on Jellyfin) deferred

What works today: an administrator adds libraries pointing at server folders, scans them, and optionally enriches metadata. Users sign in and read EPUBs and PDFs, read CBZ comics, listen to audiobooks track-by-track, browse photos, and play emulated games in the browser with their own cloud saves — with position saved per account, per file.

Six media types work: ebooks, comics, audiobooks, music, photos, games. Video is the only one not implemented, and it stays on Jellyfin on purpose.

Of the services in the existing stack, this covers Stump, Kavita, Audiobookshelf, Navidrome, and RomM. Only Jellyfin (video) is left.

Quick start

sudo dnf install golang        # once; Go is not in the base image here
make build
make run
curl localhost:7788/healthz    # {"database":"ok","status":"ok",...}

Run with media roots configured:

VELLUM_MEDIA_DIRS=/mnt/storage/media/ebooks:/mnt/storage/media/comics make run

Configuration

All configuration is environment variables. Every one has a default, so the binary runs with no configuration at all.

Variable Default Meaning
VELLUM_LISTEN :7788 Bind address
VELLUM_DATA_DIR ./data SQLite database lives here
VELLUM_CONFIG_DIR ./config Regenerable cache (covers, pages)
VELLUM_MEDIA_DIRS (none) Read-only library roots, colon-separated
VELLUM_ADMIN_USER (none) Creates the first admin, only on an empty database
VELLUM_ADMIN_PASSWORD (none) Password for that first admin (min 8 chars)
VELLUM_COOKIE_SECURE false Set true when serving over TLS
VELLUM_COMICVINE_API_KEY (none) Enables comic metadata via ComicVine
VELLUM_IGDB_CLIENT_ID (none) Enables game metadata via IGDB
VELLUM_IGDB_CLIENT_SECRET (none) IGDB client secret (both required)
VELLUM_LOG_LEVEL info debug, info, warn, error
VELLUM_LOG_FORMAT text text or json
VELLUM_SESSION_TTL 720h Login session lifetime
VELLUM_SHUTDOWN_GRACE 15s Graceful shutdown timeout

The first admin is created only when no users exist, so setting these on an established install does nothing. If the database is empty and they are unset, the server logs a warning and refuses to invent a default password.

Layout

cmd/vellum/          main — wiring and lifecycle only
internal/
  core/              config, logging, database, migration runner
  httpx/             shared JSON helpers
  httpapi/           router, middleware, static SPA serving
  auth/              argon2id, sessions, grants, bootstrap
  library/           library CRUD, media-type registry, scanner
  media/             generic item/file/cover/progress endpoints
  progress/          the polymorphic per-user progress store
  stream/            range-capable file serving
  natsort/           natural ordering for pages and tracks
  jobs/              table-backed job queue
  metadata/          provider interface, Open Library, Google Books, enricher
  search/            FTS5 queries
  modules/
    ebooks/          EPUB parsing and cover extraction
    comics/          CBZ page listing and extraction
    photos/          thumbnail generation
web/                 Svelte app; dist/ is embedded via web/embed.go
migrations/          embedded SQL, applied in order at boot

Module rule: the core never imports a feature module; modules import the core and mount their own routes, tables, and scanner rules. Adding a media type is a new package plus a migration, never a core edit.

Schema rule: media items are global; everything user-specific is a join. Nothing may hold per-user state on a media row.

Database

SQLite in WAL mode, opened through modernc.org/sqlite — a pure-Go driver, so the binary builds with CGO_ENABLED=0 and cross-compiles without a toolchain per target.

Migrations are embedded and applied in filename order; applied files are recorded only by number in PRAGMA user_version, so never edit an applied migration — add a new one.

Back up by copying the database (or its directory); VACUUM INTO 'backup.db' takes a consistent snapshot while running.

Deploy

make docker                                  # podman build
podman compose up -d                         # see compose.yml

The container mounts /data (database), /config (cache), and /media (read-only). On SELinux hosts every bind mount needs :z — compose.yml already has it.

The application never writes to /media.

Development

make dev        # backend + Vite dev server together, with hot reload
make build      # builds the frontend, then the Go binary with it embedded
make build-go   # Go only, when the frontend is unchanged
make check      # gofmt + vet + test
make run        # run the server on :7788
cd web && pnpm dev   # frontend only; needs the backend running on :7788

make dev is the one to reach for while developing: it builds and starts the backend, waits for it to answer, then starts Vite. Vite proxies /api to the backend so the browser sees a single origin and sessions behave exactly as in production. Ctrl-C stops both.

If you run Vite on its own, the backend must already be running, or every request fails with ECONNREFUSED on :7788.