09474b0e08
Authors are searchable (internal/search, migrations/0009) - The FTS index covered only title and path, so typing an artist or author found nothing even though people/item_people already held the credits. - Rather than a second index, the names are denormalised into a real media_items.authors column, so the existing external-content FTS and its triggers keep doing the work; item_people/people triggers recompute it, and the media_items UPDATE trigger re-indexes automatically. - Verified: searching "daft punk" now returns Random Access Memories, and so does the partial "punk". - The agent found and worked around a real FTS5 external-content bug: on an UPGRADE the backfill UPDATE fires the delete trigger for rowids not yet in the fresh index, raising "database disk image is malformed". A rebuild before the backfill fixes it. This would have broken existing databases while a fresh install looked fine. gzip for the SPA (internal/httpapi/static.go) - 456 KB of JS and 31 KB of CSS were served uncompressed on every cold load. - Scoped to the embedded SPA assets only, never API or media: compressing a Range response would break seeking, and wrapping every response would undo Wave 2's Flusher/ReaderFrom forwarding. - Compresses only when the client accepts gzip, the response is a compressible type, there is no Range header, and nothing already set Content-Encoding. Sets Vary: Accept-Encoding and drops the now-wrong Content-Length. - Verified by sha256 round-trip (identity vs gzip-decompressed are identical), and by confirming a Range request still returns 206 with no Content-Encoding. JS 351,835 -> 108,794 bytes (69% smaller); CSS 80% smaller. epub.js is code-split (Reader.svelte) - It was a static import, so ~350 KB was downloaded by everyone on first load even though most never open an ebook. Now loaded on demand. - Main bundle 458 KB -> 112 KB (gzipped 149 KB -> 39.5 KB). - Verified at runtime, which pnpm check cannot: only the main bundle loads initially, the epubjs chunk is fetched when a book is opened, and the reader still renders. Home screen tests (internal/media) - Coverage 23.4% -> 50.2%. (The 17.9% quoted in the brief was a stale reading; the agent measured the real baseline by moving its own files aside.) - Covers the five shelves, counts, Explore's invariants (visibility, no duplicates, every kind represented, varies across calls, wrap-around), the list offset/limit/total contract, MostViewed ordering, and — most importantly — grant scoping, which is the security chokepoint. Fixed a bug the tests found: Suggestions seeded from revoked libraries - topEngagement selected the user's most recent progress row with no library filter, so progress from a library whose grant had been revoked still steered the home screen — a visible item was labelled "More books" based on content the user could no longer read. Returned items were always correctly scoped, so this was an information-flow gap rather than a content leak. - Now scoped to visible libraries, and the test that pinned the old behaviour flipped to assert the correct reason. gofmt/vet clean, 14 packages pass, media tests run 3x for flakiness. |
||
|---|---|---|
| cmd/vellum | ||
| internal | ||
| migrations | ||
| scripts | ||
| web | ||
| .dockerignore | ||
| .gitignore | ||
| compose.yml | ||
| DEPLOY.md | ||
| Dockerfile | ||
| entrypoint.sh | ||
| go.mod | ||
| go.sum | ||
| Makefile | ||
| README.md | ||
Vellum
One application, one process, one database for every media type — ebooks, comics, audiobooks, music, photos, games, and video — behind a single account system with real per-user progress and per-library permissions.
A modular monolith. One Go binary, one SQLite file, one web frontend.
The full plan lives in ~/.opencode/plan/vellum.md.
Status
| Phase | Scope | State |
|---|---|---|
| 0 | Repo, config, migrations, health, Docker | done |
| 1 | Users, sessions, library grants | done |
| 2 | Scanner, items, assets | done |
| 2b | FTS5 search | done |
| 3 | EPUB reader + progress | done |
| 4 | CBZ comic reader | done |
| PDF reading (pdf.js) | done | |
| 5 | Job queue, metadata (Open Library, Google Books, MusicBrainz, ComicVine, IGDB) | done |
| 6a | Audiobooks + music playback | done |
| 6c | Photos + thumbnails | done |
| 7 | Games: ROM scanning, platforms, cloud saves, EmulatorJS | done |
| UI | Svelte app: home, browse, item pages, readers | done |
| 6b | Music smart playlists | not started |
| — | Video (stays on Jellyfin) | deferred |
What works today: an administrator adds libraries pointing at server folders, scans them, and optionally enriches metadata. Users sign in and read EPUBs and PDFs, read CBZ comics, listen to audiobooks track-by-track, browse photos, and play emulated games in the browser with their own cloud saves — with position saved per account, per file.
Six media types work: ebooks, comics, audiobooks, music, photos, games. Video is the only one not implemented, and it stays on Jellyfin on purpose.
Of the services in the existing stack, this covers Stump, Kavita, Audiobookshelf, Navidrome, and RomM. Only Jellyfin (video) is left.
Quick start
sudo dnf install golang # once; Go is not in the base image here
make build
make run
curl localhost:7788/healthz # {"database":"ok","status":"ok",...}
Run with media roots configured:
VELLUM_MEDIA_DIRS=/mnt/storage/media/ebooks:/mnt/storage/media/comics make run
Configuration
All configuration is environment variables. Every one has a default, so the binary runs with no configuration at all.
| Variable | Default | Meaning |
|---|---|---|
VELLUM_LISTEN |
:7788 |
Bind address |
VELLUM_DATA_DIR |
./data |
SQLite database lives here |
VELLUM_CONFIG_DIR |
./config |
Regenerable cache (covers, pages) |
VELLUM_MEDIA_DIRS |
(none) | Read-only library roots, colon-separated |
VELLUM_ADMIN_USER |
(none) | Creates the first admin, only on an empty database |
VELLUM_ADMIN_PASSWORD |
(none) | Password for that first admin (min 8 chars) |
VELLUM_COOKIE_SECURE |
false |
Set true when serving over TLS |
VELLUM_COMICVINE_API_KEY |
(none) | Enables comic metadata via ComicVine |
VELLUM_IGDB_CLIENT_ID |
(none) | Enables game metadata via IGDB |
VELLUM_IGDB_CLIENT_SECRET |
(none) | IGDB client secret (both required) |
VELLUM_LOG_LEVEL |
info |
debug, info, warn, error |
VELLUM_LOG_FORMAT |
text |
text or json |
VELLUM_SESSION_TTL |
720h |
Login session lifetime |
VELLUM_SHUTDOWN_GRACE |
15s |
Graceful shutdown timeout |
The first admin is created only when no users exist, so setting these on an established install does nothing. If the database is empty and they are unset, the server logs a warning and refuses to invent a default password.
Layout
cmd/vellum/ main — wiring and lifecycle only
internal/
core/ config, logging, database, migration runner
httpx/ shared JSON helpers
httpapi/ router, middleware, static SPA serving
auth/ argon2id, sessions, grants, bootstrap
library/ library CRUD, media-type registry, scanner
media/ generic item/file/cover/progress endpoints
progress/ the polymorphic per-user progress store
stream/ range-capable file serving
natsort/ natural ordering for pages and tracks
jobs/ table-backed job queue
metadata/ provider interface, Open Library, Google Books, enricher
search/ FTS5 queries
modules/
ebooks/ EPUB parsing and cover extraction
comics/ CBZ page listing and extraction
photos/ thumbnail generation
web/ Svelte app; dist/ is embedded via web/embed.go
migrations/ embedded SQL, applied in order at boot
Module rule: the core never imports a feature module; modules import the core and mount their own routes, tables, and scanner rules. Adding a media type is a new package plus a migration, never a core edit.
Schema rule: media items are global; everything user-specific is a join. Nothing may hold per-user state on a media row.
Database
SQLite in WAL mode, opened through modernc.org/sqlite — a pure-Go driver, so
the binary builds with CGO_ENABLED=0 and cross-compiles without a toolchain
per target.
Migrations are embedded and applied in filename order; applied files are
recorded only by number in PRAGMA user_version, so never edit an applied
migration — add a new one.
Back up by copying the database (or its directory); VACUUM INTO 'backup.db'
takes a consistent snapshot while running.
Deploy
make docker # podman build
podman compose up -d # see compose.yml
The container mounts /data (database), /config (cache), and /media
(read-only). On SELinux hosts every bind mount needs :z — compose.yml
already has it.
The application never writes to /media.
Development
make dev # backend + Vite dev server together, with hot reload
make build # builds the frontend, then the Go binary with it embedded
make build-go # Go only, when the frontend is unchanged
make check # gofmt + vet + test
make run # run the server on :7788
cd web && pnpm dev # frontend only; needs the backend running on :7788
make dev is the one to reach for while developing: it builds and starts the
backend, waits for it to answer, then starts Vite. Vite proxies /api to the
backend so the browser sees a single origin and sessions behave exactly as in
production. Ctrl-C stops both.
If you run Vite on its own, the backend must already be running, or every
request fails with ECONNREFUSED on :7788.