051169a18a
Docs - docs/ARCHITECTURE.md: how the pieces fit and why. The request path, the six enforced layers, the data model, the scanner, the job queue, direct-play playback, and the three protocol surfaces over one application. - docs/decisions/: seven ADRs with the alternatives that were rejected — SQLite-with-a-seam, direct-play-only, mark-missing-never-delete, the modular monolith, the one item-access chokepoint, the table-backed queue, and fixed-width timestamps. - README: the layout section was stale (it predated catalog, home, scanner, providers, engine, probe, playback, apitoken, arch and the dto packages) and now lists what exists, plus links to the new docs. Two real bugs, both found by verifying the docs instead of trusting them 1. The CLI could not run on a fresh install. `Config.Validate` created the data and config directories, but the CLI never called it, so `vellum user add` — the recovery path when you are locked out — failed with "unable to open database file" until the server had booted at least once. Split the directory creation into `Config.EnsureDirs`, call it from the CLI, and keep the media root check out of it: an admin command must not fail because a library is unmounted. The CLI tests now point DBPath inside DataDir, exactly as LoadConfig derives it, so they exercise the fresh-install path; removing EnsureDirs fails three of them. 2. A fresh clone could not build at all. `//go:embed all:dist` fails to compile when web/dist is absent, and web/.gitignore expected a `dist/.gitkeep` that was never created. Added it, so a clone builds and the server logs "web UI not embedded; run the frontend build to include it" — the intended behaviour for someone who only wants the API. Also: the httpapi tests depended on a built frontend and FAILED rather than skipping, so the suite was red on any fresh clone. findAsset and readAsset now skip with a message naming the missing step. They still RUN when a bundle is present (verified: zero skips here), so they have not been quietly disabled. Verified from a simulated clone of tracked files only: go build exit 0, 25 packages pass, the server boots and applies all 12 migrations, and the CLI creates a user and takes a backup. gofmt clean, vet exit 0, svelte-check 0/0.
181 lines
7.7 KiB
Markdown
181 lines
7.7 KiB
Markdown
# Vellum
|
|
|
|
One application, one process, one database for every media type — ebooks,
|
|
comics, audiobooks, music, photos, games, and video — behind a single account
|
|
system with real per-user progress and per-library permissions.
|
|
|
|
A modular monolith. One Go binary, one SQLite file, one web frontend.
|
|
|
|
The full plan lives in `~/.opencode/plan/vellum.md`.
|
|
|
|
## Status
|
|
|
|
| Phase | Scope | State |
|
|
| --- | --- | --- |
|
|
| **0** | Repo, config, migrations, health, Docker | **done** |
|
|
| **1** | Users, sessions, library grants | **done** |
|
|
| **2** | Scanner, items, assets | **done** |
|
|
| **2b** | FTS5 search | **done** |
|
|
| **3** | EPUB reader + progress | **done** |
|
|
| **4** | CBZ comic reader | **done** |
|
|
| **PDF** | PDF reading (pdf.js) | **done** |
|
|
| **5** | Job queue, metadata (Open Library, Google Books, MusicBrainz, ComicVine, IGDB) | **done** |
|
|
| **6a** | Audiobooks + music playback | **done** |
|
|
| **6c** | Photos + thumbnails | **done** |
|
|
| **7** | Games: ROM scanning, platforms, cloud saves, EmulatorJS | **done** |
|
|
| **UI** | Svelte app: home, browse, item pages, readers | **done** |
|
|
| 6b | Music smart playlists | not started |
|
|
| — | Video (stays on Jellyfin) | deferred |
|
|
|
|
What works today: an administrator adds libraries pointing at server folders,
|
|
scans them, and optionally enriches metadata. Users sign in and read EPUBs and
|
|
PDFs, read CBZ comics, listen to audiobooks track-by-track, browse photos, and
|
|
play emulated games in the browser with their own cloud saves — with position
|
|
saved per account, per file.
|
|
|
|
**Six media types work: ebooks, comics, audiobooks, music, photos, games.**
|
|
Video is the only one not implemented, and it stays on Jellyfin on purpose.
|
|
|
|
Of the services in the existing stack, this covers **Stump, Kavita,
|
|
Audiobookshelf, Navidrome, and RomM**. Only Jellyfin (video) is left.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
sudo dnf install golang # once; Go is not in the base image here
|
|
make build
|
|
make run
|
|
curl localhost:7788/healthz # {"database":"ok","status":"ok",...}
|
|
```
|
|
|
|
Run with media roots configured:
|
|
|
|
```bash
|
|
VELLUM_MEDIA_DIRS=/mnt/storage/media/ebooks:/mnt/storage/media/comics make run
|
|
```
|
|
|
|
## Configuration
|
|
|
|
All configuration is environment variables. Every one has a default, so the
|
|
binary runs with no configuration at all.
|
|
|
|
| Variable | Default | Meaning |
|
|
| --- | --- | --- |
|
|
| `VELLUM_LISTEN` | `:7788` | Bind address |
|
|
| `VELLUM_DATA_DIR` | `./data` | SQLite database lives here |
|
|
| `VELLUM_CONFIG_DIR` | `./config` | Regenerable cache (covers, pages) |
|
|
| `VELLUM_MEDIA_DIRS` | *(none)* | Read-only library roots, colon-separated |
|
|
| `VELLUM_ADMIN_USER` | *(none)* | Creates the first admin, **only on an empty database** |
|
|
| `VELLUM_ADMIN_PASSWORD` | *(none)* | Password for that first admin (min 8 chars) |
|
|
| `VELLUM_COOKIE_SECURE` | `false` | Set `true` when serving over TLS |
|
|
| `VELLUM_COMICVINE_API_KEY` | *(none)* | Enables comic metadata via ComicVine |
|
|
| `VELLUM_IGDB_CLIENT_ID` | *(none)* | Enables game metadata via IGDB |
|
|
| `VELLUM_IGDB_CLIENT_SECRET` | *(none)* | IGDB client secret (both required) |
|
|
| `VELLUM_LOG_LEVEL` | `info` | `debug`, `info`, `warn`, `error` |
|
|
| `VELLUM_LOG_FORMAT` | `text` | `text` or `json` |
|
|
| `VELLUM_SESSION_TTL` | `720h` | Login session lifetime |
|
|
| `VELLUM_SHUTDOWN_GRACE` | `15s` | Graceful shutdown timeout |
|
|
|
|
The first admin is created only when no users exist, so setting these on an
|
|
established install does nothing. If the database is empty and they are unset,
|
|
the server logs a warning and refuses to invent a default password.
|
|
|
|
## Layout
|
|
|
|
```
|
|
cmd/vellum/ main — wiring and lifecycle only, the one place packages meet
|
|
internal/
|
|
core/ config, logging, database, migrations, timestamps
|
|
httpx/ shared JSON helpers
|
|
httpapi/ router, middleware, security headers, static SPA serving
|
|
auth/ argon2id, sessions, grants, bootstrap
|
|
apitoken/ credentials for non-browser clients (Jellyfin, Subsonic)
|
|
catalog/ THE data seam: read an item, and check you may
|
|
library/ library CRUD and the media-type registry
|
|
scanner/ walk the filesystem and reconcile with the database
|
|
probe/ ffprobe wrapper — codec facts (optional dependency)
|
|
media/ generic item/file/cover/progress endpoints
|
|
home/ the five home shelves
|
|
search/ FTS5 queries
|
|
metadata/ the enrichment job and its matching policy
|
|
providers/ Open Library, Google Books, MusicBrainz, ComicVine, IGDB
|
|
playback/ playback engine selection endpoint
|
|
engine/ playback engine registry (html5, mpv, vlc, mdk)
|
|
progress/ the polymorphic per-user progress store
|
|
stream/ range-capable file serving
|
|
jobs/ table-backed job queue
|
|
natsort/ natural ordering for pages and tracks
|
|
jellyfin/ Jellyfin protocol adapter (+ dto/)
|
|
subsonic/ Subsonic protocol adapter (+ dto/)
|
|
arch/ import-rule tests; holds no code
|
|
modules/
|
|
ebooks/ EPUB parsing and cover extraction
|
|
comics/ CBZ page listing and extraction
|
|
games/ platforms, IGDB, cloud saves, EmulatorJS
|
|
photos/ thumbnail generation
|
|
web/ Svelte app; dist/ is embedded via web/embed.go
|
|
migrations/ embedded SQL, applied in order at boot
|
|
docs/ architecture guide and decision records
|
|
```
|
|
|
|
**Module rule:** the core never imports a feature module; modules import the
|
|
core and mount their own routes, tables, and scanner rules. Adding a media type
|
|
is a new package plus a migration, never a core edit.
|
|
|
|
**Layer rule:** packages are arranged in six layers and a package may never
|
|
import from a higher one. This is enforced by tests in `internal/arch`, not by
|
|
convention — see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
|
|
|
**Schema rule:** media items are global; everything user-specific is a join.
|
|
Nothing may hold per-user state on a media row.
|
|
|
|
## Documentation
|
|
|
|
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — how the pieces fit and why.
|
|
- [docs/decisions/](docs/decisions/) — the decisions that were expensive to
|
|
reverse, with the alternatives that were rejected.
|
|
|
|
## Database
|
|
|
|
SQLite in WAL mode, opened through `modernc.org/sqlite` — a pure-Go driver, so
|
|
the binary builds with `CGO_ENABLED=0` and cross-compiles without a toolchain
|
|
per target.
|
|
|
|
Migrations are embedded and applied in filename order; applied files are
|
|
recorded only by number in `PRAGMA user_version`, so **never edit an applied
|
|
migration** — add a new one.
|
|
|
|
Back up by copying the database (or its directory); `VACUUM INTO 'backup.db'`
|
|
takes a consistent snapshot while running.
|
|
|
|
## Deploy
|
|
|
|
```bash
|
|
make docker # podman build
|
|
podman compose up -d # see compose.yml
|
|
```
|
|
|
|
The container mounts `/data` (database), `/config` (cache), and `/media`
|
|
(read-only). On SELinux hosts every bind mount needs `:z` — `compose.yml`
|
|
already has it.
|
|
|
|
The application never writes to `/media`.
|
|
|
|
## Development
|
|
|
|
```bash
|
|
make dev # backend + Vite dev server together, with hot reload
|
|
make build # builds the frontend, then the Go binary with it embedded
|
|
make build-go # Go only, when the frontend is unchanged
|
|
make check # gofmt + vet + test
|
|
make run # run the server on :7788
|
|
cd web && pnpm dev # frontend only; needs the backend running on :7788
|
|
```
|
|
|
|
`make dev` is the one to reach for while developing: it builds and starts the
|
|
backend, waits for it to answer, then starts Vite. Vite proxies `/api` to the
|
|
backend so the browser sees a single origin and sessions behave exactly as in
|
|
production. Ctrl-C stops both.
|
|
|
|
If you run Vite on its own, the backend must already be running, or every
|
|
request fails with `ECONNREFUSED` on `:7788`.
|