Motivation, from the evidence rather than from taste. Five packages each
implemented "load an item, then check the caller may read it": media, comics,
games, jellyfin (with its own dbItem type and its own SQL), and subsonic. Five
copies of one authorization rule means four can drift, and the failure mode is
not a crash — it is silently serving somebody a library they were never granted.
One implementation is one thing to review.
internal/catalog now owns item and file access, and the access check.
LoadUnchecked(ctx, db, id) -- no access check; jobs and tests only
LoadFor(ctx, db, u, id) -- load AND enforce the grant, in one call
Files / FilesFor / FileByID / FirstFile / FileCount
The unchecked variant is deliberately named so that reaching for it by accident
is hard: it is the dangerous call, so it should not have the friendlier name.
This came from the interface-design rule that a good interface makes the right
thing easy and the wrong thing hard.
The security property is tested, not asserted: a forbidden item and an absent
item return byte-identical errors, so no handler can leak the existence of
another user's library through a 403. That is now enforced in one place.
Migrated all five surfaces onto it. media, comics, games and ebooks went from
four duplicate loaders to zero. jellyfin's dbItem and loadDBItem are gone.
subsonic's per-item authorization moved while its album/artist aggregate
projections stayed put, because grouping items into albums is a presentation
concern, not a data-access one. The remaining raw SQL everywhere is browse and
aggregate queries; no per-item load decides access any more.
internal/arch enforces the layering instead of documenting it. Five rules, each
proven able to fail by deliberately breaking it:
- no upward imports across six explicit layers
- internal/core may depend only on migrations
- internal/catalog must not know about HTTP, or it stops being reusable
- modules must not import each other
- every package must be placed in a layer, so the map cannot go stale
A test that only ever passes is decoration, so each rule was verified by
introducing the violation and confirming the failure.
Verified: gofmt clean, go build ./... exit 0, go vet exit 0, 20 packages pass,
svelte-check 0/0. End to end against a running server: the vellum API serves
item detail with codec facts, an unknown item 404s, Jellyfin AuthenticateByName
mints a token and /Items browses, and Subsonic ping returns status ok. The
subsonic migration was additionally A/B'd byte-identical against the previous
implementation across ~15 endpoints, and Jellyfin/Subsonic both still return the
same error for a forbidden item as for an absent one.
Known follow-ups, recorded rather than silently skipped:
- internal/jellyfin has no test files at all. Its wire contract is the most
client-visible surface in the tree and is now verified only by hand.
- catalog is item-centric, but Subsonic addresses songs file-first (mf-<id>),
so it resolves file to item through its own projection. A catalog lookup
that takes a file id would close that.
- subsonic loadAlbum now scopes the projection to the item's own library
rather than all visible audio libraries. Equivalent by construction, since
an item's kind tracks its library's kind, but a hand-edited mismatch would
differ.
Vellum now speaks two foreign protocols, so the clients people already own can
use it. Both are translation layers over queries that already exist, and both go
through the same grant chokepoint as the web app — a token is a second way to be
a user, never a second set of permissions.
Shared foundation (coordinator, written before dispatch)
- migrations/0012_api_tokens.sql + internal/apitoken. Both protocols need a
credential that is not the session cookie, and letting two agents each invent
a token table would have produced two incompatible ones.
- The table stores two shapes for a real reason: a Jellyfin token is a bearer
value so only its hash is kept, while a Subsonic key must be recoverable
because verifying `t` means computing md5(key + salt) server-side. token_plain
is therefore the one column in the schema holding a usable credential. It is a
generated random secret, never a user password, and it is revocable.
Wave 9 - Jellyfin
- /System/Info/Public, /Users/AuthenticateByName, /Users/Me, /Users/{id},
/Items, /Users/{id}/Items, /Items/{id}, /Items/{id}/Images/Primary,
/Items/{id}/PlaybackInfo, /Videos/{id}/stream, /Audio/{id}/stream,
/Sessions/Playing{,/Progress,/Stopped}.
- Auth accepts X-Emby-Token, X-MediaBrowser-Token, ?api_key=, and
Authorization: MediaBrowser Token="...". The ?api_key= form is the one that
matters: <video> elements cannot set headers.
- Honest about the one thing Vellum cannot do. PlaybackInfo reports
SupportsDirectPlay/SupportsDirectStream true and SupportsTranscoding FALSE,
with a DirectStreamUrl carrying the api_key. A client that disables both
direct modes is demanding a transcode and is refused with ErrorCode
"NoCompatibleStream" — rather than handed a TranscodingUrl that would never
resolve, which is how a client hangs forever.
Wave 10 - Subsonic
- /rest/{method} dispatch with ping, getLicense, getMusicFolders, getIndexes,
getArtists/getArtist/getAlbum, getMusicDirectory, getSong, search3, stream,
getCoverArt, getAlbumList2, getRandomSongs, scrobble, getUser, getPlaylists,
plus GET /api/v1/subsonic/credentials to hand a user their key.
- Auth via t=md5(key+salt), p=key, constant-time comparison. A bad credential
returns HTTP 200 with the envelope status "failed" and error code 40 — NOT an
HTTP 401. Subsonic clients read the envelope, and getting this wrong is the
single most common integration failure in that protocol.
- stream supports format=raw and never invokes ffmpeg.
Coordinator wired both into main.go (the line agents were told to leave alone)
and verified against real requests.
Verified end to end, 18 packages pass, gofmt/vet clean, svelte-check 0/0:
- Jellyfin: discovery, AuthenticateByName minting a token, /Items reporting 18,
PlaybackInfo showing SupportsTranscoding=false, and a Range request to
/Videos/{id}/stream using ONLY ?api_key= returning 206 with Content-Range.
- Subsonic: ping as JSON and XML, md5(key+salt) auth succeeding, and a bad token
returning the failed envelope with code 40.
- Scoping, both protocols, with a genuinely non-admin user: 0 items, 404 on an
admin-only item, 404 on its stream, 0 music folders; admin still sees 18.
- The SPA is not shadowed by the root-mounted compat paths: /, /healthz and
/api/v1/version all still answer correctly.
Both agents failed with an HTTP 400 at the REPORT stage, not during the work;
their files were complete and compile, so the work was salvaged and verified
here rather than redone.
Wave 7 - engines, chosen per file by the client's own capabilities
- internal/engine: a registry. html5 is built in and declares a CONSERVATIVE
browser baseline (h264/vp8/vp9/av1, deliberately no hevc and no mpeg4, because
that is what was measured). mpv, vlc and mdk are registered as native engines
declaring the broad support those libraries really have.
- The server never declares a file playable. It reports what is in the file and
matches it against the engines the CLIENT declares. Proven:
engines=["html5"] -> playable=false engine=null reason=vcodec:hevc
engines=["html5","mpv"] -> playable=true engine=mpv
Same file, different answer, because the client offered a different engine.
That is what makes adding a native client a registration rather than a rewrite.
- internal/playback: GET /playback/engines and POST /playback/resolve, behind
auth and the same grant chokepoint. An unreadable item is 404, not 403.
- web/src/lib/capabilities.ts: uses navigator.mediaCapabilities.decodingInfo(),
which returns {supported, smooth, powerEfficient}. That is real client-hardware
detection and it distinguishes "cannot decode" from "can decode but it will
stutter and drain the battery". Feature-detected; degrades to the server
baseline rather than throwing.
- VideoPlayer consults the server as an ADVISORY only. The element remains the
source of truth: the server's note appears first, then the element's own
confirmation replaces it once videoWidth proves the answer. Verified on HEVC —
"Playback may not work here" at 0-906ms, "Video cannot play here" from ~1208ms.
Wave 8 - missing files: visibility and an explicit purge
- migrations/0011 adds missing_since to media_items and media_files.
- It is written only on the TRANSITION into missing, so the age says when the
file first vanished rather than the last rescan. Proven byte-identical across a
rescan, and cleared when a file is restored.
- GET /api/v1/missing (grant-scoped) and POST /api/v1/libraries/{id}/purge
(admin only). The purge deletes only items that are missing AND have no
present file, and its cascade into progress/item_people/item_views/game_saves
is enumerated in a comment — that cascade is exactly why deletion is never
automatic. Proven: 19 items -> 18, only the missing one, the non-missing item
untouched and its files intact, second purge returns 0.
- MissingFiles.svelte in the Manage nav: title, kind, library, path, human age,
and a per-library Purge behind a confirmation that names the count and says
progress cannot be regenerated.
Coordinator wired playback.NewHandlers into main.go (the one line agents were
told to leave alone).
Verified end to end: 17 packages pass, gofmt/vet clean, svelte-check 0/0, and
every claim above checked against the running server.
Direct play only. The server is a headless GPU-less Xeon: nothing is ever
transcoded or remuxed. A file either plays in the client or it does not.
Measured, not assumed (ffmpeg-generated files with known codecs, played through
Vellum's own stream endpoint in Chrome 155 / Linux):
mp4 h264+aac plays mkv h264+aac PLAYS <- container is fine
mov h264+aac plays webm vp9/opus plays
mp4 av1+opus plays webm vp8/vorbis plays
mp4 hevc+aac AUDIO ONLY, videoWidth stays 0, no error
mkv hevc+flac AUDIO ONLY
avi mpeg4+mp3 fails outright
All seven audio formats play. Two conclusions drive the design: codec matters
and container barely does (mkv+h264 plays, mp4+HEVC does not), and canPlayType
is unreliable in both directions — it says "no" for mov+h264 (which plays) and
"maybe" for mkv+hevc (whose video never decodes).
Therefore playability is a property of the CLIENT, not the file. The server
never declares a file playable; it reports codec facts and each client decides.
That is what makes adding mpv/libvlc/MDK later a matter of registering an engine
rather than re-architecting.
Wave 4 - audio depth
- New MusicPlayer.svelte: real queue, shuffle that walks a fixed permutation
(so "previous" means something and nothing repeats early), repeat off/all/one,
per-track rows, album art, next-track preload.
- AudiobookPlayer: MediaSession (OS media keys + lock screen), playback speed
that SURVIVES a chapter change and persists, sleep timer with countdown that a
manual pause cancels, keyboard shortcuts, next-chapter preload.
- Shared mediaSession.ts written by the coordinator before dispatch so both
players consume one implementation instead of inventing two.
Wave 5 - ffprobe facts
- internal/probe (ffprobe wrapper, optional dependency), migration 0010 adding
vcodec/acodec/width/height/probed_at to media_files, scanner hook, API fields.
- Probing runs only for NEW or CHANGED files and BEFORE the write transaction
opens, so it can never hold SQLite's write lock and a rescan never becomes a
full-library ffprobe sweep. Proven: 797ms probing scan, then 1ms rescan with
probed_at byte-identical.
- ffprobe skips attached_pic so an mp3 with cover art reports no video codec.
- Fixed a real hole: .avi is in the video registry but had no MIME entry, so it
was served as application/octet-stream and could never play.
Wave 6 - the video player
- VideoPlayer.svelte with resume, progress + keepalive final flush, MediaSession,
keyboard shortcuts, multi-file picker, and codec facts shown.
- The honest path is the feature: an HEVC file makes Chrome play audio over a
blank frame with no error, which is the worst possible outcome. The player
detects it and says so, naming the codec and pointing at clients that can.
Coordinator fix to Wave 6: the agent's detection hung off `loadedmetadata`, but
an explicit `type` the browser cannot satisfy makes it reject the source during
selection, so NEITHER loadedmetadata NOR error fires — and the player showed a
working-looking 0:00 with a Play button, exactly the silently-broken state the
component exists to prevent. Added a load watchdog: if the element says nothing
within 2.5s, the source was rejected and the honest notice is shown. Verified
per file afterwards (h264/mkv play at 640 wide; hevc and mpeg4 show their
notices).
Verified end to end, not just compiled: 15 packages pass (including the new
internal/probe), gofmt/vet clean, svelte-check 0/0, and every claim above checked
against a running server. Note the demo's own mp3s are 100 KB of random bytes and
cannot decode; the audio players were proven against generated real audio instead.
Authors are searchable (internal/search, migrations/0009)
- The FTS index covered only title and path, so typing an artist or author
found nothing even though people/item_people already held the credits.
- Rather than a second index, the names are denormalised into a real
media_items.authors column, so the existing external-content FTS and its
triggers keep doing the work; item_people/people triggers recompute it, and
the media_items UPDATE trigger re-indexes automatically.
- Verified: searching "daft punk" now returns Random Access Memories, and so
does the partial "punk".
- The agent found and worked around a real FTS5 external-content bug: on an
UPGRADE the backfill UPDATE fires the delete trigger for rowids not yet in
the fresh index, raising "database disk image is malformed". A rebuild before
the backfill fixes it. This would have broken existing databases while a
fresh install looked fine.
gzip for the SPA (internal/httpapi/static.go)
- 456 KB of JS and 31 KB of CSS were served uncompressed on every cold load.
- Scoped to the embedded SPA assets only, never API or media: compressing a
Range response would break seeking, and wrapping every response would undo
Wave 2's Flusher/ReaderFrom forwarding.
- Compresses only when the client accepts gzip, the response is a compressible
type, there is no Range header, and nothing already set Content-Encoding.
Sets Vary: Accept-Encoding and drops the now-wrong Content-Length.
- Verified by sha256 round-trip (identity vs gzip-decompressed are identical),
and by confirming a Range request still returns 206 with no Content-Encoding.
JS 351,835 -> 108,794 bytes (69% smaller); CSS 80% smaller.
epub.js is code-split (Reader.svelte)
- It was a static import, so ~350 KB was downloaded by everyone on first load
even though most never open an ebook. Now loaded on demand.
- Main bundle 458 KB -> 112 KB (gzipped 149 KB -> 39.5 KB).
- Verified at runtime, which pnpm check cannot: only the main bundle loads
initially, the epubjs chunk is fetched when a book is opened, and the reader
still renders.
Home screen tests (internal/media)
- Coverage 23.4% -> 50.2%. (The 17.9% quoted in the brief was a stale reading;
the agent measured the real baseline by moving its own files aside.)
- Covers the five shelves, counts, Explore's invariants (visibility, no
duplicates, every kind represented, varies across calls, wrap-around),
the list offset/limit/total contract, MostViewed ordering, and — most
importantly — grant scoping, which is the security chokepoint.
Fixed a bug the tests found: Suggestions seeded from revoked libraries
- topEngagement selected the user's most recent progress row with no library
filter, so progress from a library whose grant had been revoked still steered
the home screen — a visible item was labelled "More books" based on content
the user could no longer read. Returned items were always correctly scoped, so
this was an information-flow gap rather than a content leak.
- Now scoped to visible libraries, and the test that pinned the old behaviour
flipped to assert the correct reason.
gofmt/vet clean, 14 packages pass, media tests run 3x for flakiness.
TouchSession throttle (internal/auth)
- Middleware wrote sessions.last_seen on EVERY authenticated request, taking
SQLite's single write lock each time. For media that is severe: one seek in
an audio file fires several Range requests, each taking the write lock and
competing with the scanner and progress saves.
- Now touched at most once per 5 minutes, decided from the LastSeen already
loaded by SessionByToken, so no extra query. A zero interval degrades to the
old per-request behaviour, so a hand-built Authenticator cannot silently stop
updating.
- Verified end to end with an AFTER UPDATE trigger: 20 requests now cause 0
session writes, down from 20.
Response writer forwarding (internal/httpapi)
- statusRecorder wrapped every ResponseWriter but implemented only Write and
WriteHeader, so it did not implement http.Flusher. That silently disabled
early flushing (progressive audio/video) and io.ReaderFrom, which is what
lets io.Copy use sendfile for multi-GB files. Now forwards Flush, Hijack and
ReadFrom, and WriteHeader is idempotent.
Security headers (internal/httpapi)
- Added nosniff, X-Frame-Options: DENY, Referrer-Policy and a CSP. Outermost in
the chain so a recovered panic's 500 still carries them.
- CORRECTION to an earlier claim: I said a malicious EPUB was stored XSS.
That was wrong. epub.js sets iframe sandbox to "allow-same-origin" and only
appends "allow-scripts" when settings.allowScriptedContent is true, which
defaults to false and Reader.svelte never sets. EPUB content cannot execute
script. The CSP is still worth having as defence in depth, but that specific
justification does not hold.
Final flush on teardown (AudiobookPlayer, Reader)
- The last playback/reading position was saved with a fire-and-forget fetch in
onDestroy, which the browser cancels on navigate-away — losing exactly the
thing worth persisting. Now persisted via fetch with keepalive: true, wired
to pagehide and visibilitychange(hidden), latched so the overlapping events
fire one request, and reset when the page becomes visible again so a bfcache
restore still flushes later. sendBeacon was not usable: it is POST-only and
progress is PUT-only.
- Also removed the queueMicrotask track-swap race in AudiobookPlayer: play() is
now driven by onLoadedMetadata once the new src has actually loaded.
Honest readers (Reader, ComicReader)
- registry.go advertises .mobi/.azw3/.fb2 under ebooks, but every ebooks item
was handed to epub.js, which cannot parse them. Now only .epub uses epub.js;
other containers show a message naming the format plus a download link. Items
stay catalogued and downloadable rather than silently vanishing.
- .cbr/.rar comics say the archive format is unsupported instead of surfacing
the server's raw 422 (the server reads zip only).
Verified in a real browser, not just compiled: all six reader surfaces render
(epub iframe, cbz images, audio element, photo, pdf canvas) with zero CSP
violations; .mobi shows the honest message with no epub.js iframe. gofmt/vet
clean, 14 packages pass, svelte-check 0/0.
Known issue, NOT introduced here and NOT fixed: the EPUB reader logs
"Error: No Element Provided" x3 and a TypeError in epub.js injectIdentifier
(doc.getElementsByTagName("head")[0] undefined). Proven pre-existing by A/B
against c50b3cf, and not caused by the test file (spec-correct XHTML) or by
caching (cold browser). Non-fatal: the book still renders. Needs its own
investigation.
A 50k-file library used to wedge the database. This wave fixes the four
reasons why, plus a timestamp-ordering bug the work exposed.
Chunked scans (internal/library/scan.go)
- One transaction wrapped the whole reconciliation, holding SQLite's single
write lock for the entire scan. Now 500 groups per transaction, and the two
missing-file passes are batched too.
- Verified: a 20,000-file scan reports partial progress (scanned=1500 mid-scan)
and a concurrent progress save returns 200 throughout. Under the old code
that save blocked for the whole scan.
- Behaviour change: a scan is no longer all-or-nothing. A crash now leaves a
consistent partial scan. The scan is idempotent, so re-running finishes it.
- res.Errors is capped at 100 (a broken mount used to append one string per
file, then serialise the lot into the job row); the overflow is counted.
Concurrent job workers (internal/jobs/jobs.go)
- Run() was single-threaded, so a scan stalled every enrichment job behind it.
Now 4 workers with jittered idle waits.
- Handlers run concurrently, so Register/lookup take an RWMutex, and a
panicking handler is recovered and marked failed rather than killing a worker
and stranding a row in 'running'.
- RecoverStale() requeues jobs left 'running' by a dead process; called at boot
from main.go. Previously such a job stayed 'running' forever and the UI
polled it forever.
Explore (internal/media/discover.go)
- Full-scanned and sorted the whole library on every home load, and was
deterministic, so it showed the same 30 items every visit. Now a per-kind
fair-share id range scan with a random start, wrap-around, dedup and
interleaving. The doc comment claimed per-kind sampling before; now it does.
Indexes (migrations/0008_query_indexes.sql)
- Benchmarked with ANALYZE over 60k items: browse page 1 13.37ms -> 0.03ms
(484x, full scan -> covering index), deep page 8x, recently added 3.2x.
- A candidate index for the Continue shelf was built, measured and REJECTED:
the planner never chose it because `finished = 0` selects the majority of a
user's rows, so it had no selectivity and would have cost a write on every
progress save. Recorded in the migration rather than shipped as dead weight.
Canonical timestamps (internal/core/stamp.go)
- time.RFC3339Nano trims trailing zeros, so its fraction width varies and the
strings are not lexicographically sortable: ".5Z" > ".55Z" as strings though
.500s < .550s. That broke every ORDER BY <timestamp> and MAX(timestamp),
including the Continue shelf and Most Viewed. It was wrong even in columns
written 100% by Go, so it was not a cross-format issue.
- All 14 write sites now use core.Stamp, matching SQLite's own default format
exactly. Read paths are untouched: RFC3339Nano parses both shapes, so
existing rows keep working.
Also: busy_timeout 5s -> 15s. Chunked scans still delay a concurrent write, and
measured tail latency on a first scan of 15k new files reached 3.3s, leaving
almost no margin at 5s. Waiting beats failing with SQLITE_BUSY and losing a
playback position. A 10ms inter-batch yield was implemented, measured, found to
have no effect distinguishable from noise (overlapping ranges), and removed.
Verified end to end, not just compiled: orphan recovery logged at boot, scan
returns 202 in 0.7ms, Explore varies across loads, concurrent writes survive a
scan. gofmt/vet clean, 14 packages pass, -race clean on the queue, svelte-check
0/0, fresh install applies all 8 migrations.
Five tasks, four delegated to agents on disjoint files and one on the critical
path. Board and verification log: ~/.opencode/plan/vellum-beta-tasks.md
T1 Async scans (coordinator)
- migration 0007 adds ref_id + result to jobs, indexed by (kind, ref_id, id),
so a job can be found by what it operates on and can report back.
- jobs: EnqueueRef, JobID(ctx) so a handler can attach a result, SetResult.
- POST /libraries/{id}/scan now enqueues and returns 202; a real library takes
longer than any request timeout, and a hanging request looks like a broken
server. GET /libraries/{id}/scan/status reports the latest run.
- Libraries.svelte polls and shows "Scanning…" instead of freezing.
T2 Admin CLI (agent)
- internal/cli: `user add|passwd|list`, `backup <path>` via VACUUM INTO, with
tests. Wired into main.go, which the coordinator kept.
T3 Deploy (agent)
- compose.yml now mounts all six media kinds read-only with :z and carries the
metadata env keys. DEPLOY.md covers volumes, SELinux, admin bootstrap, TLS,
backup and upgrade.
T4 Pagination (agent)
- GET /items takes offset and returns total; Browse loads 60 at a time with
"Load more" and "Showing X of Y" instead of silently truncating at 300.
T5 Password change UI (agent)
- A Password section in Settings with validation, busy and success states.
Verified end to end: scan returns 202 in 0.7 ms and polls to done; the CLI
created a user, changed a password, and produced a complete 240 KB backup while
the server ran; pagination reports total 8 and pages correctly; compose parses
with every mount :z. go vet and 14 test packages pass; svelte-check 0/0.
The landing screen now reads as a product rather than a list.
- migration 0006: item_views, per user. "Most viewed" is personal — what I
re-read is not what you re-read. Counted when an item is opened in a reader,
so the number means consumed, not merely fetched.
- Suggestions: unstarted items related to what the user has been consuming —
same author first, then same kind, then recent as a floor. Each carries the
reason it was chosen ("More by Daft Punk"), because a suggestion nobody can
understand is noise.
- Explore: a mixed sample across every kind, excluding in-progress items so the
zone is discovery rather than a repeat.
- Home fetches all five shelves in one request; the screen never pops in.
- web: Shelf gained subtitle/badge support, Card gained a subtitle, and the
home renders four shelves plus a wrapping Explore grid.
Three bugs found and fixed while verifying:
- Continue showed a multi-track audiobook once per chapter. GROUP BY item_id
with MAX(updated_at) picks one card per item (SQLite takes the bare columns
from the row that produced the max).
- Suggestions and Explore passed their SQL arguments in the wrong order: the IN
list precedes the user id in those queries, so a user id was being bound as a
library id. Covered by a scoping test that would have caught it.
- The sidebar lost its media types on a cold, logged-out load. bootstrap()
fetched media types inside a Promise.all with /auth/me, so the expected 401
rejected the whole thing and nothing refetched after login. Media types are
public and now load independently.
The RomM replacement.
- registry: a games media type with ~35 ROM extensions across NES, SNES, GB/GBA,
N64, NDS, Genesis/Master System/Game Gear, PC Engine, Neo Geo Pocket,
WonderSwan, Atari, Coleco, Intellivision, and PlayStation.
- games: platform detection from the file extension, mapping each to an
EmulatorJS core. Platforms with no browser core are recognised but marked
not-playable so the UI offers a download instead of a broken player.
- migration 0005: game_saves keyed by (user, item, slot). Save states are user
data, not media — two people playing the same ROM keep their own progress.
- save endpoints with a 16 MiB cap and a validated slot name; the upload is
read through a LimitReader so an unbounded body is refused rather than
buffered.
- igdb provider: Twitch client-credentials flow with the token cached for its
~2 month lifetime, Apicalypse query body, and cover URLs normalised from
protocol-relative t_thumb to https t_cover_big.
- searchTitle: strips "(USA)", "[!]", "(Rev 1)" and similar release tags before
searching. "Sonic the Hedgehog (USA)" matched nothing; "Sonic the Hedgehog"
matches immediately. Stored titles keep the original.
- web: GamePlayer embeds EmulatorJS and adds Load/Save-to-server buttons.
Verified live against IGDB: Sonic matched (id 316900) with real cover art; a
4 KB save uploaded, listed, downloaded byte-identical, and deleted.
- comicvine provider for comics. ComicVine has no keyless tier, so the
provider is only registered when VELLUM_COMICVINE_API_KEY is set; without it
the server logs a note and comic enrichment completes as a no-op rather than
erroring.
- BaseURL is injectable so the response parsing is covered by a test against a
local server: candidate mapping, the required User-Agent, and the
status_code=100 invalid-key case (ComicVine returns HTTP 200 for that).
- descriptions are stripped of ComicVine's embedded HTML before storage.
Verified against a fake ComicVine server, and live that a comics library
enriches cleanly with no key configured.
- musicbrainz provider: keyless, with the required descriptive User-Agent and
a 1 req/s throttle so the provider cannot get us blocked. Covers come from
the Cover Art Archive by release id.
- artistHint: directory-grouped items are stored as "Artist/Album", so the
parent directory is used as the artist when querying. Without it the search
for "Random Access Memories" matched an unrelated release by another artist.
- pickBest: providers return fuzzy matches, and taking candidates[0] blindly
is how an album gets the wrong artist. A candidate is only accepted when its
normalised title corresponds to the item's; otherwise the provider is
recorded as notfound and the next one is tried.
Verified live: after the fix the album resolves to Daft Punk, 2013, and the
correct release id, with real cover art. Before the fix it matched a Franz
Kirmann release.
- PdfReader renders PDFs to canvas with page navigation and page-number
progress, reusing the same integer progress model as comics.
- Reader routes by file container, not media kind, so a PDF in a comics
library and a PDF ebook both open in the PDF reader.
- pdf.js is dynamically imported: the main bundle stays 443 kB and the 474 kB
pdf chunk plus 1.26 MB worker load only when a PDF is actually opened.
- comics cover extractor now only handles zip archives. A PDF in a comics
library is a valid item but not a zip, so it returns no cover instead of
logging an error on every request.
- fixed the page scale to leave slack for the vertical scrollbar, which was
otherwise forcing a horizontal scrollbar on tall pages.
Verified live: a 3-page PDF renders, navigates, and reports 33% progress.
The app looked like an admin panel. This is the product people actually see
when they open their library.
- Home: a landing screen with a Continue shelf (progress bars, straight from
the progress table) and per-type shelves of recent additions. One request
feeds it, so it does not pop in piece by piece.
- Browse: cover grid per media type with type chips, plus browse-all. Powered
by a new GET /items that lists across every visible library, filtered by
kind, through the same access chokepoint.
- Item page: cover, kind, title, authors, progress, a primary action that says
Continue when you have started, description, and the file list for
multi-file items. Cards now open the item page rather than jumping straight
into a reader.
- Search: results grouped by media type, cover-forward.
- Navigation moved from "libraries" to "media types" — you browse Ebooks, not
"the Ebooks folder". Libraries became an admin concern.
- Design system: dark by default, cover-forward, one accent, tighter type
scale, horizontal shelves, skeleton loaders.
- media: GET /home, GET /items, and authors on item detail.
Verified by screenshot in a live instance: home shelves, browse grid, and item
page all render correctly.
- photos: generates a downscaled JPEG thumbnail per image, because serving a
folder of 12-megapixel originals to a grid is unusable on a TV box. Uses
CatmullRom scaling via golang.org/x/image; decodes jpeg/png/gif/webp.
- an undecodable format returns no thumbnail rather than an error, so the grid
falls back to a placeholder.
- images already smaller than the 512px ceiling are passed through untouched.
- library items endpoint now returns library_id, which the viewer needs to
find its siblings for navigation.
- web: PhotoViewer with keyboard and button navigation within the library.
Verified: 900x1200 PNG thumbnailed to a 512-long-edge JPEG and served;
unit tests assert exact dimensions, aspect preservation, and that non-images
produce no thumbnail without erroring.
A book is a folder of chapters, not a file. The scanner gained a grouping
mode so a directory of tracks becomes one item with ordered files.
- registry: MediaType.Group (file | directory). Audiobooks and music group by
directory; everything else stays one item per file.
- scanner: builds groups, upserts one item per group, marks items missing when
their directory disappears, and keeps the file-level added/updated/missing
accounting. Tracks are ordered naturally, so chapter 2 precedes chapter 10.
- natsort: one shared natural-order comparator, now used by both the scanner
and the comics module (which had its own copy).
- media: GET /items/{id}/files/{fileID} streams one specific track. It
verifies the file belongs to the item, so an item id cannot be paired with
another item's file id.
No new module was needed for playback: the polymorphic progress table already
keys on file_id, so per-chapter resume fell out of the existing design. The
web player resumes the exact track and offset.
Verified: 3 chapters grouped into one book titled "Mistborn", natural order,
range request returned 206, mismatched file id 404s, per-track progress
round-trips.
- jobs: a table-backed queue in the same SQLite file. No Redis, no second
process, included in the same backup. claim() is a single UPDATE...RETURNING
so a job cannot run twice even with several workers. Failures back off
exponentially; a missing handler fails permanently rather than retrying a
bug.
- metadata: Provider interface plus Open Library and Google Books (both
keyless). The enricher tries providers in registration order, falls back on
empty, and continues past a provider error.
- migration 0004: metadata_fetch records the attempt, not just the match, so a
rescan does not re-hammer a rate-limited API for an item that will never
match. Errors are always retried; ok/notfound are cached for 30 days.
- enrichment writes provider_ids, updates title/year, links authors through
people/item_people, and downloads the cover as an asset. The cover download
runs outside the transaction: a slow image host must not hold a write lock.
- endpoints: queue enrichment per library, and inspect an item's matches.
- web: Enrich button per library.
Verified live against Open Library: "The Hobbit" matched /works/OL27482W,
title and year 1937 written, a 301x500 JPEG cover downloaded and served.
- migration 0003: an FTS5 index over media_items as an external-content table,
so the index stores no copy of the text. Triggers keep it in sync for every
write path, including the scanner's bulk inserts.
- migrate: splitStatements now tracks BEGIN...END depth, so CREATE TRIGGER
bodies survive as one statement. Without this, the trigger's internal
semicolons would tear it apart.
- search: every whitespace term is quoted before it reaches FTS5. Unquoted
input like a bare quote, a hyphen, or the word OR is valid FTS5 syntax and
would either error or silently change the query's meaning. Tested against a
list of hostile inputs, including a DROP TABLE attempt.
- scoped through the same VisibleLibraryIDs chokepoint; an empty library set
returns nothing, never everything.
- web: search box in the topbar and a results view.
Verified live: finds items across libraries, hostile input returns 200 with no
results, index stays in sync on insert/update/delete.
- comics: reads pages straight out of the archive on demand. No extraction
pass, so a 200-page CBZ is not materialised for a reader who views three
pages. Pages are read into memory and served via http.ServeContent, which
gives Range for free.
- natural page ordering: page2 sorts before page10. Lexical ordering shuffles
a numbered archive, which is the difference between a readable book and a
scrambled one.
- skips __MACOSX and dotfiles; rejects archives with no images.
- cover is the first page.
- media.Item gained ModTime (from the backing file) so page responses carry a
Last-Modified and support conditional GETs.
- web: ComicReader with a bounded prefetch window (two pages each way), page
jump, keyboard nav, and progress stored as a page number.
Verified: 3-page CBZ served in natural order, out-of-range 404, cover from
page 1, progress round-trip. svelte-check 0/0.
The frontend the app needs to be usable at all.
- web/: Svelte 5 + Vite + TypeScript SPA. Login, app shell with per-type nav,
library list/create/scan/delete, item grid with covers, admin user
management, and an epub.js reader that syncs progress.
- Reader resumes by spine index — an integer — which is why the polymorphic
progress table needs no text column.
- Session state uses classic stores in a plain .ts module rather than runes in
a .svelte.ts module, avoiding special module resolution.
- Embedding: web/embed.go embeds web/dist. A committed .gitkeep keeps the
directive valid before the frontend is built, so `go build ./...` never
fails. Unknown non-API paths serve index.html for client routes; /api/* and
/healthz always return JSON and never fall back to HTML.
- Dockerfile builds the UI in a node stage and overlays dist into the Go stage.
- pnpm 12 needs allowBuilds in pnpm-workspace.yaml; esbuild approved, the two
unused transitive scripts explicitly denied so install exits 0.
svelte-check: 0 errors, 0 warnings. Verified served: SPA shell at /, route
fallback, hashed asset, and JSON 404 on /api/*.
- stream: serving goes through http.ServeContent, so Range/If-Range/
If-Modified-Since are correct for every media type by construction
- progress: the polymorphic store. One row per user/item/file; the PK includes
the file so multi-track audiobooks resume per track later. Finished is
inferred when position reaches duration.
- media: generic item endpoints (detail, file, cover, progress, recent). Every
one routes through auth.CanAccessLibrary; an unauthorised item 404s.
- covers: a per-kind extractor registry. Extraction is lazy and recorded in
the assets table. A missing cover is a placeholder, never a 500.
- modules/ebooks: EPUB parsing (container.xml -> OPF -> metadata, spine, cover
with three fallback strategies), cover extraction to /config, and lazy
metadata parsing so parser fixes need no rescan.
Verified against a generated EPUB: title/2 authors/language/publisher/spine,
cover served as PNG, range request returned 206 with correct Content-Range,
progress round-tripped.
- library: CRUD with root-path resolution (absolute, symlink-free, must be
readable). A library name is unique; the root is validated at creation.
- registry: media types declare their extensions; the core never imports a
module. RegisterType is the extension point.
- scanner: walks a root, reconciles media_items/media_files in one
transaction, skips unchanged files by size+mtime, and marks vanished files
missing rather than deleting them (progress is the one thing here that
cannot be regenerated). A single unreadable file never fails a scan.
- migration 0002: `missing` flags on items and files, plus indexes.
- endpoints: media-types, library list/get/create/delete, per-library items,
and scan. Library reads go through the same grant chokepoint as everything
else; an unauthorised library 404s rather than 403s so its existence is not
leaked.
Multi-user from the first migration, as planned.
- auth: argon2id password hashing (PHC encoded, parameters travel with the
hash), opaque session tokens stored only as SHA-256, login rate limiting,
and account enumeration resistance (unknown user hashes anyway)
- sessions: cookie or Bearer auth, HttpOnly + SameSite=Lax, revocation on
password change
- grants: library_grants is the single access-control chokepoint. Admins see
every library; everyone else sees exactly their can_read grants
- bootstrap: VELLUM_ADMIN_USER/PASSWORD create the first admin only on an
empty database. No default password is ever invented.
- httpapi: mounts modules on a shared mux, session middleware innermost
- httpx: shared JSON helpers, strict body decoding
Gate: two users see different libraries. Tests cover that, plus session
expiry, token-at-rest, grant upsert, and the full login/me/logout flow.
Bootstraps Vellum as a single Go binary with one SQLite database.
- core: env-driven config, slog logging, SQLite (WAL) with pooled pragmas,
embedded migration runner keyed on PRAGMA user_version
- migrations/0001_core.sql: identity, media, user state, infra. Media items
are global; all per-user state (progress, favorites, grants) is a join.
One polymorphic progress table covers every media kind.
- httpapi: request-ID, structured request logging, panic recovery, JSON
endpoints for /healthz and /api/v1/version
- multi-arch Dockerfile (pure-Go SQLite, CGO_ENABLED=0) with an entrypoint
that honours PUID/PGID like the rest of the self-hosted stack
- tests for migration idempotency, FK enforcement, WAL, config, middleware
Media does not exist yet. Nothing is exposed beyond health and version.
