Four extractions, each chosen because the two halves have different callers,
different failure modes or different testing needs. Not a line-count exercise.
internal/home out of internal/media
- media serves item RESOURCES (detail, file, cover, progress); home computes
AGGREGATES (the five shelves). The 2328-line package was doing both.
- media keeps POST /items/{id}/view because it is item-scoped and must stay in
the package that authorizes the item; it imports home.RecordView for that one
thing. home does not import media.
- Verified live: all five shelves non-empty, and two consecutive /home calls
return different explore ids, which is the non-determinism the Explore rewrite
exists for.
internal/scanner out of internal/library
- The scanner walks the filesystem and reconciles; library CRUD is an admin HTTP
surface. Different callers (a job vs a request), different failure modes.
- The registry deliberately STAYED in internal/library: three of its four callers
are not the scanner (modules register types, handlers serve the list, metadata
reads the group mode, subsonic calls CleanTitle). It is a property of the kind,
not of the scan. The one genuinely scanner-owned helper became
library.ExtensionsFor.
- Two tests moved with the code rather than being rewritten; they now exercise
the boundary by calling library.ListMissing across it.
- Verified live: first scan updates, rescan reports unchanged=N, so change
detection still short-circuits.
internal/providers out of internal/metadata
- Providers talk to third-party APIs; the enrich job owns the matching policy and
the database writes. The split's whole point is that a provider is testable
WITHOUT a database, so a DB-free test was added and OpenLibrary gained a
BaseURL override to make it possible.
- providers imports zero internal packages — a true leaf.
- The matching policy (searchTitle, artistHint, pickBest) is untouched. It exists
because real false matches were found.
dto packages out of internal/jellyfin and internal/subsonic
- The wire types and their mapping are now separate from the handler logic, so
the contract with clients we do not control is reviewable on its own.
- Both dto packages are PURE: zero references to sql.DB or net/http, verified by
grep. That is what makes them reviewable.
- No field name, tag or error code changed.
internal/arch did its job: it failed the build twice during this phase, once for
each new package that was not yet placed in a layer. That is the rule working,
not an obstacle.
Verified: gofmt clean, build exit 0, vet exit 0, 23 packages pass, svelte-check
0/0, and end to end against a running server — home shelves, a queued scan
reporting unchanged, Jellyfin /Items, and Subsonic ping.
Process note: one agent (dto) stopped mid-task and left the tree red — the dto
packages unlayered and its test still referencing constants that had moved. The
arch test caught it immediately. I finished it by hand: exported constant
references in the test, and both dto packages placed at layer 3.
Still open from Phase 1: internal/jellyfin has no test files, catalog is
item-centric while Subsonic is file-first, and subsonic loadAlbum now scopes to
the item's own library.
Wave 7 - engines, chosen per file by the client's own capabilities
- internal/engine: a registry. html5 is built in and declares a CONSERVATIVE
browser baseline (h264/vp8/vp9/av1, deliberately no hevc and no mpeg4, because
that is what was measured). mpv, vlc and mdk are registered as native engines
declaring the broad support those libraries really have.
- The server never declares a file playable. It reports what is in the file and
matches it against the engines the CLIENT declares. Proven:
engines=["html5"] -> playable=false engine=null reason=vcodec:hevc
engines=["html5","mpv"] -> playable=true engine=mpv
Same file, different answer, because the client offered a different engine.
That is what makes adding a native client a registration rather than a rewrite.
- internal/playback: GET /playback/engines and POST /playback/resolve, behind
auth and the same grant chokepoint. An unreadable item is 404, not 403.
- web/src/lib/capabilities.ts: uses navigator.mediaCapabilities.decodingInfo(),
which returns {supported, smooth, powerEfficient}. That is real client-hardware
detection and it distinguishes "cannot decode" from "can decode but it will
stutter and drain the battery". Feature-detected; degrades to the server
baseline rather than throwing.
- VideoPlayer consults the server as an ADVISORY only. The element remains the
source of truth: the server's note appears first, then the element's own
confirmation replaces it once videoWidth proves the answer. Verified on HEVC —
"Playback may not work here" at 0-906ms, "Video cannot play here" from ~1208ms.
Wave 8 - missing files: visibility and an explicit purge
- migrations/0011 adds missing_since to media_items and media_files.
- It is written only on the TRANSITION into missing, so the age says when the
file first vanished rather than the last rescan. Proven byte-identical across a
rescan, and cleared when a file is restored.
- GET /api/v1/missing (grant-scoped) and POST /api/v1/libraries/{id}/purge
(admin only). The purge deletes only items that are missing AND have no
present file, and its cascade into progress/item_people/item_views/game_saves
is enumerated in a comment — that cascade is exactly why deletion is never
automatic. Proven: 19 items -> 18, only the missing one, the non-missing item
untouched and its files intact, second purge returns 0.
- MissingFiles.svelte in the Manage nav: title, kind, library, path, human age,
and a per-library Purge behind a confirmation that names the count and says
progress cannot be regenerated.
Coordinator wired playback.NewHandlers into main.go (the one line agents were
told to leave alone).
Verified end to end: 17 packages pass, gofmt/vet clean, svelte-check 0/0, and
every claim above checked against the running server.
Direct play only. The server is a headless GPU-less Xeon: nothing is ever
transcoded or remuxed. A file either plays in the client or it does not.
Measured, not assumed (ffmpeg-generated files with known codecs, played through
Vellum's own stream endpoint in Chrome 155 / Linux):
mp4 h264+aac plays mkv h264+aac PLAYS <- container is fine
mov h264+aac plays webm vp9/opus plays
mp4 av1+opus plays webm vp8/vorbis plays
mp4 hevc+aac AUDIO ONLY, videoWidth stays 0, no error
mkv hevc+flac AUDIO ONLY
avi mpeg4+mp3 fails outright
All seven audio formats play. Two conclusions drive the design: codec matters
and container barely does (mkv+h264 plays, mp4+HEVC does not), and canPlayType
is unreliable in both directions — it says "no" for mov+h264 (which plays) and
"maybe" for mkv+hevc (whose video never decodes).
Therefore playability is a property of the CLIENT, not the file. The server
never declares a file playable; it reports codec facts and each client decides.
That is what makes adding mpv/libvlc/MDK later a matter of registering an engine
rather than re-architecting.
Wave 4 - audio depth
- New MusicPlayer.svelte: real queue, shuffle that walks a fixed permutation
(so "previous" means something and nothing repeats early), repeat off/all/one,
per-track rows, album art, next-track preload.
- AudiobookPlayer: MediaSession (OS media keys + lock screen), playback speed
that SURVIVES a chapter change and persists, sleep timer with countdown that a
manual pause cancels, keyboard shortcuts, next-chapter preload.
- Shared mediaSession.ts written by the coordinator before dispatch so both
players consume one implementation instead of inventing two.
Wave 5 - ffprobe facts
- internal/probe (ffprobe wrapper, optional dependency), migration 0010 adding
vcodec/acodec/width/height/probed_at to media_files, scanner hook, API fields.
- Probing runs only for NEW or CHANGED files and BEFORE the write transaction
opens, so it can never hold SQLite's write lock and a rescan never becomes a
full-library ffprobe sweep. Proven: 797ms probing scan, then 1ms rescan with
probed_at byte-identical.
- ffprobe skips attached_pic so an mp3 with cover art reports no video codec.
- Fixed a real hole: .avi is in the video registry but had no MIME entry, so it
was served as application/octet-stream and could never play.
Wave 6 - the video player
- VideoPlayer.svelte with resume, progress + keepalive final flush, MediaSession,
keyboard shortcuts, multi-file picker, and codec facts shown.
- The honest path is the feature: an HEVC file makes Chrome play audio over a
blank frame with no error, which is the worst possible outcome. The player
detects it and says so, naming the codec and pointing at clients that can.
Coordinator fix to Wave 6: the agent's detection hung off `loadedmetadata`, but
an explicit `type` the browser cannot satisfy makes it reject the source during
selection, so NEITHER loadedmetadata NOR error fires — and the player showed a
working-looking 0:00 with a Play button, exactly the silently-broken state the
component exists to prevent. Added a load watchdog: if the element says nothing
within 2.5s, the source was rejected and the honest notice is shown. Verified
per file afterwards (h264/mkv play at 640 wide; hevc and mpeg4 show their
notices).
Verified end to end, not just compiled: 15 packages pass (including the new
internal/probe), gofmt/vet clean, svelte-check 0/0, and every claim above checked
against a running server. Note the demo's own mp3s are 100 KB of random bytes and
cannot decode; the audio players were proven against generated real audio instead.
A 50k-file library used to wedge the database. This wave fixes the four
reasons why, plus a timestamp-ordering bug the work exposed.
Chunked scans (internal/library/scan.go)
- One transaction wrapped the whole reconciliation, holding SQLite's single
write lock for the entire scan. Now 500 groups per transaction, and the two
missing-file passes are batched too.
- Verified: a 20,000-file scan reports partial progress (scanned=1500 mid-scan)
and a concurrent progress save returns 200 throughout. Under the old code
that save blocked for the whole scan.
- Behaviour change: a scan is no longer all-or-nothing. A crash now leaves a
consistent partial scan. The scan is idempotent, so re-running finishes it.
- res.Errors is capped at 100 (a broken mount used to append one string per
file, then serialise the lot into the job row); the overflow is counted.
Concurrent job workers (internal/jobs/jobs.go)
- Run() was single-threaded, so a scan stalled every enrichment job behind it.
Now 4 workers with jittered idle waits.
- Handlers run concurrently, so Register/lookup take an RWMutex, and a
panicking handler is recovered and marked failed rather than killing a worker
and stranding a row in 'running'.
- RecoverStale() requeues jobs left 'running' by a dead process; called at boot
from main.go. Previously such a job stayed 'running' forever and the UI
polled it forever.
Explore (internal/media/discover.go)
- Full-scanned and sorted the whole library on every home load, and was
deterministic, so it showed the same 30 items every visit. Now a per-kind
fair-share id range scan with a random start, wrap-around, dedup and
interleaving. The doc comment claimed per-kind sampling before; now it does.
Indexes (migrations/0008_query_indexes.sql)
- Benchmarked with ANALYZE over 60k items: browse page 1 13.37ms -> 0.03ms
(484x, full scan -> covering index), deep page 8x, recently added 3.2x.
- A candidate index for the Continue shelf was built, measured and REJECTED:
the planner never chose it because `finished = 0` selects the majority of a
user's rows, so it had no selectivity and would have cost a write on every
progress save. Recorded in the migration rather than shipped as dead weight.
Canonical timestamps (internal/core/stamp.go)
- time.RFC3339Nano trims trailing zeros, so its fraction width varies and the
strings are not lexicographically sortable: ".5Z" > ".55Z" as strings though
.500s < .550s. That broke every ORDER BY <timestamp> and MAX(timestamp),
including the Continue shelf and Most Viewed. It was wrong even in columns
written 100% by Go, so it was not a cross-format issue.
- All 14 write sites now use core.Stamp, matching SQLite's own default format
exactly. Read paths are untouched: RFC3339Nano parses both shapes, so
existing rows keep working.
Also: busy_timeout 5s -> 15s. Chunked scans still delay a concurrent write, and
measured tail latency on a first scan of 15k new files reached 3.3s, leaving
almost no margin at 5s. Waiting beats failing with SQLITE_BUSY and losing a
playback position. A 10ms inter-batch yield was implemented, measured, found to
have no effect distinguishable from noise (overlapping ranges), and removed.
Verified end to end, not just compiled: orphan recovery logged at boot, scan
returns 202 in 0.7ms, Explore varies across loads, concurrent writes survive a
scan. gofmt/vet clean, 14 packages pass, -race clean on the queue, svelte-check
0/0, fresh install applies all 8 migrations.
Five tasks, four delegated to agents on disjoint files and one on the critical
path. Board and verification log: ~/.opencode/plan/vellum-beta-tasks.md
T1 Async scans (coordinator)
- migration 0007 adds ref_id + result to jobs, indexed by (kind, ref_id, id),
so a job can be found by what it operates on and can report back.
- jobs: EnqueueRef, JobID(ctx) so a handler can attach a result, SetResult.
- POST /libraries/{id}/scan now enqueues and returns 202; a real library takes
longer than any request timeout, and a hanging request looks like a broken
server. GET /libraries/{id}/scan/status reports the latest run.
- Libraries.svelte polls and shows "Scanning…" instead of freezing.
T2 Admin CLI (agent)
- internal/cli: `user add|passwd|list`, `backup <path>` via VACUUM INTO, with
tests. Wired into main.go, which the coordinator kept.
T3 Deploy (agent)
- compose.yml now mounts all six media kinds read-only with :z and carries the
metadata env keys. DEPLOY.md covers volumes, SELinux, admin bootstrap, TLS,
backup and upgrade.
T4 Pagination (agent)
- GET /items takes offset and returns total; Browse loads 60 at a time with
"Load more" and "Showing X of Y" instead of silently truncating at 300.
T5 Password change UI (agent)
- A Password section in Settings with validation, busy and success states.
Verified end to end: scan returns 202 in 0.7 ms and polls to done; the CLI
created a user, changed a password, and produced a complete 240 KB backup while
the server ran; pagination reports total 8 and pages correctly; compose parses
with every mount :z. go vet and 14 test packages pass; svelte-check 0/0.
The RomM replacement.
- registry: a games media type with ~35 ROM extensions across NES, SNES, GB/GBA,
N64, NDS, Genesis/Master System/Game Gear, PC Engine, Neo Geo Pocket,
WonderSwan, Atari, Coleco, Intellivision, and PlayStation.
- games: platform detection from the file extension, mapping each to an
EmulatorJS core. Platforms with no browser core are recognised but marked
not-playable so the UI offers a download instead of a broken player.
- migration 0005: game_saves keyed by (user, item, slot). Save states are user
data, not media — two people playing the same ROM keep their own progress.
- save endpoints with a 16 MiB cap and a validated slot name; the upload is
read through a LimitReader so an unbounded body is refused rather than
buffered.
- igdb provider: Twitch client-credentials flow with the token cached for its
~2 month lifetime, Apicalypse query body, and cover URLs normalised from
protocol-relative t_thumb to https t_cover_big.
- searchTitle: strips "(USA)", "[!]", "(Rev 1)" and similar release tags before
searching. "Sonic the Hedgehog (USA)" matched nothing; "Sonic the Hedgehog"
matches immediately. Stored titles keep the original.
- web: GamePlayer embeds EmulatorJS and adds Load/Save-to-server buttons.
Verified live against IGDB: Sonic matched (id 316900) with real cover art; a
4 KB save uploaded, listed, downloaded byte-identical, and deleted.
- photos: generates a downscaled JPEG thumbnail per image, because serving a
folder of 12-megapixel originals to a grid is unusable on a TV box. Uses
CatmullRom scaling via golang.org/x/image; decodes jpeg/png/gif/webp.
- an undecodable format returns no thumbnail rather than an error, so the grid
falls back to a placeholder.
- images already smaller than the 512px ceiling are passed through untouched.
- library items endpoint now returns library_id, which the viewer needs to
find its siblings for navigation.
- web: PhotoViewer with keyboard and button navigation within the library.
Verified: 900x1200 PNG thumbnailed to a 512-long-edge JPEG and served;
unit tests assert exact dimensions, aspect preservation, and that non-images
produce no thumbnail without erroring.
A book is a folder of chapters, not a file. The scanner gained a grouping
mode so a directory of tracks becomes one item with ordered files.
- registry: MediaType.Group (file | directory). Audiobooks and music group by
directory; everything else stays one item per file.
- scanner: builds groups, upserts one item per group, marks items missing when
their directory disappears, and keeps the file-level added/updated/missing
accounting. Tracks are ordered naturally, so chapter 2 precedes chapter 10.
- natsort: one shared natural-order comparator, now used by both the scanner
and the comics module (which had its own copy).
- media: GET /items/{id}/files/{fileID} streams one specific track. It
verifies the file belongs to the item, so an item id cannot be paired with
another item's file id.
No new module was needed for playback: the polymorphic progress table already
keys on file_id, so per-chapter resume fell out of the existing design. The
web player resumes the exact track and offset.
Verified: 3 chapters grouped into one book titled "Mistborn", natural order,
range request returned 206, mismatched file id 404s, per-track progress
round-trips.
- library: CRUD with root-path resolution (absolute, symlink-free, must be
readable). A library name is unique; the root is validated at creation.
- registry: media types declare their extensions; the core never imports a
module. RegisterType is the extension point.
- scanner: walks a root, reconciles media_items/media_files in one
transaction, skips unchanged files by size+mtime, and marks vanished files
missing rather than deleting them (progress is the one thing here that
cannot be regenerated). A single unreadable file never fails a scan.
- migration 0002: `missing` flags on items and files, plus indexes.
- endpoints: media-types, library list/get/create/delete, per-library items,
and scan. Library reads go through the same grant chokepoint as everything
else; an unauthorised library 404s rather than 403s so its existence is not
leaked.
mithral