Third step of ADR-008. Two routes inside the item family: the playlist starts
the session lazily and serves what ffmpeg has written so far, so playback can
begin while the rest still encodes; the segment route serves one .ts.
Segment names are validated against the exact shape ffmpeg writes before being
used as a path. Verified red-green: removing the check serves a file from
outside the session directory with a 200.
Two corrections made in review.
1. The manager was being built inside NewHandlers, which meant nothing could
ever shut it down. Its reaper would run for the process lifetime and a
running ffmpeg would outlive the server that started it. It is now created
in main.go beside the job queue and deferred, and passed in - the same
ownership the queue already has.
2. A playlist request could 404 before ffmpeg wrote the file, which a native
HLS player treats as fatal rather than retrying. The playlist handler now
waits up to four seconds for the first write. Segments are never waited for.
Verified end-to-end against a running server with real hardware: a 30-second
file produced a valid EVENT playlist in about a second (the wait doing its
job - no 404), a 658 KB segment with Content-Type video/mp2t, a 404 for a
traversal attempt, and a 401 unauthenticated.
Also confirmed at boot: all four metadata providers report their disabled
state and name their env var, including the new adult one.
Two halves of the same feature: the scanner finds tracks, the media layer
serves them as WebVTT.
Discovery walks each directory for .srt/.ass/.ssa/.vtt/.sub beside a video and
matches them by base name, including the language form "Movie.en.srt" and
"Movie.pt-BR.srt". A sidecar is never itself an item. Matching is a dot-guarded
prefix on the real video base name, so a dotted release name like
"The.Matrix.1999.1080p.en.srt" does not read the year as a language.
Serving adds two routes inside the existing item family, both grant-checked
through catalog.Store (ADR-005) so forbidden and absent are the same 404.
Browsers play WebVTT and nothing else, so .srt is converted (header plus the
comma-to-dot in timestamps), .vtt passes through, and embedded streams and
.ass/.ssa go through ffmpeg. Conversions are cached under ConfigDir, because
converting per request would spawn an ffmpeg on every seek. A track that fails
to convert fails alone - the video and the other tracks are unaffected.
Subtitle rows are a cache of what is on disk, so a vanished sidecar has its row
deleted. That is the opposite of a vanished video, which is marked instead
(ADR-003) because progress hangs off it and cannot be regenerated.
Verified red-green: disabling the sidecar call fails all four scanner tests,
and removing the VTT header fails the conversion test.
Known limits: a "Movie.forced.en.srt" records "forced.en" rather than "en",
and the conversion cache is keyed by track id, so a sidecar replaced under an
unchanged row serves the stale conversion until a rescan recreates the row.
The scanner builds library -> series -> season -> episode; this makes it
reachable. Three reads, all grant-checked through catalog.Store (ADR-005):
a series' seasons, a season's episodes, and the next unwatched episode.
Each loads the parent through ItemFor first, so forbidden and absent are the
same ErrNotFound, and the child query is scoped to the parent's own
library_id so a corrupt parent_id cannot cross a library boundary. Verified
red-green: swapping ItemFor for Item returns two rows to an ungranted
caller.
Episodes come back in episode order, not insertion order, and the ordering
is verified red-green too - dropping the clause yields [3 1 10 2].
Routes stay inside the existing family: /api/v1/items/{id}/seasons,
/episodes and /next. No parallel route surface.
Jellyfin gains SeriesDto, SeasonDto and EpisodeDto. Jellyfin's wire type is
BaseItemDto, so these carry the folder/file split plus IndexNumber and
ParentIndexNumber. ItemType now takes the item rather than the kind, because
an episode IS kind video - that is the one signature change, and the dto
tests were updated with it.
Verified end-to-end against a running server: a library holding S01E02,
1x02, a bare-number file and a film produced two series, three seasons and
six episodes, with the film left alone; the routes returned the seasons in
order, the episodes in episode order, and the next unwatched episode.
Note for later: Subsonic and the web UI are untouched, so a series browses
there as it did before. The scanner also has no separate 'tv' kind, so a
video library mixing films and shows builds both shapes side by side.
Four extractions, each chosen because the two halves have different callers,
different failure modes or different testing needs. Not a line-count exercise.
internal/home out of internal/media
- media serves item RESOURCES (detail, file, cover, progress); home computes
AGGREGATES (the five shelves). The 2328-line package was doing both.
- media keeps POST /items/{id}/view because it is item-scoped and must stay in
the package that authorizes the item; it imports home.RecordView for that one
thing. home does not import media.
- Verified live: all five shelves non-empty, and two consecutive /home calls
return different explore ids, which is the non-determinism the Explore rewrite
exists for.
internal/scanner out of internal/library
- The scanner walks the filesystem and reconciles; library CRUD is an admin HTTP
surface. Different callers (a job vs a request), different failure modes.
- The registry deliberately STAYED in internal/library: three of its four callers
are not the scanner (modules register types, handlers serve the list, metadata
reads the group mode, subsonic calls CleanTitle). It is a property of the kind,
not of the scan. The one genuinely scanner-owned helper became
library.ExtensionsFor.
- Two tests moved with the code rather than being rewritten; they now exercise
the boundary by calling library.ListMissing across it.
- Verified live: first scan updates, rescan reports unchanged=N, so change
detection still short-circuits.
internal/providers out of internal/metadata
- Providers talk to third-party APIs; the enrich job owns the matching policy and
the database writes. The split's whole point is that a provider is testable
WITHOUT a database, so a DB-free test was added and OpenLibrary gained a
BaseURL override to make it possible.
- providers imports zero internal packages — a true leaf.
- The matching policy (searchTitle, artistHint, pickBest) is untouched. It exists
because real false matches were found.
dto packages out of internal/jellyfin and internal/subsonic
- The wire types and their mapping are now separate from the handler logic, so
the contract with clients we do not control is reviewable on its own.
- Both dto packages are PURE: zero references to sql.DB or net/http, verified by
grep. That is what makes them reviewable.
- No field name, tag or error code changed.
internal/arch did its job: it failed the build twice during this phase, once for
each new package that was not yet placed in a layer. That is the rule working,
not an obstacle.
Verified: gofmt clean, build exit 0, vet exit 0, 23 packages pass, svelte-check
0/0, and end to end against a running server — home shelves, a queued scan
reporting unchanged, Jellyfin /Items, and Subsonic ping.
Process note: one agent (dto) stopped mid-task and left the tree red — the dto
packages unlayered and its test still referencing constants that had moved. The
arch test caught it immediately. I finished it by hand: exported constant
references in the test, and both dto packages placed at layer 3.
Still open from Phase 1: internal/jellyfin has no test files, catalog is
item-centric while Subsonic is file-first, and subsonic loadAlbum now scopes to
the item's own library.
Motivation, from the evidence rather than from taste. Five packages each
implemented "load an item, then check the caller may read it": media, comics,
games, jellyfin (with its own dbItem type and its own SQL), and subsonic. Five
copies of one authorization rule means four can drift, and the failure mode is
not a crash — it is silently serving somebody a library they were never granted.
One implementation is one thing to review.
internal/catalog now owns item and file access, and the access check.
LoadUnchecked(ctx, db, id) -- no access check; jobs and tests only
LoadFor(ctx, db, u, id) -- load AND enforce the grant, in one call
Files / FilesFor / FileByID / FirstFile / FileCount
The unchecked variant is deliberately named so that reaching for it by accident
is hard: it is the dangerous call, so it should not have the friendlier name.
This came from the interface-design rule that a good interface makes the right
thing easy and the wrong thing hard.
The security property is tested, not asserted: a forbidden item and an absent
item return byte-identical errors, so no handler can leak the existence of
another user's library through a 403. That is now enforced in one place.
Migrated all five surfaces onto it. media, comics, games and ebooks went from
four duplicate loaders to zero. jellyfin's dbItem and loadDBItem are gone.
subsonic's per-item authorization moved while its album/artist aggregate
projections stayed put, because grouping items into albums is a presentation
concern, not a data-access one. The remaining raw SQL everywhere is browse and
aggregate queries; no per-item load decides access any more.
internal/arch enforces the layering instead of documenting it. Five rules, each
proven able to fail by deliberately breaking it:
- no upward imports across six explicit layers
- internal/core may depend only on migrations
- internal/catalog must not know about HTTP, or it stops being reusable
- modules must not import each other
- every package must be placed in a layer, so the map cannot go stale
A test that only ever passes is decoration, so each rule was verified by
introducing the violation and confirming the failure.
Verified: gofmt clean, go build ./... exit 0, go vet exit 0, 20 packages pass,
svelte-check 0/0. End to end against a running server: the vellum API serves
item detail with codec facts, an unknown item 404s, Jellyfin AuthenticateByName
mints a token and /Items browses, and Subsonic ping returns status ok. The
subsonic migration was additionally A/B'd byte-identical against the previous
implementation across ~15 endpoints, and Jellyfin/Subsonic both still return the
same error for a forbidden item as for an absent one.
Known follow-ups, recorded rather than silently skipped:
- internal/jellyfin has no test files at all. Its wire contract is the most
client-visible surface in the tree and is now verified only by hand.
- catalog is item-centric, but Subsonic addresses songs file-first (mf-<id>),
so it resolves file to item through its own projection. A catalog lookup
that takes a file id would close that.
- subsonic loadAlbum now scopes the projection to the item's own library
rather than all visible audio libraries. Equivalent by construction, since
an item's kind tracks its library's kind, but a hand-edited mismatch would
differ.
Direct play only. The server is a headless GPU-less Xeon: nothing is ever
transcoded or remuxed. A file either plays in the client or it does not.
Measured, not assumed (ffmpeg-generated files with known codecs, played through
Vellum's own stream endpoint in Chrome 155 / Linux):
mp4 h264+aac plays mkv h264+aac PLAYS <- container is fine
mov h264+aac plays webm vp9/opus plays
mp4 av1+opus plays webm vp8/vorbis plays
mp4 hevc+aac AUDIO ONLY, videoWidth stays 0, no error
mkv hevc+flac AUDIO ONLY
avi mpeg4+mp3 fails outright
All seven audio formats play. Two conclusions drive the design: codec matters
and container barely does (mkv+h264 plays, mp4+HEVC does not), and canPlayType
is unreliable in both directions — it says "no" for mov+h264 (which plays) and
"maybe" for mkv+hevc (whose video never decodes).
Therefore playability is a property of the CLIENT, not the file. The server
never declares a file playable; it reports codec facts and each client decides.
That is what makes adding mpv/libvlc/MDK later a matter of registering an engine
rather than re-architecting.
Wave 4 - audio depth
- New MusicPlayer.svelte: real queue, shuffle that walks a fixed permutation
(so "previous" means something and nothing repeats early), repeat off/all/one,
per-track rows, album art, next-track preload.
- AudiobookPlayer: MediaSession (OS media keys + lock screen), playback speed
that SURVIVES a chapter change and persists, sleep timer with countdown that a
manual pause cancels, keyboard shortcuts, next-chapter preload.
- Shared mediaSession.ts written by the coordinator before dispatch so both
players consume one implementation instead of inventing two.
Wave 5 - ffprobe facts
- internal/probe (ffprobe wrapper, optional dependency), migration 0010 adding
vcodec/acodec/width/height/probed_at to media_files, scanner hook, API fields.
- Probing runs only for NEW or CHANGED files and BEFORE the write transaction
opens, so it can never hold SQLite's write lock and a rescan never becomes a
full-library ffprobe sweep. Proven: 797ms probing scan, then 1ms rescan with
probed_at byte-identical.
- ffprobe skips attached_pic so an mp3 with cover art reports no video codec.
- Fixed a real hole: .avi is in the video registry but had no MIME entry, so it
was served as application/octet-stream and could never play.
Wave 6 - the video player
- VideoPlayer.svelte with resume, progress + keepalive final flush, MediaSession,
keyboard shortcuts, multi-file picker, and codec facts shown.
- The honest path is the feature: an HEVC file makes Chrome play audio over a
blank frame with no error, which is the worst possible outcome. The player
detects it and says so, naming the codec and pointing at clients that can.
Coordinator fix to Wave 6: the agent's detection hung off `loadedmetadata`, but
an explicit `type` the browser cannot satisfy makes it reject the source during
selection, so NEITHER loadedmetadata NOR error fires — and the player showed a
working-looking 0:00 with a Play button, exactly the silently-broken state the
component exists to prevent. Added a load watchdog: if the element says nothing
within 2.5s, the source was rejected and the honest notice is shown. Verified
per file afterwards (h264/mkv play at 640 wide; hevc and mpeg4 show their
notices).
Verified end to end, not just compiled: 15 packages pass (including the new
internal/probe), gofmt/vet clean, svelte-check 0/0, and every claim above checked
against a running server. Note the demo's own mp3s are 100 KB of random bytes and
cannot decode; the audio players were proven against generated real audio instead.
The landing screen now reads as a product rather than a list.
- migration 0006: item_views, per user. "Most viewed" is personal — what I
re-read is not what you re-read. Counted when an item is opened in a reader,
so the number means consumed, not merely fetched.
- Suggestions: unstarted items related to what the user has been consuming —
same author first, then same kind, then recent as a floor. Each carries the
reason it was chosen ("More by Daft Punk"), because a suggestion nobody can
understand is noise.
- Explore: a mixed sample across every kind, excluding in-progress items so the
zone is discovery rather than a repeat.
- Home fetches all five shelves in one request; the screen never pops in.
- web: Shelf gained subtitle/badge support, Card gained a subtitle, and the
home renders four shelves plus a wrapping Explore grid.
Three bugs found and fixed while verifying:
- Continue showed a multi-track audiobook once per chapter. GROUP BY item_id
with MAX(updated_at) picks one card per item (SQLite takes the bare columns
from the row that produced the max).
- Suggestions and Explore passed their SQL arguments in the wrong order: the IN
list precedes the user id in those queries, so a user id was being bound as a
library id. Covered by a scoping test that would have caught it.
- The sidebar lost its media types on a cold, logged-out load. bootstrap()
fetched media types inside a Promise.all with /auth/me, so the expected 401
rejected the whole thing and nothing refetched after login. Media types are
public and now load independently.
The app looked like an admin panel. This is the product people actually see
when they open their library.
- Home: a landing screen with a Continue shelf (progress bars, straight from
the progress table) and per-type shelves of recent additions. One request
feeds it, so it does not pop in piece by piece.
- Browse: cover grid per media type with type chips, plus browse-all. Powered
by a new GET /items that lists across every visible library, filtered by
kind, through the same access chokepoint.
- Item page: cover, kind, title, authors, progress, a primary action that says
Continue when you have started, description, and the file list for
multi-file items. Cards now open the item page rather than jumping straight
into a reader.
- Search: results grouped by media type, cover-forward.
- Navigation moved from "libraries" to "media types" — you browse Ebooks, not
"the Ebooks folder". Libraries became an admin concern.
- Design system: dark by default, cover-forward, one accent, tighter type
scale, horizontal shelves, skeleton loaders.
- media: GET /home, GET /items, and authors on item detail.
Verified by screenshot in a live instance: home shelves, browse grid, and item
page all render correctly.
A book is a folder of chapters, not a file. The scanner gained a grouping
mode so a directory of tracks becomes one item with ordered files.
- registry: MediaType.Group (file | directory). Audiobooks and music group by
directory; everything else stays one item per file.
- scanner: builds groups, upserts one item per group, marks items missing when
their directory disappears, and keeps the file-level added/updated/missing
accounting. Tracks are ordered naturally, so chapter 2 precedes chapter 10.
- natsort: one shared natural-order comparator, now used by both the scanner
and the comics module (which had its own copy).
- media: GET /items/{id}/files/{fileID} streams one specific track. It
verifies the file belongs to the item, so an item id cannot be paired with
another item's file id.
No new module was needed for playback: the polymorphic progress table already
keys on file_id, so per-chapter resume fell out of the existing design. The
web player resumes the exact track and offset.
Verified: 3 chapters grouped into one book titled "Mistborn", natural order,
range request returned 206, mismatched file id 404s, per-track progress
round-trips.
- stream: serving goes through http.ServeContent, so Range/If-Range/
If-Modified-Since are correct for every media type by construction
- progress: the polymorphic store. One row per user/item/file; the PK includes
the file so multi-track audiobooks resume per track later. Finished is
inferred when position reaches duration.
- media: generic item endpoints (detail, file, cover, progress, recent). Every
one routes through auth.CanAccessLibrary; an unauthorised item 404s.
- covers: a per-kind extractor registry. Extraction is lazy and recorded in
the assets table. A missing cover is a placeholder, never a 500.
- modules/ebooks: EPUB parsing (container.xml -> OPF -> metadata, spine, cover
with three fallback strategies), cover extraction to /config, and lazy
metadata parsing so parser fixes need no rescan.
Verified against a generated EPUB: title/2 authors/language/publisher/spine,
cover served as PNG, range request returned 206 with correct Content-Range,
progress round-tripped.
mithral