Authors are searchable (internal/search, migrations/0009) - The FTS index covered only title and path, so typing an artist or author found nothing even though people/item_people already held the credits. - Rather than a second index, the names are denormalised into a real media_items.authors column, so the existing external-content FTS and its triggers keep doing the work; item_people/people triggers recompute it, and the media_items UPDATE trigger re-indexes automatically. - Verified: searching "daft punk" now returns Random Access Memories, and so does the partial "punk". - The agent found and worked around a real FTS5 external-content bug: on an UPGRADE the backfill UPDATE fires the delete trigger for rowids not yet in the fresh index, raising "database disk image is malformed". A rebuild before the backfill fixes it. This would have broken existing databases while a fresh install looked fine. gzip for the SPA (internal/httpapi/static.go) - 456 KB of JS and 31 KB of CSS were served uncompressed on every cold load. - Scoped to the embedded SPA assets only, never API or media: compressing a Range response would break seeking, and wrapping every response would undo Wave 2's Flusher/ReaderFrom forwarding. - Compresses only when the client accepts gzip, the response is a compressible type, there is no Range header, and nothing already set Content-Encoding. Sets Vary: Accept-Encoding and drops the now-wrong Content-Length. - Verified by sha256 round-trip (identity vs gzip-decompressed are identical), and by confirming a Range request still returns 206 with no Content-Encoding. JS 351,835 -> 108,794 bytes (69% smaller); CSS 80% smaller. epub.js is code-split (Reader.svelte) - It was a static import, so ~350 KB was downloaded by everyone on first load even though most never open an ebook. Now loaded on demand. - Main bundle 458 KB -> 112 KB (gzipped 149 KB -> 39.5 KB). - Verified at runtime, which pnpm check cannot: only the main bundle loads initially, the epubjs chunk is fetched when a book is opened, and the reader still renders. Home screen tests (internal/media) - Coverage 23.4% -> 50.2%. (The 17.9% quoted in the brief was a stale reading; the agent measured the real baseline by moving its own files aside.) - Covers the five shelves, counts, Explore's invariants (visibility, no duplicates, every kind represented, varies across calls, wrap-around), the list offset/limit/total contract, MostViewed ordering, and — most importantly — grant scoping, which is the security chokepoint. Fixed a bug the tests found: Suggestions seeded from revoked libraries - topEngagement selected the user's most recent progress row with no library filter, so progress from a library whose grant had been revoked still steered the home screen — a visible item was labelled "More books" based on content the user could no longer read. Returned items were always correctly scoped, so this was an information-flow gap rather than a content leak. - Now scoped to visible libraries, and the test that pinned the old behaviour flipped to assert the correct reason. gofmt/vet clean, 14 packages pass, media tests run 3x for flakiness.
120 lines
5.4 KiB
SQL
120 lines
5.4 KiB
SQL
-- 0009_search_authors.sql — make credited people (authors, artists, directors)
|
|
-- searchable.
|
|
--
|
|
-- 0003 indexed only `title` and `path`, so typing an author's name found
|
|
-- nothing even though `people` and `item_people` already held the credits.
|
|
-- Rather than make the search query join the credit tables — which would mean a
|
|
-- second MATCH against a second index, or an unindexed LIKE over a join — the
|
|
-- names are denormalised into a real column on `media_items`. The existing
|
|
-- external-content FTS5 design then keeps working unchanged in spirit: the
|
|
-- index still stores no text of its own, and the media_items triggers that
|
|
-- already keep it in sync simply gain the new column.
|
|
--
|
|
-- `authors` is maintained by triggers on item_people/people, so a credit change
|
|
-- re-indexes the item no matter which code path writes it — the metadata
|
|
-- enricher today, anything else later.
|
|
|
|
ALTER TABLE media_items ADD COLUMN authors TEXT NOT NULL DEFAULT '';
|
|
|
|
-- Drop the old two-column index and its triggers before rebuilding.
|
|
-- Triggers first: dropping the virtual table does not drop triggers that
|
|
-- reference it, and a write landing between the two drops would fire a trigger
|
|
-- against a table that no longer exists.
|
|
DROP TRIGGER media_items_ai;
|
|
DROP TRIGGER media_items_ad;
|
|
DROP TRIGGER media_items_au;
|
|
DROP TABLE media_search;
|
|
|
|
CREATE VIRTUAL TABLE media_search USING fts5(
|
|
title,
|
|
path,
|
|
authors,
|
|
content='media_items',
|
|
content_rowid='id',
|
|
tokenize='unicode61 remove_diacritics 2'
|
|
);
|
|
|
|
-- Populate the fresh index from current content before the backfill below.
|
|
-- FTS5's external-content 'delete' command assumes the rowid is already in the
|
|
-- index; issuing it for an absent rowid raises "database disk image is
|
|
-- malformed". The backfill UPDATE fires media_items_au (a delete+insert) for
|
|
-- every existing row, so the index must first hold those rows. On a fresh
|
|
-- database this indexes nothing and is a no-op.
|
|
INSERT INTO media_search(media_search) VALUES ('rebuild');
|
|
|
|
-- Recreate the media_items triggers verbatim from 0003, with `authors` added,
|
|
-- so every direct write to an item keeps the index correct.
|
|
CREATE TRIGGER media_items_ai AFTER INSERT ON media_items BEGIN
|
|
INSERT INTO media_search(rowid, title, path, authors)
|
|
VALUES (new.id, new.title, new.path, new.authors);
|
|
END;
|
|
|
|
CREATE TRIGGER media_items_ad AFTER DELETE ON media_items BEGIN
|
|
INSERT INTO media_search(media_search, rowid, title, path, authors)
|
|
VALUES ('delete', old.id, old.title, old.path, old.authors);
|
|
END;
|
|
|
|
CREATE TRIGGER media_items_au AFTER UPDATE ON media_items BEGIN
|
|
INSERT INTO media_search(media_search, rowid, title, path, authors)
|
|
VALUES ('delete', old.id, old.title, old.path, old.authors);
|
|
INSERT INTO media_search(rowid, title, path, authors)
|
|
VALUES (new.id, new.title, new.path, new.authors);
|
|
END;
|
|
|
|
-- Credit triggers. Each recomputes media_items.authors for the affected item.
|
|
-- That UPDATE is itself on media_items, so media_items_au fires and rewrites
|
|
-- the FTS row — no FTS-specific code is needed here. (This is a single level of
|
|
-- trigger nesting, not recursion; and even with PRAGMA recursive_triggers=1 the
|
|
-- chain terminates because the FTS write targets the virtual table, not
|
|
-- media_items.)
|
|
CREATE TRIGGER item_people_ai AFTER INSERT ON item_people BEGIN
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '')
|
|
WHERE id = new.item_id;
|
|
END;
|
|
|
|
CREATE TRIGGER item_people_ad AFTER DELETE ON item_people BEGIN
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '')
|
|
WHERE id = old.item_id;
|
|
END;
|
|
|
|
-- An UPDATE can move a credit between items, so both ends are recomputed.
|
|
CREATE TRIGGER item_people_au AFTER UPDATE ON item_people BEGIN
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '')
|
|
WHERE id = old.item_id;
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '')
|
|
WHERE id = new.item_id;
|
|
END;
|
|
|
|
-- A rename must not leave the old spelling in the index: recompute every item
|
|
-- credited to this person. people.name is UNIQUE COLLATE NOCASE, so renames are
|
|
-- rare, but a stale denormalised column is exactly what a rename would expose.
|
|
CREATE TRIGGER people_au AFTER UPDATE ON people BEGIN
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '')
|
|
WHERE id IN (SELECT item_id FROM item_people WHERE person_id = old.id);
|
|
END;
|
|
|
|
-- Backfill: the triggers above only fire on future writes, so populate every
|
|
-- existing row once. The final rebuild re-derives the whole index from the
|
|
-- content table, which also normalises any transient FTS state left by the
|
|
-- per-row updates above.
|
|
UPDATE media_items SET authors = COALESCE((
|
|
SELECT group_concat(p.name, ' ') FROM item_people ip
|
|
JOIN people p ON p.id = ip.person_id
|
|
WHERE ip.item_id = media_items.id), '');
|
|
|
|
INSERT INTO media_search(media_search) VALUES ('rebuild');
|