Vellum/internal/httpapi/static.go
mithral 09474b0e08 Wave 3: search depth, transfer size, and the Home screen's tests
Authors are searchable (internal/search, migrations/0009)
- The FTS index covered only title and path, so typing an artist or author
  found nothing even though people/item_people already held the credits.
- Rather than a second index, the names are denormalised into a real
  media_items.authors column, so the existing external-content FTS and its
  triggers keep doing the work; item_people/people triggers recompute it, and
  the media_items UPDATE trigger re-indexes automatically.
- Verified: searching "daft punk" now returns Random Access Memories, and so
  does the partial "punk".
- The agent found and worked around a real FTS5 external-content bug: on an
  UPGRADE the backfill UPDATE fires the delete trigger for rowids not yet in
  the fresh index, raising "database disk image is malformed". A rebuild before
  the backfill fixes it. This would have broken existing databases while a
  fresh install looked fine.

gzip for the SPA (internal/httpapi/static.go)
- 456 KB of JS and 31 KB of CSS were served uncompressed on every cold load.
- Scoped to the embedded SPA assets only, never API or media: compressing a
  Range response would break seeking, and wrapping every response would undo
  Wave 2's Flusher/ReaderFrom forwarding.
- Compresses only when the client accepts gzip, the response is a compressible
  type, there is no Range header, and nothing already set Content-Encoding.
  Sets Vary: Accept-Encoding and drops the now-wrong Content-Length.
- Verified by sha256 round-trip (identity vs gzip-decompressed are identical),
  and by confirming a Range request still returns 206 with no Content-Encoding.
  JS 351,835 -> 108,794 bytes (69% smaller); CSS 80% smaller.

epub.js is code-split (Reader.svelte)
- It was a static import, so ~350 KB was downloaded by everyone on first load
  even though most never open an ebook. Now loaded on demand.
- Main bundle 458 KB -> 112 KB (gzipped 149 KB -> 39.5 KB).
- Verified at runtime, which pnpm check cannot: only the main bundle loads
  initially, the epubjs chunk is fetched when a book is opened, and the reader
  still renders.

Home screen tests (internal/media)
- Coverage 23.4% -> 50.2%. (The 17.9% quoted in the brief was a stale reading;
  the agent measured the real baseline by moving its own files aside.)
- Covers the five shelves, counts, Explore's invariants (visibility, no
  duplicates, every kind represented, varies across calls, wrap-around),
  the list offset/limit/total contract, MostViewed ordering, and — most
  importantly — grant scoping, which is the security chokepoint.

Fixed a bug the tests found: Suggestions seeded from revoked libraries
- topEngagement selected the user's most recent progress row with no library
  filter, so progress from a library whose grant had been revoked still steered
  the home screen — a visible item was labelled "More books" based on content
  the user could no longer read. Returned items were always correctly scoped, so
  this was an information-flow gap rather than a content leak.
- Now scoped to visible libraries, and the test that pinned the old behaviour
  flipped to assert the correct reason.

gofmt/vet clean, 14 packages pass, media tests run 3x for flakiness.
2026-10-11 13:46:18 -04:00

251 lines
7.7 KiB
Go

package httpapi
import (
"bufio"
"compress/gzip"
"io"
"io/fs"
"net"
"net/http"
"path"
"strconv"
"strings"
"github.com/mithral/vellum/web"
)
// mountWeb serves the embedded single-page app at the root path.
//
// Unknown non-API paths fall back to index.html so client-side routes work on
// reload. API paths never fall back: a typo in an endpoint returns JSON, not
// an HTML page.
func (s *Server) mountWeb() {
sub, err := fs.Sub(web.Dist, "dist")
if err != nil {
s.mux.HandleFunc("/", s.handleNotFound)
return
}
index, err := fs.ReadFile(sub, "index.html")
if err != nil {
// Frontend not built into this binary. Keep JSON 404s rather than
// pretending the app exists.
s.log.Warn("web UI not embedded; run the frontend build to include it")
s.mux.HandleFunc("/", s.handleNotFound)
return
}
files := http.FileServer(http.FS(sub))
// Only the embedded static assets are wrapped for compression. API
// responses and media are served by other handlers and are never touched.
serveFiles := compressStatic(files.ServeHTTP)
serveShell := compressStatic(func(w http.ResponseWriter, r *http.Request) {
serveIndex(w, index)
})
s.mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if isAPIPath(r.URL.Path) {
s.handleNotFound(w, r)
return
}
if r.URL.Path == "/" || r.URL.Path == "" {
serveShell(w, r)
return
}
name := strings.TrimPrefix(path.Clean(r.URL.Path), "/")
if f, err := sub.Open(name); err == nil {
_ = f.Close()
serveFiles(w, r)
return
}
// SPA route: hand back the shell and let the client router decide.
serveShell(w, r)
})
}
func serveIndex(w http.ResponseWriter, index []byte) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.Header().Set("Cache-Control", "no-cache")
_, _ = w.Write(index)
}
func isAPIPath(p string) bool {
return p == "/healthz" || strings.HasPrefix(p, "/api/")
}
// ── Static-asset gzip ──────────────────────────────────────────────────
//
// The built SPA ships uncompressed: the JS bundle is ~456 KB and the CSS
// ~31 KB on every cold load. compressStatic wraps only the embedded static
// handlers with gzip negotiation; API responses and media are never wrapped.
//
// The narrow scope is deliberate. (a) Media is already compressed. (b) A Range
// response is used for seeking, and re-encoding it would invalidate the byte
// offsets. (c) server.go's statusRecorder was just fixed to forward
// http.Flusher and io.ReaderFrom for streaming; wrapping every response in a
// gzip writer would force all bodies through a compressor and undo that.
// Restricting the wrapper to the static file handler avoids all three.
// compressStatic gzips a static handler's response when the client accepts
// gzip and the response is a compressible text type. Close is called after the
// handler returns so the gzip trailer is always written.
func compressStatic(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
gz := newGzipResponseWriter(w, r)
next(gz, r)
_ = gz.Close()
}
}
// compressibleType reports whether a Content-Type should be gzip-encoded. Only
// text-like types are listed; images, fonts, wasm and other media are already
// compressed, so re-encoding them wastes CPU for no gain.
func compressibleType(contentType string) bool {
if i := strings.IndexByte(contentType, ';'); i >= 0 {
contentType = contentType[:i] // drop "charset=utf-8" and friends
}
switch strings.ToLower(strings.TrimSpace(contentType)) {
case "text/html",
"text/css",
"application/javascript",
"text/javascript",
"application/json",
"image/svg+xml",
"application/manifest+json":
return true
}
return false
}
// acceptsGzip reports whether the client's Accept-Encoding allows gzip. An
// explicit "gzip;q=0" is a refusal, so it does not count.
func acceptsGzip(header string) bool {
for _, part := range strings.Split(header, ",") {
token, params, _ := strings.Cut(part, ";")
if !strings.EqualFold(strings.TrimSpace(token), "gzip") {
continue
}
for _, p := range strings.Split(params, ";") {
if v, ok := strings.CutPrefix(strings.TrimSpace(p), "q="); ok {
if q, err := strconv.ParseFloat(strings.TrimSpace(v), 64); err == nil && q == 0 {
return false
}
}
}
return true
}
return false
}
// gzipResponseWriter wraps a static response so its body can be gzip-encoded.
//
// Like statusRecorder it forwards Flush, Hijack and ReadFrom. While the body is
// being compressed those forwards must route through the compressor (or flush
// it first), or raw bytes would be written under a gzip Content-Encoding.
type gzipResponseWriter struct {
http.ResponseWriter
eligible bool
compress bool
wroteHeader bool
gz *gzip.Writer
}
func newGzipResponseWriter(w http.ResponseWriter, r *http.Request) *gzipResponseWriter {
return &gzipResponseWriter{
ResponseWriter: w,
// A Range request must stay identity so its byte offsets remain valid.
eligible: acceptsGzip(r.Header.Get("Accept-Encoding")) && r.Header.Get("Range") == "",
}
}
// WriteHeader decides compression once the handler has set Content-Type. The
// decision has to happen before the status line is committed.
func (g *gzipResponseWriter) WriteHeader(code int) {
if g.wroteHeader {
return
}
g.wroteHeader = true
h := g.Header()
if g.eligible &&
code == http.StatusOK &&
h.Get("Content-Encoding") == "" &&
h.Get("Content-Length") != "0" &&
compressibleType(h.Get("Content-Type")) {
g.compress = true
h.Set("Content-Encoding", "gzip")
h.Add("Vary", "Accept-Encoding")
// The compressed length is not known until the body is written, and
// the handler's original Content-Length is now wrong.
h.Del("Content-Length")
}
g.ResponseWriter.WriteHeader(code)
}
// writer lazily creates the gzip stream on the first body byte. A HEAD request
// (or a bodyless response) never writes, so no empty gzip stream is emitted and
// net/http has nothing from which to fabricate a misleading Content-Length.
func (g *gzipResponseWriter) writer() *gzip.Writer {
if g.gz == nil {
g.gz = gzip.NewWriter(g.ResponseWriter)
}
return g.gz
}
func (g *gzipResponseWriter) Write(b []byte) (int, error) {
if !g.wroteHeader {
g.WriteHeader(http.StatusOK)
}
if g.compress {
return g.writer().Write(b)
}
return g.ResponseWriter.Write(b)
}
// Flush forwards to the underlying http.Flusher, after flushing any buffered
// compressed bytes so the client sees a complete gzip block.
func (g *gzipResponseWriter) Flush() {
if !g.wroteHeader {
g.WriteHeader(http.StatusOK)
}
if g.compress && g.gz != nil {
_ = g.gz.Flush()
}
if f, ok := g.ResponseWriter.(http.Flusher); ok {
f.Flush()
}
}
// Hijack forwards to the underlying http.Hijacker for protocols that take over
// the connection.
func (g *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
if hj, ok := g.ResponseWriter.(http.Hijacker); ok {
return hj.Hijack()
}
return nil, nil, http.ErrNotSupported
}
// ReadFrom forwards to the underlying io.ReaderFrom so io.Copy keeps the
// sendfile fast path. When compressing, the bytes must pass through the gzip
// writer instead; the returned count is the source byte count.
func (g *gzipResponseWriter) ReadFrom(src io.Reader) (int64, error) {
if !g.wroteHeader {
g.WriteHeader(http.StatusOK)
}
if g.compress {
return io.Copy(g.writer(), src)
}
if rf, ok := g.ResponseWriter.(io.ReaderFrom); ok {
return rf.ReadFrom(src)
}
return io.Copy(g.ResponseWriter, src)
}
// Close flushes the gzip trailer. It must run after the static handler returns,
// or the body will be missing its checksum and the client will reject it.
func (g *gzipResponseWriter) Close() error {
if g.compress && g.gz != nil {
return g.gz.Close()
}
return nil
}