ad3laid3/Vellum
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Waves 4-6: audio depth, ffprobe facts, and direct-play video

Browse source 
Direct play only. The server is a headless GPU-less Xeon: nothing is ever
transcoded or remuxed. A file either plays in the client or it does not.

Measured, not assumed (ffmpeg-generated files with known codecs, played through
Vellum's own stream endpoint in Chrome 155 / Linux):
  mp4  h264+aac   plays        mkv h264+aac  PLAYS   <- container is fine
  mov  h264+aac   plays        webm vp9/opus plays
  mp4  av1+opus   plays        webm vp8/vorbis plays
  mp4  hevc+aac   AUDIO ONLY, videoWidth stays 0, no error
  mkv  hevc+flac  AUDIO ONLY
  avi  mpeg4+mp3  fails outright
All seven audio formats play. Two conclusions drive the design: codec matters
and container barely does (mkv+h264 plays, mp4+HEVC does not), and canPlayType
is unreliable in both directions — it says "no" for mov+h264 (which plays) and
"maybe" for mkv+hevc (whose video never decodes).

Therefore playability is a property of the CLIENT, not the file. The server
never declares a file playable; it reports codec facts and each client decides.
That is what makes adding mpv/libvlc/MDK later a matter of registering an engine
rather than re-architecting.

Wave 4 - audio depth
- New MusicPlayer.svelte: real queue, shuffle that walks a fixed permutation
  (so "previous" means something and nothing repeats early), repeat off/all/one,
  per-track rows, album art, next-track preload.
- AudiobookPlayer: MediaSession (OS media keys + lock screen), playback speed
  that SURVIVES a chapter change and persists, sleep timer with countdown that a
  manual pause cancels, keyboard shortcuts, next-chapter preload.
- Shared mediaSession.ts written by the coordinator before dispatch so both
  players consume one implementation instead of inventing two.

Wave 5 - ffprobe facts
- internal/probe (ffprobe wrapper, optional dependency), migration 0010 adding
  vcodec/acodec/width/height/probed_at to media_files, scanner hook, API fields.
- Probing runs only for NEW or CHANGED files and BEFORE the write transaction
  opens, so it can never hold SQLite's write lock and a rescan never becomes a
  full-library ffprobe sweep. Proven: 797ms probing scan, then 1ms rescan with
  probed_at byte-identical.
- ffprobe skips attached_pic so an mp3 with cover art reports no video codec.
- Fixed a real hole: .avi is in the video registry but had no MIME entry, so it
  was served as application/octet-stream and could never play.

Wave 6 - the video player
- VideoPlayer.svelte with resume, progress + keepalive final flush, MediaSession,
  keyboard shortcuts, multi-file picker, and codec facts shown.
- The honest path is the feature: an HEVC file makes Chrome play audio over a
  blank frame with no error, which is the worst possible outcome. The player
  detects it and says so, naming the codec and pointing at clients that can.

Coordinator fix to Wave 6: the agent's detection hung off `loadedmetadata`, but
an explicit `type` the browser cannot satisfy makes it reject the source during
selection, so NEITHER loadedmetadata NOR error fires — and the player showed a
working-looking 0:00 with a Play button, exactly the silently-broken state the
component exists to prevent. Added a load watchdog: if the element says nothing
within 2.5s, the source was rejected and the honest notice is shown. Verified
per file afterwards (h264/mkv play at 640 wide; hevc and mpeg4 show their
notices).

Verified end to end, not just compiled: 15 packages pass (including the new
internal/probe), gofmt/vet clean, svelte-check 0/0, and every claim above checked
against a running server. Note the demo's own mp3s are 100 KB of random bytes and
cannot decode; the audio players were proven against generated real audio instead.
This commit is contained in:
mithral 2026-10-11 14:14:32 -04:00
parent 09474b0e08
commit 785d3ed5ee
13 changed files with 2246 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

101
internal/library/scan.go
View file

@ -17,6 +17,7 @@ import (
"github.com/mithral/vellum/internal/core"
"github.com/mithral/vellum/internal/jobs"
"github.com/mithral/vellum/internal/natsort"
"github.com/mithral/vellum/internal/probe"
)
const (
@ -130,6 +131,17 @@ func Scan(ctx context.Context, db *sql.DB, lib Library, log *slog.Logger) (ScanR
}
batch := groupKeys[start:end]
// Probe the changed media files in this batch BEFORE opening the write
// transaction, so a slow ffprobe (a stalled mount, a huge file) can
// never hold SQLite's write lock.
//
// The probe map is deliberately built from the same changed-only test
// the transaction uses below. Probing on the unchanged path would turn
// every rescan into a full ffprobe sweep of the entire library; the
// size+mtime skip is what keeps rescans cheap, and the probe must live
// on the changed side of it.
probed := probeBatch(ctx, lib, groups, batch, knownFiles, &res)
err := commitBatch(ctx, db, func(tx *sql.Tx) error {
for _, key := range batch {
g := groups[key]
@ -150,7 +162,8 @@ func Scan(ctx context.Context, db *sql.DB, lib Library, log *slog.Logger) (ScanR
continue
}
if err := upsertFile(ctx, tx, itemID, c); err != nil {
info, wasProbed := probed[c.relPath]
if err := upsertFile(ctx, tx, itemID, c, info, wasProbed, now); err != nil {
res.addError(fmt.Sprintf("%s: %v", c.relPath, err))
continue
}
@ -431,21 +444,97 @@ func upsertItem(ctx context.Context, tx *sql.Tx, lib Library, key, title string)
return id, nil
}
func upsertFile(ctx context.Context, tx *sql.Tx, itemID int64, c candidate) error {
func upsertFile(ctx context.Context, tx *sql.Tx, itemID int64, c candidate, info probe.Info, probed bool, now string) error {
// A non-probed file (an ebook, a photo, or any file on a host without
// ffprobe) must keep whatever codec facts a previous scan recorded, so its
// upsert never mentions those columns at all.
if !probed {
_, err := tx.ExecContext(ctx, `
INSERT INTO media_files (item_id, path, size, mtime, inode, container, missing)
VALUES (?, ?, ?, ?, ?, ?, 0)
ON CONFLICT (path)
DO UPDATE SET item_id = excluded.item_id, size = excluded.size,
mtime = excluded.mtime, inode = excluded.inode,
container = excluded.container, missing = 0`,
itemID, c.relPath, c.size, fmt.Sprint(c.mtime), c.inode, c.ext)
if err != nil {
return fmt.Errorf("upsert file: %w", err)
}
return nil
}
_, err := tx.ExecContext(ctx, `
INSERT INTO media_files (item_id, path, size, mtime, inode, container, missing)
VALUES (?, ?, ?, ?, ?, ?, 0)
INSERT INTO media_files (item_id, path, size, mtime, inode, container, missing,
duration_ms, vcodec, acodec, width, height, probed_at)
VALUES (?, ?, ?, ?, ?, ?, 0, ?, ?, ?, ?, ?, ?)
ON CONFLICT (path)
DO UPDATE SET item_id = excluded.item_id, size = excluded.size,
mtime = excluded.mtime, inode = excluded.inode,
container = excluded.container, missing = 0`,
itemID, c.relPath, c.size, fmt.Sprint(c.mtime), c.inode, c.ext)
container = excluded.container, missing = 0,
duration_ms = excluded.duration_ms, vcodec = excluded.vcodec,
acodec = excluded.acodec, width = excluded.width,
height = excluded.height, probed_at = excluded.probed_at`,
itemID, c.relPath, c.size, fmt.Sprint(c.mtime), c.inode, c.ext,
info.DurationMS, info.VCodec, info.ACodec, info.Width, info.Height, now)
if err != nil {
return fmt.Errorf("upsert file: %w", err)
}
return nil
}
// probeBatch probes every new or changed video/audio file in one batch and
// returns the facts keyed by relative path.
//
// Only files the transaction will treat as changed (a size+mtime mismatch, or a
// row that is absent or flagged missing) are probed; unchanged files are absent
// from the result, so no ffprobe runs and no codec column is rewritten for
// them. Non-media kinds (ebooks, comics, photos, games) are never probed: they
// hold no media streams, and running ffprobe over a library of them would be
// pure waste.
//
// It runs before commitBatch opens the write transaction, so the write lock is
// never held across an ffprobe. A probe failure is recorded through the bounded
// error mechanism and the file is written with empty codec fields rather than
// failing the scan.
func probeBatch(ctx context.Context, lib Library, groups map[string]group, batch []string, known map[string]knownFile, res *ScanResult) map[string]probe.Info {
if !probeable(lib.Kind) || !probe.Available() {
return nil
}
out := make(map[string]probe.Info)
for _, key := range batch {
for _, c := range groups[key].files {
if prev, ok := known[c.relPath]; ok && !prev.missing && prev.size == c.size && prev.mtime == c.mtime {
continue // unchanged: never re-probe
}
abs := filepath.Join(lib.RootPath, filepath.FromSlash(c.relPath))
info, err := probe.File(ctx, abs)
if err != nil {
res.addError(fmt.Sprintf("%s: %v", c.relPath, err))
// Record the attempt with empty fields; the codecs stay empty
// until the file changes again.
out[c.relPath] = probe.Info{}
continue
}
out[c.relPath] = info
}
}
return out
}
// probeable reports whether a library kind holds files worth probing for codec
// facts. Only video and the audio kinds have media streams; ebooks, comics,
// photos, and games are never probed.
func probeable(kind string) bool {
switch kind {
case "video", "audiobooks", "music":
return true
default:
return false
}
}
func markItemMissing(ctx context.Context, tx *sql.Tx, libraryID int64, key, now string) error {
_, err := tx.ExecContext(ctx, `
UPDATE media_items SET missing = 1

31
internal/media/handlers.go
View file

@ -305,19 +305,40 @@ func (h *Handlers) authorized(w http.ResponseWriter, r *http.Request, u auth.Use
return it, true
}
func (h *Handlers) files(r *http.Request, itemID int64) ([]FileEntry, error) {
// fileDTO is one physical file in an item's detail response. It carries the
// ffprobe codec facts alongside the container, because direct play is a
// decision each client makes for itself: the server reports what is in the file
// and never transcodes. The wire shape is frozen by MediaFile in
// web/src/lib/api.ts.
type fileDTO struct {
ID int64 `json:"id"`
Path string `json:"path"`
Size int64 `json:"size"`
Container string `json:"container"`
Missing bool `json:"missing"`
DurationMS int64 `json:"duration_ms"`
VCodec string `json:"vcodec"`
ACodec string `json:"acodec"`
Width int `json:"width"`
Height int `json:"height"`
}
func (h *Handlers) files(r *http.Request, itemID int64) ([]fileDTO, error) {
rows, err := h.db.QueryContext(r.Context(), `
SELECT id, path, size, container, missing FROM media_files WHERE item_id = ? ORDER BY id`, itemID)
SELECT id, path, size, container, missing,
duration_ms, vcodec, acodec, width, height
FROM media_files WHERE item_id = ? ORDER BY id`, itemID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []FileEntry{}
out := []fileDTO{}
for rows.Next() {
var f FileEntry
var f fileDTO
var missing int
if err := rows.Scan(&f.ID, &f.Path, &f.Size, &f.Container, &missing); err != nil {
if err := rows.Scan(&f.ID, &f.Path, &f.Size, &f.Container, &missing,
&f.DurationMS, &f.VCodec, &f.ACodec, &f.Width, &f.Height); err != nil {
return nil, err
}
f.Missing = missing == 1

133
internal/probe/probe.go Normal file
View file

@ -0,0 +1,133 @@
// Package probe reads the container and codec facts out of a media file with
// ffprobe, so a client can decide for itself whether it can direct-play the
// file.
//
// Vellum never transcodes. The server has no spare CPU, and measured against
// real files, playability turned out to be a property of the CLIENT, not the
// file: Chrome on Linux plays an MKV carrying H.264 perfectly well, yet cannot
// decode an MP4 carrying HEVC at all — it plays the audio track and reports
// videoWidth 0. So the server's only job is to expose what is actually inside
// the file; each client decides what it can handle.
//
// ffprobe is OPTIONAL. Vellum runs without it, just with less information:
// Available reports whether it is on PATH, and callers skip probing when it is
// absent rather than treating that as an error.
package probe
import (
"bytes"
"context"
"encoding/json"
"fmt"
"os/exec"
"strconv"
"strings"
"time"
)
// timeout bounds a single ffprobe invocation. A file on a stalled network mount
// must never hang a scan, so ffprobe is killed after this and the error is
// recorded like any other probe failure.
const timeout = 15 * time.Second
// Info is the subset of ffprobe facts Vellum stores per file.
//
// Zero values are meaningful: an empty VCodec means there is no video track
// (audio, or a container whose only "video" stream is embedded cover art), and
// an empty ACodec means there is no audio track. DurationMS and the dimensions
// stay zero when ffprobe could not determine them.
type Info struct {
DurationMS int64
VCodec string
ACodec string
Width int
Height int
}
// Available reports whether ffprobe is on PATH.
//
// Callers use it to skip probing entirely, so Vellum still works on a host with
// no ffmpeg installed — the library just carries no codec facts, and every
// client is left to try the file and find out.
func Available() bool {
_, err := exec.LookPath("ffprobe")
return err == nil
}
// File probes one file and returns its codec facts. The returned context
// deadline is capped at timeout regardless of the caller's deadline.
func File(ctx context.Context, path string) (Info, error) {
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
cmd := exec.CommandContext(ctx, "ffprobe",
"-v", "error",
"-print_format", "json",
"-show_format",
"-show_streams",
path,
)
var stdout, stderr bytes.Buffer
cmd.Stdout = &stdout
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
if ctx.Err() != nil {
return Info{}, fmt.Errorf("probe %s: %w", path, ctx.Err())
}
msg := strings.TrimSpace(stderr.String())
if msg == "" {
msg = err.Error()
}
return Info{}, fmt.Errorf("probe %s: %s", path, msg)
}
var out ffprobeOutput
if err := json.Unmarshal(stdout.Bytes(), &out); err != nil {
return Info{}, fmt.Errorf("probe %s: parse ffprobe output: %w", path, err)
}
info := Info{}
for _, s := range out.Streams {
switch s.CodecType {
case "video":
// Embedded cover art (an mp3's album image) is reported as a video
// stream. It is not a playable video track, so ignore it and leave
// VCodec empty for audio files.
if s.Disposition.AttachedPic != 0 || info.VCodec != "" {
continue
}
info.VCodec = s.CodecName
info.Width = s.Width
info.Height = s.Height
case "audio":
if info.ACodec == "" {
info.ACodec = s.CodecName
}
}
}
// ffprobe reports the container duration in seconds as a decimal string.
// It is absent for some live/streamed inputs; leave DurationMS zero then.
if d, err := strconv.ParseFloat(out.Format.Duration, 64); err == nil && d > 0 {
info.DurationMS = int64(d * 1000)
}
return info, nil
}
// ffprobeOutput mirrors only the JSON fields Vellum reads from
// `ffprobe -print_format json -show_format -show_streams`.
type ffprobeOutput struct {
Format struct {
Duration string `json:"duration"`
} `json:"format"`
Streams []struct {
CodecType string `json:"codec_type"`
CodecName string `json:"codec_name"`
Width int `json:"width"`
Height int `json:"height"`
Disposition struct {
AttachedPic int `json:"attached_pic"`
} `json:"disposition"`
} `json:"streams"`
}

96
internal/probe/probe_test.go Normal file
View file

@ -0,0 +1,96 @@
package probe
import (
"context"
"os/exec"
"path/filepath"
"testing"
)
// TestAvailable never skips: the answer is a plain bool and must not panic on a
// host without ffprobe.
func TestAvailable(t *testing.T) {
_ = Available()
}
// TestFile probes a fixture generated on the fly. It skips cleanly when ffprobe
// or ffmpeg is missing, so the suite passes on a machine with neither.
func TestFile(t *testing.T) {
if !Available() {
t.Skip("ffprobe not on PATH; skipping probe test")
}
ffmpeg, err := exec.LookPath("ffmpeg")
if err != nil {
t.Skip("ffmpeg not on PATH; cannot generate a fixture")
}
path := filepath.Join(t.TempDir(), "clip.mp4")
// One second of H.264 video and AAC audio in an MP4.
gen := exec.Command(ffmpeg, "-v", "error", "-y",
"-f", "lavfi", "-i", "testsrc=size=64x48:rate=10:duration=1",
"-f", "lavfi", "-i", "sine=frequency=440:duration=1",
"-c:v", "libx264", "-c:a", "aac", "-shortest", path,
)
if out, err := gen.CombinedOutput(); err != nil {
t.Skipf("ffmpeg cannot produce a fixture here: %v\n%s", err, out)
}
info, err := File(context.Background(), path)
if err != nil {
t.Fatalf("File: %v", err)
}
if info.VCodec != "h264" {
t.Errorf("VCodec = %q, want h264", info.VCodec)
}
if info.ACodec != "aac" {
t.Errorf("ACodec = %q, want aac", info.ACodec)
}
if info.Width != 64 || info.Height != 48 {
t.Errorf("dimensions = %dx%d, want 64x48", info.Width, info.Height)
}
if info.DurationMS < 500 || info.DurationMS > 2000 {
t.Errorf("DurationMS = %d, want ~1000", info.DurationMS)
}
}
// TestFileAudioOnly confirms an audio file reports no video codec, so a client
// does not mistake an mp3 for a video with a missing track.
func TestFileAudioOnly(t *testing.T) {
if !Available() {
t.Skip("ffprobe not on PATH; skipping probe test")
}
ffmpeg, err := exec.LookPath("ffmpeg")
if err != nil {
t.Skip("ffmpeg not on PATH; cannot generate a fixture")
}
path := filepath.Join(t.TempDir(), "tone.m4a")
gen := exec.Command(ffmpeg, "-v", "error", "-y",
"-f", "lavfi", "-i", "sine=frequency=440:duration=1",
"-c:a", "aac", path,
)
if out, err := gen.CombinedOutput(); err != nil {
t.Skipf("ffmpeg cannot produce a fixture here: %v\n%s", err, out)
}
info, err := File(context.Background(), path)
if err != nil {
t.Fatalf("File: %v", err)
}
if info.VCodec != "" {
t.Errorf("VCodec = %q, want empty for audio-only", info.VCodec)
}
if info.ACodec != "aac" {
t.Errorf("ACodec = %q, want aac", info.ACodec)
}
}
// TestFileMissing confirms a missing file is an error, not a silent zero Info.
func TestFileMissing(t *testing.T) {
if !Available() {
t.Skip("ffprobe not on PATH; skipping probe test")
}
if _, err := File(context.Background(), filepath.Join(t.TempDir(), "nope.mp4")); err == nil {
t.Fatal("File on a missing path: want error, got nil")
}
}

6
internal/stream/stream.go
View file

@ -38,11 +38,17 @@ var contentTypes = map[string]string{
".ogg": "audio/ogg",
".wav": "audio/wav",
// Video.
//
// This table and the library extension registry (internal/library/registry.go)
// must agree: an extension in the video registry with no MIME entry here is
// served as application/octet-stream, which browsers download instead of
// playing. .avi was in the registry but missing here.
".mp4": "video/mp4",
".m4v": "video/mp4",
".mkv": "video/x-matroska",
".webm": "video/webm",
".mov": "video/quicktime",
".avi": "video/x-msvideo",
// Images (comic pages, covers, photos).
".jpg": "image/jpeg",
".jpeg": "image/jpeg",

28
migrations/0010_media_probe.sql Normal file
View file

@ -0,0 +1,28 @@
-- 0010_media_probe.sql — record the codec facts a client needs to choose direct
-- play.
--
-- Vellum does not transcode. The server has no spare CPU, and measured against
-- real files, playability turned out to be a property of the CLIENT, not the
-- file: Chrome on Linux plays an MKV carrying H.264 perfectly well, yet cannot
-- decode an MP4 carrying HEVC at all — it plays the audio track and reports
-- videoWidth 0. So the server's job is only to expose what is actually inside
-- each file and let every client decide for itself. These columns are that
-- exposure; without them a client has nothing to base the decision on.
--
-- The values come from ffprobe at scan time. They are empty/zero until the
-- scanner probes the file, and stay that way when ffprobe is not installed:
-- Vellum still works without it, just with less information. duration_ms
-- already exists on media_files from 0001, so it is reused rather than re-added.
-- ffprobe codec name, e.g. "h264", "hevc", "vp9", "av1". "" when there is no
-- video track (audio, or embedded cover art only).
ALTER TABLE media_files ADD COLUMN vcodec TEXT NOT NULL DEFAULT '';
-- ffprobe codec name, e.g. "aac", "opus", "flac", "mp3". "" when there is no
-- audio track.
ALTER TABLE media_files ADD COLUMN acodec TEXT NOT NULL DEFAULT '';
-- Pixel dimensions of the video track; 0 when there is none.
ALTER TABLE media_files ADD COLUMN width INTEGER NOT NULL DEFAULT 0;
ALTER TABLE media_files ADD COLUMN height INTEGER NOT NULL DEFAULT 0;
-- When the codec facts above were last written, so a rescan can be proven not
-- to have re-probed an unchanged file. '' means never probed.
ALTER TABLE media_files ADD COLUMN probed_at TEXT NOT NULL DEFAULT '';

23
web/src/lib/api.ts
View file

@ -269,9 +269,30 @@ export interface Home {
total_items: number
}
// MediaFile carries the ffprobe facts about one file on disk.
//
// The codec fields are what make "direct play only" workable: playability is a
// property of the CLIENT, not the file, so the server reports what is in the
// file and each client decides for itself. An empty vcodec means there is no
// video track (audio, or a comic).
export interface MediaFile {
id: number
path: string
size: number
container: string
missing: boolean
duration_ms: number
/** ffprobe codec name, e.g. "h264", "hevc", "vp9", "av1". "" when absent. */
vcodec: string
/** ffprobe codec name, e.g. "aac", "opus", "flac", "mp3". "" when absent. */
acodec: string
width: number
height: number
}
export interface ItemDetail {
item: Item
files: { id: number; path: string; size: number; container: string; missing: boolean }[]
files: MediaFile[]
has_cover: boolean
authors: string[] | null
progress?: Progress

279
web/src/lib/components/AudiobookPlayer.svelte
View file

@ -1,11 +1,35 @@
<script lang="ts">
import { onMount, onDestroy } from 'svelte'
import { api, ApiError, type Item } from '../api'
import {
setNowPlaying,
setHandlers,
setPosition,
setPlaybackState,
clear as clearMediaSession,
} from '../mediaSession'
let { itemId }: { itemId: number } = $props()
// Playback speed steps. 1 is the default; the choice is per-device and
// survives both chapter changes and opening a different book.
const SPEEDS = [0.75, 1, 1.25, 1.5, 1.75, 2] as const
type Speed = (typeof SPEEDS)[number]
const SPEED_KEY = 'vellum.audiobook.speed'
// Seek step for the keyboard shortcuts and the OS seek actions, in seconds.
// 15s is the audiobook convention (a sentence or two), not the 10s of music.
const SEEK_STEP = 15
const SLEEP_CHOICES = [5, 15, 30, 45, 60] as const
type SleepMode = 'off' | 'chapter' | number
let audio = $state<HTMLAudioElement>()
// Hidden element that warms the next chapter's metadata so the hand-off on
// 'ended' is as close to gapless as the platform allows.
let preloader = $state<HTMLAudioElement>()
let item = $state<Item | null>(null)
let authors = $state<string[] | null>(null)
let tracks = $state<{ id: number; path: string; size: number }[]>([])
let index = $state(0)
let loading = $state(true)
@ -15,19 +39,29 @@
let currentTime = $state(0)
let duration = $state(0)
let speed = $state<Speed>(readSpeed())
let sleepMode = $state<SleepMode>('off')
let sleepRemaining = $state(0)
let sleepInterval: number | undefined
let saveTimer: number | undefined
let resumeAt = 0
let pendingPlay = false
let finalFlushed = false
// Set when the user pauses on purpose (button, keyboard or OS control) so a
// running sleep timer can be cancelled without reacting to our own pause.
let manualPause = false
onMount(async () => {
// Persist the final position even if the page is torn down: keepalive lets a
// PUT outlive the document, which sendBeacon (POST-only) cannot.
window.addEventListener('pagehide', finalFlush)
document.addEventListener('visibilitychange', onVisibilityChange)
window.addEventListener('keydown', onKeydown)
try {
const detail = await api.item(itemId)
item = detail.item
authors = detail.authors
tracks = detail.files ?? []
if (tracks.length === 0) {
error = 'This item has no audio tracks.'
@ -41,6 +75,8 @@
resumeAt = detail.progress.position / 1000
}
}
publishNowPlaying()
registerHandlers()
} catch (err) {
error = err instanceof ApiError ? err.message : 'could not open this audiobook'
} finally {
@ -51,6 +87,13 @@
onDestroy(() => {
window.removeEventListener('pagehide', finalFlush)
document.removeEventListener('visibilitychange', onVisibilityChange)
window.removeEventListener('keydown', onKeydown)
clearSleep()
clearMediaSession()
if (preloader) {
preloader.removeAttribute('src')
preloader.load()
}
finalFlush()
})
@ -59,9 +102,45 @@
else finalFlushed = false
}
// ── MediaSession ─────────────────────────────────────────────────────
// The OS media keys, lock screen and browser media popup read from here; the
// helper guards every call so an unsupported browser never breaks playback.
function publishNowPlaying() {
setNowPlaying({
title: item?.title ?? 'Audiobook',
artist: authors?.length ? authors.join(', ') : undefined,
album: 'Audiobook',
artwork: api.coverUrl(itemId),
duration,
})
}
function registerHandlers() {
setHandlers({
play: () => {
if (audio) void audio.play().catch(() => {})
},
pause: () => pauseManual(),
next: () => goToTrack(index + 1),
previous: () => goToTrack(index - 1),
seekBackward: (d) => seekBy(-(d?.seekOffset ?? SEEK_STEP)),
seekForward: (d) => seekBy(d?.seekOffset ?? SEEK_STEP),
seekTo: (d) => {
if (audio && d?.seekTime != null) {
audio.currentTime = d.seekTime
currentTime = d.seekTime
}
},
})
}
function onLoadedMetadata() {
if (!audio) return
duration = audio.duration || 0
// Re-apply the chosen speed to the freshly-loaded source, otherwise the
// element silently resets to 1x on every chapter change.
audio.playbackRate = speed
if (resumeAt > 0 && resumeAt < duration) {
audio.currentTime = resumeAt
resumeAt = 0
@ -71,25 +150,42 @@
pendingPlay = false
void audio.play().catch(() => {})
}
publishNowPlaying()
setPosition(audio.currentTime, duration, speed)
}
function onTimeUpdate() {
if (!audio) return
currentTime = audio.currentTime
setPosition(currentTime, duration, speed)
scheduleSave()
}
function toggle() {
if (!audio) return
if (audio.paused) void audio.play()
else audio.pause()
else pauseManual()
}
function pauseManual() {
if (!audio || audio.paused) return
manualPause = true
audio.pause()
}
function onPlay() {
playing = true
setPlaybackState('playing')
}
function onPause() {
playing = false
setPlaybackState('paused')
// A deliberate pause cancels a pending sleep timer so it cannot fire later
// against a session the user resumed on their own.
if (manualPause) {
manualPause = false
if (sleepMode !== 'off') clearSleep()
}
}
function goToTrack(i: number) {
@ -107,10 +203,123 @@
}
function onEnded() {
// "End of chapter" sleep: stop here instead of rolling into the next file.
if (sleepMode === 'chapter') {
clearSleep()
if (audio) audio.pause()
return
}
if (index + 1 < tracks.length) goToTrack(index + 1)
else flush(true)
}
// ── Playback speed ───────────────────────────────────────────────────
function readSpeed(): Speed {
try {
const raw = localStorage.getItem(SPEED_KEY)
const n = raw ? Number(raw) : NaN
if ((SPEEDS as readonly number[]).includes(n)) return n as Speed
} catch {
// Private browsing or no storage: fall back to 1x for this session.
}
return 1
}
function cycleSpeed() {
const i = SPEEDS.indexOf(speed)
speed = SPEEDS[(i + 1) % SPEEDS.length]
try {
localStorage.setItem(SPEED_KEY, String(speed))
} catch {
// Keep the change for this session even if it cannot be persisted.
}
if (audio) audio.playbackRate = speed
}
// ── Sleep timer ──────────────────────────────────────────────────────
function setSleep(mode: SleepMode) {
clearSleep()
if (mode === 'off') return
sleepMode = mode
if (mode === 'chapter') return
sleepRemaining = mode * 60
sleepInterval = window.setInterval(tickSleep, 1000)
}
function tickSleep() {
sleepRemaining -= 1
if (sleepRemaining > 0) return
const wasPlaying = !!audio && !audio.paused
clearSleep()
if (wasPlaying) audio!.pause()
}
function clearSleep() {
if (sleepInterval !== undefined) {
clearInterval(sleepInterval)
sleepInterval = undefined
}
sleepMode = 'off'
sleepRemaining = 0
}
function onSleepChange(event: Event) {
const value = (event.currentTarget as HTMLSelectElement).value
if (value === 'off') setSleep('off')
else if (value === 'chapter') setSleep('chapter')
else setSleep(Number(value))
}
// ── Keyboard ─────────────────────────────────────────────────────────
// Space = play/pause, ←/→ = seek ∓15s, , / . = previous / next track.
// Ignored while typing so the shortcuts never eat a search or form field.
function onKeydown(event: KeyboardEvent) {
const target = event.target as HTMLElement | null
if (
target &&
(target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.tagName === 'SELECT' ||
target.isContentEditable)
) {
return
}
switch (event.key) {
case ' ':
case 'Spacebar':
event.preventDefault()
toggle()
break
case 'ArrowLeft':
event.preventDefault()
seekBy(-SEEK_STEP)
break
case 'ArrowRight':
event.preventDefault()
seekBy(SEEK_STEP)
break
case ',':
event.preventDefault()
goToTrack(index - 1)
break
case '.':
event.preventDefault()
goToTrack(index + 1)
break
}
}
function seekBy(delta: number) {
if (!audio) return
const limit = isFinite(audio.duration) && audio.duration > 0 ? audio.duration : Infinity
const next = Math.min(Math.max(audio.currentTime + delta, 0), limit)
audio.currentTime = next
currentTime = next
}
function scheduleSave() {
if (saveTimer) clearTimeout(saveTimer)
saveTimer = window.setTimeout(() => flush(), 2000)
@ -204,6 +413,17 @@
onended={onEnded}
></audio>
{#if index + 1 < tracks.length}
<audio
bind:this={preloader}
class="preload"
src={api.fileUrlById(itemId, tracks[index + 1].id)}
preload="metadata"
aria-hidden="true"
tabindex="-1"
></audio>
{/if}
<input
class="seek"
type="range"
@ -228,8 +448,35 @@
onclick={() => goToTrack(index + 1)}
disabled={index >= tracks.length - 1}>⏭</button
>
<button class="btn btn-ghost speed" onclick={cycleSpeed} title="Playback speed — saved across chapters and books"
>{speed}×</button
>
</div>
<div class="sleep">
<label class="muted" for="sleep-timer">Sleep timer</label>
<select
id="sleep-timer"
value={sleepMode === 'off' ? 'off' : String(sleepMode)}
onchange={onSleepChange}
>
<option value="off">Off</option>
{#each SLEEP_CHOICES as minutes (minutes)}
<option value={String(minutes)}>{minutes} min</option>
{/each}
<option value="chapter">End of chapter</option>
</select>
{#if sleepMode !== 'off'}
<span class="countdown mono" aria-live="polite">
{sleepMode === 'chapter' ? 'at chapter end' : `pauses in ${fmt(sleepRemaining)}`}
</span>
{/if}
</div>
<p class="hint muted">
Space play/pause · ←/→ seek {SEEK_STEP}s · , / . previous / next
</p>
{#if tracks.length > 1}
<ol class="tracks">
{#each tracks as t, i (t.id)}
@ -276,6 +523,14 @@
display: flex;
flex-direction: column;
}
/* Warm the next chapter without rendering a second control surface. */
.preload {
position: absolute;
width: 0;
height: 0;
opacity: 0;
pointer-events: none;
}
.seek {
width: 100%;
accent-color: var(--accent);
@ -295,6 +550,28 @@
min-width: 96px;
justify-content: center;
}
.speed {
min-width: 52px;
justify-content: center;
font-variant-numeric: tabular-nums;
}
.sleep {
display: flex;
align-items: center;
gap: var(--space-2);
font-size: .857rem;
}
.sleep select {
padding: 4px var(--space-2);
}
.countdown {
color: var(--accent);
}
.hint {
margin: 0;
font-size: .786rem;
text-align: center;
}
.tracks {
list-style: none;
margin: var(--space-2) 0 0;

580
web/src/lib/components/MusicPlayer.svelte Normal file
View file

@ -0,0 +1,580 @@
<script lang="ts">
import { onMount, onDestroy } from 'svelte'
import { api, ApiError, type Item } from '../api'
import * as mediaSession from '../mediaSession'
let { itemId }: { itemId: number } = $props()
type Track = { id: number; path: string; size: number }
type RepeatMode = 'off' | 'all' | 'one'
let audio = $state<HTMLAudioElement>()
let preload = $state<HTMLAudioElement>()
let item = $state<Item | null>(null)
let authors = $state<string[]>([])
let tracks = $state<Track[]>([])
let loading = $state(true)
let error = $state<string | null>(null)
// Play order is a permutation of track indices, walked by `pos`. Shuffling
// builds that permutation once so "previous" is meaningful and no track
// repeats until the queue is exhausted — rather than picking at random per
// step.
let order = $state<number[]>([])
let pos = $state(0)
let shuffle = $state(false)
let repeat = $state<RepeatMode>('off')
let playing = $state(false)
let currentTime = $state(0)
let duration = $state(0)
// Learned from loadedmetadata as tracks are visited (and from the preloader
// for the upcoming track), so durations fill in as playback moves along.
let durations = $state<Record<number, number>>({})
let saveTimer: number | undefined
let pendingPlay = false
let finalFlushed = false
const currentTrack = $derived(tracks[order[pos]] ?? null)
const currentIndex = $derived(order[pos] ?? -1)
// What to warm up in the hidden element. Wrapping honours repeat so the
// metadata for the track that will actually follow is already in memory.
const nextTrack = $derived.by(() => {
if (tracks.length === 0) return null
if (repeat === 'one') return tracks[order[pos]] ?? null
const n = order[pos + 1]
if (n !== undefined) return tracks[n] ?? null
if (repeat === 'all') return tracks[order[0]] ?? null
return null
})
onMount(async () => {
// Persist the final position even if the page is torn down: keepalive lets a
// PUT outlive the document, which sendBeacon (POST-only) cannot.
window.addEventListener('pagehide', finalFlush)
document.addEventListener('visibilitychange', onVisibilityChange)
// OS/lock-screen controls. Registered up front so media keys work as soon
// as playback starts.
mediaSession.setHandlers({
play: () => {
if (audio) void audio.play().catch(() => {})
},
pause: () => audio?.pause(),
next: () => next(),
previous: () => previous(),
seekTo: (details) => {
if (audio && typeof details?.seekTime === 'number') {
audio.currentTime = details.seekTime
currentTime = details.seekTime
}
},
seekBackward: (details) => seekBy(-(details?.seekOffset ?? 10)),
seekForward: (details) => seekBy(details?.seekOffset ?? 10),
})
try {
const detail = await api.item(itemId)
item = detail.item
authors = detail.authors ?? []
tracks = detail.files ?? []
if (tracks.length === 0) {
error = 'This album has no tracks.'
return
}
order = tracks.map((_, i) => i)
} catch (err) {
error = err instanceof ApiError ? err.message : 'could not open this album'
} finally {
loading = false
}
})
onDestroy(() => {
window.removeEventListener('pagehide', finalFlush)
document.removeEventListener('visibilitychange', onVisibilityChange)
finalFlush()
audio?.pause()
// Release the preloader's decoder and network handle.
if (preload) {
preload.removeAttribute('src')
preload.load()
}
mediaSession.clear()
})
// Keep the OS metadata in step with the current track. Runs once per track
// change; the helper is guarded, so an unsupported browser is harmless.
$effect(() => {
const t = currentTrack
if (!t) return
mediaSession.setNowPlaying({
title: cleanTitle(t.path),
album: item?.title ?? '',
artist: authors.join(', '),
artwork: api.coverUrl(itemId),
})
})
function onVisibilityChange() {
if (document.visibilityState === 'hidden') finalFlush()
else finalFlushed = false
}
function onLoadedMetadata() {
if (!audio) return
duration = audio.duration || 0
const i = currentIndex
if (i >= 0 && isFinite(duration) && duration > 0) durations[tracks[i].id] = duration
// The new source is loaded: autoplay a track we just switched to.
if (pendingPlay) {
pendingPlay = false
void audio.play().catch(() => {})
}
}
function onPreloadMetadata(event: Event) {
const el = event.currentTarget as HTMLAudioElement
const id = Number(el.dataset.track)
const d = el.duration
if (id && isFinite(d) && d > 0) durations[id] = d
}
function onTimeUpdate() {
if (!audio) return
currentTime = audio.currentTime
mediaSession.setPosition(currentTime, audio.duration || duration)
scheduleSave()
}
function toggle() {
if (!audio) return
if (audio.paused) void audio.play()
else audio.pause()
}
function onPlay() {
playing = true
mediaSession.setPlaybackState('playing')
}
function onPause() {
playing = false
mediaSession.setPlaybackState('paused')
}
// Build a fresh walk with `first` at the front and everything else shuffled.
function shuffledOrder(first: number): number[] {
const rest = tracks.map((_, i) => i).filter((i) => i !== first)
for (let i = rest.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1))
;[rest[i], rest[j]] = [rest[j], rest[i]]
}
return [first, ...rest]
}
function toggleShuffle() {
const cur = currentIndex
shuffle = !shuffle
if (shuffle) {
order = shuffledOrder(cur < 0 ? 0 : cur)
pos = 0
} else {
order = tracks.map((_, i) => i)
pos = cur < 0 ? 0 : cur
}
}
function cycleRepeat() {
repeat = repeat === 'off' ? 'all' : repeat === 'all' ? 'one' : 'off'
}
function goTo(i: number) {
if (i < 0 || i >= tracks.length) return
flush()
if (i === currentIndex) {
if (audio) void audio.play().catch(() => {})
return
}
// Clicking a row plays it and the queue continues from there: list order
// when unshuffled, or a fresh shuffled walk when shuffle is on.
if (shuffle) {
order = shuffledOrder(i)
pos = 0
} else {
pos = i
}
// Autoplay once Svelte has actually swapped the src and it has loaded;
// playing from a microtask can race the DOM update and replay the old track.
pendingPlay = true
}
function step(delta: number) {
if (order.length === 0) return
const n = pos + delta
let target = -1
if (n >= 0 && n < order.length) target = n
else if (repeat === 'all') target = delta > 0 ? 0 : order.length - 1
if (target < 0) return
flush()
// A single-track queue (or a wrap back onto the current track) does not
// change the src, so no loadedmetadata will fire to autoplay it: restart
// in place instead.
if (target === pos) {
if (audio) {
audio.currentTime = 0
void audio.play().catch(() => {})
}
return
}
pos = target
pendingPlay = true
}
function next() {
step(1)
}
function previous() {
// Restart the current track first, the way a hardware player does.
if (audio && audio.currentTime > 3) {
audio.currentTime = 0
currentTime = 0
return
}
step(-1)
}
function onEnded() {
if (repeat === 'one') {
if (audio) {
audio.currentTime = 0
void audio.play().catch(() => {})
}
return
}
if (pos + 1 < order.length || repeat === 'all') step(1)
else flush(true)
}
function scheduleSave() {
if (saveTimer) clearTimeout(saveTimer)
saveTimer = window.setTimeout(() => flush(), 2000)
}
// Progress is milliseconds and is keyed by file, so the album's last-played
// track and offset round-trip.
function progressBody(finished: boolean) {
if (!audio || !currentTrack) return null
const at = Math.floor(audio.currentTime * 1000)
const dur = Math.floor((audio.duration || 0) * 1000)
return {
file_id: currentTrack.id,
position: at,
duration: dur,
percent: dur > 0 ? at / dur : 0,
finished,
}
}
function flush(finished = false) {
const body = progressBody(finished)
if (!body) return
void api.putProgress(itemId, body).catch(() => {})
}
// Last-chance save on teardown. Uses fetch with keepalive so the PUT survives
// the document being unloaded, and latches so the overlapping pagehide /
// visibilitychange / onDestroy events only fire one request. Progress is
// PUT-only, so sendBeacon cannot be used.
function finalFlush() {
if (finalFlushed) return
const body = progressBody(false)
if (!body) return
finalFlushed = true
if (saveTimer) {
clearTimeout(saveTimer)
saveTimer = undefined
}
void fetch(`/api/v1/items/${itemId}/progress`, {
method: 'PUT',
keepalive: true,
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
}).catch(() => {})
}
function seek(event: Event) {
const value = Number((event.currentTarget as HTMLInputElement).value)
if (audio) audio.currentTime = value
currentTime = value
}
function seekBy(delta: number) {
if (!audio) return
const t = Math.min(Math.max(audio.currentTime + delta, 0), audio.duration || 0)
audio.currentTime = t
currentTime = t
}
// Mirrors the server's CleanTitle: strip the extension, turn separators into
// spaces, collapse runs.
function cleanTitle(path: string): string {
const base = path.split('/').pop() ?? path
const dot = base.lastIndexOf('.')
const stem = dot > 0 ? base.slice(0, dot) : base
return stem.replace(/[_.]+/g, ' ').split(/\s+/).filter(Boolean).join(' ')
}
function fmt(seconds: number): string {
if (!isFinite(seconds) || seconds < 0) seconds = 0
const h = Math.floor(seconds / 3600)
const m = Math.floor((seconds % 3600) / 60)
const s = Math.floor(seconds % 60)
const mm = String(m).padStart(2, '0')
const ss = String(s).padStart(2, '0')
return h > 0 ? `${h}:${mm}:${ss}` : `${m}:${ss}`
}
</script>
{#if error}
<div class="alert">{error}</div>
{:else if loading}
<div class="center"><div class="spinner"></div></div>
{:else}
<div class="player card">
<div class="head">
<img
class="cover"
src={api.coverUrl(itemId)}
alt=""
onerror={(e) => (e.currentTarget as HTMLImageElement).style.display = 'none'}
/>
<div class="meta">
<strong>{item?.title}</strong>
{#if authors.length}<span class="muted">{authors.join(', ')}</span>{/if}
<span class="muted">{[tracks.length, tracks.length === 1 ? 'track' : 'tracks'].join(' ')}</span>
</div>
</div>
<audio
bind:this={audio}
src={currentTrack ? api.fileUrlById(itemId, currentTrack.id) : undefined}
preload="metadata"
onloadedmetadata={onLoadedMetadata}
ontimeupdate={onTimeUpdate}
onplay={onPlay}
onpause={onPause}
onended={onEnded}
></audio>
{#if nextTrack}
<audio
bind:this={preload}
src={api.fileUrlById(itemId, nextTrack.id)}
data-track={nextTrack.id}
preload="metadata"
onloadedmetadata={onPreloadMetadata}
hidden
></audio>
{/if}
<div class="now">
<span class="mono idx">{currentIndex >= 0 ? String(currentIndex + 1).padStart(2, '0') : '--'}</span>
<strong class="now-title">{currentTrack ? cleanTitle(currentTrack.path) : ''}</strong>
<span class="queue mono muted">{order.length ? `${pos + 1}/${order.length}` : ''}</span>
</div>
<input
class="seek"
type="range"
min="0"
max={duration || 0}
step="1"
value={currentTime}
oninput={seek}
aria-label="Seek"
/>
<div class="times mono">
<span>{fmt(currentTime)}</span>
<span class="muted">{fmt(duration)}</span>
</div>
<div class="controls">
<button
class="btn"
class:btn-primary={shuffle}
class:btn-ghost={!shuffle}
onclick={toggleShuffle}
aria-pressed={shuffle}
title="Shuffle">Shuffle</button
>
<button class="btn btn-ghost" onclick={() => previous()} title="Previous">⏮</button>
<button class="btn btn-primary play" onclick={toggle}>{playing ? 'Pause' : 'Play'}</button>
<button class="btn btn-ghost" onclick={() => next()} title="Next">⏭</button>
<button
class="btn"
class:btn-primary={repeat !== 'off'}
class:btn-ghost={repeat === 'off'}
onclick={cycleRepeat}
title="Repeat"
>
{repeat === 'off' ? 'Repeat off' : repeat === 'all' ? 'Repeat all' : 'Repeat one'}
</button>
</div>
<ol class="tracks">
{#each tracks as t, i (t.id)}
<li>
<button class="track" class:current={i === currentIndex} onclick={() => goTo(i)}>
<span class="mono num">{String(i + 1).padStart(2, '0')}</span>
<span class="ind" aria-hidden="true">
{#if i === currentIndex}
{playing ? '▶' : '❚❚'}
{/if}
</span>
<span class="name">{cleanTitle(t.path)}</span>
<span class="dur mono muted">{durations[t.id] ? fmt(durations[t.id]) : ''}</span>
</button>
</li>
{/each}
</ol>
</div>
{/if}
<style>
.center {
display: grid;
place-items: center;
padding: var(--space-6);
}
.player {
max-width: 560px;
margin: 0 auto;
padding: var(--space-5);
display: flex;
flex-direction: column;
gap: var(--space-3);
}
.head {
display: flex;
align-items: center;
gap: var(--space-4);
}
.cover {
width: 72px;
height: 72px;
object-fit: cover;
border-radius: var(--radius);
border: 1px solid var(--border);
background: var(--bg);
}
.meta {
display: flex;
flex-direction: column;
min-width: 0;
}
.meta strong {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.meta .muted {
font-size: var(--fs-sm);
}
.now {
display: flex;
align-items: baseline;
gap: var(--space-3);
min-width: 0;
}
.idx {
color: var(--faint);
}
.now-title {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-size: var(--fs-md);
}
.queue {
flex: none;
}
.seek {
width: 100%;
accent-color: var(--accent);
}
.times {
display: flex;
justify-content: space-between;
font-size: var(--fs-sm);
}
.controls {
display: flex;
align-items: center;
justify-content: center;
gap: var(--space-2);
flex-wrap: wrap;
}
.play {
min-width: 96px;
justify-content: center;
}
.tracks {
list-style: none;
margin: var(--space-2) 0 0;
padding: 0;
max-height: 320px;
overflow-y: auto;
border-top: 1px solid var(--border);
}
.track {
display: flex;
align-items: center;
gap: var(--space-3);
width: 100%;
padding: 7px var(--space-2);
border: none;
border-radius: var(--radius);
background: transparent;
color: var(--muted);
text-align: left;
cursor: pointer;
}
.track:hover {
background: var(--surface-2);
color: var(--text);
}
.track.current {
color: var(--accent);
}
.num {
flex: none;
color: var(--faint);
}
.track.current .num {
color: var(--accent);
}
.ind {
flex: none;
width: 1.1em;
font-size: var(--fs-xs);
color: var(--accent);
}
.name {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.dur {
flex: none;
color: var(--faint);
}
</style>

8
web/src/lib/components/Reader.svelte
View file

@ -3,9 +3,11 @@
import { api, ApiError, type Item, type EbookMetadata } from '../api'
import ComicReader from './ComicReader.svelte'
import AudiobookPlayer from './AudiobookPlayer.svelte'
import MusicPlayer from './MusicPlayer.svelte'
import PhotoViewer from './PhotoViewer.svelte'
import PdfReader from './PdfReader.svelte'
import GamePlayer from './GamePlayer.svelte'
import VideoPlayer from './VideoPlayer.svelte'
let { itemId }: { itemId: number } = $props()
@ -139,12 +141,16 @@
<div class="alert">{error}</div>
{:else if item && item.kind === 'games'}
<GamePlayer {itemId} />
{:else if item && item.kind === 'video'}
<VideoPlayer {itemId} />
{:else if item && fileContainer === 'pdf'}
<PdfReader {itemId} />
{:else if item && item.kind === 'comics'}
<ComicReader {itemId} />
{:else if item && (item.kind === 'audiobooks' || item.kind === 'music')}
{:else if item && item.kind === 'audiobooks'}
<AudiobookPlayer {itemId} />
{:else if item && item.kind === 'music'}
<MusicPlayer {itemId} />
{:else if item && item.kind === 'photos'}
<PhotoViewer {itemId} />
{:else if item && item.kind !== 'ebooks'}

698
web/src/lib/components/VideoPlayer.svelte Normal file
View file

@ -0,0 +1,698 @@
<script lang="ts">
import { onMount, onDestroy } from 'svelte'
import { api, ApiError, type Item, type MediaFile } from '../api'
import * as playback from '../playback'
import {
setNowPlaying,
setHandlers,
setPosition,
setPlaybackState,
clear as clearMediaSession,
} from '../mediaSession'
let { itemId }: { itemId: number } = $props()
// Seek step for the keyboard shortcuts and the OS seek actions, in seconds.
// 10s is the video convention (a few seconds of a scene), not the 15s of an
// audiobook chapter.
const SEEK_STEP = 10
// How close to the end counts as "finished", so a video that is not watched
// to the last frame still drops off "Continue watching".
const FINISH_WINDOW_MS = 10_000
// How long to let a freshly loaded element publish its dimensions before we
// conclude the video track is undecodable. Long enough for a slow start, short
// enough that a black frame is never mistaken for a working picture.
const DECODE_GRACE_MS = 1200
// How long to wait for the element to say ANYTHING after load() before we
// conclude it rejected the source outright. Needed because an explicit `type`
// attribute the browser cannot satisfy makes it refuse the resource during
// selection: neither `loadedmetadata` NOR `error` ever fires, so a player that
// only reacts to those two events shows 0:00 and a Play button forever — the
// silently-broken state this component exists to avoid.
const LOAD_GRACE_MS = 2500
let video = $state<HTMLVideoElement>()
let item = $state<Item | null>(null)
let files = $state<MediaFile[]>([])
let index = $state(0)
let loading = $state(true)
let error = $state<string | null>(null)
let playing = $state(false)
let muted = $state(false)
let currentTime = $state(0)
let duration = $state(0)
// ── Honest-path state ────────────────────────────────────────────────
// `notice` is non-empty only when this client genuinely cannot decode the
// video track; `undecodable` drives the "audio only" presentation. Both are
// cleared the moment a real picture appears.
let notice = $state('')
let undecodable = $state(false)
let videoDecoded = $state(false)
let saveTimer: number | undefined
let verifyTimer: number | undefined
let loadWatchdog: number | undefined
let resumeAt = 0
let pendingPlay = false
let finalFlushed = false
const file = $derived(files[index] ?? null)
const src = $derived(file ? api.fileUrlById(itemId, file.id) : undefined)
const srcType = $derived(file ? playback.sourceType(file) : '')
// A <source> child only feeds the element's resource-selection algorithm when
// it is first attached; swapping the src afterwards needs an explicit load().
// This is what lets the picker switch between a multi-file item's files.
$effect(() => {
src
const el = video
if (!el) return
resetDecodeState()
el.load()
// Arm the watchdog: if the element neither loads metadata nor errors, the
// source was rejected during selection and only this timer will notice.
clearWatchdog()
loadWatchdog = window.setTimeout(() => {
loadWatchdog = undefined
const v = video
if (!v) return
if (playback.hasDecodedVideo(v)) return
// readyState < HAVE_METADATA means it never even parsed the container.
if (v.readyState < 1) showUndecodable(file)
}, LOAD_GRACE_MS)
})
function clearWatchdog() {
if (loadWatchdog !== undefined) {
clearTimeout(loadWatchdog)
loadWatchdog = undefined
}
}
onMount(async () => {
// Persist the final position even if the page is torn down: keepalive lets a
// PUT outlive the document, which sendBeacon (POST-only) cannot. Progress is
// PUT-only, so sendBeacon is not an option.
window.addEventListener('pagehide', finalFlush)
document.addEventListener('visibilitychange', onVisibilityChange)
window.addEventListener('keydown', onKeydown)
try {
const detail = await api.item(itemId)
item = detail.item
files = detail.files ?? []
if (files.length === 0) {
error = 'This item has no video file.'
return
}
// Resume on the file that was playing, at the saved offset.
if (detail.progress) {
const i = files.findIndex((f) => f.id === detail.progress!.file_id)
if (i >= 0) {
index = i
resumeAt = detail.progress.position / 1000
}
}
publishNowPlaying()
registerHandlers()
} catch (err) {
error = err instanceof ApiError ? err.message : 'could not open this video'
} finally {
loading = false
}
})
onDestroy(() => {
window.removeEventListener('pagehide', finalFlush)
document.removeEventListener('visibilitychange', onVisibilityChange)
window.removeEventListener('keydown', onKeydown)
if (saveTimer) clearTimeout(saveTimer)
if (verifyTimer !== undefined) clearTimeout(verifyTimer)
clearWatchdog()
video?.pause()
clearMediaSession()
finalFlush()
})
function onVisibilityChange() {
if (document.visibilityState === 'hidden') finalFlush()
else finalFlushed = false
}
// ── MediaSession ─────────────────────────────────────────────────────
// The OS media keys, lock screen and browser media popup read from here; the
// helper guards every call so an unsupported browser never breaks playback.
function publishNowPlaying() {
setNowPlaying({
title: item?.title ?? 'Video',
artwork: api.coverUrl(itemId),
duration,
})
}
function registerHandlers() {
setHandlers({
play: () => {
if (video) void video.play().catch(() => {})
},
pause: () => video?.pause(),
seekTo: (d) => {
if (video && d?.seekTime != null) {
video.currentTime = d.seekTime
currentTime = d.seekTime
}
},
seekBackward: (d) => seekBy(-(d?.seekOffset ?? SEEK_STEP)),
seekForward: (d) => seekBy(d?.seekOffset ?? SEEK_STEP),
})
}
// ── Honest path ──────────────────────────────────────────────────────
// `canPlayType` is a hint and is wrong in both directions (it says "no" for
// mov+h264, which plays, and "maybe" for mkv+hevc, whose video never
// decodes). The only reliable answer is the loaded element's own dimensions,
// so the player always confirms by actually loading the file.
function resetDecodeState() {
if (verifyTimer !== undefined) {
clearTimeout(verifyTimer)
verifyTimer = undefined
}
notice = ''
undecodable = false
videoDecoded = false
}
function honestNotice(f: MediaFile | null): string {
const note = f ? playback.playbackNote(f) : ''
if (note) return note
return 'This browser cannot decode the video track in this file, so you would only hear audio over a blank picture. Vellum does not transcode.'
}
function showUndecodable(f: MediaFile | null, fallback = '') {
notice = honestNotice(f) || fallback
undecodable = true
videoDecoded = false
}
// Called after metadata loads. If the file carries a video codec and this
// client cannot decode it, the element plays AUDIO while videoWidth stays 0 —
// exactly what HEVC does in Chrome. A file with no probed codec gets the
// benefit of the doubt: it is only condemned once the element itself confirms
// 0x0 or errors, never on missing metadata.
function checkDecoded(f: MediaFile | null) {
const el = video
if (!el || !f) return
if (playback.hasDecodedVideo(el)) {
markDecoded()
return
}
// No picture yet: give a slow start a moment before declaring failure.
if (verifyTimer !== undefined) return
verifyTimer = window.setTimeout(() => {
verifyTimer = undefined
const v = video
if (!v) return
if (playback.hasDecodedVideo(v)) markDecoded()
else showUndecodable(f)
}, DECODE_GRACE_MS)
}
function markDecoded() {
if (verifyTimer !== undefined) {
clearTimeout(verifyTimer)
verifyTimer = undefined
}
videoDecoded = true
undecodable = false
notice = ''
}
function onResize() {
// The element fires `resize` when videoWidth/videoHeight first become
// non-zero, which is the moment a picture actually appears.
if (video && playback.hasDecodedVideo(video)) markDecoded()
}
function onError() {
const el = video
clearWatchdog()
const code = el?.error?.code
// 4 is MEDIA_ERR_SRC_NOT_SUPPORTED — the common one for avi/mpeg4 and for
// an mkv whose tracks are not understood. It means the file will never play
// here, so say so rather than showing a spinner forever.
const suffix = code === 4 ? ' (the file format or codec is not supported here)' : ''
showUndecodable(file, (el?.error?.message || 'This video failed to load.') + suffix)
}
// ── Playback ─────────────────────────────────────────────────────────
function onLoadedMetadata() {
const el = video
if (!el) return
clearWatchdog()
duration = el.duration || 0
resetDecodeState()
checkDecoded(file)
if (resumeAt > 0 && isFinite(el.duration) && resumeAt < el.duration) {
el.currentTime = resumeAt
resumeAt = 0
}
// The new source is loaded: autoplay a file we just switched to.
if (pendingPlay) {
pendingPlay = false
void el.play().catch(() => {})
}
publishNowPlaying()
setPosition(el.currentTime, duration)
}
function onTimeUpdate() {
const el = video
if (!el) return
currentTime = el.currentTime
setPosition(currentTime, el.duration || duration)
scheduleSave()
}
function toggle() {
if (!video) return
if (video.paused) void video.play().catch(() => {})
else video.pause()
}
function onPlay() {
playing = true
setPlaybackState('playing')
}
function onPause() {
playing = false
setPlaybackState('paused')
}
function onEnded() {
flush(true)
}
function seekBy(delta: number) {
const el = video
if (!el) return
const limit = isFinite(el.duration) && el.duration > 0 ? el.duration : Infinity
const next = Math.min(Math.max(el.currentTime + delta, 0), limit)
el.currentTime = next
currentTime = next
}
function toggleMute() {
const el = video
if (!el) return
el.muted = !el.muted
muted = el.muted
}
function toggleFullscreen() {
const el = video
if (!el) return
if (document.fullscreenElement) void document.exitFullscreen().catch(() => {})
else void el.requestFullscreen().catch(() => {})
}
// ── File picker ──────────────────────────────────────────────────────
// A video library groups by file, so an item can carry more than one. The
// picker switches the element's source between them.
function selectFile(i: number) {
if (i < 0 || i >= files.length || i === index) return
flush()
index = i
resumeAt = 0
resetDecodeState()
// Autoplay once Svelte has swapped the src and it has loaded; playing from a
// microtask can race the DOM update and replay the old file.
pendingPlay = true
}
// ── Keyboard ─────────────────────────────────────────────────────────
// Space = play/pause, ←/→ = seek ∓10s, F = fullscreen, M = mute. Ignored
// while typing so the shortcuts never eat a search or form field.
function onKeydown(event: KeyboardEvent) {
const target = event.target as HTMLElement | null
if (
target &&
(target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.tagName === 'SELECT' ||
target.isContentEditable)
) {
return
}
switch (event.key) {
case ' ':
case 'Spacebar':
event.preventDefault()
toggle()
break
case 'ArrowLeft':
event.preventDefault()
seekBy(-SEEK_STEP)
break
case 'ArrowRight':
event.preventDefault()
seekBy(SEEK_STEP)
break
case 'f':
case 'F':
event.preventDefault()
toggleFullscreen()
break
case 'm':
case 'M':
event.preventDefault()
toggleMute()
break
}
}
// ── Progress ─────────────────────────────────────────────────────────
// Progress is milliseconds and is keyed by file, so a multi-file item
// resumes exactly where it stopped.
function scheduleSave() {
if (saveTimer) clearTimeout(saveTimer)
saveTimer = window.setTimeout(() => flush(), 2000)
}
function progressBody(forceFinished = false) {
const el = video
const f = file
if (!el || !f) return null
const pos = Math.floor(el.currentTime * 1000)
const dur = Math.floor((el.duration || 0) * 1000)
const nearEnd = dur > 0 && dur - pos <= FINISH_WINDOW_MS
return {
file_id: f.id,
position: pos,
duration: dur,
percent: dur > 0 ? pos / dur : 0,
finished: forceFinished || nearEnd,
}
}
function flush(finished = false) {
const body = progressBody(finished)
if (!body) return
void api.putProgress(itemId, body).catch(() => {})
}
// Last-chance save on teardown. Uses fetch with keepalive so the PUT survives
// the document being unloaded, and latches so the overlapping pagehide /
// visibilitychange / onDestroy events only fire one request. Progress is
// PUT-only, so sendBeacon cannot be used.
function finalFlush() {
if (finalFlushed) return
const body = progressBody(false)
if (!body) return
finalFlushed = true
if (saveTimer) {
clearTimeout(saveTimer)
saveTimer = undefined
}
void fetch(`/api/v1/items/${itemId}/progress`, {
method: 'PUT',
keepalive: true,
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
}).catch(() => {})
}
function seek(event: Event) {
const value = Number((event.currentTarget as HTMLInputElement).value)
if (video) video.currentTime = value
currentTime = value
}
function fmt(seconds: number): string {
if (!isFinite(seconds) || seconds < 0) seconds = 0
const h = Math.floor(seconds / 3600)
const m = Math.floor((seconds % 3600) / 60)
const s = Math.floor(seconds % 60)
const mm = String(m).padStart(2, '0')
const ss = String(s).padStart(2, '0')
return h > 0 ? `${h}:${mm}:${ss}` : `${m}:${ss}`
}
function codecSummary(f: MediaFile): string {
const parts: string[] = []
if (f.vcodec) parts.push(f.vcodec)
if (f.acodec) parts.push(f.acodec)
if (f.width && f.height) parts.push(`${f.width}×${f.height}`)
return parts.join(' · ')
}
</script>
{#if error}
<div class="alert">{error}</div>
{:else if loading}
<div class="center"><div class="spinner"></div></div>
{:else if files.length === 0}
<div class="empty">
<h3>{item?.title ?? 'Video'}</h3>
<p>This item has no video file to play.</p>
<a class="btn" href={api.fileUrl(itemId)} download>Download the file</a>
</div>
{:else}
<div class="player card" class:undecodable>
<div class="head">
<div class="meta">
<strong>{item?.title}</strong>
<span class="muted">
{file?.path.split('/').pop() ?? ''}
{#if files.length > 1}· {index + 1} of {files.length}{/if}
</span>
{#if file && codecSummary(file)}
<span class="badge mono">{codecSummary(file)}</span>
{/if}
</div>
<a class="btn btn-ghost dl" href={api.fileUrl(itemId)} download>Download</a>
</div>
{#if notice}
<div class="notice" role="alert">
<strong>Video cannot play here</strong>
<p>{notice}</p>
<a class="btn" href={api.fileUrl(itemId)} download>Download the file</a>
</div>
{/if}
<div class="stage">
<!-- svelte-ignore a11y_media_has_caption -->
<!-- Direct play only: Vellum serves the file untouched and has no subtitle
tracks to offer, so there is nothing to caption. -->
<video
bind:this={video}
class="video"
controls
playsinline
preload="metadata"
onloadedmetadata={onLoadedMetadata}
ontimeupdate={onTimeUpdate}
onplay={onPlay}
onpause={onPause}
onended={onEnded}
onerror={onError}
onresize={onResize}
>
<source src={src} type={srcType || undefined} />
</video>
</div>
{#if videoDecoded}
<p class="ok-note" aria-live="polite">Playing video directly — no transcoding.</p>
{/if}
<input
class="seek"
type="range"
min="0"
max={duration || 0}
step="1"
value={currentTime}
oninput={seek}
aria-label="Seek"
/>
<div class="times mono">
<span>{fmt(currentTime)}</span>
<span class="muted">{fmt(duration)}</span>
</div>
<div class="controls">
<button class="btn btn-primary play" onclick={toggle}>{playing ? 'Pause' : 'Play'}</button>
<button class="btn btn-ghost" onclick={toggleMute} aria-pressed={muted} title="Mute (M)">
{muted ? 'Unmute' : 'Mute'}
</button>
<button class="btn btn-ghost" onclick={toggleFullscreen} title="Fullscreen (F)">Fullscreen</button>
</div>
<p class="hint muted">Space play/pause · ←/→ seek {SEEK_STEP}s · F fullscreen · M mute</p>
{#if files.length > 1}
<ol class="files">
{#each files as f, i (f.id)}
<li>
<button class="file" class:current={i === index} onclick={() => selectFile(i)}>
<span class="mono">{String(i + 1).padStart(2, '0')}</span>
<span class="name">{f.path.split('/').pop()}</span>
<span class="muted mono">{codecSummary(f)}</span>
</button>
</li>
{/each}
</ol>
{/if}
</div>
{/if}
<style>
.center {
display: grid;
place-items: center;
padding: var(--space-6);
}
.player {
max-width: 960px;
margin: 0 auto;
padding: var(--space-5);
display: flex;
flex-direction: column;
gap: var(--space-3);
}
.head {
display: flex;
align-items: center;
gap: var(--space-4);
}
.meta {
display: flex;
flex-direction: column;
align-items: flex-start;
gap: 2px;
min-width: 0;
flex: 1;
}
.meta strong {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.meta .muted {
font-size: var(--fs-sm);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 100%;
}
.dl {
flex: none;
}
.notice {
display: flex;
flex-direction: column;
align-items: flex-start;
gap: var(--space-2);
padding: 10px 14px;
border: 1px solid color-mix(in srgb, var(--danger) 45%, var(--border));
border-radius: var(--radius);
background: color-mix(in srgb, var(--danger) 10%, transparent);
font-size: var(--fs-md);
}
.notice p {
margin: 0;
color: var(--muted);
}
.stage {
background: #000;
border: 1px solid var(--border);
border-radius: var(--radius);
overflow: hidden;
}
.video {
display: block;
width: 100%;
max-height: calc(100vh - 52px - 220px);
background: #000;
}
/* The file loaded but only audio decodes: keep the audio reachable, but make
the black frame read as a known limitation rather than a working picture. */
.undecodable .stage {
border-color: color-mix(in srgb, var(--danger) 45%, var(--border));
}
.ok-note {
margin: 0;
font-size: var(--fs-sm);
color: var(--success);
}
.seek {
width: 100%;
accent-color: var(--accent);
}
.times {
display: flex;
justify-content: space-between;
font-size: var(--fs-sm);
}
.controls {
display: flex;
align-items: center;
justify-content: center;
gap: var(--space-3);
}
.play {
min-width: 96px;
justify-content: center;
}
.hint {
margin: 0;
font-size: var(--fs-xs);
text-align: center;
}
.files {
list-style: none;
margin: var(--space-2) 0 0;
padding: 0;
max-height: 260px;
overflow-y: auto;
border-top: 1px solid var(--border);
}
.file {
display: flex;
align-items: center;
gap: var(--space-3);
width: 100%;
padding: 7px var(--space-2);
border: none;
border-radius: var(--radius);
background: transparent;
color: var(--muted);
text-align: left;
cursor: pointer;
}
.file:hover {
background: var(--surface-2);
color: var(--text);
}
.file.current {
color: var(--accent);
}
.file .name {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
</style>

130
web/src/lib/mediaSession.ts Normal file
View file

@ -0,0 +1,130 @@
// MediaSession wiring for the audio players.
//
// The OS media keys, the lock screen and the browser's own media popup all read
// from `navigator.mediaSession`. Without this, pressing the play/pause key on a
// keyboard or a pair of headphones does nothing, and the lock screen shows a
// blank entry — which for an audiobook or an album is the difference between
// "a web page playing audio" and "a player".
//
// Everything here is best-effort: the API is absent in some browsers, and an
// unsupported action handler must never break playback. Every call is guarded
// and every registration is wrapped.
export interface NowPlaying {
title: string
artist?: string
album?: string
/** Cover art URL. Same-origin, so the browser can fetch it. */
artwork?: string
/** Seconds. */
duration?: number
}
type ActionHandler = (details?: { seekTime?: number; seekOffset?: number }) => void
function session(): MediaSession | undefined {
// Feature-detected rather than assumed: Firefox shipped this late and some
// embedded webviews never did.
return typeof navigator !== 'undefined' ? navigator.mediaSession : undefined
}
/** Whether the OS-level media controls are available at all. */
export function supported(): boolean {
return session() !== undefined && typeof MediaMetadata !== 'undefined'
}
/**
* Publish what is playing. Safe to call on every track change; callers do not
* need to check support first.
*/
export function setNowPlaying(np: NowPlaying): void {
const s = session()
if (!s || typeof MediaMetadata === 'undefined') return
try {
s.metadata = new MediaMetadata({
title: np.title,
artist: np.artist ?? '',
album: np.album ?? '',
artwork: np.artwork
? [{ src: np.artwork, sizes: '512x512', type: 'image/png' }]
: [],
})
} catch {
// A malformed metadata object must not stop playback.
}
}
/** Register the actions the player can honour. Unlisted actions stay absent. */
export function setHandlers(handlers: {
play?: ActionHandler
pause?: ActionHandler
next?: ActionHandler
previous?: ActionHandler
seekBackward?: ActionHandler
seekForward?: ActionHandler
seekTo?: ActionHandler
}): void {
const s = session()
if (!s) return
const pairs: [MediaSessionAction, ActionHandler | undefined][] = [
['play', handlers.play],
['pause', handlers.pause],
['nexttrack', handlers.next],
['previoustrack', handlers.previous],
['seekbackward', handlers.seekBackward],
['seekforward', handlers.seekForward],
['seekto', handlers.seekTo],
]
for (const [action, fn] of pairs) {
try {
// null clears a previously registered handler; passing undefined would
// throw in some browsers.
s.setActionHandler(action, fn ?? null)
} catch {
// Unsupported action for this browser — skip it.
}
}
}
/**
* Tell the OS where playback is, so the lock-screen scrubber is accurate.
* Some browsers reject a position without a duration, so this is skipped
* until the track has loaded metadata.
*/
export function setPosition(position: number, duration: number, rate = 1): void {
const s = session()
if (!s || typeof s.setPositionState !== 'function') return
if (!isFinite(duration) || duration <= 0) return
const pos = Math.min(Math.max(position, 0), duration)
try {
s.setPositionState({ duration, playbackRate: rate, position: pos })
} catch {
// Out-of-range or unsupported; the scrubber just stays as it was.
}
}
/** Mirror the element's play/pause state onto the OS controls. */
export function setPlaybackState(state: 'playing' | 'paused' | 'none'): void {
const s = session()
if (!s) return
try {
s.playbackState = state
} catch {
// Ignore.
}
}
/** Drop metadata and handlers when the player unmounts. */
export function clear(): void {
const s = session()
if (!s) return
try {
s.metadata = null
s.playbackState = 'none'
} catch {
// Ignore.
}
setHandlers({})
}

147
web/src/lib/playback.ts Normal file
View file

@ -0,0 +1,147 @@
// Direct-play decisions.
//
// Vellum does not transcode — the server has no spare CPU for it — so a file
// either plays in the client or it does not. Which one it is depends on the
// CLIENT, not the file: Chrome on Linux cannot decode HEVC, Safari can, and the
// native Jellyfin apps handle far more than any browser. So nothing here is a
// server-side verdict; it is a per-client guess, and the player always confirms
// it by actually loading the file.
//
// The mappings below were measured, not guessed. Files generated with known
// codecs and played through the server's own stream endpoint in Chrome 155 on
// Linux gave this:
//
// mp4 h264 + aac plays mkv h264 + aac PLAYS (container is fine!)
// webm vp9 + opus plays mov h264 + aac plays
// mp4 av1 + opus plays webm vp8 + vorbis plays
// mp4 hevc + aac AUDIO ONLY — videoWidth stays 0
// mkv hevc + flac AUDIO ONLY
// avi mpeg4 + mp3 fails outright
//
// Two things that follow, and that the UI must respect:
//
// 1. Container matters far less than codec. mkv plays fine when the video
// track is h264; mp4 fails when the track is HEVC. Judging a file by its
// extension is wrong.
// 2. `canPlayType` is a hint, not an answer. It said "no" for mov+h264 (which
// plays) and "maybe" for mkv+hevc (whose video never decodes). The only
// reliable test is to load the file and check `videoWidth`.
import type { MediaFile } from './api'
/** Container extension → MIME type. Mirrors internal/stream's table. */
const CONTAINER_MIME: Record<string, string> = {
mp4: 'video/mp4',
m4v: 'video/mp4',
mov: 'video/quicktime',
mkv: 'video/x-matroska',
webm: 'video/webm',
avi: 'video/x-msvideo',
mp3: 'audio/mpeg',
m4a: 'audio/mp4',
m4b: 'audio/mp4',
aac: 'audio/aac',
flac: 'audio/flac',
opus: 'audio/ogg',
ogg: 'audio/ogg',
wav: 'audio/wav',
}
/**
* ffprobe codec name → an RFC 6381 codec parameter good enough for
* `canPlayType`. The full parameter string (profile, level, tier) is not
* recoverable from a bare codec name, so these are the shortest forms browsers
* accept.
*/
const VCODEC: Record<string, string> = {
h264: 'avc1.42E01E',
avc1: 'avc1.42E01E',
hevc: 'hvc1.1.6.L93.B0',
h265: 'hvc1.1.6.L93.B0',
vp8: 'vp8',
vp9: 'vp09.00.10.08',
av1: 'av01.0.04M.08',
mpeg4: 'mp4v.20.9',
mpeg2video: 'mp2v',
vc1: 'vc-1',
theora: 'theora',
}
const ACODEC: Record<string, string> = {
aac: 'mp4a.40.2',
mp3: 'mp3',
opus: 'opus',
vorbis: 'vorbis',
flac: 'flac',
ac3: 'ac-3',
eac3: 'ec-3',
dts: 'dts',
pcm_s16le: 'pcm',
}
export type Verdict = 'probably' | 'maybe' | 'no'
/** The MIME type for a file's container, or '' if it is unknown. */
export function mimeFor(file: MediaFile): string {
return CONTAINER_MIME[(file.container || '').toLowerCase()] ?? ''
}
/**
* Build the `type` string for a media element, e.g.
* `video/mp4; codecs="avc1.42E01E, mp4a.40.2"`. Returns '' when the container
* is unknown, in which case the caller should let the element sniff the file.
*/
export function sourceType(file: MediaFile): string {
const mime = mimeFor(file)
if (!mime) return ''
const codecs: string[] = []
const v = VCODEC[(file.vcodec || '').toLowerCase()]
const a = ACODEC[(file.acodec || '').toLowerCase()]
if (v) codecs.push(v)
if (a) codecs.push(a)
return codecs.length ? `${mime}; codecs="${codecs.join(', ')}"` : mime
}
/**
* Ask this browser whether it can play the file. A hint only — see the note at
* the top of this file about how wrong `canPlayType` is in both directions.
*/
export function canDirectPlay(file: MediaFile): Verdict {
if (typeof document === 'undefined') return 'maybe'
const type = sourceType(file)
if (!type) return 'maybe'
const el = document.createElement('video')
const verdict = el.canPlayType(type)
if (verdict === 'probably') return 'probably'
if (verdict === 'maybe') return 'maybe'
return 'no'
}
/**
* A short human explanation for a file that is not expected to play, or '' when
* there is nothing worth saying. The wording avoids blaming the user's machine
* and points at the actual cause.
*/
export function playbackNote(file: MediaFile): string {
if (canDirectPlay(file) !== 'no') return ''
const codec = (file.vcodec || '').toLowerCase()
if (codec === 'hevc' || codec === 'h265') {
return 'This file uses HEVC video, which this browser cannot decode. Safari and the native apps can.'
}
if (codec === 'mpeg4' || codec === 'mpeg2video' || codec === 'vc1') {
return `This file uses ${codec} video, which browsers no longer support.`
}
return 'This browser cannot play this file directly, and Vellum does not transcode.'
}
/**
* Whether a loaded media element actually decoded a video track.
*
* This is the reliable test, and the reason the player never trusts
* `canPlayType`: an element whose metadata loaded but whose video track is
* undecodable reports a duration and plays AUDIO while `videoWidth` stays 0.
* That is exactly what HEVC does in Chrome.
*/
export function hasDecodedVideo(el: HTMLVideoElement): boolean {
return el.videoWidth > 0 && el.videoHeight > 0
}