Use Jellyfin as media catalog source

README.md

@@ -8,7 +8,7 @@ It does not need Discord gateway intents or slash commands for the status module
 
 The bot also includes a small web dashboard for editing monitored services and forcing immediate Discord refreshes.
 
-It also has a media catalog module. From the dashboard you can upload `Movies.csv` and/or `Shows.csv`, choose a Discord channel ID, and publish a formatted catalog embed set.
+It also has a media catalog module. The dashboard syncs movies and shows from Jellyfin, tracks additions/removals between syncs, and publishes a compact Discord catalog post.
 
 ## Discord Bot Setup
 
@@ -187,15 +187,16 @@ The dashboard currently supports:
 - editing check URL, display URL, expected statuses, timeout, and keyword
 - saving `services.json`
 - forcing an immediate check and Discord message update
-- uploading `Movies.csv` and `Shows.csv`
+- syncing movies and shows from Jellyfin
-- editing imported movies and shows before publishing
+- tracking additions and removals between Jellyfin syncs
+- editing synced movies and shows before publishing
 - publishing a compact Markdown media catalog to a selected Discord channel
 
 The sidebar leaves room for future modules like polls, webhooks, automations, and service integrations without changing the bot shape later.
 
 ## Media Catalog
 
-Open the dashboard and switch to `Media`. Set the target Discord channel ID, choose `Movies.csv`, `Shows.csv`, or both, then import the CSVs into the library editor.
+Open the dashboard and switch to `Media`. Set the target Discord channel ID, configure Jellyfin, then run `Sync now`.
 
 The editor supports adding, editing, and deleting movie/show rows before saving or publishing. Publishing sends the currently edited dashboard library, not the raw uploaded files.
 
@@ -203,7 +204,7 @@ Discord publishing uses one message with an attached `media-catalog.md` file so
 
 The dashboard also serves a read-only catalog page at `/catalog`. Set the public reverse-proxy URL for that page in the `Media` tab’s `Catalog URL` field, or with `PUBLIC_CATALOG_URL`, and Discord posts will include an `Open Catalog` button.
 
-You can also sync directly from Jellyfin instead of using CSVs. In Jellyfin, create an API key from the admin dashboard, then enter the Jellyfin URL and key in the `Media` tab. `Sync now` replaces the editable library with the current Jellyfin movies and shows. `Auto-sync changes` checks Jellyfin periodically and republishes only when the catalog fingerprint changes. Jellyfin results are deduplicated across libraries using provider IDs first, then normalized title and year.
+In Jellyfin, create an API key from the admin dashboard, then enter the Jellyfin URL and key in the `Media` tab. `Sync now` replaces the editable library with the current Jellyfin movies and shows while comparing against the previously saved library. `Auto-sync changes` checks Jellyfin periodically and republishes only when the catalog fingerprint changes. Jellyfin results are deduplicated across libraries using provider IDs first, then normalized title and year.
 
 Channel selections are stored in:
 
@@ -211,8 +212,6 @@ Channel selections are stored in:
 BOT_SETTINGS_STATE=state/bot-settings.json
 ```
 
-The parser accepts common column names such as `title`, `name`, `year`, `genre`, `genres`, `rating`, `runtime`, `summary`, `overview`, `season`, and `episode`. Show exports that contain one row per episode are grouped by show title where possible.
-
 The bot stores media catalog message IDs in:
 
 ```env

dashboard.html

@@ -323,7 +323,7 @@
 
     .media-status {
       display: grid;
-      grid-template-columns: repeat(4, minmax(0, 1fr));
+      grid-template-columns: repeat(5, minmax(0, 1fr));
       gap: 1px;
       background: var(--line);
     }
@@ -488,7 +488,7 @@
       <div class="brand">Archive Bot</div>
       <nav aria-label="Bot modules">
         <button class="nav-item active" type="button" data-view="status"><span>Status</span><span>Ready</span></button>
-        <button class="nav-item" type="button" data-view="media"><span>Media</span><span>CSV</span></button>
+        <button class="nav-item" type="button" data-view="media"><span>Media</span><span>Jellyfin</span></button>
         <div class="nav-item disabled"><span>Polls</span><span>Later</span></div>
         <div class="nav-item disabled"><span>Automations</span><span>Later</span></div>
       </nav>
@@ -580,7 +580,7 @@
 
         <section class="panel">
           <div class="panel-header">
-            <div class="panel-title">CSV Upload</div>
+            <div class="panel-title">Publish Settings</div>
             <div id="mediaMessage" class="message"></div>
           </div>
           <div class="media-form">
@@ -592,17 +592,8 @@
               <label for="catalogUrl">Catalog URL</label>
               <input id="catalogUrl" placeholder="https://archive.example.com/catalog">
             </div>
-            <div class="media-field">
-              <label for="moviesCsv">Movies.csv</label>
-              <input id="moviesCsv" type="file" accept=".csv,text/csv">
-            </div>
-            <div class="media-field">
-              <label for="showsCsv">Shows.csv</label>
-              <input id="showsCsv" type="file" accept=".csv,text/csv">
-            </div>
           </div>
           <div class="media-toolbar">
-            <button id="importMediaCsv" type="button">Import CSV</button>
             <button id="addMovie" type="button">Add movie</button>
             <button id="addShow" type="button">Add show</button>
           </div>
@@ -623,6 +614,10 @@
               <span>Published</span>
               <strong id="mediaPublishedAt">Never</strong>
             </div>
+            <div>
+              <span>Last changes</span>
+              <strong id="mediaChangeCount">0 added · 0 removed</strong>
+            </div>
           </div>
         </section>
 
@@ -870,6 +865,7 @@
       document.querySelector("#mediaPublishedAt").textContent = payload.publishedAt
         ? new Date(payload.publishedAt).toLocaleString([], { dateStyle: "short", timeStyle: "short" })
         : "Never";
+      renderMediaChanges(payload.changes || payload.lastChanges || (payload.jellyfin && payload.jellyfin.lastChanges));
     }
 
     async function loadMediaStatus() {
@@ -933,9 +929,16 @@
       document.querySelector("#mediaShowCount").textContent = payload.showCount;
       renderJellyfinStatus(payload);
       const action = payload.published ? "published" : payload.changed ? "updated library" : "no changes";
+      renderMediaChanges(payload.changes);
       setJellyfinMessage(`Synced ${payload.movieCount} movies and ${payload.showCount} shows; ${action}.`);
     }
 
+    function renderMediaChanges(changes = {}) {
+      const added = Number(changes.addedCount || (changes.added && changes.added.length) || 0);
+      const removed = Number(changes.removedCount || (changes.removed && changes.removed.length) || 0);
+      document.querySelector("#mediaChangeCount").textContent = `${added} added · ${removed} removed`;
+    }
+
     function normalizeMediaLibrary(library = {}) {
       return {
         movies: Array.isArray(library.movies) ? library.movies.map((item) => normalizeMediaItem(item, "movie")) : [],
@@ -1055,41 +1058,6 @@
       return channels;
     }
 
-    function readFileText(input) {
-      const file = input.files && input.files[0];
-      if (!file) return Promise.resolve({ name: "", text: "" });
-      return file.text().then((text) => ({ name: file.name, text }));
-    }
-
-    async function importMediaCsv() {
-      const moviesInput = document.querySelector("#moviesCsv");
-      const showsInput = document.querySelector("#showsCsv");
-      const [movies, shows] = await Promise.all([
-        readFileText(moviesInput),
-        readFileText(showsInput)
-      ]);
-
-      if (!movies.text.trim() && !shows.text.trim()) {
-        throw new Error("Choose a Movies.csv or Shows.csv file before importing.");
-      }
-
-      setMediaMessage("Importing CSV files...");
-      const payload = await api("/api/media/import", {
-        method: "POST",
-        body: JSON.stringify({
-          moviesCsv: movies.text,
-          showsCsv: shows.text,
-          movieFileName: movies.name || "Movies.csv",
-          showFileName: shows.name || "Shows.csv"
-        })
-      });
-      mediaLibrary = normalizeMediaLibrary(payload.library);
-      renderMediaLibrary();
-      document.querySelector("#mediaMovieCount").textContent = payload.movieCount;
-      document.querySelector("#mediaShowCount").textContent = payload.showCount;
-      setMediaMessage(`Imported ${payload.movieCount} movies and ${payload.showCount} shows.`);
-    }
-
     async function saveMediaLibrary() {
       collectMediaLibrary();
       const payload = await api("/api/media/library", {
@@ -1227,10 +1195,6 @@
       syncJellyfin(true).catch((error) => setJellyfinMessage(error.message, true));
     });
 
-    document.querySelector("#importMediaCsv").addEventListener("click", () => {
-      importMediaCsv().catch((error) => setMediaMessage(error.message, true));
-    });
-
     document.querySelector("#saveMediaLibrary").addEventListener("click", () => {
       saveMediaLibrary().catch((error) => setMediaMessage(error.message, true));
     });

status_bot.py

@@ -5,11 +5,9 @@ from __future__ import annotations
 
 import base64
 import asyncio
-import csv
 import hashlib
 import hmac
 import html
-import io
 import json
 import os
 import re
@@ -646,6 +644,7 @@ def jellyfin_settings(runtime: BotRuntime) -> dict[str, Any]:
         "lastPublishedAt": data.get("jellyfin_last_published_at"),
         "lastFingerprint": data.get("jellyfin_last_fingerprint"),
         "lastPublishedFingerprint": data.get("jellyfin_last_published_fingerprint"),
+        "lastChanges": data.get("jellyfin_last_changes"),
     }
 
 
@@ -677,18 +676,6 @@ def save_catalog_url_setting(runtime: BotRuntime, catalog_url: str) -> dict[str,
     return channel_settings(runtime)
 
 
-def normalize_csv_key(value: str) -> str:
-    return "".join(character for character in value.lower() if character.isalnum())
-
-
-def value_from_row(row: dict[str, str], aliases: list[str]) -> str:
-    for alias in aliases:
-        value = row.get(alias, "").strip()
-        if value:
-            return value
-    return ""
-
-
 def parse_int_text(value: str) -> int | None:
     digits = "".join(character for character in value if character.isdigit())
     if not digits:
@@ -731,150 +718,6 @@ def format_runtime(value: str) -> str | None:
     return f"{remainder}m"
 
 
-def parse_media_csv(csv_text: str, media_type: str, filename: str) -> list[MediaItem]:
-    text = csv_text.lstrip("\ufeff")
-    if not text.strip():
-        return []
-
-    try:
-        dialect = csv.Sniffer().sniff(text[:4096], delimiters=",;\t|")
-    except csv.Error:
-        dialect = csv.excel
-
-    reader = csv.DictReader(io.StringIO(text), dialect=dialect)
-    if not reader.fieldnames:
-        raise ValueError(f"{filename} does not have a header row")
-
-    title_aliases = [
-        "title",
-        "name",
-        "movie",
-        "movietitle",
-        "sorttitle",
-        "originaltitle",
-    ]
-    if media_type == "show":
-        title_aliases = [
-            "showtitle",
-            "seriestitle",
-            "series",
-            "show",
-            "grandparenttitle",
-            "parenttitle",
-            "title",
-            "name",
-        ]
-
-    year_aliases = ["year", "releaseyear", "productionyear", "releasedate", "premiered", "date"]
-    genre_aliases = ["genres", "genre", "tags", "categories"]
-    rating_aliases = ["rating", "contentrating", "agerating", "certification", "mpaarating"]
-    runtime_aliases = ["runtime", "duration", "runtimeminutes", "length", "durationminutes"]
-    summary_aliases = ["summary", "overview", "description", "plot", "tagline"]
-    season_count_aliases = ["seasoncount", "seasons"]
-    season_number_aliases = ["season", "seasonnumber", "seasonindex", "parentindex"]
-    episode_count_aliases = ["episodecount", "episodes"]
-    episode_number_aliases = ["episode", "episodenumber", "episodeindex", "index"]
-
-    items: list[MediaItem] = []
-    show_groups: dict[tuple[str, str], dict[str, Any]] = {}
-
-    for raw_row in reader:
-        row = {
-            normalize_csv_key(str(key)): str(value or "")
-            for key, value in raw_row.items()
-            if key is not None
-        }
-        title = clean_media_text(value_from_row(row, title_aliases), 120)
-        if not title:
-            continue
-
-        year = parse_year_text(value_from_row(row, year_aliases))
-        genres = clean_media_text(value_from_row(row, genre_aliases), 120)
-        rating = clean_media_text(value_from_row(row, rating_aliases), 32)
-        runtime = format_runtime(value_from_row(row, runtime_aliases))
-        summary = clean_media_text(value_from_row(row, summary_aliases))
-        season_count = parse_int_text(value_from_row(row, season_count_aliases))
-        season_number = parse_int_text(value_from_row(row, season_number_aliases))
-        episode_count = parse_int_text(value_from_row(row, episode_count_aliases))
-        episode_number = parse_int_text(value_from_row(row, episode_number_aliases))
-
-        if media_type == "show":
-            key = (title.casefold(), year or "")
-            group = show_groups.setdefault(
-                key,
-                {
-                    "title": title,
-                    "year": year,
-                    "genres": genres,
-                    "rating": rating,
-                    "runtime": runtime,
-                    "summary": summary,
-                    "seasons": set(),
-                    "episodes": 0,
-                    "explicit_season_count": None,
-                    "explicit_episode_count": None,
-                },
-            )
-            if not group.get("genres") and genres:
-                group["genres"] = genres
-            if not group.get("rating") and rating:
-                group["rating"] = rating
-            if not group.get("runtime") and runtime:
-                group["runtime"] = runtime
-            if not group.get("summary") and summary:
-                group["summary"] = summary
-            if season_number is not None:
-                group["seasons"].add(season_number)
-            if episode_number is not None:
-                group["episodes"] += 1
-            if season_count is not None:
-                current = group.get("explicit_season_count")
-                group["explicit_season_count"] = max(current or 0, season_count)
-            if episode_count is not None:
-                current = group.get("explicit_episode_count")
-                group["explicit_episode_count"] = max(current or 0, episode_count)
-            continue
-
-        items.append(
-            MediaItem(
-                title=title,
-                media_type=media_type,
-                year=year,
-                genres=genres,
-                rating=rating,
-                runtime=runtime,
-                summary=summary,
-            )
-        )
-
-    if media_type == "show":
-        for group in show_groups.values():
-            seasons = group.get("explicit_season_count") or len(group["seasons"]) or None
-            episodes = group.get("explicit_episode_count") or group["episodes"] or None
-            items.append(
-                MediaItem(
-                    title=group["title"],
-                    media_type=media_type,
-                    year=group["year"],
-                    genres=group["genres"],
-                    rating=group["rating"],
-                    runtime=group["runtime"],
-                    summary=group["summary"],
-                    seasons=seasons,
-                    episodes=episodes,
-                )
-            )
-
-    deduped: dict[tuple[str, str], MediaItem] = {}
-    for item in items:
-        deduped.setdefault((item.title.casefold(), item.year or ""), item)
-
-    parsed = sorted(deduped.values(), key=lambda item: (item.title.casefold(), item.year or ""))
-    if not parsed:
-        raise ValueError(f"{filename} did not contain any rows with a recognizable title/name column")
-    return parsed
-
-
 def media_item_to_jsonable(item: MediaItem) -> dict[str, Any]:
     data: dict[str, Any] = {
         "title": item.title,
@@ -952,24 +795,6 @@ def save_media_library(runtime: BotRuntime, movies: list[MediaItem], shows: list
     return payload
 
 
-def import_media_csvs(
-    runtime: BotRuntime,
-    movies_csv: str,
-    shows_csv: str,
-    movie_filename: str,
-    show_filename: str,
-) -> dict[str, Any]:
-    current_movies, current_shows = load_media_library(runtime)
-    movies = parse_media_csv(movies_csv, "movie", movie_filename) if movies_csv.strip() else current_movies
-    shows = parse_media_csv(shows_csv, "show", show_filename) if shows_csv.strip() else current_shows
-    save_media_library(runtime, movies, shows)
-    return {
-        "library": media_library_to_jsonable(movies, shows),
-        "movieCount": len(movies),
-        "showCount": len(shows),
-    }
-
-
 def jellyfin_runtime(runtime_ticks: Any) -> str | None:
     try:
         ticks = int(runtime_ticks or 0)
@@ -1151,12 +976,45 @@ def media_library_fingerprint(movies: list[MediaItem], shows: list[MediaItem]) -
     return hashlib.sha256(body).hexdigest()
 
 
+def media_item_tracking_key(item: MediaItem) -> str:
+    return f"{item.media_type}:{item.title.casefold()}:{item.year or ''}"
+
+
+def media_change_entry(item: MediaItem) -> dict[str, Any]:
+    return {
+        "title": item.title,
+        "year": item.year or "",
+        "mediaType": item.media_type,
+    }
+
+
+def media_library_changes(
+    previous_movies: list[MediaItem],
+    previous_shows: list[MediaItem],
+    current_movies: list[MediaItem],
+    current_shows: list[MediaItem],
+) -> dict[str, Any]:
+    previous = {media_item_tracking_key(item): item for item in [*previous_movies, *previous_shows]}
+    current = {media_item_tracking_key(item): item for item in [*current_movies, *current_shows]}
+    added_keys = sorted(set(current) - set(previous), key=lambda key: (current[key].media_type, current[key].title.casefold()))
+    removed_keys = sorted(set(previous) - set(current), key=lambda key: (previous[key].media_type, previous[key].title.casefold()))
+    added = [media_change_entry(current[key]) for key in added_keys]
+    removed = [media_change_entry(previous[key]) for key in removed_keys]
+    return {
+        "added": added,
+        "removed": removed,
+        "addedCount": len(added),
+        "removedCount": len(removed),
+    }
+
+
 def update_jellyfin_sync_state(
     runtime: BotRuntime,
     *,
     fingerprint: str | None = None,
     error: str | None = None,
     published: bool = False,
+    changes: dict[str, Any] | None = None,
 ) -> None:
     state = load_state(runtime.settings_path)
     now = datetime.now(timezone.utc).isoformat()
@@ -1168,11 +1026,15 @@ def update_jellyfin_sync_state(
         state["jellyfin_last_published_at"] = now
         if fingerprint is not None:
             state["jellyfin_last_published_fingerprint"] = fingerprint
+    if changes is not None:
+        state["jellyfin_last_changes"] = changes
     save_state(runtime.settings_path, state)
 
 
 def sync_jellyfin_library(runtime: BotRuntime, force_publish: bool = False) -> dict[str, Any]:
+    previous_movies, previous_shows = load_media_library(runtime)
     movies, shows = fetch_jellyfin_library(runtime)
+    changes = media_library_changes(previous_movies, previous_shows, movies, shows)
     fingerprint = media_library_fingerprint(movies, shows)
     settings_state = load_state(runtime.settings_path)
     changed = fingerprint != str(settings_state.get("jellyfin_last_fingerprint", ""))
@@ -1190,10 +1052,11 @@ def sync_jellyfin_library(runtime: BotRuntime, force_publish: bool = False) -> d
         )
         published = True
 
-    update_jellyfin_sync_state(runtime, fingerprint=fingerprint, published=published)
+    update_jellyfin_sync_state(runtime, fingerprint=fingerprint, published=published, changes=changes)
     return {
         "changed": changed,
         "published": published,
+        "changes": changes,
         "movieCount": len(movies),
         "showCount": len(shows),
         "library": media_library_to_jsonable(movies, shows),
@@ -1583,27 +1446,6 @@ def publish_media_items(
     }
 
 
-def publish_media_catalog(
-    runtime: BotRuntime,
-    channel_id: str,
-    movies_csv: str,
-    shows_csv: str,
-    movie_filename: str,
-    show_filename: str,
-) -> dict[str, Any]:
-    movies_all = parse_media_csv(movies_csv, "movie", movie_filename) if movies_csv.strip() else []
-    shows_all = parse_media_csv(shows_csv, "show", show_filename) if shows_csv.strip() else []
-    save_media_library(runtime, movies_all, shows_all)
-    return publish_media_items(
-        runtime=runtime,
-        channel_id=channel_id,
-        movies_all=movies_all,
-        shows_all=shows_all,
-        movie_source=movie_filename or "Movies.csv",
-        show_source=show_filename or "Shows.csv",
-    )
-
-
 def media_catalog_status(runtime: BotRuntime) -> dict[str, Any]:
     state = load_state(runtime.media_state_path)
     settings = channel_settings(runtime)
@@ -1616,6 +1458,7 @@ def media_catalog_status(runtime: BotRuntime) -> dict[str, Any]:
         "showCount": state.get("show_count"),
         "format": state.get("format", "markdown"),
         "publishedAt": state.get("published_at"),
+        "changes": jellyfin_settings(runtime).get("lastChanges"),
         "library": media_library_to_jsonable(movies, shows),
     }
 
@@ -1922,9 +1765,6 @@ def make_dashboard_handler(runtime: BotRuntime, auth: DashboardAuth | None) -> t
             if path == "/api/media":
                 self.handle_media_catalog()
                 return
-            if path == "/api/media/import":
-                self.handle_media_import()
-                return
             if path == "/api/media/library":
                 self.handle_media_library()
                 return
@@ -2136,30 +1976,7 @@ def make_dashboard_handler(runtime: BotRuntime, auth: DashboardAuth | None) -> t
                     self.send_json(HTTPStatus.OK, result)
                     return
 
-                result = publish_media_catalog(
+                raise ValueError("Publish requires the saved Jellyfin media library")
-                    runtime=runtime,
-                    channel_id=str(data.get("channelId", "")),
-                    movies_csv=str(data.get("moviesCsv", "")),
-                    shows_csv=str(data.get("showsCsv", "")),
-                    movie_filename=str(data.get("movieFileName", "Movies.csv") or "Movies.csv"),
-                    show_filename=str(data.get("showFileName", "Shows.csv") or "Shows.csv"),
-                )
-            except Exception as exc:
-                self.send_json(HTTPStatus.BAD_REQUEST, {"error": str(exc)})
-                return
-
-            self.send_json(HTTPStatus.OK, result)
-
-        def handle_media_import(self) -> None:
-            try:
-                data = self.read_json()
-                result = import_media_csvs(
-                    runtime=runtime,
-                    movies_csv=str(data.get("moviesCsv", "")),
-                    shows_csv=str(data.get("showsCsv", "")),
-                    movie_filename=str(data.get("movieFileName", "Movies.csv") or "Movies.csv"),
-                    show_filename=str(data.get("showFileName", "Shows.csv") or "Shows.csv"),
-                )
             except Exception as exc:
                 self.send_json(HTTPStatus.BAD_REQUEST, {"error": str(exc)})
                 return