From fe53b7cbef4ff02a904a4ba2007041201cee7329 Mon Sep 17 00:00:00 2001 From: Natalie Date: Tue, 9 Jun 2026 05:50:02 -0700 Subject: [PATCH] chore: build runner, gitignore, project manifest, objectives + history docs --- .gitignore | 23 +-- .../20260608_fleet-manager-mesh-design.md | 105 ++++++++++ .project/objectives.md | 183 ++++++++++++++++++ build-install.sh | 11 +- project.yml | 3 + run | 43 ++++ 6 files changed, 352 insertions(+), 16 deletions(-) create mode 100644 .project/objectives.md create mode 100755 run diff --git a/.gitignore b/.gitignore index 64f8b66..9e6b9cb 100644 --- a/.gitignore +++ b/.gitignore @@ -13,20 +13,21 @@ build/ *.xcuserstate xcuserdata/ -# Dependencies (never belongs in git) -node_modules/ +# Dependencies + build artifacts of the in-repo subsystems (governor, mcp, search, +# recommender) — the SOURCE of these trees is tracked; only their generated output +# is ignored. (They were formerly separate repos nested + ignored here; now folded +# into this single repo — see .project/history/20260608_*.) +**/node_modules/ +**/.venv/ +**/__pycache__/ +**/.pytest_cache/ +recommender/data/ +recommender/out/ +search/torrents/ +**/*.log # macOS .DS_Store -# Foreign project trees that physically sit in this directory but belong to their -# OWN repos (portable-net-tv governor, plum-control-mcp, media-recommender, fleet -# design) — never part of the TVAnarchy app repo. Ignored so the auto-commit -# service can't accidentally sweep them into tv-anarchy. -governor/ -mcp/ -fleet/ -recommender/ - # Agent session state + git worktrees .claude/ diff --git a/.project/history/20260608_fleet-manager-mesh-design.md b/.project/history/20260608_fleet-manager-mesh-design.md index 06356ca..e5675c6 100644 --- a/.project/history/20260608_fleet-manager-mesh-design.md +++ b/.project/history/20260608_fleet-manager-mesh-design.md @@ -251,3 +251,108 @@ Discord planes (keep separate): Watch parties: synchronized LOCAL playback (Discord carries only voice + timestamps), never stream copyrighted video through Discord. + +--- + +# Networking — two independent planes (added 2026-06-09) + +Two VPN/overlay concerns that are **separate systems on separate interfaces** and must +never be conflated or allowed to collide. Plane 1 connects *your own devices to each +other*; plane 2 is *how a public-swarm-facing node leaves the building*. + +``` + fleet WireGuard (tva0) commercial VPN (.ovpn → tun) + plum ───────────────┐ ┌────────────────────► public swarms + black ───── internal mesh, scoped ──── always-on node ───┤ (only public_swarm_face egress) + phone ───────────────┘ AllowedIPs = fleet subnet only └────────────────────► + every device gets ONE stable fleet IP, home IP stays dark + reachable home OR away (off-home exit) +``` + +## Plane 1 — Fleet WireGuard (internal mesh): auto-provisioned, tv-anarchy-only, collision-proof + +**Why foundational, not stage-4.** Today the app juggles black's home-LAN address +(`10.0.0.11`) vs an overlay address (`10.9.0.4`) and breaks when a device is away +(see [[black-endpoint-lan-vpn]] — the LAN→VPN transmission fallback shipped 2026-06-09 +is the *interim* stopgap). A fleet WG fabric gives every device **one stable fleet IP, +reachable home or away**, which collapses `devices.json` to a single IP per device and +removes the juggling entirely. So WG-fabric is promoted to an **early foundational +stage** (before friend-mesh), even though cross-fleet F2F still lands at stage 4. + +**Split brain: the package decides, a thin per-platform actuator brings the interface up.** +The fleet package owns all logic — generates each device's Curve25519 keypair (REUSED as +the device's mesh identity, per the anchor-trust model), assigns a stable fleet IP, +distributes the peer list via the broadcast/rendezvous anchor (the public-IP always-on +node), and writes the interface config. Raising the interface is per-OS: +- **Linux** (black, apricot, seedbox): `wg-quick` + a `systemd` unit, root, fully + scriptable end-to-end. Buildable first. +- **macOS** (plum): no kernel WireGuard — wireguard-go over a named `utun`. The app + shells `wg-quick up` **under sudo** (same precedent as the mpv `"sudo": true` path in + `devices.json`); a sandboxed app cannot otherwise raise an interface. The design states + this explicitly rather than assuming `wg-quick up` "just works". +- **iOS** (phone): auto-provisioning is not possible without the WireGuard app + manual + import/MDM. Phone is **manual-config / out of the auto-path** — it's a pure consumer + sink anyway and never holds a duty. + +**tv-anarchy-only — a scoped mesh, NOT a VPN.** `AllowedIPs` = the **fleet subnet only** +(never `0.0.0.0/0`), `Table=off`, no default route, no DNS takeover. Only traffic *to +fleet peers* enters the tunnel; the user's general internet and any other VPN are +untouched. + +**"Never conflict" is a runtime probe, not a lucky default** (the user said *never*): +at bring-up the actuator enumerates existing routes/interfaces (`netstat -rn` on macOS / +`ip route` + `ip link` on Linux), picks a fleet subnet that overlaps **nothing** present +(then persists it for stability), auto-picks a free UDP listen port (51820 may be the +user's other WG), and uses a dedicated interface name (`tva0`). It refuses to clobber an +existing interface/route and shifts its own subnet instead. + +**Single-fleet trust bootstrap is trivial** — the user owns every device, so NO M-of-N +anchor co-signing (that's the cross-fleet stage-4 problem). Keys are minted and trusted +locally within the one fleet. + +**Open question (blocks finalizing this plane):** what is `10.9.0.4` today — a +general-purpose VPN/WG the user runs for *other* infra (→ fleet WG is purely additive and +must never route through it), or just an ad-hoc tv-anarchy overlay (→ fleet WG *replaces* +it; a migration)? The answer changes whether plane 1 coexists-with or supersedes the +current overlay. + +## Plane 2 — Commercial VPN via uploaded `.ovpn` (the public-swarm exit) + +This is the "+ VPN on the public-swarm leg" from the privacy note (line 28) and the +mechanism behind the `public_swarm_face` duty (a node — seedbox FIRST, never a consumer, +prefer `!on_home_ip` — is the only one touching public swarms, keeping the rest of the +fleet dark). + +**App setting — VPN configs (import + manage):** +- **Import** individual `.ovpn` files OR a `.zip` bundle. Providers (Mullvad, PIA, AirVPN, + ProtonVPN, …) ship a **zip of per-server `.ovpn` files** (one per location); on zip + import, unpack every `.ovpn` entry (recurse nested folders), ignore non-ovpn files. +- **Parse** each config for: display name (filename, or a `#`-comment / the `remote` + host), `remote` endpoint(s) + `proto` (udp/tcp), and whether certs/keys are **inline** + (`…`, ``, ``, ``) or **sidecar files** shared across the + bundle (a zip often carries one `ca.crt` + an `auth-user-pass` reference for all + servers). Keep sidecars together with the configs that reference them. +- **Credentials** (`auth-user-pass`): a secure username/password field stored in the + **Keychain**, never written beside the config in plaintext. +- **Manage:** a single managed dir (`~/.config/tv-anarchy/vpn/`); list / rename / group by + provider / delete; pick the active config(s) for the device(s) holding + `public_swarm_face`. Show parsed endpoint/proto/location so the user picks sensibly. + +**Actuation (where it runs):** `public_swarm_face` lives on an always-on, ideally +off-home node (seedbox / black — Linux), so OpenVPN bring-up is a Linux concern +(`openvpn` + systemd, root). The egress is **split-routed** — only the torrent client's +public-swarm traffic leaves through the `tun` interface (policy routing / a network +namespace on Linux), so it is **not** a full-device VPN and the box stays reachable on +its normal + fleet interfaces. + +## Coexistence — the two planes run simultaneously without collision + +A `public_swarm_face` node runs **both** at once: fleet WireGuard on `tva0` (internal +mesh, scoped `AllowedIPs`) AND a commercial-VPN `tun` (public-swarm egress, split-routed). +They occupy different interfaces and different routing scopes: +- fleet-peer traffic → `tva0` (WG) +- public-swarm torrent egress → `tun` (.ovpn) +- everything else → the box's normal route +No interface, subnet, port, or default-route overlap — by the runtime collision-probe +(plane 1) + split-routing (plane 2). This is the structural guarantee behind the user's +"never conflict with other wg or vpn". diff --git a/.project/objectives.md b/.project/objectives.md new file mode 100644 index 0000000..23b31b7 --- /dev/null +++ b/.project/objectives.md @@ -0,0 +1,183 @@ +# TV Anarchy — Objectives (living backlog) + +Running list of agreed objectives. Status: ✅ done (built+tested+staged) · 🟡 in +progress · ⬜ planned · 💭 designed (fleet package, future). Full specs in +`~/.claude/plans/woolly-tumbling-pelican.md`; fleet design in `.project/history/`. + +## ✅ Done — built, tested, staged (green) +- **Consolidate** governor/mcp/recommender/search into the single repo; repoint via + `RepoPaths`; in-repo `mcp→search` verified returning live magnets. (Part A) +- **Hosts → Devices** rename + 5 device types — cellphone · laptop · storage · seed · + **Broadcast Station** — each with an overridable `DeviceServices` set (stream · + offlineCache · ttlSeed · custody · publicSwarmFace · f2fRelay · meshAnchor), + mirroring the fleet duties; legacy `hosts.json` infers type from kind. (Part B1) +- **Offline cache** — pull next-Y-episodes-of-recent-Z-shows to local disk + (continue-watching / recently-added rules), rsync from black, capped. (Part B2) +- **Media-control forwarding** — Mac media keys / Control Center / lock screen / + AirPods drive the active device + Now Playing; `forwardMediaKeys` setting. (Part B3) +- **Unified playlist** — playing a series episode queues the rest of the show from + there (resume the first, rest from 0). **Continue Watching + Home rails now route + through it too** (was the "S3 didn't queue after S2 Daria" bug — those paths + single-launched one episode and queued nothing). +- **Season 0 = specials/movies**, ordered LAST (Daria's two movies after the run); + shown as "Specials & Movies"; the unified queue puts them at the end. +- **Continue Watching rethink** — was severely broken (one row per *watched episode*, + never advanced, junk `:Zone.Identifier` rows). Now **per-show**, pointing at the + **next** episode (library-ordered → crosses season boundaries, specials last), + most-recent **sustained** play wins (fly-by filter, mirrors the governor), finished + shows drop off, junk filtered. +- **Watch-state core + UI** — `WatchState` (unwatched/inProgress/watched) + + `nextUnwatched` + `isWatched`; **watched badges** on the Library grid + the + **thumbnail-tap = "watch next"** / **title-tap = open detail** split (movie plays, + series plays its next unwatched episode and queues the rest). *(Still pending: + rewatch button = reset watch state; searchable top tags; indicators on the other + list formats — Home rails, detail.)* + +## ⬜ Planned — specified, not yet built +- **Download manager (Part E):** `TransferHealth` ✅ → smart sort (attention + ETA) ✅ + (`downloadingSorted` + "needs attention" count + health pills) → ✅ system+in-app + surfacing (ready-to-watch / stuck) → ✅ **per-torrent debug detail + + reannounce/verify/pause actions** (mcp `tx-detail`/`tx-reannounce`/`tx-verify`/ + `tx-start`/`tx-stop`; `TorrentDetail`/`TrackerStat` models; `DownloadsController` + detail+act; "Inspect / fix…" sheet showing error/peers/per-tracker announce + + Reannounce/Verify/Pause-Resume — verified live against black) → ⬜ dead-torrent + re-search swap ("Find a better release": search title → rank by seeders → swap). +- ✅ **Settings UI** — media keys, offline (Y/Z/source), combine + local-LLM, hover + previews, adult, all in the (renamed) **Settings** pane, one-persist on change. +- ✅ **Sidebar subnav** — Library → category links, auto-collapsed (shown only while + Library is the active nav). +- **Search results as collections (Part E):** per-title summary cards (not flat rows), + anchored to library/TMDB; collection page with **resolution selector** — + movie = pick resolution → auto-pick best-seeded (one tap); series = **resolution × + season matrix** showing owned/available/downloading. +- **Search robustness (Part E):** launch-time warmup, per-source partial results, + dependency health row, retry-after-warmup, prune dead sources. +- **Settings hub (Part F):** summary/home page + dedicated pages; everything + toggleable/tuneable via `AppSettings`; **device editing lives in Settings** + (Devices view deep-links to it). +- **Auto-preview pre-warm (Part F):** generate hover previews ahead of time for + likely-previewed content — rules: recently-added · visible-on-screen · + continue-watching · recommended; capped per pass, low-priority. +- **Watch-state + card interaction (new):** + - **Watched indicators** in every list format (grid/rows/collection) — unwatched / + in-progress / watched. + - **Card tap split:** tapping the **thumbnail** = "**watch next**" (play the next + unwatched episode straight away); tapping the **text/title** area = open the + detail page (breadcrumb). Two targets on one card. + - **Rewatch button** per collection/item — resets watch state so "watch next" + starts over. + - **Searchable top tags** — surface top tags/genres as chips that search on tap. +- **Dynamic show combination (new):** merge the split/duplicate library entries of + one show into a single combined show/collection. Today `mergeSeriesByName` only + merges *exact normalized name + same category + disjoint seasons*, so e.g. + **Dandadan** stays as 3–4 entries: spacing variants ("DAN DA DAN" vs "DANDADAN"), + cross-category (anime vs cartoons), and duplicate S1 releases (REPACK/Remux/BDRip) + whose overlapping seasons make the merge treat them as different shows. **Rethink: + metadata-anchored grouping** — resolve each entry to a canonical TMDB/enrich work + (title+year+id); same work → combine + dedup episodes per season×episode (alt + releases reachable via the quality switcher); different work → stay separate (so an + anime + its live-action remake don't fuse). Same anchor model as search collections; + feeds the shared catalog's "media definitions." + - ✅ **Cheap stage + seam built:** `ShowGrouping.candidateClusters` blocks variants + by a canonical key (alphanumeric — "DAN DA DAN"/"DANDADAN"/"Dan.Da.Dan" → one + cluster); `ambiguousClusters` is the cheap→reasoner hand-off; the `ShowGrouper` + seam (provider-agnostic: local LLM / TMDB) makes the final same-work call. Two- + stage per the design: **easy algorithm for the bulk + LLM light reasoning on the + ambiguous tail** (the same role the unfilled `TitleRefiner` MLX seam was built for). + - ✅ **Local-LLM reasoner BUILT + wired (no TMDB):** installed **MLX** + (`mlx-lm` + `Qwen2.5-1.5B-Instruct-4bit`) into the recommender uv env — plum now + has a working local LLM. `recommender/media_rec/grouper.py` reasons over a + candidate cluster (correctly groups Dandadan's cross-category dupes); Swift + `LocalLLMGrouper` shells into it like `enrich`; `ShowGroupCache`/ + `CachedGroupDecider` persist decisions (once per cluster); + `ShowGrouping.combine`+`merge` apply them (dedup episodes per season×episode, a + year-gap guard against a small model over-merging a remake); + `LibraryController.combineSplitShows()` runs it off-main after load/scan, gated by + the `combineSplitShows` setting. + - ✅ **Shipped default is now deterministic (zero MB), LLM optional.** The measured + 1.5B model is 839 MB — too big to ship (no git-lfs; repo .git is 2.5 MB). The + same-work signal is *structural*: `DeterministicGrouper` combines same-canonical- + key entries with close release years (dupes + split seasons) and excludes + year-distant remakes — no model. It's the default in `combineSplitShows`; the MLX + `LocalLLMGrouper` is gated behind `useLLMGrouper` (off) for the rare same-year + ambiguous tail. *Future tiny-ship option:* a Model2Vec static embedding (~30 MB, + CPU-instant) for fuzzy title recall ("Frieren" ≈ "Sousou no Frieren") — beats a + generative model for this task and is genuinely shippable. +- **Sidebar subnav (Part G):** Library → category links, Settings → its pages, with + auto-collapse per nav. +- **Cleanup (Part C, LAST):** remove the stale worktrees + delete the external repos + (plum-control-mcp, portable-net-tv, media-recommender, torrent-search-mcp) once the + port is verified. Nothing irreversible until then. + +## 💭 Designed — fleet PACKAGE (future, not app core) +> Keep the mesh *engine* out of `TVAnarchyCore`; the app consumes its outputs. Lives +> in `fleet/` (package). +- **Networking — two planes** (spec: `.project/history/20260608_fleet-manager-mesh-design.md` + → "Networking — two independent planes"). Separate interfaces, never conflated: + - **Plane 1 — Fleet WireGuard (internal mesh):** auto-provisioned (package mints keys/ + subnet/peers, thin per-OS actuator raises the interface — Linux `wg-quick`+systemd; + macOS `wg-quick` under sudo over a utun; iOS manual/out-of-path), **tv-anarchy-only** + (`AllowedIPs`=fleet subnet, `Table=off`, no default route — a scoped mesh, NOT a VPN), + **never-conflict via a runtime route/interface/port collision-probe** + dedicated + `tva0` name. Promoted to an **early foundational stage** because one stable fleet IP + per device (home-or-away) kills today's LAN-vs-VPN juggling ([[black-endpoint-lan-vpn]]; + the transmission LAN→VPN fallback is the interim). Single-fleet trust = trivial (own + all devices, no M-of-N). *Open Q: is `10.9.0.4` general infra (additive) or just the + tv-anarchy overlay (migration)?* + - **Plane 2 — Commercial VPN via uploaded `.ovpn` (public-swarm exit):** the + `public_swarm_face` egress that keeps the home IP dark. + - ✅ **Import + manage UI BUILT** (Settings → "VPN (public-swarm exit)"): import + individual `.ovpn` OR a provider **zip** (`ditto` unpack → harvest every `.ovpn` + + its sidecar certs → group by zip name in `~/.config/tv-anarchy/vpn/`), + `OVPNParser` surfaces remote/proto/needs-login/inline-vs-external-certs, + per-provider login stored in the **Keychain** (`VPNCredentialStore`, never on + disk), list/group/delete. `VPNController` + `VPNConfigStore`; parser/scan/zip + tested (`OVPNParserTests`, 8). Management only — no actuation yet. + - ⬜ **Select active config per `public_swarm_face` device** + **actuation** = + OpenVPN on the always-on Linux node, **split-routed** (only public-swarm torrent + egress through `tun`, not a full-device VPN). Coexists with plane-1 WG on a + separate interface/route scope. (Needs the fleet duty engine.) +- **Mesh registry + duty assignment** — `peers_for(infohash)` / `custodians_of(title)`, + auto-built from activity (watch = auto-seed, completion = custody). +- **Shared catalog / "public data drive"** — registry lists + media/collection + definitions (e.g. "The Matrix collection") + optional cover art; **written only by + mesh anchors (Broadcast Stations), read by all, distributed AS a torrent**; a + signed, **versioned** catalog pointer. +- **Anchor trust** — M-of-N existing anchors co-sign a new writer's key (vouching); + reuse WireGuard peer identity; signed revocation list; founder-key bootstrap. +- **Custody floor + reaper** — ≥N copies per wanted title; re-pin before the last + copy vanishes; reaper resurrects dead titles from the mesh first, public re-search + fallback. (System-level "improve health".) +- **Cover-art sharing** across the mesh (a payload of the shared catalog). +- **Friend subscription + mutual promotion** — A invites B, B accepts → a bidirectional, + **signed, revocable** `Friendship` (on the WireGuard-key identity + Discord) carrying + a per-pair share/promote policy: serve each other custody/streaming (bandwidth tier 2), + union each other into `peers_for`, relay each other's F2F requests, and boost each + other's content in discovery/recs. Either side revokes. +- **Anonymized adult sharing** — help friends watch porn you hold *without it being + attributable to you*: serve it **content-addressed (by `ContentID`/infohash, not "my + files")**, **via F2F relay** (source identity stripped hop-by-hop), with a **raised + k-anonymity threshold** (never surfaced below K distinct holders). The friend gets the + bytes by hash; the mesh never reveals *who* has it. A per-`porn` sharing mode on top + of the data boundaries. +- ✅ **Content hashing (`ContentID`) — BUILT.** Canonical, library-agnostic episode id + (`canonicalKey(work)/sNNeNN`, quality opt-in) so the same episode across libraries + (different rips/folders) normalizes to one id; + a SHA-256 short digest that leaks no + title. The keystone for dedup, `peers_for(id)`, k-anonymity counting, and the + anonymized content-addressed serving above. (Byte-exact swarm identity stays the + torrent infohash at the transmission layer.) +- **Data boundaries + bandwidth tiers** ✅ pure models built (`DataDomain`: + tvanarchy/user/publicMedia + shareability; `BandwidthPolicy`: you > friends-when-idle + > public, Travel-Mode = all-to-you). ⬜ Actuation: mcp upload-control verbs + (`transmission-remote -u/-U`, per-torrent), laptop magnet-add for swarm augmentation, + governor enforcement. + +## 💭 For later — recommendation engine +- **"Collection ideas like this"** — per-collection "more like this" recommendations + (recommender / media-recommender). Surfaced from a collection item. + +## Standing directives +- Keep this list current as objectives are added/finished. +- Keep building while waiting for responses; don't stall on questions. +- Separate core app code from ideas that should be packages. +- Thoroughly test each feature; keep the tree green at every boundary. diff --git a/build-install.sh b/build-install.sh index 0d72005..71e6a09 100755 --- a/build-install.sh +++ b/build-install.sh @@ -1,14 +1,15 @@ #!/usr/bin/env bash -# Build TVAnarchy (Release) and install it to ~/Applications, so the running app -# and the built app can't silently drift. The one irreducible manual step is -# quitting + relaunching — you can't hot-swap a running native app — and the -# sidebar build stamp (v · ·