From ac1c983ddd100691b154fd5d96be192ba498798a Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 17 Jul 2026 16:20:38 +0200 Subject: [PATCH] feat(schema): fields the client needs to sync watch state / watchlist / prefs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The #9 schema sketched these collections before the client existed. Reconciling two devices turned up three gaps: - `meta` (json) on watch_state/watchlist. Local rows carry display fields — episode, title, cover art, MAL id, completed/dismissed — with nowhere to live server-side, so a freshly-signed-in device pulled resume points it couldn't render. - A client-owned clock. `updatedAt`/`addedAt` were autodate, i.e. stamped on server receipt, so a device that edited offline and pushed a day later beat a device that edited afterwards and synced at once. LWW needs the time the user acted. The server autodate lives on as `updated` and is now the pull cursor — that one must be server-side, or a skewed client clock would write rows behind another device's cursor and stay invisible to it. - `deletedAt` tombstones on watchlist. Hard deletes are an absence, and an absence isn't pullable — the next device to push just resurrects the title. The type change means dropping and re-adding the fields, which drops their column data, so the migration snapshots the old timestamps and writes them back. Live is believed empty, but blanking `updated` would leave rows invisible to every future pull, which is too quiet a failure to risk on an assumption. Verified against a local docker compose on both paths — a fresh install, and an upgrade over the init-only schema with rows already in it (timestamps preserved). scripts/verify.py grows the field-shape assertions plus behavioural checks: the client clock round-trips unrestamped, duplicate (profile,itemId) is rejected, and delete/re-add reuses the row instead of duplicating. 36/36 pass. Refs richiexec/myanime-app#11 --- README.md | 30 ++++- pb_migrations/1784250000_sync_fields.js | 155 ++++++++++++++++++++++++ scripts/verify.py | 66 +++++++++- 3 files changed, 245 insertions(+), 6 deletions(-) create mode 100644 pb_migrations/1784250000_sync_fields.js diff --git a/README.md b/README.md index c9b4fc8..a829ea0 100644 --- a/README.md +++ b/README.md @@ -29,9 +29,9 @@ users ─┬─* profiles ─┬─* watch_state (resume points; 1 row per pro |---------------|------|------------| | `users` | auth | `username` (unique when set), `email`, `password`, **`nsfwEnabled`** (admin-only) | | `profiles` | base | `user`→users, `name`, `avatar` (file), `isChild`, `pinHash` (hidden) | -| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `updatedAt` | -| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `addedAt` | -| `prefs` | base | `profile`→profiles, `data` (json) | +| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `meta` (json), `updatedAt` (client), `updated` (server) | +| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `meta` (json), `addedAt` (client), `updated` (server), `deletedAt` (tombstone) | +| `prefs` | base | `profile`→profiles, `data` (json), `updatedAt` (client), `updated` (server) | **Access rules.** A user only ever reads/writes their own `users` row, their own `profiles`, and rows whose `profile.user` is them. `nsfwEnabled` is never @@ -41,6 +41,30 @@ update) — only a **superuser** sets it, from the admin UI. Relations Unique indexes keep one resume/watchlist row per `(profile, itemId)` and one `prefs` row per profile. +### The two clocks (issue #11) + +Sync rows carry **two** timestamps, and they are not interchangeable: + +- **`updatedAt` / `addedAt` — set by the client.** When the user actually acted. + This is what resolves conflicts (last-write-wins). It has to be the client's, + because a device that edits offline on Monday and pushes on Friday must not + beat a device that edited on Tuesday and synced immediately. +- **`updated` — a server autodate.** When we heard about it. This is the **pull + cursor** (`filter=updated > {lastCursor}`). It has to be the server's: a device + with a skewed clock would otherwise stamp rows *behind* another device's cursor + and stay invisible to it forever. + +`itemId` encodes the client's local key — `anime::`, +`movie::0`, `tv::` — so the unique +`(profile, itemId)` index is what makes a push idempotent. `meta` is an opaque +JSON blob of the client's display fields (title, cover art, episode, flags); the +server never reads into it. + +**Deletes are soft.** A watchlist removal sets `deletedAt` rather than deleting +the row, because an absence is not something another device can pull — it would +just re-push the title and resurrect it. Re-adding clears `deletedAt` on the same +row, so a delete/re-add round-trip can never leave a duplicate. + ## Local run / verify ```bash diff --git a/pb_migrations/1784250000_sync_fields.js b/pb_migrations/1784250000_sync_fields.js new file mode 100644 index 0000000..f5f229a --- /dev/null +++ b/pb_migrations/1784250000_sync_fields.js @@ -0,0 +1,155 @@ +/// + +// Fields the client needs to actually sync watch state / watchlist / prefs +// (epic #6, issue #11). The init schema (#9) sketched the collections before the +// client existed; three things were missing once it came to reconciling two +// devices: +// +// 1. `meta` — the app's local rows carry display fields (episode, title, cover +// art, MAL id, completed/dismissed flags) with nowhere to live here. Without +// them a freshly-signed-in device pulls resume points it can't render. +// 2. A client-owned clock. `updatedAt`/`addedAt` were autodate, i.e. stamped on +// server *receipt*, so a device that edited while offline and pushed a day +// later would beat a device that edited afterwards but synced immediately. +// Last-write-wins needs the time the user acted, not the time we heard it. +// 3. Tombstones. Deleting a watchlist row outright means the next device to +// push simply re-creates it — a removal on one device rebounds from another. +// +// Both clocks now exist and mean different things: `updatedAt` (client) resolves +// conflicts, `updated` (server autodate) is the pull cursor. The cursor must be +// server-side — a device with a skewed clock would otherwise write rows that sort +// before another device's cursor and stay invisible to it forever. + +// Dropping a field drops its column data with it, so any timestamp already on a +// row is snapshotted here and written back afterwards. Against an empty +// collection this is a no-op; against a populated one it's the difference +// between a clean upgrade and blanking every timestamp. A row left with an empty +// `updated` cursor would be worse than merely wrong — it'd be invisible, skipped +// by every `updated > cursor` pull forever. +function snapshot(app, table, column) { + const rows = arrayOf(new DynamicModel({ id: "", value: "" })) + app.db() + .newQuery(`SELECT id, ${column} as value FROM ${table} WHERE ${column} != ''`) + .all(rows) + return rows +} + +// Raw SQL rather than app.save(), so the autodate hooks don't immediately +// restamp the very columns being restored. +function restore(app, table, rows, columns) { + for (const row of rows) { + const sets = columns.map((c) => `${c} = {:value}`).join(", ") + app.db() + .newQuery(`UPDATE ${table} SET ${sets} WHERE id = {:id}`) + .bind({ value: row.value, id: row.id }) + .execute() + } +} + +migrate((app) => { + // ---- watch_state ------------------------------------------------------- + const watchState = app.findCollectionByNameOrId("watch_state") + const wsStamps = snapshot(app, "watch_state", "updatedAt") + + // autodate -> client-set date. Recreated rather than mutated: PocketBase + // treats type as immutable, so the field has to be dropped and re-added. + watchState.fields.removeByName("updatedAt") + watchState.fields.add(new Field({ + type: "date", + name: "updatedAt", + required: true, + })) + watchState.fields.add(new Field({ + type: "autodate", + name: "updated", + onCreate: true, + onUpdate: true, + })) + // episode, title, coverUrl, malId, isMovie, completed, dismissedAt — whatever + // the client needs to rebuild its local row verbatim. Opaque to the server. + watchState.fields.add(new Field({ + type: "json", + name: "meta", + maxSize: 100000, + })) + app.save(watchState) + // The old autodate value is the best evidence we have of when the row last + // changed, so it seeds both clocks. + restore(app, "watch_state", wsStamps, ["updatedAt", "updated"]) + + // ---- watchlist --------------------------------------------------------- + const watchlist = app.findCollectionByNameOrId("watchlist") + const wlStamps = snapshot(app, "watchlist", "addedAt") + + // Client-set so a title migrated from local storage keeps the day it was + // actually added — the list is ordered newest-first off this. + watchlist.fields.removeByName("addedAt") + watchlist.fields.add(new Field({ + type: "date", + name: "addedAt", + required: true, + })) + watchlist.fields.add(new Field({ + type: "autodate", + name: "updated", + onCreate: true, + onUpdate: true, + })) + watchlist.fields.add(new Field({ + type: "json", + name: "meta", + maxSize: 100000, + })) + // Soft delete: removals have to be a *visible* state other devices can pull, + // not an absence. Re-adding clears this back to null; the unique + // (profile, itemId) index means that reuses the same row, never a duplicate. + watchlist.fields.add(new Field({ + type: "date", + name: "deletedAt", + })) + app.save(watchlist) + restore(app, "watchlist", wlStamps, ["addedAt", "updated"]) + + // ---- prefs ------------------------------------------------------------- + // Already has `updated` (autodate) to serve as the cursor; it just needs the + // client-owned clock to resolve two devices editing prefs while offline. + const prefs = app.findCollectionByNameOrId("prefs") + prefs.fields.add(new Field({ + type: "date", + name: "updatedAt", + required: true, + })) + app.save(prefs) + // prefs rows predate the client clock; seed it from the server's own so the + // first sync has something ordered to compare against. + restore(app, "prefs", snapshot(app, "prefs", "updated"), ["updatedAt"]) +}, (app) => { + // ---- Down migration ---------------------------------------------------- + const watchState = app.findCollectionByNameOrId("watch_state") + watchState.fields.removeByName("meta") + watchState.fields.removeByName("updated") + watchState.fields.removeByName("updatedAt") + watchState.fields.add(new Field({ + type: "autodate", + name: "updatedAt", + onCreate: true, + onUpdate: true, + })) + app.save(watchState) + + const watchlist = app.findCollectionByNameOrId("watchlist") + watchlist.fields.removeByName("deletedAt") + watchlist.fields.removeByName("meta") + watchlist.fields.removeByName("updated") + watchlist.fields.removeByName("addedAt") + watchlist.fields.add(new Field({ + type: "autodate", + name: "addedAt", + onCreate: true, + })) + app.save(watchlist) + + const prefs = app.findCollectionByNameOrId("prefs") + prefs.fields.removeByName("updatedAt") + app.save(prefs) +}) diff --git a/scripts/verify.py b/scripts/verify.py index 88d34cb..6a026cc 100644 --- a/scripts/verify.py +++ b/scripts/verify.py @@ -64,6 +64,30 @@ st, ws = req("GET", "/api/collections/watch_state", admin) check("watch_state has unique (profile,itemId) index", st, any("idx_watch_state_profile_item" in i for i in ws["indexes"])) +# --- sync fields (issue #11) --- +def field(coll, name): + return next((f for f in coll["fields"] if f["name"] == name), None) + +check("watch_state.updatedAt is a client-set date, not autodate", st, + (field(ws, "updatedAt") or {}).get("type") == "date") +check("watch_state.updated is a server autodate (pull cursor)", st, + (field(ws, "updated") or {}).get("type") == "autodate") +check("watch_state has meta json", st, + (field(ws, "meta") or {}).get("type") == "json") + +st, wl = req("GET", "/api/collections/watchlist", admin) +check("watchlist.addedAt is a client-set date, not autodate", st, + (field(wl, "addedAt") or {}).get("type") == "date") +check("watchlist.updated is a server autodate (pull cursor)", st, + (field(wl, "updated") or {}).get("type") == "autodate") +check("watchlist has meta json", st, (field(wl, "meta") or {}).get("type") == "json") +check("watchlist has deletedAt tombstone", st, + (field(wl, "deletedAt") or {}).get("type") == "date") + +st, pf = req("GET", "/api/collections/prefs", admin) +check("prefs.updatedAt is a client-set date", st, + (field(pf, "updatedAt") or {}).get("type") == "date") + # --- create two users (superuser create; nsfwEnabled set by admin) --- def make_user(email, uname): st, r = req("POST", "/api/collections/users/records", admin, { @@ -89,11 +113,46 @@ bob = login("bob@example.com") st, pa = req("POST", "/api/collections/profiles/records", alice, {"user": ua["id"], "name": "Alice Main"}) check("alice creates her own profile", st, st == 200) +CLIENT_TS = "2026-01-02 03:04:05.000Z" st, wsrow = req("POST", "/api/collections/watch_state/records", alice, - {"profile": pa["id"], "itemId": "tt0111161", "itemType": "movie", - "position": 120, "duration": 8520}) + {"profile": pa["id"], "itemId": "movie:278:0", "itemType": "movie", + "position": 120, "duration": 8520, "updatedAt": CLIENT_TS, + "meta": {"title": "The Shawshank Redemption", "episode": 0}}) check("alice creates watch_state under her profile", st, st == 200) +# The client's clock must survive the round-trip untouched — LWW is decided on +# it, so a server that restamped it would silently break conflict resolution. +check("watch_state.updatedAt round-trips the client's value", st, + wsrow.get("updatedAt", "").startswith("2026-01-02 03:04:05")) +check("watch_state.meta round-trips", st, + (wsrow.get("meta") or {}).get("title") == "The Shawshank Redemption") +check("watch_state.updated (cursor) is stamped by the server", st, + bool(wsrow.get("updated")) and not wsrow["updated"].startswith("2026-01-02")) + +# A second push of the same item must collide, not duplicate. +st, dup = req("POST", "/api/collections/watch_state/records", alice, + {"profile": pa["id"], "itemId": "movie:278:0", "itemType": "movie", + "position": 500, "duration": 8520, "updatedAt": CLIENT_TS}) +check("duplicate (profile,itemId) watch_state is rejected", st, st == 400) + +# --- watchlist tombstone: remove then re-add must reuse the row ------------- +st, wlrow = req("POST", "/api/collections/watchlist/records", alice, + {"profile": pa["id"], "itemId": "movie:278", "itemType": "movie", + "addedAt": CLIENT_TS, "meta": {"title": "Shawshank"}}) +check("alice creates watchlist row", st, st == 200) +check("watchlist.addedAt round-trips the client's value", st, + wlrow.get("addedAt", "").startswith("2026-01-02 03:04:05")) +st, r = req("PATCH", f"/api/collections/watchlist/records/{wlrow['id']}", alice, + {"deletedAt": "2026-01-03 00:00:00.000Z"}) +check("watchlist row soft-deletes", st, st == 200 and bool(r.get("deletedAt"))) +st, r = req("PATCH", f"/api/collections/watchlist/records/{wlrow['id']}", alice, + {"deletedAt": None, "addedAt": "2026-01-04 00:00:00.000Z"}) +check("re-add clears the tombstone on the same row (no duplicate)", st, + st == 200 and not r.get("deletedAt") and r["id"] == wlrow["id"]) +st, r = req("GET", "/api/collections/watchlist/records", alice) +check("exactly one watchlist row after delete+re-add", st, + st == 200 and r.get("totalItems") == 1) + # --- ISOLATION: bob must not see or touch alice's data --- st, r = req("GET", f"/api/collections/profiles/records/{pa['id']}", bob) check("bob CANNOT view alice's profile (404)", st, st == 404) @@ -118,7 +177,8 @@ check("alice's nsfwEnabled still True after attempt", st, me.get("nsfwEnabled") # --- alice cannot create watch_state under bob's (nonexistent-to-her) profile --- st, pb = req("POST", "/api/collections/profiles/records", bob, {"user": ub["id"], "name": "Bob Main"}) st, r = req("POST", "/api/collections/watch_state/records", alice, - {"profile": pb["id"], "itemId": "tt0068646", "itemType": "movie"}) + {"profile": pb["id"], "itemId": "movie:238:0", "itemType": "movie", + "updatedAt": CLIENT_TS}) check("alice CANNOT attach a row to bob's profile", st, st in (400, 403, 404)) print("\n" + ("ALL CHECKS PASSED" if allpass else "SOME CHECKS FAILED"))