Compare commits
No commits in common. "f9300ceec5d3f3005c72ae1abfa19ffc0e4f0970" and "cf299e06dd3cb9a48afca4abd11428238e3586fe" have entirely different histories.
f9300ceec5
...
cf299e06dd
3 changed files with 6 additions and 245 deletions
30
README.md
30
README.md
|
|
@ -29,9 +29,9 @@ users ─┬─* profiles ─┬─* watch_state (resume points; 1 row per pro
|
||||||
|---------------|------|------------|
|
|---------------|------|------------|
|
||||||
| `users` | auth | `username` (unique when set), `email`, `password`, **`nsfwEnabled`** (admin-only) |
|
| `users` | auth | `username` (unique when set), `email`, `password`, **`nsfwEnabled`** (admin-only) |
|
||||||
| `profiles` | base | `user`→users, `name`, `avatar` (file), `isChild`, `pinHash` (hidden) |
|
| `profiles` | base | `user`→users, `name`, `avatar` (file), `isChild`, `pinHash` (hidden) |
|
||||||
| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `meta` (json), `updatedAt` (client), `updated` (server) |
|
| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `updatedAt` |
|
||||||
| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `meta` (json), `addedAt` (client), `updated` (server), `deletedAt` (tombstone) |
|
| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `addedAt` |
|
||||||
| `prefs` | base | `profile`→profiles, `data` (json), `updatedAt` (client), `updated` (server) |
|
| `prefs` | base | `profile`→profiles, `data` (json) |
|
||||||
|
|
||||||
**Access rules.** A user only ever reads/writes their own `users` row, their own
|
**Access rules.** A user only ever reads/writes their own `users` row, their own
|
||||||
`profiles`, and rows whose `profile.user` is them. `nsfwEnabled` is never
|
`profiles`, and rows whose `profile.user` is them. `nsfwEnabled` is never
|
||||||
|
|
@ -41,30 +41,6 @@ update) — only a **superuser** sets it, from the admin UI. Relations
|
||||||
Unique indexes keep one resume/watchlist row per `(profile, itemId)` and one
|
Unique indexes keep one resume/watchlist row per `(profile, itemId)` and one
|
||||||
`prefs` row per profile.
|
`prefs` row per profile.
|
||||||
|
|
||||||
### The two clocks (issue #11)
|
|
||||||
|
|
||||||
Sync rows carry **two** timestamps, and they are not interchangeable:
|
|
||||||
|
|
||||||
- **`updatedAt` / `addedAt` — set by the client.** When the user actually acted.
|
|
||||||
This is what resolves conflicts (last-write-wins). It has to be the client's,
|
|
||||||
because a device that edits offline on Monday and pushes on Friday must not
|
|
||||||
beat a device that edited on Tuesday and synced immediately.
|
|
||||||
- **`updated` — a server autodate.** When we heard about it. This is the **pull
|
|
||||||
cursor** (`filter=updated > {lastCursor}`). It has to be the server's: a device
|
|
||||||
with a skewed clock would otherwise stamp rows *behind* another device's cursor
|
|
||||||
and stay invisible to it forever.
|
|
||||||
|
|
||||||
`itemId` encodes the client's local key — `anime:<anilistId>:<episode>`,
|
|
||||||
`movie:<tmdbId>:0`, `tv:<tmdbId>:<seasonEpisodeKey>` — so the unique
|
|
||||||
`(profile, itemId)` index is what makes a push idempotent. `meta` is an opaque
|
|
||||||
JSON blob of the client's display fields (title, cover art, episode, flags); the
|
|
||||||
server never reads into it.
|
|
||||||
|
|
||||||
**Deletes are soft.** A watchlist removal sets `deletedAt` rather than deleting
|
|
||||||
the row, because an absence is not something another device can pull — it would
|
|
||||||
just re-push the title and resurrect it. Re-adding clears `deletedAt` on the same
|
|
||||||
row, so a delete/re-add round-trip can never leave a duplicate.
|
|
||||||
|
|
||||||
## Local run / verify
|
## Local run / verify
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|
|
||||||
|
|
@ -1,155 +0,0 @@
|
||||||
/// <reference path="../pb_data/types.d.ts" />
|
|
||||||
|
|
||||||
// Fields the client needs to actually sync watch state / watchlist / prefs
|
|
||||||
// (epic #6, issue #11). The init schema (#9) sketched the collections before the
|
|
||||||
// client existed; three things were missing once it came to reconciling two
|
|
||||||
// devices:
|
|
||||||
//
|
|
||||||
// 1. `meta` — the app's local rows carry display fields (episode, title, cover
|
|
||||||
// art, MAL id, completed/dismissed flags) with nowhere to live here. Without
|
|
||||||
// them a freshly-signed-in device pulls resume points it can't render.
|
|
||||||
// 2. A client-owned clock. `updatedAt`/`addedAt` were autodate, i.e. stamped on
|
|
||||||
// server *receipt*, so a device that edited while offline and pushed a day
|
|
||||||
// later would beat a device that edited afterwards but synced immediately.
|
|
||||||
// Last-write-wins needs the time the user acted, not the time we heard it.
|
|
||||||
// 3. Tombstones. Deleting a watchlist row outright means the next device to
|
|
||||||
// push simply re-creates it — a removal on one device rebounds from another.
|
|
||||||
//
|
|
||||||
// Both clocks now exist and mean different things: `updatedAt` (client) resolves
|
|
||||||
// conflicts, `updated` (server autodate) is the pull cursor. The cursor must be
|
|
||||||
// server-side — a device with a skewed clock would otherwise write rows that sort
|
|
||||||
// before another device's cursor and stay invisible to it forever.
|
|
||||||
|
|
||||||
// Dropping a field drops its column data with it, so any timestamp already on a
|
|
||||||
// row is snapshotted here and written back afterwards. Against an empty
|
|
||||||
// collection this is a no-op; against a populated one it's the difference
|
|
||||||
// between a clean upgrade and blanking every timestamp. A row left with an empty
|
|
||||||
// `updated` cursor would be worse than merely wrong — it'd be invisible, skipped
|
|
||||||
// by every `updated > cursor` pull forever.
|
|
||||||
function snapshot(app, table, column) {
|
|
||||||
const rows = arrayOf(new DynamicModel({ id: "", value: "" }))
|
|
||||||
app.db()
|
|
||||||
.newQuery(`SELECT id, ${column} as value FROM ${table} WHERE ${column} != ''`)
|
|
||||||
.all(rows)
|
|
||||||
return rows
|
|
||||||
}
|
|
||||||
|
|
||||||
// Raw SQL rather than app.save(), so the autodate hooks don't immediately
|
|
||||||
// restamp the very columns being restored.
|
|
||||||
function restore(app, table, rows, columns) {
|
|
||||||
for (const row of rows) {
|
|
||||||
const sets = columns.map((c) => `${c} = {:value}`).join(", ")
|
|
||||||
app.db()
|
|
||||||
.newQuery(`UPDATE ${table} SET ${sets} WHERE id = {:id}`)
|
|
||||||
.bind({ value: row.value, id: row.id })
|
|
||||||
.execute()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
migrate((app) => {
|
|
||||||
// ---- watch_state -------------------------------------------------------
|
|
||||||
const watchState = app.findCollectionByNameOrId("watch_state")
|
|
||||||
const wsStamps = snapshot(app, "watch_state", "updatedAt")
|
|
||||||
|
|
||||||
// autodate -> client-set date. Recreated rather than mutated: PocketBase
|
|
||||||
// treats type as immutable, so the field has to be dropped and re-added.
|
|
||||||
watchState.fields.removeByName("updatedAt")
|
|
||||||
watchState.fields.add(new Field({
|
|
||||||
type: "date",
|
|
||||||
name: "updatedAt",
|
|
||||||
required: true,
|
|
||||||
}))
|
|
||||||
watchState.fields.add(new Field({
|
|
||||||
type: "autodate",
|
|
||||||
name: "updated",
|
|
||||||
onCreate: true,
|
|
||||||
onUpdate: true,
|
|
||||||
}))
|
|
||||||
// episode, title, coverUrl, malId, isMovie, completed, dismissedAt — whatever
|
|
||||||
// the client needs to rebuild its local row verbatim. Opaque to the server.
|
|
||||||
watchState.fields.add(new Field({
|
|
||||||
type: "json",
|
|
||||||
name: "meta",
|
|
||||||
maxSize: 100000,
|
|
||||||
}))
|
|
||||||
app.save(watchState)
|
|
||||||
// The old autodate value is the best evidence we have of when the row last
|
|
||||||
// changed, so it seeds both clocks.
|
|
||||||
restore(app, "watch_state", wsStamps, ["updatedAt", "updated"])
|
|
||||||
|
|
||||||
// ---- watchlist ---------------------------------------------------------
|
|
||||||
const watchlist = app.findCollectionByNameOrId("watchlist")
|
|
||||||
const wlStamps = snapshot(app, "watchlist", "addedAt")
|
|
||||||
|
|
||||||
// Client-set so a title migrated from local storage keeps the day it was
|
|
||||||
// actually added — the list is ordered newest-first off this.
|
|
||||||
watchlist.fields.removeByName("addedAt")
|
|
||||||
watchlist.fields.add(new Field({
|
|
||||||
type: "date",
|
|
||||||
name: "addedAt",
|
|
||||||
required: true,
|
|
||||||
}))
|
|
||||||
watchlist.fields.add(new Field({
|
|
||||||
type: "autodate",
|
|
||||||
name: "updated",
|
|
||||||
onCreate: true,
|
|
||||||
onUpdate: true,
|
|
||||||
}))
|
|
||||||
watchlist.fields.add(new Field({
|
|
||||||
type: "json",
|
|
||||||
name: "meta",
|
|
||||||
maxSize: 100000,
|
|
||||||
}))
|
|
||||||
// Soft delete: removals have to be a *visible* state other devices can pull,
|
|
||||||
// not an absence. Re-adding clears this back to null; the unique
|
|
||||||
// (profile, itemId) index means that reuses the same row, never a duplicate.
|
|
||||||
watchlist.fields.add(new Field({
|
|
||||||
type: "date",
|
|
||||||
name: "deletedAt",
|
|
||||||
}))
|
|
||||||
app.save(watchlist)
|
|
||||||
restore(app, "watchlist", wlStamps, ["addedAt", "updated"])
|
|
||||||
|
|
||||||
// ---- prefs -------------------------------------------------------------
|
|
||||||
// Already has `updated` (autodate) to serve as the cursor; it just needs the
|
|
||||||
// client-owned clock to resolve two devices editing prefs while offline.
|
|
||||||
const prefs = app.findCollectionByNameOrId("prefs")
|
|
||||||
prefs.fields.add(new Field({
|
|
||||||
type: "date",
|
|
||||||
name: "updatedAt",
|
|
||||||
required: true,
|
|
||||||
}))
|
|
||||||
app.save(prefs)
|
|
||||||
// prefs rows predate the client clock; seed it from the server's own so the
|
|
||||||
// first sync has something ordered to compare against.
|
|
||||||
restore(app, "prefs", snapshot(app, "prefs", "updated"), ["updatedAt"])
|
|
||||||
}, (app) => {
|
|
||||||
// ---- Down migration ----------------------------------------------------
|
|
||||||
const watchState = app.findCollectionByNameOrId("watch_state")
|
|
||||||
watchState.fields.removeByName("meta")
|
|
||||||
watchState.fields.removeByName("updated")
|
|
||||||
watchState.fields.removeByName("updatedAt")
|
|
||||||
watchState.fields.add(new Field({
|
|
||||||
type: "autodate",
|
|
||||||
name: "updatedAt",
|
|
||||||
onCreate: true,
|
|
||||||
onUpdate: true,
|
|
||||||
}))
|
|
||||||
app.save(watchState)
|
|
||||||
|
|
||||||
const watchlist = app.findCollectionByNameOrId("watchlist")
|
|
||||||
watchlist.fields.removeByName("deletedAt")
|
|
||||||
watchlist.fields.removeByName("meta")
|
|
||||||
watchlist.fields.removeByName("updated")
|
|
||||||
watchlist.fields.removeByName("addedAt")
|
|
||||||
watchlist.fields.add(new Field({
|
|
||||||
type: "autodate",
|
|
||||||
name: "addedAt",
|
|
||||||
onCreate: true,
|
|
||||||
}))
|
|
||||||
app.save(watchlist)
|
|
||||||
|
|
||||||
const prefs = app.findCollectionByNameOrId("prefs")
|
|
||||||
prefs.fields.removeByName("updatedAt")
|
|
||||||
app.save(prefs)
|
|
||||||
})
|
|
||||||
|
|
@ -64,30 +64,6 @@ st, ws = req("GET", "/api/collections/watch_state", admin)
|
||||||
check("watch_state has unique (profile,itemId) index", st,
|
check("watch_state has unique (profile,itemId) index", st,
|
||||||
any("idx_watch_state_profile_item" in i for i in ws["indexes"]))
|
any("idx_watch_state_profile_item" in i for i in ws["indexes"]))
|
||||||
|
|
||||||
# --- sync fields (issue #11) ---
|
|
||||||
def field(coll, name):
|
|
||||||
return next((f for f in coll["fields"] if f["name"] == name), None)
|
|
||||||
|
|
||||||
check("watch_state.updatedAt is a client-set date, not autodate", st,
|
|
||||||
(field(ws, "updatedAt") or {}).get("type") == "date")
|
|
||||||
check("watch_state.updated is a server autodate (pull cursor)", st,
|
|
||||||
(field(ws, "updated") or {}).get("type") == "autodate")
|
|
||||||
check("watch_state has meta json", st,
|
|
||||||
(field(ws, "meta") or {}).get("type") == "json")
|
|
||||||
|
|
||||||
st, wl = req("GET", "/api/collections/watchlist", admin)
|
|
||||||
check("watchlist.addedAt is a client-set date, not autodate", st,
|
|
||||||
(field(wl, "addedAt") or {}).get("type") == "date")
|
|
||||||
check("watchlist.updated is a server autodate (pull cursor)", st,
|
|
||||||
(field(wl, "updated") or {}).get("type") == "autodate")
|
|
||||||
check("watchlist has meta json", st, (field(wl, "meta") or {}).get("type") == "json")
|
|
||||||
check("watchlist has deletedAt tombstone", st,
|
|
||||||
(field(wl, "deletedAt") or {}).get("type") == "date")
|
|
||||||
|
|
||||||
st, pf = req("GET", "/api/collections/prefs", admin)
|
|
||||||
check("prefs.updatedAt is a client-set date", st,
|
|
||||||
(field(pf, "updatedAt") or {}).get("type") == "date")
|
|
||||||
|
|
||||||
# --- create two users (superuser create; nsfwEnabled set by admin) ---
|
# --- create two users (superuser create; nsfwEnabled set by admin) ---
|
||||||
def make_user(email, uname):
|
def make_user(email, uname):
|
||||||
st, r = req("POST", "/api/collections/users/records", admin, {
|
st, r = req("POST", "/api/collections/users/records", admin, {
|
||||||
|
|
@ -113,46 +89,11 @@ bob = login("bob@example.com")
|
||||||
st, pa = req("POST", "/api/collections/profiles/records", alice,
|
st, pa = req("POST", "/api/collections/profiles/records", alice,
|
||||||
{"user": ua["id"], "name": "Alice Main"})
|
{"user": ua["id"], "name": "Alice Main"})
|
||||||
check("alice creates her own profile", st, st == 200)
|
check("alice creates her own profile", st, st == 200)
|
||||||
CLIENT_TS = "2026-01-02 03:04:05.000Z"
|
|
||||||
st, wsrow = req("POST", "/api/collections/watch_state/records", alice,
|
st, wsrow = req("POST", "/api/collections/watch_state/records", alice,
|
||||||
{"profile": pa["id"], "itemId": "movie:278:0", "itemType": "movie",
|
{"profile": pa["id"], "itemId": "tt0111161", "itemType": "movie",
|
||||||
"position": 120, "duration": 8520, "updatedAt": CLIENT_TS,
|
"position": 120, "duration": 8520})
|
||||||
"meta": {"title": "The Shawshank Redemption", "episode": 0}})
|
|
||||||
check("alice creates watch_state under her profile", st, st == 200)
|
check("alice creates watch_state under her profile", st, st == 200)
|
||||||
|
|
||||||
# The client's clock must survive the round-trip untouched — LWW is decided on
|
|
||||||
# it, so a server that restamped it would silently break conflict resolution.
|
|
||||||
check("watch_state.updatedAt round-trips the client's value", st,
|
|
||||||
wsrow.get("updatedAt", "").startswith("2026-01-02 03:04:05"))
|
|
||||||
check("watch_state.meta round-trips", st,
|
|
||||||
(wsrow.get("meta") or {}).get("title") == "The Shawshank Redemption")
|
|
||||||
check("watch_state.updated (cursor) is stamped by the server", st,
|
|
||||||
bool(wsrow.get("updated")) and not wsrow["updated"].startswith("2026-01-02"))
|
|
||||||
|
|
||||||
# A second push of the same item must collide, not duplicate.
|
|
||||||
st, dup = req("POST", "/api/collections/watch_state/records", alice,
|
|
||||||
{"profile": pa["id"], "itemId": "movie:278:0", "itemType": "movie",
|
|
||||||
"position": 500, "duration": 8520, "updatedAt": CLIENT_TS})
|
|
||||||
check("duplicate (profile,itemId) watch_state is rejected", st, st == 400)
|
|
||||||
|
|
||||||
# --- watchlist tombstone: remove then re-add must reuse the row -------------
|
|
||||||
st, wlrow = req("POST", "/api/collections/watchlist/records", alice,
|
|
||||||
{"profile": pa["id"], "itemId": "movie:278", "itemType": "movie",
|
|
||||||
"addedAt": CLIENT_TS, "meta": {"title": "Shawshank"}})
|
|
||||||
check("alice creates watchlist row", st, st == 200)
|
|
||||||
check("watchlist.addedAt round-trips the client's value", st,
|
|
||||||
wlrow.get("addedAt", "").startswith("2026-01-02 03:04:05"))
|
|
||||||
st, r = req("PATCH", f"/api/collections/watchlist/records/{wlrow['id']}", alice,
|
|
||||||
{"deletedAt": "2026-01-03 00:00:00.000Z"})
|
|
||||||
check("watchlist row soft-deletes", st, st == 200 and bool(r.get("deletedAt")))
|
|
||||||
st, r = req("PATCH", f"/api/collections/watchlist/records/{wlrow['id']}", alice,
|
|
||||||
{"deletedAt": None, "addedAt": "2026-01-04 00:00:00.000Z"})
|
|
||||||
check("re-add clears the tombstone on the same row (no duplicate)", st,
|
|
||||||
st == 200 and not r.get("deletedAt") and r["id"] == wlrow["id"])
|
|
||||||
st, r = req("GET", "/api/collections/watchlist/records", alice)
|
|
||||||
check("exactly one watchlist row after delete+re-add", st,
|
|
||||||
st == 200 and r.get("totalItems") == 1)
|
|
||||||
|
|
||||||
# --- ISOLATION: bob must not see or touch alice's data ---
|
# --- ISOLATION: bob must not see or touch alice's data ---
|
||||||
st, r = req("GET", f"/api/collections/profiles/records/{pa['id']}", bob)
|
st, r = req("GET", f"/api/collections/profiles/records/{pa['id']}", bob)
|
||||||
check("bob CANNOT view alice's profile (404)", st, st == 404)
|
check("bob CANNOT view alice's profile (404)", st, st == 404)
|
||||||
|
|
@ -177,8 +118,7 @@ check("alice's nsfwEnabled still True after attempt", st, me.get("nsfwEnabled")
|
||||||
# --- alice cannot create watch_state under bob's (nonexistent-to-her) profile ---
|
# --- alice cannot create watch_state under bob's (nonexistent-to-her) profile ---
|
||||||
st, pb = req("POST", "/api/collections/profiles/records", bob, {"user": ub["id"], "name": "Bob Main"})
|
st, pb = req("POST", "/api/collections/profiles/records", bob, {"user": ub["id"], "name": "Bob Main"})
|
||||||
st, r = req("POST", "/api/collections/watch_state/records", alice,
|
st, r = req("POST", "/api/collections/watch_state/records", alice,
|
||||||
{"profile": pb["id"], "itemId": "movie:238:0", "itemType": "movie",
|
{"profile": pb["id"], "itemId": "tt0068646", "itemType": "movie"})
|
||||||
"updatedAt": CLIENT_TS})
|
|
||||||
check("alice CANNOT attach a row to bob's profile", st, st in (400, 403, 404))
|
check("alice CANNOT attach a row to bob's profile", st, st in (400, 403, 404))
|
||||||
|
|
||||||
print("\n" + ("ALL CHECKS PASSED" if allpass else "SOME CHECKS FAILED"))
|
print("\n" + ("ALL CHECKS PASSED" if allpass else "SOME CHECKS FAILED"))
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue