Claude
c3635f0e7c
The nightly trimClientLogs cron fetched one page of 500 records older than 14 days and deleted those, while about 1275 rows arrive a day. Measured on production (read-only) on 2026-09-13: 30213 rows, 14420 of them past the cutoff, the oldest from 2026-08-19, so the backlog grew by roughly 775 rows a day and the documented 14-day retention never held. The job now deletes in batches of 2000 with plain SQL until a batch comes back short, so one run clears everything past the cutoff. Plain SQL instead of $app.delete() per record is safe here: client_logs has no file fields, no collection has a relation pointing at it and no hook watches its deletes, so the record path would only add a transaction and a cascade lookup per row. Batches keep each write lock short. The run logs "trimClientLogs" with the deleted count and the cutoff to PocketBase's logs, and a failure is logged instead of swallowed silently as before. scripts/test_client_logs_trim.py starts a throwaway PocketBase 0.39.6 with this repo's migrations and hooks, seeds stale and fresh rows, fires the job through POST /api/crons/trimClientLogs and checks what is left. Against the old hook it failed: 800 of 1300 stale rows left, no log line. With the fix: 0 of 1300 left, all 40 in-window rows kept (one just inside the cutoff), and 0 of 15000 stale rows left in 0.4 s at production scale. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
179 lines
9.6 KiB
Markdown
179 lines
9.6 KiB
Markdown
# myanime-backend
|
|
|
|
**PocketBase** backend for the myanime app — auth, database, realtime, file
|
|
storage and admin UI in a single Go binary. This is the foundation the account
|
|
features build on: cloud sync, codeless device sign-in, profiles, child
|
|
profiles, recommendations, and gated auto-update (epic
|
|
[#6](https://projects.petruzalekr.cz/richiexec/myanime-app/issues/6)).
|
|
|
|
- **No E2E.** TLS in transit + at-rest on disk. The server is deliberately
|
|
readable so recommendations, the admin `nsfwEnabled` flag, and painless
|
|
password reset all work.
|
|
- **Schema as code.** The collections live in [`pb_migrations/`](pb_migrations/)
|
|
and auto-apply on boot — never hand-click the schema in the admin UI, or the
|
|
next deploy will drift.
|
|
- **Version-pinned.** The [`Dockerfile`](Dockerfile) downloads the official
|
|
PocketBase **v0.39.6** static binary from GitHub releases (no third-party
|
|
image). Bump `PB_VERSION` there and in `docker-compose.yml` to upgrade.
|
|
|
|
## Data model
|
|
|
|
```
|
|
users ─┬─* profiles ─┬─* watch_state (resume points; 1 row per profile+item)
|
|
(auth)│ ├─* watchlist (1 row per profile+item)
|
|
│ ├─1 prefs (playback/audio/subtitle blob)
|
|
│ └─1 addon_config (client-encrypted addon blob; ciphertext only)
|
|
├─* device_auth (codeless sign-in requests / approved devices, issue #12)
|
|
└ nsfwEnabled (admin-only bool → gates adult vs clean build, issue #16)
|
|
```
|
|
|
|
| Collection | Type | Key fields |
|
|
|----------------|------|------------|
|
|
| `users` | auth | `username` (unique when set), `email`, `password`, **`nsfwEnabled`** (admin-only) |
|
|
| `profiles` | base | `user`→users, `name`, `avatar` (file), `isChild`, `pinHash` (hidden) |
|
|
| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `meta` (json), `updatedAt` (client), `updated` (server) |
|
|
| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `meta` (json), `addedAt` (client), `updated` (server), `deletedAt` (tombstone) |
|
|
| `prefs` | base | `profile`→profiles, `data` (json), `updatedAt` (client), `updated` (server) |
|
|
| `addon_config` | base | `profile`→profiles, `blob` (**ciphertext**), `salt`, `kdf`, `updatedAt` (client), `updated` (server) |
|
|
| `device_auth` | base | `code` (unique), `deviceName`, `status`, `user`→users, `expiresAt`, + hidden `authToken`/`keyCiphertext`/`pollSecret`/`devicePubKey` — codeless device sign-in (#12) |
|
|
|
|
**Access rules.** A user only ever reads/writes their own `users` row, their own
|
|
`profiles`, and rows whose `profile.user` is them. `nsfwEnabled` is never
|
|
accepted from a client (`@request.body.nsfwEnabled:isset = false` on create and
|
|
update) — only a **superuser** sets it, from the admin UI. Relations
|
|
`cascadeDelete`, so deleting a user removes their profiles and all child rows.
|
|
Unique indexes keep one resume/watchlist row per `(profile, itemId)` and one
|
|
`prefs`/`addon_config` row per profile.
|
|
|
|
### Encrypted addon config (issue #20)
|
|
|
|
The `addon_config` blob is the one piece of account data that is **not** readable
|
|
by the server. The addon "tokens" are base64'd *plaintext credentials* (prehraj.to
|
|
login, TorBox API key) plus the TMDB key, so — unlike watch history — they are
|
|
encrypted **client-side** before upload. The server stores only:
|
|
|
|
- `blob` — AES-GCM ciphertext (nonce ‖ ciphertext ‖ tag), base64; opaque here.
|
|
- `salt` — the per-account KDF salt (base64). Not secret; a salt never is. It
|
|
lives server-side so a second device can derive the **same** key from the
|
|
user's password. The key itself is never sent and never leaves the device.
|
|
- `kdf` — the key-derivation descriptor (e.g. `pbkdf2-sha256-210000`).
|
|
|
|
Because the key is derived from the password, a **password reset** (done on
|
|
PocketBase's hosted page, off-device) leaves the blob undecryptable — expected and
|
|
acceptable for re-enterable addon config, and the client detects it, clears it, and
|
|
prompts a re-entry. A normal in-app *change password* re-encrypts the blob, so only
|
|
reset loses it. This does **not** change the no-E2E stance for the rest of the data.
|
|
|
|
### Codeless device sign-in (issue #12)
|
|
|
|
A new device (a TV) signs into an account with **no password typing and no code
|
|
transcription**: it shows a QR + short code, and an already-logged-in phone/PC
|
|
approves it from `GET /device`. On approval the TV gets a minted session token and
|
|
— **end-to-end, the server never sees it** — the addon-config vault key, so it can
|
|
decrypt the `addon_config` blob it then pulls. This lives in the `device_auth`
|
|
collection (`pb_migrations/1786000000_device_auth.js`) plus two JS hooks
|
|
(`pb_hooks/device_auth.pb.js` routes + cron, `pb_hooks/device_page.pb.js` approve
|
|
page). All create/poll/approve logic is in the hooks (the TV is unauthenticated
|
|
when it requests and polls); the collection exposes only the owner-scoped device
|
|
**list + revoke**. **JWT revocation caveat:** deleting a device row removes it from
|
|
the list but can't invalidate a token already minted (stateless JWTs) — a
|
|
documented v1 limitation.
|
|
|
|
**Full contract** (endpoints, record shape, the exact crypto — PBKDF2 params +
|
|
the X25519/HKDF/AES-GCM key transfer): [`docs/device-auth-contract.md`](docs/device-auth-contract.md).
|
|
|
|
### The two clocks (issue #11)
|
|
|
|
Sync rows carry **two** timestamps, and they are not interchangeable:
|
|
|
|
- **`updatedAt` / `addedAt` — set by the client.** When the user actually acted.
|
|
This is what resolves conflicts (last-write-wins). It has to be the client's,
|
|
because a device that edits offline on Monday and pushes on Friday must not
|
|
beat a device that edited on Tuesday and synced immediately.
|
|
- **`updated` — a server autodate.** When we heard about it. This is the **pull
|
|
cursor** (`filter=updated > {lastCursor}`). It has to be the server's: a device
|
|
with a skewed clock would otherwise stamp rows *behind* another device's cursor
|
|
and stay invisible to it forever.
|
|
|
|
`itemId` encodes the client's local key — `anime:<anilistId>:<episode>`,
|
|
`movie:<tmdbId>:0`, `tv:<tmdbId>:<seasonEpisodeKey>` — so the unique
|
|
`(profile, itemId)` index is what makes a push idempotent. `meta` is an opaque
|
|
JSON blob of the client's display fields (title, cover art, episode, flags); the
|
|
server never reads into it.
|
|
|
|
**Deletes are soft.** A watchlist removal sets `deletedAt` rather than deleting
|
|
the row, because an absence is not something another device can pull — it would
|
|
just re-push the title and resurrect it. Re-adding clears `deletedAt` on the same
|
|
row, so a delete/re-add round-trip can never leave a duplicate.
|
|
|
|
## Local run / verify
|
|
|
|
```bash
|
|
docker compose up --build # http://localhost:8090
|
|
# create the first superuser (one-time):
|
|
docker compose exec pocketbase pocketbase superuser upsert you@example.com 'a-strong-pass' --dir=/pb_data
|
|
# admin UI: http://localhost:8090/_/
|
|
```
|
|
|
|
The end-to-end access-rule check (two users can't see each other's data; a user
|
|
can't set their own `nsfwEnabled`; a superuser can) lives in
|
|
[`scripts/verify.py`](scripts/verify.py):
|
|
|
|
```bash
|
|
python scripts/verify.py # expects the server on :8090 + the superuser above
|
|
```
|
|
|
|
The `client_logs` retention check starts its own throwaway PocketBase (the
|
|
binary pinned in the Dockerfile), seeds rows past the 14-day cutoff, fires the
|
|
nightly `trimClientLogs` cron through the superuser API and asserts none is left:
|
|
|
|
```bash
|
|
PB_BIN=/path/to/pocketbase python scripts/test_client_logs_trim.py
|
|
```
|
|
|
|
## Deploy on Coolify
|
|
|
|
The schema and image are automated; the steps below need your hands, Coolify
|
|
access and secrets (which never live in this repo).
|
|
|
|
1. **Push this repo** to a remote Coolify can reach (e.g. GitHub, like
|
|
`myanime-pair-server`).
|
|
2. **New Resource → Dockerfile application**, point it at this repo. It listens
|
|
on `8090`.
|
|
3. **Persistent storage:** add a volume mounted at **`/pb_data`** (DB + uploaded
|
|
files). Without this you lose data on redeploy.
|
|
4. **Domain + TLS:** set the domain to `pb.petruzalekr.cz`; Coolify/Traefik
|
|
provisions the Let's Encrypt cert and terminates TLS in front of the plain
|
|
`:8090`. Force-HTTPS on.
|
|
5. **Bootstrap the superuser** once, from the container terminal:
|
|
`pocketbase superuser upsert you@example.com 'a-strong-pass' --dir=/pb_data`
|
|
(or open the printed `/_/#/pbinstall/...` link on first boot). **Lock down**
|
|
the admin UI — strong password; the `_superusers` collection is the only way
|
|
in.
|
|
6. **SMTP** (admin UI → *Settings → Mail settings*): point at your SMTP host so
|
|
verification / password-reset emails send. Send the test email to confirm.
|
|
7. **Automated backups** (admin UI → *Settings → Backups*): enable the schedule
|
|
and, ideally, S3 off-site upload. Coolify volume backups are a second layer.
|
|
8. **App wiring** (later issues, not here): point the app's API base at
|
|
`https://pb.petruzalekr.cz`.
|
|
|
|
### Verify the deploy
|
|
|
|
- Admin UI reachable over **HTTPS** only; HTTP redirects up.
|
|
- Create a test user + profile + a couple of rows via admin or REST; confirm a
|
|
second user can't read them (run `scripts/verify.py` against the live URL by
|
|
editing `BASE`).
|
|
- Trigger a password-reset email and confirm delivery.
|
|
- Confirm a backup ran and can be downloaded/restored.
|
|
|
|
## Security
|
|
|
|
Account/session tokens and the addon plaintext-credential "tokens" are **never
|
|
logged or echoed** (see the app's `CLAUDE.md`). `pinHash` is a hidden field and
|
|
never leaves the API. `nsfwEnabled` is admin-only by rule, not just by UI.
|
|
|
|
## Upgrading PocketBase
|
|
|
|
Bump `PB_VERSION` in the `Dockerfile` (and the compose arg), rebuild, redeploy.
|
|
Check the PocketBase release notes for migration-API or schema changes first,
|
|
and test locally with `docker compose up --build` before deploying.
|