amber-backend/README.md

237 lines
12 KiB
Markdown
Raw Normal View History

# amber-backend
**PocketBase** backend for Amber: auth, database, realtime, file storage and admin
UI in a single Go binary. This is the foundation the account features build on:
cloud sync, codeless device sign-in, profiles, child profiles, recommendations,
and gated auto-update (epic
[#6](https://projects.petruzalekr.cz/richiexec/myanime-app/issues/6)).
The repo used to be called **myanime-backend**. On Forgejo it is now
`richiexec/amber-backend`; the old name still redirects to it, and Coolify's git
source still uses the old URL.
- **No E2E.** TLS in transit + at-rest on disk. The server is deliberately
readable so recommendations and painless password reset work.
- **Schema as code.** The collections live in [`pb_migrations/`](pb_migrations/)
and auto-apply on boot — never hand-click the schema in the admin UI, or the
next deploy will drift.
- **Version-pinned.** The [`Dockerfile`](Dockerfile) downloads the official
PocketBase **v0.39.6** static binary from GitHub releases (no third-party
image). Bump `PB_VERSION` there and in `docker-compose.yml` to upgrade.
## Data model
```
users ─┬─* profiles ─┬─* watch_state (resume points; 1 row per profile+item)
(auth)│ ├─* watchlist (1 row per profile+item)
│ ├─1 prefs (playback/audio/subtitle blob)
│ └─1 addon_config (client-encrypted addon blob; ciphertext only)
├─* device_auth (codeless sign-in requests / approved devices, issue #12)
└ nsfwEnabled (admin-only bool, legacy: see below)
```
| Collection | Type | Key fields |
|----------------|------|------------|
| `users` | auth | `username` (unique when set), `email`, `password`, **`nsfwEnabled`** (admin-only, legacy) |
| `profiles` | base | `user`→users, `name`, `avatar` (file), `isChild`, `pinHash` (hidden) |
| `watch_state` | base | `profile`→profiles, `itemId`, `itemType`, `position`, `duration`, `meta` (json), `updatedAt` (client), `updated` (server) |
| `watchlist` | base | `profile`→profiles, `itemId`, `itemType`, `meta` (json), `addedAt` (client), `updated` (server), `deletedAt` (tombstone) |
| `prefs` | base | `profile`→profiles, `data` (json), `updatedAt` (client), `updated` (server) |
| `addon_config` | base | `profile`→profiles, `blob` (**ciphertext**), `salt`, `kdf`, `updatedAt` (client), `updated` (server) |
| `device_auth` | base | `code` (unique), `deviceName`, `status`, `user`→users, `expiresAt`, + hidden `authToken`/`keyCiphertext`/`pollSecret`/`devicePubKey` — codeless device sign-in (#12) |
**Access rules.** A user only ever reads/writes their own `users` row, their own
`profiles`, and rows whose `profile.user` is them. `nsfwEnabled` is never
accepted from a client (`@request.body.nsfwEnabled:isset = false` on create and
update) — only a **superuser** sets it, from the admin UI. Relations
`cascadeDelete`, so deleting a user removes their profiles and all child rows.
Unique indexes keep one resume/watchlist row per `(profile, itemId)` and one
`prefs`/`addon_config` row per profile.
### `nsfwEnabled` is legacy
There is **one Amber build**. The flag used to choose between an 18+ build and the
clean one; the 18+ section left Amber to become a separate app, and none of that
gating is left in this repo:
- `GET /api/update/manifest` (`pb_hooks/update.pb.js`) always serves variant
`clean`, whatever the flag says. The `variant` field stays in the response
because every installed Amber silently refuses a mismatch.
- The `releases` read rules no longer mention it
(`pb_migrations/1793000000_releases_drop_adult_rule.js`): any signed-in account
reads releases.
- `/api/amber/whoami` no longer returns it, and the status dashboard is gated on
superuser credentials, never on this flag.
- The Amber app neither reads nor sends it.
What remains: the field is still on `users`, still admin-only by rule, and has
exactly one reader, the family web page (`pb_public/index.html`). For an account
with the flag set, that page shows an adult addon URL field on the sources tab
(saved inside the encrypted addon config) and mentions 18+ in the child-profile
label. Nothing else reads it.
### Encrypted addon config (issue #20)
The `addon_config` blob is the one piece of account data that is **not** readable
by the server. The addon "tokens" are base64'd *plaintext credentials* (prehraj.to
login, TorBox API key) plus the TMDB key, so — unlike watch history — they are
encrypted **client-side** before upload. The server stores only:
- `blob` — AES-GCM ciphertext (nonce ‖ ciphertext ‖ tag), base64; opaque here.
- `salt` — the per-account KDF salt (base64). Not secret; a salt never is. It
lives server-side so a second device can derive the **same** key from the
user's password. The key itself is never sent and never leaves the device.
- `kdf` — the key-derivation descriptor (e.g. `pbkdf2-sha256-210000`).
Because the key is derived from the password, a **password reset** (done on
PocketBase's hosted page, off-device) leaves the blob undecryptable — expected and
acceptable for re-enterable addon config, and the client detects it, clears it, and
prompts a re-entry. A normal in-app *change password* re-encrypts the blob, so only
reset loses it. This does **not** change the no-E2E stance for the rest of the data.
### Codeless device sign-in (issue #12)
A new device (a TV) signs into an account with **no password typing and no code
transcription**: it shows a QR + short code, and an already-logged-in phone/PC
approves it from `GET /device`. On approval the TV gets a minted session token and
— **end-to-end, the server never sees it** — the addon-config vault key, so it can
decrypt the `addon_config` blob it then pulls. This lives in the `device_auth`
collection (`pb_migrations/1786000000_device_auth.js`) plus two JS hooks
(`pb_hooks/device_auth.pb.js` routes + cron, `pb_hooks/device_page.pb.js` approve
page). All create/poll/approve logic is in the hooks (the TV is unauthenticated
when it requests and polls); the collection exposes only the owner-scoped device
**list + revoke**. **JWT revocation caveat:** deleting a device row removes it from
the list but can't invalidate a token already minted (stateless JWTs) — a
documented v1 limitation.
**Full contract** (endpoints, record shape, the exact crypto — PBKDF2 params +
the X25519/HKDF/AES-GCM key transfer): [`docs/device-auth-contract.md`](docs/device-auth-contract.md).
feat(schema): fields the client needs to sync watch state / watchlist / prefs The #9 schema sketched these collections before the client existed. Reconciling two devices turned up three gaps: - `meta` (json) on watch_state/watchlist. Local rows carry display fields — episode, title, cover art, MAL id, completed/dismissed — with nowhere to live server-side, so a freshly-signed-in device pulled resume points it couldn't render. - A client-owned clock. `updatedAt`/`addedAt` were autodate, i.e. stamped on server receipt, so a device that edited offline and pushed a day later beat a device that edited afterwards and synced at once. LWW needs the time the user acted. The server autodate lives on as `updated` and is now the pull cursor — that one must be server-side, or a skewed client clock would write rows behind another device's cursor and stay invisible to it. - `deletedAt` tombstones on watchlist. Hard deletes are an absence, and an absence isn't pullable — the next device to push just resurrects the title. The type change means dropping and re-adding the fields, which drops their column data, so the migration snapshots the old timestamps and writes them back. Live is believed empty, but blanking `updated` would leave rows invisible to every future pull, which is too quiet a failure to risk on an assumption. Verified against a local docker compose on both paths — a fresh install, and an upgrade over the init-only schema with rows already in it (timestamps preserved). scripts/verify.py grows the field-shape assertions plus behavioural checks: the client clock round-trips unrestamped, duplicate (profile,itemId) is rejected, and delete/re-add reuses the row instead of duplicating. 36/36 pass. Refs richiexec/myanime-app#11
2026-07-17 14:20:38 +00:00
### The two clocks (issue #11)
Sync rows carry **two** timestamps, and they are not interchangeable:
- **`updatedAt` / `addedAt` — set by the client.** When the user actually acted.
This is what resolves conflicts (last-write-wins). It has to be the client's,
because a device that edits offline on Monday and pushes on Friday must not
beat a device that edited on Tuesday and synced immediately.
- **`updated` — a server autodate.** When we heard about it. This is the **pull
cursor** (`filter=updated > {lastCursor}`). It has to be the server's: a device
with a skewed clock would otherwise stamp rows *behind* another device's cursor
and stay invisible to it forever.
`itemId` encodes the client's local key — `anime:<anilistId>:<episode>`,
`movie:<tmdbId>:0`, `tv:<tmdbId>:<seasonEpisodeKey>` — so the unique
`(profile, itemId)` index is what makes a push idempotent. `meta` is an opaque
JSON blob of the client's display fields (title, cover art, episode, flags); the
server never reads into it.
**Deletes are soft.** A watchlist removal sets `deletedAt` rather than deleting
the row, because an absence is not something another device can pull — it would
just re-push the title and resurrect it. Re-adding clears `deletedAt` on the same
row, so a delete/re-add round-trip can never leave a duplicate.
## Local run / verify
```bash
docker compose up --build # http://localhost:8090
# create the first superuser (one-time):
docker compose exec pocketbase pocketbase superuser upsert you@example.com 'a-strong-pass' --dir=/pb_data
# admin UI: http://localhost:8090/_/
```
The end-to-end access-rule check (two users can't see each other's data; a user
can't set their own `nsfwEnabled`; a superuser can) lives in
[`scripts/verify.py`](scripts/verify.py):
```bash
python scripts/verify.py # expects the server on :8090 + the superuser above
```
2026-09-13 20:32:34 +00:00
The `client_logs` retention check starts its own throwaway PocketBase (the
binary pinned in the Dockerfile), seeds rows past the 14-day cutoff, fires the
nightly `trimClientLogs` cron through the superuser API and asserts none is left:
```bash
PB_BIN=/path/to/pocketbase python scripts/test_client_logs_trim.py
```
## Deploy
Production is a Coolify "Dockerfile application" on `https://coolify.petruzalekr.cz`:
app `amber-backend`, uuid `eu3wk4j0zf0uczhbew0m5c9p`, serving
`pb.petruzalekr.cz` and `amber.petruzalekr.cz` on port 8090, with a persistent
volume at `/pb_data`.
**A push to `main` deploys it to production within seconds.** Coolify's git source
is `https://projects.petruzalekr.cz/richiexec/myanime-backend.git` (the old name,
redirected to this repo), branch `main`.
- Its deployment records say `is_api` rather than webhook, but every one since
2026-09-13 started 2 to 4 seconds after its commit.
- If no deployment follows a push, deploy by hand: the app's **Deploy** button in
Coolify, or `GET /api/v1/deploy?uuid=eu3wk4j0zf0uczhbew0m5c9p` with the Coolify
API token.
Migrations and hooks are baked into the image and apply when the new container
boots, so a schema or hook change reaches production only with a deploy.
A deploy is done when live requests show it, not when Coolify says `finished`:
```bash
curl -s https://pb.petruzalekr.cz/api/health # 200: the binary is up
curl -s -o /dev/null -w '%{http_code}\n' https://amber.petruzalekr.cz/status # 302: the hooks loaded
```
and then a request that shows the change itself. There is no commit endpoint, so
compare the commit on Coolify's deployment record with `origin/main`. How to wait
for a deploy, recover a wedged deployment queue and verify every service:
[`docs/runbooks/deploy.md` in amber-app](https://projects.petruzalekr.cz/richiexec/amber-app/src/branch/main/docs/runbooks/deploy.md).
### Setting it up from scratch
Only needed to rebuild the Coolify app. Secrets never live in this repo.
1. **Git source:** Coolify's "GitHub App" source cannot talk to Forgejo (it forces
`api.github.com`), so add the repo as a plain git source: its public HTTPS URL,
or SSH on Forgejo's port 22222.
2. **New Resource → Dockerfile application**, point it at this repo. It listens
on `8090`.
3. **Persistent storage:** add a volume mounted at **`/pb_data`** (DB + uploaded
files). Without this you lose data on redeploy.
4. **Domain + TLS:** set the domains to `pb.petruzalekr.cz` and
`amber.petruzalekr.cz`; Coolify/Traefik provisions the Let's Encrypt cert and
terminates TLS in front of the plain `:8090`. Force-HTTPS on.
5. **Bootstrap the superuser** once, from the container terminal:
`pocketbase superuser upsert you@example.com 'a-strong-pass' --dir=/pb_data`
(or open the printed `/_/#/pbinstall/...` link on first boot). **Lock down**
the admin UI — strong password; the `_superusers` collection is the only way
in.
6. **SMTP** (admin UI → *Settings → Mail settings*): point at your SMTP host so
verification / password-reset emails send. Send the test email to confirm.
7. **Automated backups** (admin UI → *Settings → Backups*): enable the schedule
and, ideally, S3 off-site upload. Coolify volume backups are a second layer.
8. **App wiring:** the Amber app's PocketBase base URL defaults to
`https://pb.petruzalekr.cz` (`lib/core/config.dart` in amber-app).
#### Verify a fresh setup
- Admin UI reachable over **HTTPS** only; HTTP redirects up.
- Create a test user + profile + a couple of rows via admin or REST; confirm a
second user can't read them (run `scripts/verify.py` against the live URL by
editing `BASE`).
- Trigger a password-reset email and confirm delivery.
- Confirm a backup ran and can be downloaded/restored.
## Security
Account/session tokens and the addon plaintext-credential "tokens" are **never
logged or echoed** (see the app's `CLAUDE.md`). `pinHash` is a hidden field and
never leaves the API. `nsfwEnabled` is admin-only by rule, not just by UI.
## Upgrading PocketBase
Bump `PB_VERSION` in the `Dockerfile` (and the compose arg), then deploy
explicitly as described above (a push alone does not). Check the PocketBase
release notes for migration-API or schema changes first, and test locally with
`docker compose up --build` before deploying.