Kessinen/Levyraati26_go
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Replace Postgres with SQLite

Browse Source 
Ten members and a handful of songs a week never needed a database server,
and the server was the last thing making this a two-container deployment.
modernc.org/sqlite is pure Go, so CGO_ENABLED=0 survives and the dependency
count is unchanged: pgx out, sqlite in.

The port stayed small because the driver matches $1-style placeholders
against argument ordinals exactly as pgx does, so no query needed rewriting
for parameters. What did change:

- timestamptz becomes timestamp holding UTC 'YYYY-MM-DD HH:MM:SS'. The
  declared type is what makes the driver return time.Time, and the
  fixed-width UTC string is what makes ordering and comparison against
  datetime('now') mean what they say.
- interval has no equivalent: sessions.idle_ttl is seconds, and the review
  edit window travels as a SQLite date modifier string.
- No stddev_pop, so the divisive and unified boards spell the population
  formula out, guarded with max(0.0, ...) because cancellation returns a
  tiny negative when every score is identical.
- foreign_keys is off by default, so the cascades only exist because the
  pragma is set on every connection.

Drops the postgres service, its healthcheck, the depends_on gate, the
startup retry loop and POSTGRES_PASSWORD. ./storage is now the whole
backup. Tests get a fresh database file per test and run everywhere
instead of skipping without TEST_DATABASE_URL.
This commit is contained in:
Esa Kataja
2026-08-02 20:47:41 +03:00
parent a9776c6dde
commit 1fe5211ae6
28 changed files with 440 additions and 389 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/spec.md
+29 -20
View File
@@ -327,12 +327,13 @@ and the average.
Always public, ignores the reveal rule. Minimum **3 reviews** for a song to qualify for any ranking;
`min_reviews` is published, not hardcoded in a client.
- Songs: Top 10 all-time, Bottom 10, Most Divisive (highest `stddev_pop`), Most Unified (lowest),
- Songs: Top 10 all-time, Bottom 10, Most Divisive (highest score spread), Most Unified (lowest),
Most Reviewed.
- Reviewers: Harshest Critic (lowest average given), Most Generous, Most Active, Most Prolific
Submitter.
- Postgres does all of it: `avg()`, `count()`, `stddev_pop()`, `HAVING count(*) >= 3`. Order and
limit in SQL, never in Go.
- SQL does all of it: `avg()`, `count()`, `HAVING count(*) >= 3`. Order and limit in SQL, never in
Go. SQLite has no `stddev_pop`, so the divisive/unified boards spell the population formula out —
see `stddevPop` in `stats.go`.
- **Every leaderboard needs a deterministic tie-break** — `ORDER BY value DESC, review_count DESC,
id ASC`. Ties are common in a ten-person club, and without one the list reshuffles between reloads
for no reason.
@@ -385,7 +386,7 @@ func requireAdmin(next http.Handler) http.Handler {
```
No bcrypt here: hashing protects *stored* passwords against a database leak, and this one lives in
the env file next to the Postgres password already. The constant-time compare is the part that
the env file already. The constant-time compare is the part that
matters. **Fatal at startup if `ADMIN_PASSWORD` is unset** — an admin panel that silently opens is
worse than one that will not boot.
@@ -519,7 +520,7 @@ the first endpoint is one line over a data function that already exists.
- `snake_case` field names, matching the SQL columns.
- Timestamps are RFC 3339 UTC strings (`2026-08-01T10:00:00Z`). Never preformatted, never a locale
string, never a unix int.
- Ids are JSON numbers (`bigserial`, safely under 2⁵³).
- Ids are JSON numbers (SQLite rowids, safely under 2⁵³).
- Nullable fields are present and `null`. **No `omitempty`** — a stable key set is worth more than a
few bytes, and "missing" versus "null" is a distinction clients get wrong.
- Scores are integers, averages are floats.
@@ -642,33 +643,41 @@ reason the count-based lists stay until they are proven useless.
## 9. Data model
Ids are `bigserial`. Session tokens stay random — those are secrets, ids are not, and enumerable ids
are not a threat model for a login-walled app for ten friends.
Ids are `integer primary key autoincrement` — never reused, because `storage/audio/<song_id>.ogg` is
named after one. Session tokens stay random — those are secrets, ids are not, and enumerable ids are
not a threat model for a login-walled app for ten friends.
Timestamps are declared `timestamp` and hold UTC `YYYY-MM-DD HH:MM:SS`: the declared type is what
makes the driver return `time.Time`, and the fixed-width UTC string is what makes ordering and
comparison against `datetime('now')` mean what they say. Booleans are `integer`, 0 or 1.
```sql
users (id bigserial pk, name, email unique, password_hash, avatar, banned, created_at)
sessions (token pk, user_id fk not null, idle_ttl interval not null,
users (id pk, name, email unique, password_hash, avatar, banned, created_at)
sessions (token pk, user_id fk not null, idle_ttl integer not null, -- seconds
expires_at, created_at) -- token: 32 random bytes, hex
songs (id bigserial pk, title, artist, genre, description, audio_file,
duration_seconds int,
songs (id pk, title, artist, genre, description, lyrics, audio_file,
duration_seconds integer,
source_url, -- nullable, for YouTube submissions
submitted_by fk users, created_at)
submissions (id bigserial pk, user_id fk not null,
submissions (id pk, user_id fk not null,
status text not null default 'queued', -- queued|downloading|converting|ready|failed
status_msg text,
source_url, tmp_path,
title, artist, genre, description,
title, artist, genre, description, lyrics,
created_at)
reviews (id bigserial pk, song_id fk on delete cascade, reviewer_id fk users,
score int, text, created_at, updated_at,
reviews (id pk, song_id fk on delete cascade, reviewer_id fk users,
score integer, text, created_at, updated_at,
unique (song_id, reviewer_id))
invites (id bigserial pk, code unique, is_valid bool, created_at)
reports (id bigserial pk, user_id fk not null, body text not null,
invites (id pk, code unique, is_valid integer, created_at)
reports (id pk, user_id fk not null, body text not null,
page text, user_agent text,
resolved_at timestamptz, -- null = open
resolved_at timestamp, -- null = open
created_at)
```
`foreign_keys` is off by default in SQLite, so the cascades above only exist because the pragma is
set on every connection — see `openDB` in `main.go`.
No `role` column (§6). No `status` on `songs` (§4).
Also: `CHECK (score BETWEEN 1 AND 100)`, `NOT NULL` on everything required, an index on
@@ -719,8 +728,8 @@ Everything else is forms and `INSERT`s. No framework, no fixtures beyond a test
Each step leaves something runnable. Registration needs an invite and invites come from the admin
panel, so the admin surface comes first — before a single member can exist.
1. **Skeleton** — `main.go`, embedded migrations at startup, pgxpool, slog, Docker Compose, the two
listeners.
1. **Skeleton** — `main.go`, embedded migrations at startup, `database/sql`, slog, Docker Compose,
the two listeners.
2. **Admin, invites, auth** — Basic Auth listener, mint an invite, register, log in, sessions, ban.
3. **Submission pipeline, upload path only** — submit, convert, waiting page, publish. No yt-dlp yet,
so the hard parts (worker, publish transaction, restart recovery) are proven without a network