Replace Postgres with SQLite
Ten members and a handful of songs a week never needed a database server,
and the server was the last thing making this a two-container deployment.
modernc.org/sqlite is pure Go, so CGO_ENABLED=0 survives and the dependency
count is unchanged: pgx out, sqlite in.
The port stayed small because the driver matches $1-style placeholders
against argument ordinals exactly as pgx does, so no query needed rewriting
for parameters. What did change:
- timestamptz becomes timestamp holding UTC 'YYYY-MM-DD HH:MM:SS'. The
declared type is what makes the driver return time.Time, and the
fixed-width UTC string is what makes ordering and comparison against
datetime('now') mean what they say.
- interval has no equivalent: sessions.idle_ttl is seconds, and the review
edit window travels as a SQLite date modifier string.
- No stddev_pop, so the divisive and unified boards spell the population
formula out, guarded with max(0.0, ...) because cancellation returns a
tiny negative when every score is identical.
- foreign_keys is off by default, so the cascades only exist because the
pragma is set on every connection.
Drops the postgres service, its healthcheck, the depends_on gate, the
startup retry loop and POSTGRES_PASSWORD. ./storage is now the whole
backup. Tests get a fresh database file per test and run everywhere
instead of skipping without TEST_DATABASE_URL.
This commit is contained in:
@@ -209,3 +209,30 @@ says so.
|
||||
|
||||
The coverage assumption that shaped the earlier sketch was wrong and is corrected in `later.md`:
|
||||
LRCLIB has synced lyrics for a good share of Finnish rock, not almost none.
|
||||
46. **SQLite instead of Postgres — this reverses entries 1 and 3** (2026-08-02). Ten members and a
|
||||
handful of songs a week never needed a database server, and the server was the last thing making
|
||||
this a two-container deployment. `modernc.org/sqlite` is pure Go, so `CGO_ENABLED=0` survives and
|
||||
the dependency count does not change: `pgx` out, `sqlite` in. What it buys: one container, one
|
||||
bind mount that is the entire backup, no `pgdata`, no healthcheck-gated `depends_on`, no startup
|
||||
retry loop, and tests that run anywhere instead of skipping without `TEST_DATABASE_URL`.
|
||||
|
||||
The port was smaller than expected, because the driver matches `$1`-style placeholders against
|
||||
argument ordinals exactly as pgx does — so no query needed rewriting for parameters. What did
|
||||
change:
|
||||
- **`timestamptz` → `timestamp` holding UTC `YYYY-MM-DD HH:MM:SS`.** The declared type is what
|
||||
makes the driver return `time.Time`; the fixed-width UTC string is what makes `order by
|
||||
created_at` and `expires_at > datetime('now')` mean what they say. `_time_format=datetime` and
|
||||
`_timezone=UTC` on the DSN make Go write exactly the shape `datetime('now')` produces, so the
|
||||
two sources of a timestamp are comparable.
|
||||
- **`interval` has no equivalent.** `sessions.idle_ttl` is seconds as an integer, and the review
|
||||
edit window travels as a SQLite date modifier string (`-1800 seconds`).
|
||||
- **No `stddev_pop`.** The divisive and unified boards use the population formula written out,
|
||||
guarded with `max(0.0, …)` because floating-point cancellation returns a tiny negative when
|
||||
every score is identical, and `sqrt` of that is null.
|
||||
- **`foreign_keys` is off by default in SQLite**, so every `on delete cascade` in the schema is
|
||||
decoration without the pragma. It is set on the DSN alongside WAL, `busy_timeout` and
|
||||
`_txlock=immediate`.
|
||||
|
||||
Taken while there was still no data: the tables were recreated rather than converted, same as
|
||||
entry 45. What would reverse this: enough concurrent writers that one writer is a real limit, or
|
||||
wanting the database on a different box from the audio files.
|
||||
|
||||
+1
-1
@@ -84,7 +84,7 @@ The constraint that shapes every idea: **E2B-class multimodal models are speech
|
||||
encoder targets ASR and spoken-audio QA. Music is out of distribution — genre calls are near
|
||||
coin-flips, "describe this track" produces beige copy, and singing over instrumentation is a worst
|
||||
case for ASR. Encoders also work in ~30 s windows, so a four-minute song is a chunk loop, and on CPU
|
||||
beside Postgres and ffmpeg that is minutes per submission.
|
||||
beside ffmpeg that is minutes per submission.
|
||||
|
||||
So the ideas that use the model to be *correct* are the weak ones, and the idea that uses it to be
|
||||
*entertaining* is the strong one:
|
||||
|
||||
+29
-20
@@ -327,12 +327,13 @@ and the average.
|
||||
Always public, ignores the reveal rule. Minimum **3 reviews** for a song to qualify for any ranking;
|
||||
`min_reviews` is published, not hardcoded in a client.
|
||||
|
||||
- Songs: Top 10 all-time, Bottom 10, Most Divisive (highest `stddev_pop`), Most Unified (lowest),
|
||||
- Songs: Top 10 all-time, Bottom 10, Most Divisive (highest score spread), Most Unified (lowest),
|
||||
Most Reviewed.
|
||||
- Reviewers: Harshest Critic (lowest average given), Most Generous, Most Active, Most Prolific
|
||||
Submitter.
|
||||
- Postgres does all of it: `avg()`, `count()`, `stddev_pop()`, `HAVING count(*) >= 3`. Order and
|
||||
limit in SQL, never in Go.
|
||||
- SQL does all of it: `avg()`, `count()`, `HAVING count(*) >= 3`. Order and limit in SQL, never in
|
||||
Go. SQLite has no `stddev_pop`, so the divisive/unified boards spell the population formula out —
|
||||
see `stddevPop` in `stats.go`.
|
||||
- **Every leaderboard needs a deterministic tie-break** — `ORDER BY value DESC, review_count DESC,
|
||||
id ASC`. Ties are common in a ten-person club, and without one the list reshuffles between reloads
|
||||
for no reason.
|
||||
@@ -385,7 +386,7 @@ func requireAdmin(next http.Handler) http.Handler {
|
||||
```
|
||||
|
||||
No bcrypt here: hashing protects *stored* passwords against a database leak, and this one lives in
|
||||
the env file next to the Postgres password already. The constant-time compare is the part that
|
||||
the env file already. The constant-time compare is the part that
|
||||
matters. **Fatal at startup if `ADMIN_PASSWORD` is unset** — an admin panel that silently opens is
|
||||
worse than one that will not boot.
|
||||
|
||||
@@ -519,7 +520,7 @@ the first endpoint is one line over a data function that already exists.
|
||||
- `snake_case` field names, matching the SQL columns.
|
||||
- Timestamps are RFC 3339 UTC strings (`2026-08-01T10:00:00Z`). Never preformatted, never a locale
|
||||
string, never a unix int.
|
||||
- Ids are JSON numbers (`bigserial`, safely under 2⁵³).
|
||||
- Ids are JSON numbers (SQLite rowids, safely under 2⁵³).
|
||||
- Nullable fields are present and `null`. **No `omitempty`** — a stable key set is worth more than a
|
||||
few bytes, and "missing" versus "null" is a distinction clients get wrong.
|
||||
- Scores are integers, averages are floats.
|
||||
@@ -642,33 +643,41 @@ reason the count-based lists stay until they are proven useless.
|
||||
|
||||
## 9. Data model
|
||||
|
||||
Ids are `bigserial`. Session tokens stay random — those are secrets, ids are not, and enumerable ids
|
||||
are not a threat model for a login-walled app for ten friends.
|
||||
Ids are `integer primary key autoincrement` — never reused, because `storage/audio/<song_id>.ogg` is
|
||||
named after one. Session tokens stay random — those are secrets, ids are not, and enumerable ids are
|
||||
not a threat model for a login-walled app for ten friends.
|
||||
|
||||
Timestamps are declared `timestamp` and hold UTC `YYYY-MM-DD HH:MM:SS`: the declared type is what
|
||||
makes the driver return `time.Time`, and the fixed-width UTC string is what makes ordering and
|
||||
comparison against `datetime('now')` mean what they say. Booleans are `integer`, 0 or 1.
|
||||
|
||||
```sql
|
||||
users (id bigserial pk, name, email unique, password_hash, avatar, banned, created_at)
|
||||
sessions (token pk, user_id fk not null, idle_ttl interval not null,
|
||||
users (id pk, name, email unique, password_hash, avatar, banned, created_at)
|
||||
sessions (token pk, user_id fk not null, idle_ttl integer not null, -- seconds
|
||||
expires_at, created_at) -- token: 32 random bytes, hex
|
||||
songs (id bigserial pk, title, artist, genre, description, audio_file,
|
||||
duration_seconds int,
|
||||
songs (id pk, title, artist, genre, description, lyrics, audio_file,
|
||||
duration_seconds integer,
|
||||
source_url, -- nullable, for YouTube submissions
|
||||
submitted_by fk users, created_at)
|
||||
submissions (id bigserial pk, user_id fk not null,
|
||||
submissions (id pk, user_id fk not null,
|
||||
status text not null default 'queued', -- queued|downloading|converting|ready|failed
|
||||
status_msg text,
|
||||
source_url, tmp_path,
|
||||
title, artist, genre, description,
|
||||
title, artist, genre, description, lyrics,
|
||||
created_at)
|
||||
reviews (id bigserial pk, song_id fk on delete cascade, reviewer_id fk users,
|
||||
score int, text, created_at, updated_at,
|
||||
reviews (id pk, song_id fk on delete cascade, reviewer_id fk users,
|
||||
score integer, text, created_at, updated_at,
|
||||
unique (song_id, reviewer_id))
|
||||
invites (id bigserial pk, code unique, is_valid bool, created_at)
|
||||
reports (id bigserial pk, user_id fk not null, body text not null,
|
||||
invites (id pk, code unique, is_valid integer, created_at)
|
||||
reports (id pk, user_id fk not null, body text not null,
|
||||
page text, user_agent text,
|
||||
resolved_at timestamptz, -- null = open
|
||||
resolved_at timestamp, -- null = open
|
||||
created_at)
|
||||
```
|
||||
|
||||
`foreign_keys` is off by default in SQLite, so the cascades above only exist because the pragma is
|
||||
set on every connection — see `openDB` in `main.go`.
|
||||
|
||||
No `role` column (§6). No `status` on `songs` (§4).
|
||||
|
||||
Also: `CHECK (score BETWEEN 1 AND 100)`, `NOT NULL` on everything required, an index on
|
||||
@@ -719,8 +728,8 @@ Everything else is forms and `INSERT`s. No framework, no fixtures beyond a test
|
||||
Each step leaves something runnable. Registration needs an invite and invites come from the admin
|
||||
panel, so the admin surface comes first — before a single member can exist.
|
||||
|
||||
1. **Skeleton** — `main.go`, embedded migrations at startup, pgxpool, slog, Docker Compose, the two
|
||||
listeners.
|
||||
1. **Skeleton** — `main.go`, embedded migrations at startup, `database/sql`, slog, Docker Compose,
|
||||
the two listeners.
|
||||
2. **Admin, invites, auth** — Basic Auth listener, mint an invite, register, log in, sessions, ban.
|
||||
3. **Submission pipeline, upload path only** — submit, convert, waiting page, publish. No yt-dlp yet,
|
||||
so the hard parts (worker, publish transaction, restart recovery) are proven without a network
|
||||
|
||||
Reference in New Issue
Block a user