docs: describe SQLite logging (logs.db), drop stale log-file docs
Logging moved to a logs.db SQLite store, but MANUAL.md, MANUAL.html and SPEC.md still documented info_*.log / error_*.log / structured.json and a midnight-rollover limitation that no longer exist. Replace those sections with the logs.db reality: levels (info/error to db+stdout/stderr, debug db-only), progress-to-stdout-only, retention via log_retention_days, and that recent events are viewable in the dashboard / GET /status. Closes #6
This commit is contained in:
+8
-8
@@ -505,19 +505,19 @@ old.broadcast.tvrip.tt0066026.mkv</code></pre></div>
|
|||||||
<section id="logs">
|
<section id="logs">
|
||||||
<p class="eyebrow">08 — Observability</p>
|
<p class="eyebrow">08 — Observability</p>
|
||||||
<h2>Logs</h2>
|
<h2>Logs</h2>
|
||||||
<p>Three log files are written to the <strong>current working directory</strong> (not the config paths):</p>
|
<p>All logs are written to a <strong>SQLite database, <code>logs.db</code></strong>, in the current working directory (not the config paths). Run the program from where you want it to land — in Docker that's the mounted <code>/data</code>. The <code>logs</code> table has columns <code>id</code>, <code>ts</code>, <code>level</code>, <code>message</code>, <code>file</code>, <code>extra</code>.</p>
|
||||||
<div class="tablewrap">
|
<div class="tablewrap">
|
||||||
<table>
|
<table>
|
||||||
<thead><tr><th>File</th><th>Contents</th></tr></thead>
|
<thead><tr><th>Level</th><th>Where</th><th>Contents</th></tr></thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
<tr><td><code>info_YYYY-MM-DD.log</code></td><td>INFO messages, dated</td></tr>
|
<tr><td><code>info</code></td><td>db + stdout</td><td>Processing / metadata hits / completions</td></tr>
|
||||||
<tr><td><code>error_YYYY-MM-DD.log</code></td><td>ERROR messages, dated</td></tr>
|
<tr><td><code>error</code></td><td>db + stderr</td><td>Failures, with the source <code>file</code> recorded</td></tr>
|
||||||
<tr><td><code>structured.json</code></td><td>One JSON object per line — every entry, with <code>timestamp</code>, <code>level</code>, <code>message</code>, optional <code>error</code> & <code>file</code></td></tr>
|
<tr><td><code>debug</code></td><td>db only</td><td>ffprobe output, zscale width, full ffmpeg/opusenc command, OMDb/TVmaze responses (API key redacted) in <code>extra</code></td></tr>
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
</div>
|
||||||
<p>Run the program from the directory where you want the logs to land. A number of <code>DEBUG</code> lines also print to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command) — intentional, but not written to the log files.</p>
|
<p>Live encode progress (<code>encoding … · 47% · 3.2fps · …</code>) prints to <strong>stdout only</strong> and is deliberately not stored, so it can't flood the database.</p>
|
||||||
<div class="note warn"><span class="tag">Known limitation</span><p>The date in <code>info_*.log</code> / <code>error_*.log</code> filenames is computed at daemon start and does not roll over at midnight. Left running across days, all writes continue into the start-day's file — restart to rotate. <code>structured.json</code> does not rotate at all.</p></div>
|
<div class="note"><span class="tag">Retention</span><p>Rows older than <code>log_retention_days</code> (default <code>7</code>) are purged on startup and shutdown. Editable live from the settings page (<code>/settings</code>); applies at the next purge. Recent non-debug events are viewable in the dashboard and via <code>GET /status</code> when <code>http_addr</code> is set.</p></div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
<section id="failures">
|
<section id="failures">
|
||||||
@@ -527,7 +527,7 @@ old.broadcast.tvrip.tt0066026.mkv</code></pre></div>
|
|||||||
<ul>
|
<ul>
|
||||||
<li>The source <code>.mkv</code> is moved to <code>paths.failed</code> (<code>os.Stat</code>-guarded — if the source is already gone the move is skipped and logged; a move error is logged too). This guarantees the source leaves <code>paths.input</code> on every failure path, so the watcher won't retry it next tick.</li>
|
<li>The source <code>.mkv</code> is moved to <code>paths.failed</code> (<code>os.Stat</code>-guarded — if the source is already gone the move is skipped and logged; a move error is logged too). This guarantees the source leaves <code>paths.input</code> on every failure path, so the watcher won't retry it next tick.</li>
|
||||||
<li>The per-job work directory (partial wav/opus/output.mkv) is deleted unconditionally.</li>
|
<li>The per-job work directory (partial wav/opus/output.mkv) is deleted unconditionally.</li>
|
||||||
<li>The error is logged to <code>error_*.log</code> and <code>structured.json</code> with the source path.</li>
|
<li>The error is logged to <code>logs.db</code> (level <code>error</code>) with the source path, and printed to stderr.</li>
|
||||||
</ul>
|
</ul>
|
||||||
<p>The watcher continues with the next file; one bad rip won't stop the daemon.</p>
|
<p>The watcher continues with the next file; one bad rip won't stop the daemon.</p>
|
||||||
<div class="note"><span class="tag">Collisions</span><p>If a finished encode would land on a name that already exists in <code>paths.output</code>, the move is refused (no silent overwrite) and the source is routed to <code>paths.failed</code>. This is the path you hit when two sources sanitize to the same name — two re-rips of the same release, or two episodes that both resolve to <code>SxxExx</code>.</p></div>
|
<div class="note"><span class="tag">Collisions</span><p>If a finished encode would land on a name that already exists in <code>paths.output</code>, the move is refused (no silent overwrite) and the source is routed to <code>paths.failed</code>. This is the path you hit when two sources sanitize to the same name — two re-rips of the same release, or two episodes that both resolve to <code>SxxExx</code>.</p></div>
|
||||||
|
|||||||
@@ -225,17 +225,19 @@ A few things worth knowing:
|
|||||||
|
|
||||||
## 8. Logs
|
## 8. Logs
|
||||||
|
|
||||||
Three log files are written to the **current working directory** (not the config paths):
|
All logs are written to a **SQLite database, `logs.db`**, in the current working directory (not the config paths). Run the program from the directory where you want it to land — in Docker that's the mounted `/data`.
|
||||||
|
|
||||||
- `info_YYYY-MM-DD.log` — INFO messages, dated.
|
The `logs` table has columns `id`, `ts`, `level`, `message`, `file`, `extra`. Three levels are recorded:
|
||||||
- `error_YYYY-MM-DD.log` — ERROR messages, dated.
|
|
||||||
- `structured.json` — one JSON object per line, every entry (info + error), with `timestamp`, `level`, `message`, optional `error` and `file` fields.
|
|
||||||
|
|
||||||
Run the program from the directory where you want the logs to land.
|
- `info` — INFO messages; also printed to stdout.
|
||||||
|
- `error` — ERROR messages; also printed to stderr, with the source `file` recorded.
|
||||||
|
- `debug` — verbose diagnostics (ffprobe output, calculated zscale width, the full ffmpeg/opusenc command, OMDb/TVmaze responses with the API key redacted) in the `extra` column. **Database only** — not printed.
|
||||||
|
|
||||||
Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command, etc.). These are intentional but not written to the log files.
|
Live encode progress (`encoding … · 47% · 3.2fps · …`) prints to **stdout only** and is deliberately *not* stored, so it can't flood the database.
|
||||||
|
|
||||||
**Known limitation:** the date in `info_*.log` / `error_*.log` filenames is computed when the daemon starts and does not roll over at midnight. If the daemon is left running across days, all writes continue into the start-day's file. Restart the daemon to rotate. `structured.json` does not rotate at all.
|
**Retention:** rows older than `log_retention_days` (default `7`) are purged on startup and on shutdown. The value is editable live from the settings page (`/settings`) and applies at the next purge.
|
||||||
|
|
||||||
|
Recent non-debug events are also viewable in the web dashboard and via `GET /status` (when `http_addr` is set).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -245,7 +247,7 @@ If any step from probing through encoding through renaming fails:
|
|||||||
|
|
||||||
- The source `.mkv` is moved to `paths.failed` (the move itself is `os.Stat`-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves `paths.input` on every failure path, so the watcher doesn't retry the same file on the next tick.
|
- The source `.mkv` is moved to `paths.failed` (the move itself is `os.Stat`-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves `paths.input` on every failure path, so the watcher doesn't retry the same file on the next tick.
|
||||||
- The per-job work directory under `paths.work` (containing partial wav/opus/output.mkv) is deleted unconditionally.
|
- The per-job work directory under `paths.work` (containing partial wav/opus/output.mkv) is deleted unconditionally.
|
||||||
- The error is logged to `error_*.log` and `structured.json` with the source file path.
|
- The error is logged to `logs.db` (level `error`) with the source file path, and printed to stderr.
|
||||||
|
|
||||||
The watcher continues with the next file; one bad rip won't stop the daemon.
|
The watcher continues with the next file; one bad rip won't stop the daemon.
|
||||||
|
|
||||||
|
|||||||
@@ -99,9 +99,10 @@ Regex patterns:
|
|||||||
|
|
||||||
## Logging
|
## Logging
|
||||||
|
|
||||||
- info.log: Human-readable info level
|
- All logs are stored in a SQLite database `logs.db` in the working directory.
|
||||||
- error.log: Human-readable error level
|
- Levels: `debug`, `info`, `error`. `info`/`error` also print to stdout/stderr; `debug` (ffprobe/ffmpeg/API dumps) is database-only. Live encode progress prints to stdout only (not stored).
|
||||||
- structured.json: JSON structured logs
|
- Retention: rows older than `log_retention_days` (default 7) are purged on startup/shutdown; editable from the settings UI.
|
||||||
|
- Recent non-debug events are viewable in the web dashboard and via `GET /status`.
|
||||||
|
|
||||||
## Polling
|
## Polling
|
||||||
- Scan input folder every 10-30 seconds for `.mkv` files
|
- Scan input folder every 10-30 seconds for `.mkv` files
|
||||||
|
|||||||
Reference in New Issue
Block a user