Kessinen/av1dae
1
0
Fork 0
Code Issues 11 Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: Kessinen/av1dae:71f7ef7bca063ad6389dc81d7cd610ca19b0cfb3
Branches Tags
Kessinen/av1dae:main Kessinen/av1dae:dev
..
compare: Kessinen/av1dae:dev
Branches Tags
Kessinen/av1dae:dev Kessinen/av1dae:main
25 Commits
71f7ef7bca .. dev
Author SHA1 Message Date
Esa Kataja 31e22bd4a7 docs: describe SQLite logging (logs.db), drop stale log-file docs 
Logging moved to a logs.db SQLite store, but MANUAL.md, MANUAL.html and
SPEC.md still documented info_*.log / error_*.log / structured.json and a
midnight-rollover limitation that no longer exist. Replace those sections with
the logs.db reality: levels (info/error to db+stdout/stderr, debug db-only),
progress-to-stdout-only, retention via log_retention_days, and that recent
events are viewable in the dashboard / GET /status.

Closes #6
 2026-06-21 20:51:34 +03:00
Esa Kataja dfbd74b0d2 add: queue total ETA from current encode speed 
The queue card now shows an estimated time to clear: current job remaining +
sum(pending source durations) / current speed. Durations come from a
path+mtime-keyed cache that probes each file once in the background (ffprobe
format=duration, header-only), so the 1s poll never re-probes; entries are
pruned to the live queue. No speed (idle/held) -> no estimate; files still
being probed mark it partial (shown as '~Xh+').

Closes #9
 2026-06-21 20:48:37 +03:00
Esa Kataja 04d6c85712 add: processing controls — start/hold, pause/resume, retry 
Three lifecycle controls on the dashboard, sharing one /api surface and the
status feed:

- Start/hold gate (#11): the watcher tracks the queue but holds processing
  until POST /api/start. Held by default; AV1DAE_AUTOSTART=1 restores start-
  on-boot. NOTE: flips the previous auto-start default.
- Pause/resume the active encode (#10): SIGSTOP/SIGCONT on the ffmpeg process
  (libsvtav1 is in-process, so one signal suspends all its threads). Resumes
  exactly where it left off; the tracker excludes paused time from elapsed.
- Retry a failed file (#3): POST /api/retry?file=NAME moves it from failed/
  back to input/, with a base-name guard against path traversal.

/status now reports running + the failed list; snapshots carry a paused flag.
Verified live: held queue, retry move, traversal -> 400, and a real encode
suspending to process state T on pause and S on resume.

Closes #3
Closes #10
Closes #11
 2026-06-21 20:38:21 +03:00
Esa Kataja 8fc00c96b9 add: DB-backed settings UI for live-editable tunables 
Add a settings page (/settings) and API (/api/settings GET/PUT) to edit
encoding profiles (crf/preset per media type), lp, OMDb key, log retention,
and delete-originals from the browser, persisted to a settings table in
logs.db. config.yaml seeds the store on first run; afterwards the DB is the
source of truth (paths.* and http_addr stay config-only). Each job snapshots
the current settings, so changes apply to the next encode with no restart.

PUT validates ranges (crf 0-63, preset 0-13, lp >=0, retention >=1) before
persisting. No auth, by design for now (isolated network) — see #2.

Closes #1
 2026-06-21 19:51:35 +03:00
Esa Kataja bef58f480b fix: anchor av1dae binary gitignore to repo root 
The unanchored 'av1dae' pattern (from the rename) also matched the cmd/av1dae
source dir. Anchor to /av1dae and /av1dae-dev so only the root-level built
binaries are ignored.
 2026-06-21 19:39:05 +03:00
Esa Kataja 80b602fbbb add: SVT-AV1 lp (thread cap) config knob 
Add encoding.lp (0 = auto) so a long encode can be capped to N logical
processors instead of pinning every core, as a lighter-weight alternative to
the compose cgroup limit. Threaded config -> types.Job -> -svtav1-params via a
testable svtav1Params helper. Settings-UI surfacing stays with #1.

refs #5
 2026-06-21 19:37:26 +03:00
Esa Kataja 9bb7c72c1a refactor: rename project videnc-vibe -> av1dae 
Rename the Go module, the cmd/ entrypoint dir, the binary, the default config
dir (~/.config/av1dae/), the Docker image/compose service, and all docs and
the dashboard wordmark. No behavioral change — import paths and identifiers
only.
 2026-06-21 19:02:06 +03:00
Esa Kataja a6dc0018ac add: throughput sparkline and ETA-first status hero 
Restructure the current-job card around the question the monitor exists to
answer: time remaining is now the hero, with percent riding the bar. Add a
client-side throughput sparkline (90 s rolling buffer of encode speed against
a fixed 1.0x realtime reference) and a health-colored speed gauge so stalls
and sub-realtime encoding are visible at a glance. Collapse the repetitive
input queue into a count plus shared-prefix rows.
 2026-06-21 18:37:29 +03:00
Esa Kataja 35193c695e add: show metadata, streams, and clock-time ETA in status UI 
Surface the fetched job metadata (show/episode or movie title, season,
episode, release/airdate, media type) and the source audio/subtitle streams
through the tracker and /status, and render them in the dashboard's current-job
card. Audio channel counts are now probed and shown as 5.1/2.0/etc.

ETA now reads as a wall-clock finish time plus remaining duration, e.g.
"17:49 (3h52m)". Recent events drop the per-row date for a WhatsApp-style
Today/Yesterday/date divider with time-only rows. All dynamic strings are
HTML-escaped, since metadata and filenames are user-controlled.
 2026-06-21 18:23:36 +03:00
Esa Kataja d38c3ad568 chore: expose status port in Docker/compose and document http_addr 
EXPOSE 8080 in the image, publish 8080 in compose, document http_addr in the
example config, and add the CPU/memory limits so a long encode can't starve
the host.
 2026-06-21 17:43:13 +03:00
Esa Kataja aa8a341069 add: live encode progress tracking and read-only status web UI 
Stream ffmpeg -progress during the video encode into an in-memory tracker
(internal/status) so the long-running step is no longer a black box: percent,
fps, speed, and ETA are derived from the source duration and updated ~1/s.
Progress prints to stdout only (no logs.db row) to avoid burying real events.

Expose it over HTTP (internal/server, default :8080, set http_addr to "" to
disable): GET /status returns the live snapshot, the pending input queue, and
recent non-debug events; GET / serves an embedded dashboard that polls /status
every second. The server shuts down on the same SIGINT/SIGTERM context as the
watcher.
 2026-06-21 17:43:00 +03:00
Esa Kataja 4147cb9470 docs: add HTML user manual 2026-06-21 17:08:32 +03:00
Esa Kataja ef52d7856c add: Docker image and compose for containerized deploy 
Multi-stage build (cgo for the SQLite logger) on debian-slim with ffmpeg,
opus-tools, and ca-certificates bundled. Compose mounts ./data/config.yaml
and ./data/media; SIGTERM cancels in-flight encodes cleanly.
 2026-06-21 17:08:32 +03:00
Esa Kataja 3170f3bbb5 chore: ignore dev binary, mp4 inputs, db files, and data dir 2026-06-21 17:08:32 +03:00
Esa Kataja f39f5b1b16 add: SQLite-backed structured logging with retention 
Replace JSON/file logging with a logs.db (WAL) store. Thread the logger
through the encoder and metadata client for debug instrumentation of every
ffmpeg/ffprobe/opusenc invocation and OMDb/TVmaze request. API keys are
redacted before request URLs are logged. Retention defaults to 7 days,
overridable via log_retention_days in config.
 2026-06-21 17:08:23 +03:00
Esa Kataja 6a564aabb2 fix: detect mp4 and other common containers in watcher 
The watcher only globbed *.mkv, silently ignoring every other source
container. Expand the glob to a list of common input extensions
(mkv, mp4, m4v, mov, avi, ts, m2ts, mts, mpg, mpeg, vob, webm, wmv,
flv). Downstream is unaffected: workdir naming uses filepath.Ext, the
encoder always emits .mkv output, and the metadata regexes are not
end-anchored.
 2026-05-18 08:34:55 +03:00
Esa Kataja 321f11dd8f chore: document SIGINT cancellation in §4 
After bug #9's context plumbing, a signal interrupts the current encode
immediately rather than waiting for the poll cycle. Document the new
shutdown semantics: children killed, work dir cleaned via the deferred
RemoveAll, source routed to failed/.
 2026-05-16 20:39:20 +03:00
Esa Kataja 459ab30d94 fix: cancel in-flight encode on SIGINT/SIGTERM via context plumbing 
Thread context.Context from main through watcher, encoder, and metadata
clients so a Ctrl-C during a multi-hour encode immediately kills the
ffmpeg/ffprobe/opusenc children instead of waiting for them to finish.

- main: signal.NotifyContext replaces the manual sigChan + done goroutine
- watcher.Start: takes ctx, exits on ctx.Done(); processFn signature is
  now func(context.Context, string) error
- encoder: Transcode and every helper (extractAudio, encodeOpus,
  encodeVideo, GetMediaInfo, GetStreamLanguages, calculateZscaleWidth)
  take ctx; every exec.Command becomes exec.CommandContext so the child
  is SIGKILL'd on cancel
- metadata: FetchMovieMetadata, FetchSeriesMetadata, fetchTVMazeJSON
  take ctx and use http.NewRequestWithContext

Mover stays ctx-free intentionally: a rename is fast enough that
mid-cancel cleanup is the next-restart's problem. processFile's
deferred RemoveAll(workDir) and failToFailed still run after cancel,
so partial output dies in the work dir and the source moves to failed/.
 2026-05-16 20:38:26 +03:00
Esa Kataja 191bc955e4 chore: document watcher stability, collision refusal, EXDEV fallback 
§9: add notes on destination-collision refusal (silent overwrite was
bug #14) and cross-filesystem move fallback (bug #13). Both are now
implemented but the manual didn't reflect them.

§10: drop the "no atomic-write detection" warning. The watcher now
requires mtime+size stability across two ticks before processing (bug
#22), and quarantines a failing file for 5 minutes — so the
copy-then-mv workaround is no longer a correctness requirement.
 2026-05-16 20:34:11 +03:00
Esa Kataja 706cd22b05 fix: isolate per-job intermediates in paths.work and harden failure cleanup 
Adds a per-job scratch directory under the new `paths.work` config (default
`./work`) so audio.<n>.wav, audio.<n>.opus and output.mkv no longer live
beside the user's sources in paths.input. The work subdir is named after
the input basename and unconditionally removed when processFile returns
(success or failure), which kills bug #4 (intermediates leaking into the
user-owned input folder; output.mkv getting re-picked by the 15-second
watcher tick on rename failure; fixed-name collision risk for any future
concurrency).

Tightens the failure paths in main.processFile too (bug #25):
- mover.MoveToFailed return values are now surfaced via a new
  failToFailed helper.
- The helper os.Stats the source first; missing -> log and skip instead
  of the previous silent no-op when MoveToFailed was called on
  output.mkv before it existed.
- On rename-output failure, the source is now routed to paths.failed
  (it was previously left in paths.input, causing an infinite re-encode
  loop on the next watcher tick). The old MoveToFailed on the work-dir
  output is dropped — the deferred RemoveAll covers it.

Mechanical changes:
- PathsConfig gains `Work string \`yaml:"work"\`` with default ./work,
  included in EnsureDirs.
- Encoder.Transcode signature now takes workDir; extractAudio,
  encodeOpus and encodeVideo all write into workDir. The internal
  cleanupWavs/cleanupOpus defers are gone (RemoveAll in main is the
  one cleanup path).
- MANUAL.md updated: example config, field reference, §6 pipeline
  step wording, §9 failure handling description, §11 runtime
  directories block.
 2026-05-16 20:32:23 +03:00
Esa Kataja 22b3a73109 fix: refuse destination collisions and fall back on EXDEV in mover 
Centralize all move helpers through a single safeRename function that
stats the destination first and refuses to overwrite (bug #14), and
falls back to copy+fsync+remove when os.Rename returns EXDEV across
mounted filesystems (bug #13). Drops the dead MoveToOutput helper.
 2026-05-16 20:32:19 +03:00
Esa Kataja 6b72ef03ac fix: add mtime+size stability check and failure quarantine to watcher 
Two related hardening fixes for the input folder poller (bug #22):

1. Partial-write protection. A freshly-dropped .mkv now has to present
   the same (mtime, size) on two consecutive ticks before processFn is
   called. A file mid-rsync that grows or has a moving mtime is skipped
   until it settles. The per-file (mtime, size) state lives in
   Watcher.seen and is rebuilt from the current glob each tick so the
   map cannot grow unboundedly.

2. Failure quarantine. When processFn returns an error, the file's
   (failedAt, mtime) is recorded in Watcher.failed and subsequent ticks
   skip it until either the backoff elapses or its mtime changes (user
   replaced or touched it). Previously a permanently-broken input -- e.g.
   a video-only mkv that trips the "no audio streams found" path added
   in the multi-audio fix -- would be retried and logged every 15 s
   forever.

Backoff is 5 minutes: comfortably longer than the 10-30 s polling
interval clamp so we are not effectively retrying every tick, but
short enough that an operator fixing the underlying problem by
replacing the file sees it picked up promptly on the next stable scan.
 2026-05-16 20:32:16 +03:00
Esa Kataja 8b38986690 chore: bring MANUAL.md in line with current encoder behavior 
§6 step 2: drop the codec-name allowlist, describe the bounded idet run
and the Multi-frame summary parse.

§6 step 6/7: per-stream audio.<n>.wav / audio.<n>.opus now.

§6 step 8/9: zscale is conditional and width rounds to mod-2; -vf is
omitted entirely when no filter applies. Audio language tags are indexed
by output position. TITLE/COLLECTION values are unquoted.

§6 step 11: series filename no longer requires an IMDb mapping.

§7: added a token example and notes about what the token does and does
not influence in the output filename.

§8: noted the known log-rotation limitation (filename frozen at start;
restart to rotate; structured.json does not rotate).
 2026-05-16 20:20:05 +03:00
Esa Kataja 244bdce586 add: WebDL / TVRip media types and filename-token override 
Source kind is now declared per-file via a `dvd` / `bluray` / `webdl` /
`tvrip` token in the basename (case-insensitive, word-bounded). Matches
the existing `tt…` / `tvm…` filename convention. When no token is
present, falls back to the pixel-count guess in DetectMediaType, which
still only chooses between DVD and Blu-ray.

The parsed media type drives both ORIGINAL_MEDIA_TYPE in the muxed
metadata and the CRF/preset profile used for encoding. EncodingConfig
gains `webdl` and `tvrip` sub-blocks with conservative defaults
(WebDL 30/3, TVRip 32/2); user can override in config.yaml.

Manual updated with the new tokens, config fields, and pipeline
description.
 2026-05-16 20:17:10 +03:00
Esa Kataja 022d131cd6 fix: preserve all audio tracks and align language metadata indices 
Bug #5: extractAudio invoked ffmpeg without `-map`, so default stream
selection kept only one audio track from sources with multiple audio
streams (e.g. eng/fra/jpn Blu-rays). It now enumerates audio streams
from the streamLangs already fetched by the caller and runs one
`ffmpeg -map 0🅰️<n>` per stream, writing audio.<n>.wav.

Bug #11: the audio language metadata loop in encodeVideo computed the
output index by counting source-side audio streams with a lower Index,
which drifted when some source streams lacked a language tag. It now
walks opusFiles in output order and looks up the language at the
matching source-audio position via a sorted helper.

These ship together because #11 was masked by #5: when only one audio
track survived extraction, the broken index calculation never produced
a visible misalignment. Fixing #5 alone would have caused multi-track
outputs with shuffled language tags; both fixes are required to land
correct multi-track output.
 2026-05-16 20:07:59 +03:00
29 changed files with 3531 additions and 465 deletions
Expand all files Collapse all files
.dockerignore
+16
View File
@@ -0,0 +1,16 @@
av1dae
*.mkv
*.mp4
*.wav
*.opus
*.log
logs.db
*.json
/input
/output
/originals
/failed
/work
/data
.git
MANUAL.html
.gitignore
+6 -1
View File
@@ -1,11 +1,16 @@
videnc-vibe /av1dae
/av1dae-dev
*.mkv *.mkv
*.mp4
*.wav *.wav
*.opus *.opus
*.log *.log
*.json *.json
*.db
/input /input
/output /output
/originals /originals
/failed /failed
BUGS.md BUGS.md
data/*
CLAUDE.md
+8 -8
View File
@@ -1,6 +1,6 @@
# AGENTS.md # AGENTS.md
This file provides guidance for agents operating in the videnc-vibe repository. This file provides guidance for agents operating in the av1dae repository.
## Project Overview ## Project Overview
@@ -10,14 +10,14 @@ Go-based CLI tool for transcoding DVD/Blu-ray to SVT-AV1 with automatic metadata
```bash ```bash
# Build binary # Build binary
go build -o videnc-vibe ./cmd/videnc/ go build -o av1dae ./cmd/av1dae/
# Run binary # Run binary
./videnc-vibe ./av1dae
# With flags # With flags
./videnc-vibe -d # Delete original after encode ./av1dae -d # Delete original after encode
./videnc-vibe -c /path/to/config.yaml ./av1dae -c /path/to/config.yaml
``` ```
## Code Style Guidelines ## Code Style Guidelines
@@ -29,7 +29,7 @@ go build -o videnc-vibe ./cmd/videnc/
### Imports ### Imports
- Group imports: standard library, external packages, internal packages - Group imports: standard library, external packages, internal packages
- Use aliases for packages: `"videnc-vibe/pkg/types"` - Use aliases for packages: `"av1dae/pkg/types"`
```go ```go
import ( import (
@@ -37,7 +37,7 @@ import (
"os" "os"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"videnc-vibe/pkg/types" "av1dae/pkg/types"
) )
``` ```
@@ -84,7 +84,7 @@ Example: `add: add command line argument support`
## Project Structure ## Project Structure
``` ```
cmd/videnc/main.go # CLI entrypoint cmd/av1dae/main.go # CLI entrypoint
internal/ internal/
config/ # Config loading config/ # Config loading
watcher/ # Folder polling watcher/ # Folder polling
Dockerfile
+18
View File
@@ -0,0 +1,18 @@
# Build needs cgo (the logger uses github.com/mattn/go-sqlite3), so no scratch/static image.
FROM golang:1.26-bookworm AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=1 go build -ldflags="-s -w" -o /av1dae ./cmd/av1dae/
FROM debian:bookworm-slim
# ffmpeg gives ffmpeg+ffprobe; opus-tools gives opusenc; ca-certificates for OMDb/TVmaze HTTPS.
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg opus-tools ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=build /av1dae /usr/local/bin/av1dae
# logs.db + log files land in the working dir — make it the mounted /data so they persist.
WORKDIR /data
EXPOSE 8080
ENTRYPOINT ["av1dae", "-c", "/config/config.yaml"]
MANUAL.html
+662
View File
@@ -0,0 +1,662 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>av1dae — User Manual</title>
<style>
:root {
--bg:#0d1117; --panel:#151b23; --panel-2:#1a2230; --line:#26303f;
--text:#cdd5df; --muted:#7d8896; --dim:#5a6573;
--amber:#ffb454; --amber-soft:#ffd9a0; --cyan:#56c7e8; --green:#5cc98b; --red:#f0816a;
--mono:"SF Mono",ui-monospace,"JetBrains Mono","Cascadia Code",Menlo,Consolas,monospace;
--sans:"Inter",system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;
--measure:64ch;
}
* { box-sizing:border-box; }
html { scroll-behavior:smooth; }
@media (prefers-reduced-motion:reduce){ html{scroll-behavior:auto;} *{transition:none!important;} }
body {
margin:0; background:var(--bg); color:var(--text);
font-family:var(--sans); font-size:16px; line-height:1.65;
-webkit-font-smoothing:antialiased;
}
a { color:var(--cyan); text-decoration:none; }
a:hover { text-decoration:underline; }
code { font-family:var(--mono); font-size:.86em; }
:focus-visible { outline:2px solid var(--amber); outline-offset:3px; border-radius:3px; }
/* ---- hero ---- */
.hero {
border-bottom:1px solid var(--line);
background:
radial-gradient(120% 140% at 85% -10%, rgba(255,180,84,.10), transparent 55%),
radial-gradient(90% 120% at 5% 0%, rgba(86,199,232,.06), transparent 50%);
padding:clamp(2.5rem,6vw,4.5rem) clamp(1.25rem,5vw,3rem) clamp(2rem,4vw,3rem);
}
.hero-inner { max-width:1180px; margin:0 auto; }
.wordmark {
font-family:var(--mono); font-weight:600;
font-size:clamp(2.1rem,6vw,3.6rem); letter-spacing:-.02em; line-height:1;
margin:0; color:var(--text);
}
.wordmark .prompt { color:var(--dim); }
.wordmark .vibe { color:var(--amber); }
.thesis {
max-width:60ch; margin:1.4rem 0 0; font-size:clamp(1.05rem,2.2vw,1.3rem);
line-height:1.5; color:var(--text);
}
.thesis b { color:var(--amber-soft); font-weight:600; }
.flow {
display:flex; flex-wrap:wrap; align-items:center; gap:.35rem .15rem;
margin:2rem 0 0; font-family:var(--mono); font-size:.8rem;
}
.flow .node {
background:var(--panel); border:1px solid var(--line); color:var(--text);
padding:.35rem .7rem; border-radius:5px; white-space:nowrap;
}
.flow .node.accent { border-color:var(--amber); color:var(--amber); }
.flow .arr { color:var(--dim); padding:0 .15rem; }
.specstrip {
display:flex; flex-wrap:wrap; gap:.5rem .6rem; margin:1.6rem 0 0;
font-family:var(--mono); font-size:.74rem; color:var(--muted);
}
.specstrip span { display:inline-flex; align-items:center; gap:.4rem; }
.specstrip span::before { content:""; width:6px; height:6px; border-radius:50%; background:var(--green); }
/* ---- layout ---- */
.shell { max-width:1180px; margin:0 auto; display:grid; grid-template-columns:240px 1fr; gap:0; }
.toc {
border-right:1px solid var(--line);
align-self:start; position:sticky; top:0; max-height:100vh; overflow-y:auto;
padding:2rem 1.25rem 3rem;
}
.toc > summary { display:none; }
.toc .toc-title {
font-family:var(--mono); font-size:.7rem; letter-spacing:.18em; text-transform:uppercase;
color:var(--dim); margin:0 0 1rem .5rem;
}
.toc ol { list-style:none; margin:0; padding:0; counter-reset:toc; }
.toc li { counter-increment:toc; }
.toc a {
display:flex; gap:.6rem; padding:.32rem .5rem; border-radius:6px;
color:var(--muted); font-size:.88rem; line-height:1.3; border-left:2px solid transparent;
}
.toc a::before { content:counter(toc,decimal-leading-zero); color:var(--dim); font-family:var(--mono); font-size:.72rem; }
.toc a:hover { color:var(--text); background:var(--panel); text-decoration:none; }
.toc a.active { color:var(--amber); border-left-color:var(--amber); background:var(--panel); }
.toc a.active::before { color:var(--amber); }
main { padding:2.5rem clamp(1.25rem,4vw,3.5rem) 6rem; min-width:0; }
/* ---- sections ---- */
section { padding:2.5rem 0; border-top:1px solid var(--line); scroll-margin-top:1.5rem; }
section:first-child { border-top:0; padding-top:.5rem; }
.eyebrow {
font-family:var(--mono); font-size:.72rem; letter-spacing:.14em; text-transform:uppercase;
color:var(--amber); margin:0 0 .5rem;
}
h2 { font-size:clamp(1.5rem,3vw,2rem); line-height:1.15; margin:0 0 1.2rem; letter-spacing:-.01em; }
h3 { font-size:1.12rem; margin:2rem 0 .6rem; color:var(--amber-soft); }
p, li { max-width:var(--measure); }
main ul, main ol { padding-left:1.3rem; }
main li { margin:.3rem 0; }
strong { color:#e7edf4; }
.lead { font-size:1.08rem; color:var(--text); }
/* inline code */
p code, li code, td code, h3 code, summary code {
background:var(--panel-2); border:1px solid var(--line); color:var(--amber-soft);
padding:.08em .4em; border-radius:4px;
}
/* ---- code blocks ---- */
.pre {
position:relative; margin:1.2rem 0; background:#0a0e14; border:1px solid var(--line);
border-radius:9px; overflow:hidden;
}
.pre::before {
content:"› shell"; display:block; font-family:var(--mono); font-size:.68rem; letter-spacing:.1em;
color:var(--dim); padding:.5rem .9rem; border-bottom:1px solid var(--line); background:var(--panel);
}
.pre.is-file::before { content:"⌗ file"; }
.pre pre { margin:0; padding:1rem 1.1rem; overflow-x:auto; font-family:var(--mono); font-size:.84rem; line-height:1.6; color:var(--text); }
.pre .cm { color:var(--dim); }
.copy {
position:absolute; top:.42rem; right:.5rem; font-family:var(--mono); font-size:.68rem;
background:var(--panel-2); color:var(--muted); border:1px solid var(--line); border-radius:5px;
padding:.25rem .55rem; cursor:pointer; transition:color .15s,border-color .15s;
}
.copy:hover { color:var(--amber); border-color:var(--amber); }
.copy.ok { color:var(--green); border-color:var(--green); }
/* ---- tables ---- */
.tablewrap { overflow-x:auto; margin:1.2rem 0; border:1px solid var(--line); border-radius:9px; }
table { border-collapse:collapse; width:100%; font-size:.9rem; }
th, td { text-align:left; padding:.6rem .85rem; border-bottom:1px solid var(--line); vertical-align:top; }
thead th {
background:var(--panel); font-family:var(--mono); font-size:.7rem; letter-spacing:.08em;
text-transform:uppercase; color:var(--muted); font-weight:500;
}
tbody tr:last-child td { border-bottom:0; }
tbody tr:hover { background:rgba(255,180,84,.03); }
td:first-child code { white-space:nowrap; }
/* ---- profile cards ---- */
.profiles { display:grid; grid-template-columns:repeat(auto-fit,minmax(150px,1fr)); gap:.8rem; margin:1.4rem 0; }
.card {
background:var(--panel); border:1px solid var(--line); border-radius:10px; padding:1rem 1.1rem;
}
.card .pname { font-family:var(--mono); font-size:.78rem; letter-spacing:.06em; color:var(--amber); text-transform:uppercase; }
.card .pdesc { font-size:.78rem; color:var(--muted); margin:.2rem 0 .9rem; }
.card .pval { display:flex; justify-content:space-between; font-family:var(--mono); font-size:.82rem; padding:.2rem 0; }
.card .pval span { color:var(--dim); }
.card .pval b { color:var(--text); font-weight:600; }
/* ---- pipeline (signature) ---- */
.pipe { list-style:none; margin:1.6rem 0 0; padding:0; counter-reset:step; position:relative; }
.pipe > li {
position:relative; padding:0 0 1.6rem 3.2rem; counter-increment:step; max-width:none;
}
.pipe > li::before {
content:counter(step,decimal-leading-zero);
position:absolute; left:0; top:0; width:2.2rem; height:2.2rem; border-radius:7px;
display:grid; place-items:center; font-family:var(--mono); font-size:.78rem; font-weight:600;
color:var(--amber); background:var(--panel); border:1px solid var(--line); z-index:1;
}
.pipe > li::after {
content:""; position:absolute; left:1.1rem; top:2.2rem; bottom:-.2rem; width:1px; background:var(--line);
}
.pipe > li:last-child::after { display:none; }
.pipe .step-title { font-weight:600; color:#e7edf4; margin:.25rem 0 .35rem; }
.pipe .step-title b { color:var(--amber-soft); }
.pipe ul { margin:.4rem 0 0; padding-left:1.1rem; }
.pipe ul li { font-size:.92rem; color:var(--text); max-width:var(--measure); }
/* ---- callouts ---- */
.note {
border-left:3px solid var(--cyan); background:rgba(86,199,232,.05);
padding:.85rem 1.1rem; border-radius:0 8px 8px 0; margin:1.2rem 0; max-width:var(--measure);
}
.note.warn { border-left-color:var(--amber); background:rgba(255,180,84,.06); }
.note .tag { font-family:var(--mono); font-size:.7rem; letter-spacing:.1em; text-transform:uppercase; color:var(--cyan); display:block; margin-bottom:.3rem; }
.note.warn .tag { color:var(--amber); }
.note p { margin:0; max-width:none; }
details.facts { margin:1.2rem 0; border:1px solid var(--line); border-radius:9px; background:var(--panel); max-width:var(--measure); }
details.facts > summary { cursor:pointer; padding:.75rem 1rem; font-weight:600; color:var(--amber-soft); list-style:none; }
details.facts > summary::-webkit-details-marker { display:none; }
details.facts > summary::before { content:"▸ "; color:var(--amber); }
details.facts[open] > summary::before { content:"▾ "; }
details.facts .body { padding:0 1rem 1rem; }
details.facts ul { margin:.3rem 0 0; }
footer { border-top:1px solid var(--line); padding:2rem clamp(1.25rem,4vw,3.5rem); color:var(--dim); font-family:var(--mono); font-size:.78rem; max-width:1180px; margin:0 auto; }
/* ---- responsive ---- */
@media (max-width:860px){
.shell { grid-template-columns:1fr; }
.toc {
position:sticky; top:0; z-index:5; border-right:0; border-bottom:1px solid var(--line);
max-height:none; padding:0; background:var(--bg);
}
.toc > summary {
display:block; cursor:pointer; padding:.9rem 1.25rem; font-family:var(--mono);
font-size:.78rem; letter-spacing:.1em; text-transform:uppercase; color:var(--amber); list-style:none;
}
.toc > summary::-webkit-details-marker { display:none; }
.toc > summary::after { content:" ▾"; color:var(--dim); }
.toc[open] > summary::after { content:" ▴"; }
.toc .toc-title { display:none; }
.toc ol { padding:0 1rem 1rem; }
}
</style>
</head>
<body>
<header class="hero">
<div class="hero-inner">
<h1 class="wordmark"><span class="prompt">$ </span>av<span class="vibe">1</span>dae</h1>
<p class="thesis">A folder-watching daemon that turns <b>.mkv</b> rips into <b>SVT-AV1 + Opus</b>, tags them with metadata from OMDb and TVmaze, and files the results — untouched by you after the drop.</p>
<div class="flow" aria-label="Processing flow">
<span class="node">drop .mkv</span><span class="arr">→</span>
<span class="node">ffprobe</span><span class="arr">→</span>
<span class="node">extract wav</span><span class="arr">→</span>
<span class="node">opus 128k</span><span class="arr">→</span>
<span class="node accent">SVT-AV1</span><span class="arr">→</span>
<span class="node">mux + tag</span><span class="arr">→</span>
<span class="node">output/</span>
</div>
<div class="specstrip">
<span>libsvtav1 · yuv420p10le</span>
<span>opus 128k</span>
<span>OMDb · TVmaze</span>
<span>polls every 15s</span>
<span>docker ready</span>
</div>
</div>
</header>
<div class="shell">
<details class="toc" id="toc" open>
<summary>Contents</summary>
<p class="toc-title">User Manual</p>
<ol>
<li><a href="#requirements">Requirements</a></li>
<li><a href="#build">Build</a></li>
<li><a href="#config">Configuration</a></li>
<li><a href="#running">Running</a></li>
<li><a href="#filenames">Filename convention</a></li>
<li><a href="#pipeline">Processing pipeline</a></li>
<li><a href="#naming">Output naming</a></li>
<li><a href="#logs">Logs</a></li>
<li><a href="#failures">Failure handling</a></li>
<li><a href="#polling">Polling behavior</a></li>
<li><a href="#layout">Project layout</a></li>
<li><a href="#docker">Deploy with Docker</a></li>
</ol>
</details>
<main>
<section id="requirements">
<p class="eyebrow">01 — Prerequisites</p>
<h2>Requirements</h2>
<p>These external binaries must be on <code>$PATH</code>. The program checks them at startup and exits if any are missing:</p>
<div class="tablewrap">
<table>
<thead><tr><th>Binary</th><th>Role</th></tr></thead>
<tbody>
<tr><td><code>ffmpeg</code></td><td>Video transcode + interlace detection</td></tr>
<tr><td><code>ffprobe</code></td><td>Stream + language probing</td></tr>
<tr><td><code>opusenc</code></td><td>Audio encoding</td></tr>
</tbody>
</table>
</div>
<p><strong>To build:</strong> Go 1.x with module support.</p>
<p><strong>API keys:</strong> an OMDb API key (free at <a href="https://www.omdbapi.com/">omdbapi.com</a>) is required for movie metadata. TVmaze is unauthenticated.</p>
<div class="note"><span class="tag">Running in Docker?</span><p>The three binaries above are baked into the image — the only host requirement is Docker. Skip to <a href="#docker">§12 — Deploy with Docker</a>.</p></div>
</section>
<section id="build">
<p class="eyebrow">02 — Compile</p>
<h2>Build</h2>
<div class="pre"><pre><code>go build -o av1dae ./cmd/av1dae/</code></pre></div>
<p>This produces a self-contained binary <code>./av1dae</code> (cgo is used for the SQLite logger, so it links against the system libc). To skip installing Go and the encoders on the host entirely, build the container instead — <a href="#docker">§12</a>.</p>
</section>
<section id="config">
<p class="eyebrow">03 — Setup</p>
<h2>Configuration</h2>
<p>By default the config loads from <code>~/.config/av1dae/config.yaml</code>. Pass <code>-c /path/to/config.yaml</code> to override.</p>
<div class="pre is-file"><pre><code>omdb_api_key: "YOUR_API_KEY_HERE"
encoding:
dvd: { crf: 30, preset: 2 }
bluray: { crf: 29, preset: 3 }
webdl: { crf: 30, preset: 3 }
tvrip: { crf: 32, preset: 2 }
paths:
input: "./input"
output: "./output"
originals: "./originals"
failed: "./failed"
work: "./work"</code></pre></div>
<h3>Encoding profiles</h3>
<p>Each source type carries its own SVT-AV1 quality pair. The type is chosen by a filename token, or guessed from pixel count (see <a href="#filenames">§5</a>).</p>
<div class="profiles">
<div class="card">
<div class="pname">dvd</div><div class="pdesc">SD sources</div>
<div class="pval"><span>crf</span><b>30</b></div>
<div class="pval"><span>preset</span><b>2</b></div>
</div>
<div class="card">
<div class="pname">bluray</div><div class="pdesc">HD sources</div>
<div class="pval"><span>crf</span><b>29</b></div>
<div class="pval"><span>preset</span><b>3</b></div>
</div>
<div class="card">
<div class="pname">webdl</div><div class="pdesc">Token only</div>
<div class="pval"><span>crf</span><b>30</b></div>
<div class="pval"><span>preset</span><b>3</b></div>
</div>
<div class="card">
<div class="pname">tvrip</div><div class="pdesc">Token only</div>
<div class="pval"><span>crf</span><b>32</b></div>
<div class="pval"><span>preset</span><b>2</b></div>
</div>
</div>
<h3>Field reference</h3>
<div class="tablewrap">
<table>
<thead><tr><th>Field</th><th>Meaning</th><th>Default</th></tr></thead>
<tbody>
<tr><td><code>omdb_api_key</code></td><td>OMDb key for movie metadata lookup</td><td><em>none — movies get no metadata without it</em></td></tr>
<tr><td><code>paths.input</code></td><td>Folder polled for new <code>.mkv</code> files</td><td><code>./input</code></td></tr>
<tr><td><code>paths.output</code></td><td>Destination for finished encodes</td><td><code>./output</code></td></tr>
<tr><td><code>paths.originals</code></td><td>Where sources move on success (unless <code>-d</code>)</td><td><code>./originals</code></td></tr>
<tr><td><code>paths.failed</code></td><td>Where sources go on failure</td><td><code>./failed</code></td></tr>
<tr><td><code>paths.work</code></td><td>Per-job scratch (wav/opus/output.mkv); wiped each job</td><td><code>./work</code></td></tr>
</tbody>
</table>
</div>
<div class="note"><span class="tag">Note</span><p>All five directories are created on startup if they don't already exist.</p></div>
</section>
<section id="running">
<p class="eyebrow">04 — Operate</p>
<h2>Running</h2>
<div class="pre"><pre><code>./av1dae <span class="cm"># default config, keep originals</span>
./av1dae -d <span class="cm"># delete originals after success</span>
./av1dae -c /etc/av1dae.yaml <span class="cm"># custom config</span>
./av1dae -c /etc/av1dae.yaml -d</code></pre></div>
<div class="tablewrap">
<table>
<thead><tr><th>Flag</th><th>Effect</th></tr></thead>
<tbody>
<tr><td><code>-d</code></td><td>Delete the source <code>.mkv</code> after a successful encode, instead of moving it to <code>originals/</code>.</td></tr>
<tr><td><code>-c PATH</code></td><td>Path to config file.</td></tr>
</tbody>
</table>
</div>
<div class="note warn"><span class="tag">Shutdown</span><p>Stop with <code>Ctrl+C</code> (SIGINT) or <code>SIGTERM</code>. The signal cancels any in-flight encode immediately — the ffmpeg / ffprobe / opusenc children are killed, the per-job work directory is removed by its deferred cleanup, and the source <code>.mkv</code> is routed to <code>paths.failed</code> so the next run sees a clean <code>paths.input</code>.</p></div>
<p>The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter.</p>
</section>
<section id="filenames">
<p class="eyebrow">05 — Input</p>
<h2>Filename convention</h2>
<p class="lead">The base name of each <code>.mkv</code> in <code>paths.input</code> is parsed to decide what metadata to fetch. Three forms are recognized.</p>
<h3>5.1 — Movies: IMDb ID</h3>
<p>Filename must contain an IMDb tag of the form <code>tt&lt;digits&gt;</code>.</p>
<div class="pre is-file"><pre><code>Heat.tt0113277.mkv
some-rip-tt0114369.mkv</code></pre></div>
<p>→ OMDb is queried for that IMDb ID. Title, release date, and IMDb ID are embedded.</p>
<h3>5.2 — Series: TVmaze ID + season/episode</h3>
<p>Filename must contain <strong>both</strong> <code>tvm&lt;digits&gt;</code> (the TVmaze show ID) and <code>s&lt;digits&gt;e&lt;digits&gt;</code> (season/episode). Both are case-insensitive.</p>
<div class="pre is-file"><pre><code>Breaking.Bad.tvm169.S01E01.mkv
the-wire.TVM75.s2e5.mkv</code></pre></div>
<p>→ TVmaze is queried for that show/season/episode. Show name (Collection), episode title, season, episode, airdate, and the show's IMDb ID are embedded.</p>
<h3>5.3 — No recognizable tags</h3>
<p>If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an <code>.nometadata.mkv</code> suffix.</p>
<h3>5.4 — Optional: source media type</h3>
<p>Any filename can additionally carry a media-type token (case-insensitive, word-bounded): <code>dvd</code>, <code>bluray</code>, <code>webdl</code>, or <code>tvrip</code>.</p>
<div class="pre is-file"><pre><code>Heat.tt0113277.bluray.mkv
some-rip.tvm169.S01E01.webdl.mkv
old.broadcast.tvrip.tt0066026.mkv</code></pre></div>
<p>The token controls <strong>both</strong> the <code>ORIGINAL_MEDIA_TYPE</code> metadata tag and which <code>encoding.&lt;type&gt;</code> crf/preset pair is used.</p>
<div class="note warn"><span class="tag">Fallback</span><p>With no token, the type is guessed from pixel count: <code>width × height &lt; 600,000</code> → DVD, otherwise Blu-ray. <strong>WebDL and TVRip are never auto-detected</strong> — they must be declared via the token.</p></div>
</section>
<section id="pipeline">
<p class="eyebrow">06 — Signature flow</p>
<h2>The processing pipeline</h2>
<p>For every <code>.mkv</code> in <code>paths.input</code>, these steps run in order. A per-job scratch subdir under <code>paths.work</code> (named after the input base name) holds all intermediates and is deleted unconditionally at the end. Any error sends the source to <code>paths.failed</code>; the work subdir is wiped regardless.</p>
<ol class="pipe">
<li>
<p class="step-title">Parse filename</p>
Determines whether this is a movie or series, and what IDs to use.
</li>
<li>
<p class="step-title">Probe video <b>(ffprobe)</b></p>
<ul>
<li>Picks the first stream with <code>codec_type=video</code>, regardless of codec. Errors out if there isn't one.</li>
<li>Records width, height, and sample aspect ratio (SAR).</li>
<li>Detects interlacing via <code>ffmpeg -vf idet -frames:v 400 -an -sn -f null -</code>, parsing the <code>Multi frame detection</code> summary. Interlaced only when <code>TFF+BFF &gt; Progressive</code>; undetermined frames are ignored, a missing line defaults to progressive.</li>
</ul>
</li>
<li>
<p class="step-title">Probe stream languages <b>(ffprobe)</b></p>
Collects <code>language</code> tags for every audio/subtitle stream so they survive — <code>-map_metadata -1</code> strips them otherwise.
</li>
<li>
<p class="step-title">Detect media type</p>
<ul>
<li>Filename token (<code>dvd</code>/<code>bluray</code>/<code>webdl</code>/<code>tvrip</code>) wins if present.</li>
<li>Otherwise pixel count: <code>w × h &lt; 600,000</code> → DVD, else Blu-ray.</li>
</ul>
The chosen profile selects the crf/preset pair and is written to <code>ORIGINAL_MEDIA_TYPE</code>.
</li>
<li>
<p class="step-title">Fetch metadata</p>
From OMDb or TVmaze per the parsed filename. Failures here are logged but <strong>do not abort</strong> the encode — the file is just encoded without metadata.
</li>
<li>
<p class="step-title">Extract audio</p>
One PCM wav per source audio stream → <code>audio.0.wav</code>, <code>audio.1.wav</code>, … (PCM s16le, 48 kHz). Errors out if there are no audio streams.
</li>
<li>
<p class="step-title">Encode audio <b>(opusenc)</b></p>
Each wav → <code>audio.&lt;n&gt;.opus</code> at <code>--bitrate 128k</code>.
</li>
<li>
<p class="step-title">Calculate display width from SAR</p>
Rescale only when there's work to do — <code>zscale</code> is skipped for square-pixel sources (SAR <code>1:1</code>, <code>N/A</code>, empty, <code>0:N</code>) and for any SAR whose width rounds back to the source width. When rescaling, width is rounded to the nearest even number (mod-2, preferred by AV1).
</li>
<li>
<p class="step-title">Encode video <b>(ffmpeg → libsvtav1)</b></p>
<ul>
<li>Filter chain built conditionally: <code>bwdif=mode=0:par=-1:-1</code> prepended when interlaced; <code>zscale</code> appended only when a rescale is needed. Neither → <code>-vf</code> omitted.</li>
<li>Codec <code>libsvtav1</code>, <code>-pix_fmt yuv420p10le</code>, crf/preset from the profile.</li>
<li><code>-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s</code></li>
<li>Maps: video <code>0:v</code>, subtitles <code>0:s?</code> (optional), one audio per opus file (<code>1:a</code>, <code>2:a</code>, …). Audio <code>-c:a copy</code>, subtitles <code>-c:s copy</code>.</li>
<li><code>-map_metadata -1</code> strips global metadata, then language tags are re-applied — audio indexed by <strong>output position</strong>, so missing-language streams don't shift the index.</li>
<li>Container tags: <code>TITLE</code>, <code>DATE_RELEASED</code>, <code>IMDBID</code>, <code>ORIGINAL_MEDIA_TYPE</code> — plus <code>COLLECTION</code>, <code>SEASON</code>, <code>EPISODE</code>, <code>TVMAZE_ID</code> for series. Values unquoted. Output → <code>output.mkv</code> in the work dir.</li>
</ul>
</li>
<li>
<p class="step-title">Clean up the work directory</p>
Intermediates and <code>output.mkv</code> together — once the move below succeeds, or on any failure via deferred cleanup.
</li>
<li>
<p class="step-title">Rename and move to <b>paths.output</b></p>
<ul>
<li>Series with a known show name → <code>&lt;show&gt;.S&lt;NN&gt;E&lt;NN&gt;.mkv</code> (no IMDb mapping required).</li>
<li>Movie with title + IMDb ID → <code>&lt;title&gt;.&lt;imdbID&gt;.mkv</code>.</li>
<li>Anything else → <code>&lt;8-hex&gt;.nometadata.mkv</code>.</li>
</ul>
Sanitization keeps <code>a–z A–Z 0–9 - ä ö Ä Ö</code> only.
</li>
<li>
<p class="step-title">Dispose of the source</p>
<code>-d</code> set → delete the original; otherwise → move it to <code>paths.originals</code>.
</li>
</ol>
</section>
<section id="naming">
<p class="eyebrow">07 — Reference</p>
<h2>Output naming examples</h2>
<div class="tablewrap">
<table>
<thead><tr><th>Input</th><th>Result in output/</th><th>Notes</th></tr></thead>
<tbody>
<tr><td><code>Heat.tt0113277.mkv</code></td><td><code>Heat.tt0113277.mkv</code></td><td>Pixel-count fallback → Blu-ray profile + tag</td></tr>
<tr><td><code>Heat.tt0113277.bluray.mkv</code></td><td><code>Heat.tt0113277.mkv</code></td><td>Same name; <code>ORIGINAL_MEDIA_TYPE</code> now from token, not guess</td></tr>
<tr><td><code>Breaking.Bad.tvm169.S01E01.webdl.mkv</code></td><td><code>BreakingBad.S01E01.mkv</code></td><td>WebDL profile + tag</td></tr>
<tr><td><code>unrecognized-rip.mkv</code></td><td><code>a1b2c3d4.nometadata.mkv</code></td><td>No IDs at all</td></tr>
</tbody>
</table>
</div>
<details class="facts">
<summary>A few things worth knowing</summary>
<div class="body">
<ul>
<li>The media-type token affects the muxed <code>ORIGINAL_MEDIA_TYPE</code> tag and the crf/preset profile, but <strong>not</strong> the output filename.</li>
<li>The IMDb ID in the filename is the one returned by the API, not the one in the input — a typo in the source name will surface in the output name.</li>
<li>A TVmaze show with no IMDb mapping still gets a <code>&lt;Collection&gt;.S&lt;NN&gt;E&lt;NN&gt;.mkv</code> filename; only the <code>IMDBID</code> tag is left empty.</li>
</ul>
</div>
</details>
</section>
<section id="logs">
<p class="eyebrow">08 — Observability</p>
<h2>Logs</h2>
<p>All logs are written to a <strong>SQLite database, <code>logs.db</code></strong>, in the current working directory (not the config paths). Run the program from where you want it to land — in Docker that's the mounted <code>/data</code>. The <code>logs</code> table has columns <code>id</code>, <code>ts</code>, <code>level</code>, <code>message</code>, <code>file</code>, <code>extra</code>.</p>
<div class="tablewrap">
<table>
<thead><tr><th>Level</th><th>Where</th><th>Contents</th></tr></thead>
<tbody>
<tr><td><code>info</code></td><td>db + stdout</td><td>Processing / metadata hits / completions</td></tr>
<tr><td><code>error</code></td><td>db + stderr</td><td>Failures, with the source <code>file</code> recorded</td></tr>
<tr><td><code>debug</code></td><td>db only</td><td>ffprobe output, zscale width, full ffmpeg/opusenc command, OMDb/TVmaze responses (API key redacted) in <code>extra</code></td></tr>
</tbody>
</table>
</div>
<p>Live encode progress (<code>encoding … · 47% · 3.2fps · …</code>) prints to <strong>stdout only</strong> and is deliberately not stored, so it can't flood the database.</p>
<div class="note"><span class="tag">Retention</span><p>Rows older than <code>log_retention_days</code> (default <code>7</code>) are purged on startup and shutdown. Editable live from the settings page (<code>/settings</code>); applies at the next purge. Recent non-debug events are viewable in the dashboard and via <code>GET /status</code> when <code>http_addr</code> is set.</p></div>
</section>
<section id="failures">
<p class="eyebrow">09 — Resilience</p>
<h2>Failure handling</h2>
<p>If any step from probing through encoding through renaming fails:</p>
<ul>
<li>The source <code>.mkv</code> is moved to <code>paths.failed</code> (<code>os.Stat</code>-guarded — if the source is already gone the move is skipped and logged; a move error is logged too). This guarantees the source leaves <code>paths.input</code> on every failure path, so the watcher won't retry it next tick.</li>
<li>The per-job work directory (partial wav/opus/output.mkv) is deleted unconditionally.</li>
<li>The error is logged to <code>logs.db</code> (level <code>error</code>) with the source path, and printed to stderr.</li>
</ul>
<p>The watcher continues with the next file; one bad rip won't stop the daemon.</p>
<div class="note"><span class="tag">Collisions</span><p>If a finished encode would land on a name that already exists in <code>paths.output</code>, the move is refused (no silent overwrite) and the source is routed to <code>paths.failed</code>. This is the path you hit when two sources sanitize to the same name — two re-rips of the same release, or two episodes that both resolve to <code>SxxExx</code>.</p></div>
<div class="note"><span class="tag">Cross-filesystem</span><p>Every move (input→output, →failed, →originals) transparently falls back to copy + delete when source and destination live on different mounts. Put each path on a different drive without breaking the pipeline.</p></div>
</section>
<section id="polling">
<p class="eyebrow">10 — Watcher</p>
<h2>Polling behavior</h2>
<ul>
<li>Input directory is scanned every <strong>15 seconds</strong> (clamped to a 10–30 s range).</li>
<li>Only <code>*.mkv</code> directly in <code>paths.input</code> are picked up — no recursion.</li>
<li>Files are processed <strong>sequentially</strong>, one at a time, in the order <code>filepath.Glob</code> returns (alphabetical on Linux).</li>
<li><strong>Partial-write protection:</strong> a file's <code>mtime</code> and <code>size</code> must match on two consecutive scans before processing. A freshly dropped or still-copying file waits at least one full tick (~15 s). Copying a large file straight into <code>paths.input</code> is safe — no need to land it under a different name and <code>mv</code> into place (though that still works and skips the tick delay).</li>
<li><strong>Failure quarantine:</strong> if <code>processFile</code> errors, that file is skipped for 5 minutes (or until its <code>mtime</code> changes — e.g. you replace or <code>touch</code> it). Stops a permanently-broken input from spamming the logs. In-memory only; restarting clears it.</li>
</ul>
</section>
<section id="layout">
<p class="eyebrow">11 — Source map</p>
<h2>Project layout</h2>
<div class="pre is-file"><pre><code>cmd/av1dae/main.go <span class="cm">CLI entrypoint and per-file orchestration</span>
internal/config/ <span class="cm">YAML config load + defaults + mkdir</span>
internal/watcher/ <span class="cm">Polling loop, media-type detection by pixel count</span>
internal/encoder/ <span class="cm">ffprobe/ffmpeg/opusenc wrapper, transcode pipeline</span>
internal/metadata/ <span class="cm">Filename parsing, OMDb + TVmaze clients</span>
internal/mover/ <span class="cm">File rename/move/delete helpers</span>
internal/logger/ <span class="cm">Plain + JSON logging</span>
pkg/types/types.go <span class="cm">Shared structs (Config, Job, Metadata, …)</span></code></pre></div>
<p>Runtime directories (from <code>paths.*</code> in the config):</p>
<div class="pre is-file"><pre><code>paths.input <span class="cm">Drop new .mkv here; polled every 15 s</span>
paths.output <span class="cm">Finished encodes land here under their final name</span>
paths.originals <span class="cm">Encoded sources end up here (unless -d)</span>
paths.failed <span class="cm">Sources of failed jobs end up here</span>
paths.work <span class="cm">Per-job scratch subdir (basename); wiped per job</span></code></pre></div>
</section>
<section id="docker">
<p class="eyebrow">12 — Deploy</p>
<h2>Deploy with Docker</h2>
<p class="lead">Run it as a container on your server. <code>ffmpeg</code>, <code>ffprobe</code>, and <code>opusenc</code> are baked into the image, so the only host requirement is Docker — nothing to install, nothing on <code>$PATH</code>.</p>
<h3>Quick start</h3>
<div class="pre"><pre><code><span class="cm"># one-time: create the media tree and your config</span>
mkdir -p media/input media/output media/originals media/failed media/work
cp config.example.yaml config.yaml <span class="cm"># then fill in omdb_api_key</span>
docker compose up -d --build</code></pre></div>
<p>Drop <code>.mkv</code> files into <code>media/input/</code>; finished encodes appear in <code>media/output/</code>. Follow the logs with <code>docker compose logs -f</code>.</p>
<h3>How the volumes map</h3>
<p>Two mounts (defined in <code>docker-compose.yml</code>) are all it needs:</p>
<div class="tablewrap">
<table>
<thead><tr><th>Host</th><th>Container</th><th>Holds</th></tr></thead>
<tbody>
<tr><td><code>./config.yaml</code></td><td><code>/config/config.yaml</code> <em>(ro)</em></td><td>Your config — passed via <code>-c</code></td></tr>
<tr><td><code>./media</code></td><td><code>/data</code></td><td><code>input/ output/ originals/ failed/ work/</code> + <code>logs.db</code> and log files</td></tr>
</tbody>
</table>
</div>
<div class="note"><span class="tag">Paths just work</span><p>The container's working directory is <code>/data</code>, so the <strong>relative</strong> paths in <code>config.example.yaml</code> (<code>./input</code>, <code>./output</code>, …) resolve to <code>/data/input</code>, <code>/data/output</code>, … inside the mount. No path edits needed — only the <code>omdb_api_key</code>.</p></div>
<h3>What's in the image</h3>
<div class="tablewrap">
<table>
<thead><tr><th>Layer</th><th>Detail</th></tr></thead>
<tbody>
<tr><td>Build stage</td><td><code>golang:1.26-bookworm</code>, <code>CGO_ENABLED=1</code> (the SQLite logger needs cgo — no <code>scratch</code> image)</td></tr>
<tr><td>Runtime</td><td><code>debian:bookworm-slim</code></td></tr>
<tr><td>Bundled</td><td><code>ffmpeg</code> (ships <code>ffprobe</code>), <code>opus-tools</code> (<code>opusenc</code>), <code>ca-certificates</code> (for OMDb/TVmaze HTTPS)</td></tr>
</tbody>
</table>
</div>
<div class="note warn"><span class="tag">Delete originals</span><p>To delete sources after a successful encode (the <code>-d</code> flag), uncomment <code>command: ["-d"]</code> in <code>docker-compose.yml</code> and re-run <code>docker compose up -d</code>.</p></div>
<div class="note"><span class="tag">Clean shutdown</span><p><code>docker stop</code> sends <code>SIGTERM</code>, which cancels any in-flight encode and routes the source to <code>failed/</code> (see <a href="#running">§4</a>). Restart with <code>docker compose restart</code>; the daemon re-scans <code>input/</code> on boot.</p></div>
<h3>Without compose</h3>
<p>Same thing with plain <code>docker</code>:</p>
<div class="pre"><pre><code>docker build -t av1dae .
docker run -d --name av1dae --restart unless-stopped \
-v "$PWD/config.yaml:/config/config.yaml:ro" \
-v "$PWD/media:/data" \
av1dae</code></pre></div>
</section>
</main>
</div>
<footer>av1dae — DVD/Blu-ray → SVT-AV1 transcoding daemon · single-file manual, no dependencies</footer>
<script>
// Copy buttons on every code block — native clipboard, no deps.
document.querySelectorAll('.pre').forEach(box => {
const code = box.querySelector('code');
const btn = document.createElement('button');
btn.className = 'copy'; btn.type = 'button'; btn.textContent = 'copy';
btn.setAttribute('aria-label', 'Copy code to clipboard');
btn.addEventListener('click', async () => {
try {
await navigator.clipboard.writeText(code.innerText);
btn.textContent = 'copied'; btn.classList.add('ok');
setTimeout(() => { btn.textContent = 'copy'; btn.classList.remove('ok'); }, 1200);
} catch { btn.textContent = 'failed'; }
});
box.appendChild(btn);
});
// Scroll-spy: highlight the TOC entry for the section in view.
const links = [...document.querySelectorAll('.toc a')];
const byId = new Map(links.map(l => [l.getAttribute('href').slice(1), l]));
const spy = new IntersectionObserver(entries => {
entries.forEach(e => {
if (e.isIntersecting) {
links.forEach(l => l.classList.remove('active'));
byId.get(e.target.id)?.classList.add('active');
}
});
}, { rootMargin: '-15% 0px -75% 0px' });
document.querySelectorAll('section[id]').forEach(s => spy.observe(s));
// On mobile, collapse the TOC after a jump.
const toc = document.getElementById('toc');
links.forEach(l => l.addEventListener('click', () => {
if (window.matchMedia('(max-width:860px)').matches) toc.removeAttribute('open');
}));
</script>
</body>
</html>
MANUAL.md
+107 -50
View File
@@ -1,4 +1,4 @@
# videnc-vibe — User Manual # av1dae — User Manual
A Go CLI that watches a folder for `.mkv` rips, transcodes them to SVT-AV1 video + Opus audio, embeds metadata fetched from OMDb (movies) or TVmaze (series), and files the results into output/originals/failed directories. A Go CLI that watches a folder for `.mkv` rips, transcodes them to SVT-AV1 video + Opus audio, embeds metadata fetched from OMDb (movies) or TVmaze (series), and files the results into output/originals/failed directories.
@@ -25,16 +25,16 @@ API keys:
## 2. Build ## 2. Build
```bash ```bash
go build -o videnc-vibe ./cmd/videnc/ go build -o av1dae ./cmd/av1dae/
``` ```
This produces a single static binary `./videnc-vibe`. This produces a single static binary `./av1dae`.
--- ---
## 3. Configuration ## 3. Configuration
By default the config is loaded from `~/.config/videnc-vibe/config.yaml`. Pass `-c /path/to/config.yaml` to override. By default the config is loaded from `~/.config/av1dae/config.yaml`. Pass `-c /path/to/config.yaml` to override.
Example (`config.example.yaml`): Example (`config.example.yaml`):
@@ -48,12 +48,19 @@ encoding:
bluray: bluray:
crf: 29 crf: 29
preset: 3 preset: 3
webdl:
crf: 30
preset: 3
tvrip:
crf: 32
preset: 2
paths: paths:
input: "./input" input: "./input"
output: "./output" output: "./output"
originals: "./originals" originals: "./originals"
failed: "./failed" failed: "./failed"
work: "./work"
``` ```
### Field reference ### Field reference
@@ -65,22 +72,27 @@ paths:
| `encoding.dvd.preset` | SVT-AV1 preset for SD sources | `2` | | `encoding.dvd.preset` | SVT-AV1 preset for SD sources | `2` |
| `encoding.bluray.crf` | SVT-AV1 CRF for HD sources | `29` | | `encoding.bluray.crf` | SVT-AV1 CRF for HD sources | `29` |
| `encoding.bluray.preset` | SVT-AV1 preset for HD sources | `3` | | `encoding.bluray.preset` | SVT-AV1 preset for HD sources | `3` |
| `encoding.webdl.crf` | SVT-AV1 CRF for WebDL sources | `30` |
| `encoding.webdl.preset` | SVT-AV1 preset for WebDL sources | `3` |
| `encoding.tvrip.crf` | SVT-AV1 CRF for TVRip sources | `32` |
| `encoding.tvrip.preset` | SVT-AV1 preset for TVRip sources | `2` |
| `paths.input` | Folder polled for new `.mkv` files | `./input` | | `paths.input` | Folder polled for new `.mkv` files | `./input` |
| `paths.output` | Destination for finished encodes | `./output` | | `paths.output` | Destination for finished encodes | `./output` |
| `paths.originals` | Where source files are moved on success (unless `-d`) | `./originals` | | `paths.originals` | Where source files are moved on success (unless `-d`) | `./originals` |
| `paths.failed` | Where source + partial output go on failure | `./failed` | | `paths.failed` | Where source files go on failure | `./failed` |
| `paths.work` | Scratch directory for per-job intermediates (wav/opus/output.mkv); deleted after every job | `./work` |
All four directories are created on startup if they don't exist. All five directories are created on startup if they don't exist.
--- ---
## 4. Running ## 4. Running
```bash ```bash
./videnc-vibe # default config path, keep originals ./av1dae # default config path, keep originals
./videnc-vibe -d # delete originals after successful encode ./av1dae -d # delete originals after successful encode
./videnc-vibe -c /etc/videnc.yaml # custom config ./av1dae -c /etc/av1dae.yaml # custom config
./videnc-vibe -c /etc/videnc.yaml -d ./av1dae -c /etc/av1dae.yaml -d
``` ```
Flags: Flags:
@@ -88,7 +100,7 @@ Flags:
- `-d` — delete the source `.mkv` after a successful encode instead of moving it to `originals/`. - `-d` — delete the source `.mkv` after a successful encode instead of moving it to `originals/`.
- `-c PATH` — path to config file. - `-c PATH` — path to config file.
Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. A signal triggers a clean shutdown after the current poll cycle. Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. The signal cancels any in-flight encode immediately — the ffmpeg / ffprobe / opusenc children are killed, the per-job work directory is removed by its deferred cleanup, and the source `.mkv` is routed to `paths.failed` so the next run sees a clean `paths.input`.
The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter. The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter.
@@ -131,41 +143,62 @@ the-wire.TVM75.s2e5.mkv
If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an `.nometadata.mkv` suffix. If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an `.nometadata.mkv` suffix.
### 5.4 Optional: source media type
Any filename can additionally contain a media-type token (case-insensitive, word-bounded):
- `dvd`
- `bluray`
- `webdl`
- `tvrip`
Examples:
```
Heat.tt0113277.bluray.mkv
some-rip.tvm169.S01E01.webdl.mkv
old.broadcast.tvrip.tt0066026.mkv
```
The token controls **both** the `ORIGINAL_MEDIA_TYPE` metadata tag written into the output and which `encoding.<type>.crf` / `encoding.<type>.preset` pair is used.
If no token is present, the program falls back to guessing from pixel count: `width × height < 600,000` → DVD, otherwise Blu-ray. WebDL and TVRip are never auto-detected — they must be declared via the token.
--- ---
## 6. The processing pipeline ## 6. The processing pipeline
For every `.mkv` found in `paths.input`, the program runs these steps in order. Any error sends the source file (and the partial `output.mkv`, if any) to `paths.failed`. For every `.mkv` found in `paths.input`, the program runs these steps in order. A per-job scratch subdirectory under `paths.work` (named after the input base name without `.mkv`) holds all intermediates, and is deleted unconditionally at the end of the job. Any error sends the source file to `paths.failed`; the work subdirectory is wiped regardless of outcome.
1. **Parse filename** → determines whether this is a movie or series, and what IDs to use. 1. **Parse filename** → determines whether this is a movie or series, and what IDs to use.
2. **Probe video** with `ffprobe`: 2. **Probe video** with `ffprobe`:
- Picks the first video stream whose codec is `mpeg2video`, `h264`, or `hevc`. - Picks the first stream with `codec_type=video`, regardless of codec name. Errors out if there isn't one.
- Records width, height, and sample aspect ratio (SAR). - Records width, height, and sample aspect ratio (SAR).
- Runs `ffmpeg -vf idet` to detect interlacing (presence of `TFF`/`BFF` in stderr). - Detects interlacing by running `ffmpeg -vf idet -frames:v 400 -an -sn -f null -` and parsing the `Multi frame detection: TFF: a BFF: b Progressive: c Undetermined: d` summary line. The source is treated as interlaced only when `a+b > c`; undetermined frames are ignored, and a missing summary line defaults to progressive.
3. **Probe stream languages** with `ffprobe` — collects `language` tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's `-map_metadata -1` strips them otherwise). 3. **Probe stream languages** with `ffprobe` — collects `language` tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's `-map_metadata -1` strips them otherwise).
4. **Detect media type** from pixel count: 4. **Detect media type**:
- `width × height < 600,000` → **DVD** profile. - First, check the filename for a `dvd` / `bluray` / `webdl` / `tvrip` token (case-insensitive). If present, that wins.
- Otherwise → **Blu-ray** profile. - Otherwise, fall back to pixel count: `width × height < 600,000` → DVD, else Blu-ray.
The chosen profile selects which CRF/preset pair from the config to use. The chosen profile selects which `encoding.<type>.crf` / `encoding.<type>.preset` pair from the config to use, and is written into the `ORIGINAL_MEDIA_TYPE` metadata tag.
5. **Fetch metadata** from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata. 5. **Fetch metadata** from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata.
6. **Extract audio** to `audio.wav` (PCM s16le, 48 kHz) in the input directory. 6. **Extract audio** — one PCM wav per source audio stream, written to the per-job work directory as `audio.0.wav`, `audio.1.wav`, … in source order (PCM s16le, 48 kHz). Errors out if the source has no audio streams.
7. **Encode audio** with `opusenc --bitrate 128k` → `audio.opus`. 7. **Encode audio** — each wav is converted with `opusenc --bitrate 128k` to a matching `audio.<n>.opus` in the same work directory.
8. **Calculate display width** from SAR. If SAR ≠ `1:1`, the width is rescaled so the output has square pixels, using a `zscale` filter (`spline36`). Height is preserved. 8. **Calculate display width** from SAR. The width is rescaled so the output has square pixels only when there's actually work to do — `zscale` is skipped entirely for square-pixel sources (SAR `1:1`, `N/A`, empty, `0:N`), and for any SAR whose calculated width rounds to the source width. When rescaling, the width is rounded to the nearest even number (mod-2, preferred by AV1).
9. **Encode video** with FFmpeg: 9. **Encode video** with FFmpeg:
- Video filter chain: `bwdif=mode=0:par=-1:-1,zscale=w=W:h=H:filter=spline36` if interlaced, else just the `zscale` step. - Video filter chain is built conditionally. `bwdif=mode=0:par=-1:-1` is prepended when the source is interlaced; the `zscale` step is appended only when a rescale is actually needed (see step 8). If neither applies, `-vf` is omitted entirely.
- Codec: `libsvtav1`, `-pix_fmt yuv420p10le`. - Codec: `libsvtav1`, `-pix_fmt yuv420p10le`.
- `-crf` and `-preset` from the selected profile. - `-crf` and `-preset` from the selected profile (step 4).
- `-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s`. - `-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s`.
- Streams mapped: video from input, subtitles from input (`0:s?` — optional), audio from the re-encoded Opus files. - Streams mapped: video from input (`0:v`), subtitles from input (`0:s?` — optional), and one audio stream per Opus file (`1:a`, `2:a`, …).
- Audio copied (`-c:a copy`), subtitles copied (`-c:s copy`). - Audio re-muxed (`-c:a copy` — copies the already-Opus-encoded streams), subtitles copied (`-c:s copy`).
- `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied from the ffprobe pass. - `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied. Audio language tags are indexed by **output position** (the position in the opus-file list), not by counting source streams, so missing-language streams don't shift the index.
- Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`. - Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`. Values are unquoted (literal value, no wrapping `"…"`).
- Output written to `output.mkv` in the input directory. - Output written to `output.mkv` in the per-job work directory.
10. **Clean up** intermediate `.wav` and `.opus` files. 10. **Clean up** the entire per-job work directory (intermediates and `output.mkv` together) once the rename/move below succeeds — or, on any failure, when the deferred cleanup runs.
11. **Rename and move** `output.mkv` to `paths.output` with a final name: 11. **Rename and move** `output.mkv` from the work directory to `paths.output` with a final name:
- Series → `<sanitized-show-name>.S<NN>E<NN>.mkv` - Series with a known show name → `<sanitized-show-name>.S<NN>E<NN>.mkv` (the series no longer needs a populated IMDb mapping — a TVmaze show with no external IMDb link still gets a useful filename).
- Movie → `<sanitized-title>.<imdbID>.mkv` - Movie with a known title and IMDb ID → `<sanitized-title>.<imdbID>.mkv`.
- No metadata → `<8-hex-chars>.nometadata.mkv` - Anything else → `<8-hex-chars>.nometadata.mkv`.
Sanitization keeps `a–z A–Z 0–9 - ä ö Ä Ö` only. Sanitization keeps `a–z A–Z 0–9 - ä ö Ä Ö` only.
12. **Dispose of the source**: 12. **Dispose of the source**:
- `-d` flag set → delete the original `.mkv`. - `-d` flag set → delete the original `.mkv`.
@@ -175,27 +208,36 @@ For every `.mkv` found in `paths.input`, the program runs these steps in order.
## 7. Output naming examples ## 7. Output naming examples
| Input filename | Result in `paths.output` | | Input filename | Result in `paths.output` | Notes |
|---|---| |---|---|---|
| `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` | | `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` | Pixel-count fallback → Blu-ray profile + tag |
| `Breaking.Bad.tvm169.S01E01.mkv` | `BreakingBad.S01E01.mkv` | | `Heat.tt0113277.bluray.mkv` | `Heat.tt0113277.mkv` | Same output filename; muxed `ORIGINAL_MEDIA_TYPE=Blu-ray` is now from the token, not the guess |
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` | | `Breaking.Bad.tvm169.S01E01.webdl.mkv` | `BreakingBad.S01E01.mkv` | WebDL profile + tag |
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` | No IDs at all |
The IMDb ID returned by the API is used, not the one from the filename, so a typo in the filename would surface there. A few things worth knowing:
- The media-type token affects the muxed `ORIGINAL_MEDIA_TYPE` tag and the CRF/preset profile, but **not** the output filename.
- The IMDb ID written into the filename is the one returned by the API, not the one in the input filename, so a typo in the source filename will surface in the output name.
- A TVmaze show with no IMDb mapping still gets a `<Collection>.S<NN>E<NN>.mkv` filename (only the `IMDBID` metadata tag is left empty).
--- ---
## 8. Logs ## 8. Logs
Three log files are written to the **current working directory** (not the config paths): All logs are written to a **SQLite database, `logs.db`**, in the current working directory (not the config paths). Run the program from the directory where you want it to land — in Docker that's the mounted `/data`.
- `info_YYYY-MM-DD.log` — INFO messages, dated. The `logs` table has columns `id`, `ts`, `level`, `message`, `file`, `extra`. Three levels are recorded:
- `error_YYYY-MM-DD.log` — ERROR messages, dated.
- `structured.json` — one JSON object per line, every entry (info + error), with `timestamp`, `level`, `message`, optional `error` and `file` fields.
Run the program from the directory where you want the logs to land. - `info` — INFO messages; also printed to stdout.
- `error` — ERROR messages; also printed to stderr, with the source `file` recorded.
- `debug` — verbose diagnostics (ffprobe output, calculated zscale width, the full ffmpeg/opusenc command, OMDb/TVmaze responses with the API key redacted) in the `extra` column. **Database only** — not printed.
Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command, etc.). These are intentional but not written to the log files. Live encode progress (`encoding … · 47% · 3.2fps · …`) prints to **stdout only** and is deliberately *not* stored, so it can't flood the database.
**Retention:** rows older than `log_retention_days` (default `7`) are purged on startup and on shutdown. The value is editable live from the settings page (`/settings`) and applies at the next purge.
Recent non-debug events are also viewable in the web dashboard and via `GET /status` (when `http_addr` is set).
--- ---
@@ -203,12 +245,16 @@ Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, ca
If any step from probing through encoding through renaming fails: If any step from probing through encoding through renaming fails:
- The source `.mkv` is moved to `paths.failed`. - The source `.mkv` is moved to `paths.failed` (the move itself is `os.Stat`-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves `paths.input` on every failure path, so the watcher doesn't retry the same file on the next tick.
- Any partial `output.mkv` left in the input directory is also moved to `paths.failed`. - The per-job work directory under `paths.work` (containing partial wav/opus/output.mkv) is deleted unconditionally.
- The error is logged to `error_*.log` and `structured.json` with the source file path. - The error is logged to `logs.db` (level `error`) with the source file path, and printed to stderr.
The watcher continues with the next file; one bad rip won't stop the daemon. The watcher continues with the next file; one bad rip won't stop the daemon.
**Destination collisions:** if a finished encode would land on a name that already exists in `paths.output`, the move is refused (no silent overwrite) and the source is routed to `paths.failed`. This is the path you'll hit when two sources sanitize to the same output filename — e.g. two re-rips of the same release, or two episodes that both come out as `SxxExx`.
**Cross-filesystem moves:** every move (`paths.input → paths.output`, `… → paths.failed`, `… → paths.originals`) transparently falls back to copy + delete when the source and destination live on different mounts. You can put each path on a different drive without breaking the pipeline.
--- ---
## 10. Polling behavior ## 10. Polling behavior
@@ -216,14 +262,15 @@ The watcher continues with the next file; one bad rip won't stop the daemon.
- Input directory is scanned every **15 seconds** (clamped to a 10–30 s range). - Input directory is scanned every **15 seconds** (clamped to a 10–30 s range).
- Only files matching `*.mkv` directly in `paths.input` are picked up — no recursion. - Only files matching `*.mkv` directly in `paths.input` are picked up — no recursion.
- Files are processed **sequentially**, one at a time, in the order `filepath.Glob` returns them (alphabetical on Linux). - Files are processed **sequentially**, one at a time, in the order `filepath.Glob` returns them (alphabetical on Linux).
- There is no atomic-write detection. If you're copying a large file into `paths.input`, copy it to a different name first and `mv` it into place once complete, otherwise the watcher may try to encode a half-written file. - **Partial-write protection:** the watcher requires a file's `mtime` and `size` to be identical on two consecutive scans before processing. A freshly dropped or still-copying file therefore waits at least one full tick (~15 s) before encoding begins. Copying a large file directly into `paths.input` is now safe; you no longer have to land it under a different name and `mv` into place (though doing so still works and shaves off the tick delay).
- **Failure quarantine:** if `processFile` returns an error, that file is skipped for 5 minutes (or until its `mtime` changes — e.g. you replace or `touch` it). Prevents a permanently-broken input from spamming the logs every 15 s. The quarantine is in-memory only; restarting the daemon clears it.
--- ---
## 11. Project layout ## 11. Project layout
``` ```
cmd/videnc/main.go CLI entrypoint and per-file orchestration cmd/av1dae/main.go CLI entrypoint and per-file orchestration
internal/config/ YAML config load + defaults + mkdir internal/config/ YAML config load + defaults + mkdir
internal/watcher/ Polling loop, media-type detection by pixel count internal/watcher/ Polling loop, media-type detection by pixel count
internal/encoder/ ffprobe/ffmpeg/opusenc wrapper, transcode pipeline internal/encoder/ ffprobe/ffmpeg/opusenc wrapper, transcode pipeline
@@ -232,3 +279,13 @@ internal/mover/ File rename/move/delete helpers
internal/logger/ Plain + JSON logging internal/logger/ Plain + JSON logging
pkg/types/types.go Shared structs (Config, Job, Metadata, …) pkg/types/types.go Shared structs (Config, Job, Metadata, …)
``` ```
Runtime directories (from `paths.*` in the config):
```
paths.input User drops new .mkv files here; watcher polls every 15 s
paths.output Finished encodes land here under their final name
paths.originals Successfully-encoded sources end up here (unless -d)
paths.failed Sources of failed jobs end up here
paths.work Per-job scratch subdir (basename of input); wiped per job
```
SPEC.md
+7 -6
View File
@@ -1,11 +1,11 @@
# videnc-vibe Specification # av1dae Specification
## Overview ## Overview
FFmpeg-based video transcoder for DVD/Blu-ray to SVT-AV1 with automatic metadata fetching. FFmpeg-based video transcoder for DVD/Blu-ray to SVT-AV1 with automatic metadata fetching.
## Configuration ## Configuration
Config path: `-c` flag or `~/.config/videnc-vibe/config.yaml` Config path: `-c` flag or `~/.config/av1dae/config.yaml`
```yaml ```yaml
omdb_api_key: "your-key-here" omdb_api_key: "your-key-here"
@@ -27,7 +27,7 @@ paths:
## Flags ## Flags
- `-d` : Delete original after successful encode (default: move to originals/) - `-d` : Delete original after successful encode (default: move to originals/)
- `-c` : Config file path (default: ~/.config/videnc-vibe/config.yaml) - `-c` : Config file path (default: ~/.config/av1dae/config.yaml)
## Input Parsing ## Input Parsing
@@ -99,9 +99,10 @@ Regex patterns:
## Logging ## Logging
- info.log: Human-readable info level - All logs are stored in a SQLite database `logs.db` in the working directory.
- error.log: Human-readable error level - Levels: `debug`, `info`, `error`. `info`/`error` also print to stdout/stderr; `debug` (ffprobe/ffmpeg/API dumps) is database-only. Live encode progress prints to stdout only (not stored).
- structured.json: JSON structured logs - Retention: rows older than `log_retention_days` (default 7) are purged on startup/shutdown; editable from the settings UI.
- Recent non-debug events are viewable in the web dashboard and via `GET /status`.
## Polling ## Polling
- Scan input folder every 10-30 seconds for `.mkv` files - Scan input folder every 10-30 seconds for `.mkv` files
cmd/av1dae/main.go
+330
View File
@@ -0,0 +1,330 @@
package main
import (
"context"
"crypto/rand"
"encoding/hex"
"flag"
"fmt"
"net/http"
"os"
"os/signal"
"path/filepath"
"regexp"
"strings"
"syscall"
"time"
"av1dae/internal/config"
"av1dae/internal/encoder"
"av1dae/internal/logger"
"av1dae/internal/metadata"
"av1dae/internal/mover"
"av1dae/internal/server"
"av1dae/internal/settings"
"av1dae/internal/status"
"av1dae/internal/watcher"
"av1dae/pkg/types"
)
var (
deleteOrigin bool
configPath string
)
func init() {
flag.BoolVar(&deleteOrigin, "d", false, "Delete original after successful encode")
flag.StringVar(&configPath, "c", "", "Config file path (default: ~/.config/av1dae/config.yaml)")
}
func main() {
flag.Parse()
cfg, err := config.Load(configPath)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to load config: %v\n", err)
os.Exit(1)
}
if err := config.EnsureDirs(cfg); err != nil {
fmt.Fprintf(os.Stderr, "Failed to create directories: %v\n", err)
os.Exit(1)
}
log, err := logger.New(".", cfg.LogRetentionDays)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to initialize logger: %v\n", err)
os.Exit(1)
}
defer log.Close()
tracker := status.New()
enc := encoder.New(log, tracker)
if err := enc.CheckDeps(); err != nil {
log.Error("Dependency check failed", err.Error())
fmt.Fprintf(os.Stderr, "Dependency check failed: %v\n", err)
os.Exit(1)
}
// Live-editable settings: config.yaml seeds the store on first run; after
// that the DB (logs.db) is the source of truth for these values.
store, err := settings.New(log.DB(), settings.Settings{
DVD: settings.Profile{CRF: cfg.Encoding.DVD.CRF, Preset: cfg.Encoding.DVD.Preset},
Bluray: settings.Profile{CRF: cfg.Encoding.Bluray.CRF, Preset: cfg.Encoding.Bluray.Preset},
WebDL: settings.Profile{CRF: cfg.Encoding.WebDL.CRF, Preset: cfg.Encoding.WebDL.Preset},
TVRip: settings.Profile{CRF: cfg.Encoding.TVRip.CRF, Preset: cfg.Encoding.TVRip.Preset},
LP: cfg.Encoding.LP,
OMDBAPIKey: cfg.OMDBAPIKey,
LogRetentionDays: cfg.LogRetentionDays,
DeleteOriginals: deleteOrigin,
})
if err != nil {
log.Error("Settings init failed", err.Error())
fmt.Fprintf(os.Stderr, "Failed to init settings: %v\n", err)
os.Exit(1)
}
// A persisted retention value (DB) overrides the config-seeded one.
log.SetRetention(store.Get().LogRetentionDays)
metaClient := metadata.NewClient(cfg.OMDBAPIKey, log)
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer cancel()
// Start/hold gate: held by default so the user clicks Start; AV1DAE_AUTOSTART
// (truthy) restores start-on-boot.
autostart := envTruthy(os.Getenv("AV1DAE_AUTOSTART"))
w := watcher.New(cfg.Paths.Input, 15, autostart)
if !autostart {
log.Info("Queue held on startup — click Start (set AV1DAE_AUTOSTART=1 to auto-start)")
}
controls := server.Controls{
Running: w.Running,
SetRunning: w.SetRunning,
Pause: enc.Pause,
Resume: enc.Resume,
RetryFailed: func(name string) error {
if name == "" || name != filepath.Base(name) {
return fmt.Errorf("invalid file name")
}
return mover.Rename(filepath.Join(cfg.Paths.Failed, name), filepath.Join(cfg.Paths.Input, name))
},
}
if addr := *cfg.HTTPAddr; addr != "" {
probeDuration := func(p string) (float64, error) {
pctx, pcancel := context.WithTimeout(context.Background(), 10*time.Second)
defer pcancel()
return enc.GetDuration(pctx, p)
}
srv := &http.Server{Addr: addr, Handler: server.New(tracker, log, store, controls, probeDuration, cfg.Paths.Input, cfg.Paths.Failed).Handler()}
go func() {
log.Info(fmt.Sprintf("Status server listening on %s", addr))
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Error("Status server", err.Error())
}
}()
go func() {
<-ctx.Done()
shutCtx, shutCancel := context.WithTimeout(context.Background(), 5*time.Second)
defer shutCancel()
_ = srv.Shutdown(shutCtx)
}()
}
w.Start(ctx, func(ctx context.Context, inputPath string) error {
return processFile(ctx, inputPath, cfg, enc, metaClient, log, tracker, store)
})
log.Info("av1dae started")
}
func processFile(ctx context.Context, inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger, tracker *status.Tracker, store *settings.Store) error {
log.Info(fmt.Sprintf("Processing: %s", inputPath))
// Snapshot the live settings once for the whole job, so a save mid-encode
// doesn't change anything until the next file.
cur := store.Get()
metaClient.SetAPIKey(cur.OMDBAPIKey)
// Mark this file as the active job (phase: probing) and clear the tracker
// back to idle on every exit path — success, failure, or cancellation.
tracker.Begin(inputPath)
defer tracker.Idle()
filename := filepath.Base(inputPath)
isSeries, imdbID, tvmazeID, season, episode := metadata.ParseFilename(filename)
// Per-job work subdirectory under paths.work. Uses the source base name
// (without ".mkv") as the subdir name. MkdirAll is idempotent, so a
// leftover dir from a crashed previous run is harmless to overwrite.
workDirName := strings.TrimSuffix(filename, filepath.Ext(filename))
workDir := filepath.Join(cfg.Paths.Work, workDirName)
if err := os.MkdirAll(workDir, 0755); err != nil {
log.ErrorFile(inputPath, "Creating work directory", err.Error())
failToFailed(inputPath, cfg.Paths.Failed, log)
return err
}
// Cleanup the work directory on every exit path, success or failure.
defer os.RemoveAll(workDir)
width, height, interlaced, err := enc.GetMediaInfo(ctx, inputPath)
if err != nil {
log.ErrorFile(inputPath, "Getting media info", err.Error())
failToFailed(inputPath, cfg.Paths.Failed, log)
return err
}
streamLangs, err := enc.GetStreamLanguages(ctx, inputPath)
if err != nil {
log.ErrorFile(inputPath, "Getting stream languages", err.Error())
}
tracker.SetStreams(toStatusStreams(streamLangs))
mediaType := metadata.ParseMediaType(filename)
if mediaType == "" {
mediaType = watcher.DetectMediaType(width, height)
}
profile := cur.ProfileFor(mediaType)
crf := profile.CRF
preset := profile.Preset
var meta *types.Metadata
if isSeries && tvmazeID != "" && season != "" && episode != "" {
meta, err = metaClient.FetchSeriesMetadata(ctx, tvmazeID, season, episode)
if err != nil {
log.ErrorFile(inputPath, "Fetching series metadata", err.Error())
} else if meta != nil {
log.Info(fmt.Sprintf("TVmaze hit: %s S%sE%s - %s", meta.Collection, meta.Season, meta.Episode, meta.Title))
}
} else if imdbID != "" {
meta, err = metaClient.FetchMovieMetadata(ctx, imdbID)
if err != nil {
log.ErrorFile(inputPath, "Fetching movie metadata", err.Error())
} else if meta != nil {
log.Info(fmt.Sprintf("OMDb hit: %s (%s)", meta.Title, meta.IMDBID))
}
}
if meta == nil {
meta = &types.Metadata{
Title: "Unknown",
DateReleased: "",
IMDBID: "",
OriginalMedia: mediaType,
}
}
meta.OriginalMedia = mediaType
tracker.SetMeta(status.JobMeta{
IsSeries: meta.IsSeries,
Title: meta.Title,
Collection: meta.Collection,
Season: meta.Season,
Episode: meta.Episode,
DateReleased: meta.DateReleased,
MediaType: string(mediaType),
})
job := &types.Job{
InputPath: inputPath,
MediaType: mediaType,
CRF: crf,
Preset: preset,
LP: cur.LP,
DeleteOrigin: cur.DeleteOriginals,
}
if err := enc.Transcode(ctx, inputPath, workDir, job, meta, interlaced, streamLangs); err != nil {
log.ErrorFile(inputPath, "Transcoding", err.Error())
failToFailed(inputPath, cfg.Paths.Failed, log)
return err
}
outputPath := filepath.Join(workDir, "output.mkv")
var outFilename string
switch {
case isSeries && meta.Collection != "":
outFilename = fmt.Sprintf("%s.S%sE%s.mkv", sanitizeFilename(meta.Collection), season, episode)
case !isSeries && meta.IMDBID != "" && meta.Title != "Unknown":
outFilename = fmt.Sprintf("%s.%s.mkv", sanitizeFilename(meta.Title), meta.IMDBID)
default:
outFilename = fmt.Sprintf("%s.nometadata.mkv", generateRandomString(8))
}
finalOutput := filepath.Join(cfg.Paths.Output, outFilename)
if err := mover.Rename(outputPath, finalOutput); err != nil {
// Move the source to failed/ so it doesn't get re-encoded on the next
// tick. The deferred RemoveAll(workDir) takes care of the partial
// output.mkv left in the work dir.
log.ErrorFile(inputPath, "Moving output", err.Error())
failToFailed(inputPath, cfg.Paths.Failed, log)
return err
}
if cur.DeleteOriginals {
mover.Delete(inputPath)
log.Info(fmt.Sprintf("Deleted original: %s", inputPath))
} else {
mover.MoveToOriginals(inputPath, cfg.Paths.Originals)
log.Info(fmt.Sprintf("Moved original to originals: %s", inputPath))
}
log.Info(fmt.Sprintf("Completed: %s -> %s", inputPath, finalOutput))
return nil
}
// failToFailed stats path before invoking MoveToFailed: if missing, logs and
// skips; if present, surfaces any move error to the logger. This avoids the
// silent no-op pattern where MoveToFailed was called on a non-existent file.
func failToFailed(path, failedDir string, log *logger.Logger) {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
log.Info(fmt.Sprintf("Skip move to failed; not present: %s", path))
return
}
log.ErrorFile(path, "Stat before move to failed", err.Error())
return
}
if err := mover.MoveToFailed(path, failedDir); err != nil {
log.ErrorFile(path, "Moving to failed", err.Error())
}
}
// envTruthy reports whether an env var is set to a truthy value.
func envTruthy(s string) bool {
switch strings.ToLower(strings.TrimSpace(s)) {
case "1", "true", "yes", "on":
return true
}
return false
}
// toStatusStreams maps probed source streams to the display shape, keeping only
// audio and subtitle streams (the video stream isn't shown).
func toStatusStreams(streams []encoder.StreamMetadata) []status.Stream {
var out []status.Stream
for _, s := range streams {
switch s.CodecType {
case "audio":
out = append(out, status.Stream{Kind: "audio", Language: s.Language, Codec: s.CodecName, Channels: s.Channels, Title: s.Title})
case "subtitle":
out = append(out, status.Stream{Kind: "subtitle", Language: s.Language, Codec: s.CodecName, Title: s.Title})
}
}
return out
}
func generateRandomString(length int) string {
bytes := make([]byte, length/2+1)
rand.Read(bytes)
return hex.EncodeToString(bytes)[:length]
}
func sanitizeFilename(name string) string {
reg := regexp.MustCompile(`[^a-zA-Z0-9\-äöÄÖ]`)
return reg.ReplaceAllString(name, "")
}
cmd/videnc/main.go
-186
View File
@@ -1,186 +0,0 @@
package main
import (
"crypto/rand"
"encoding/hex"
"flag"
"fmt"
"os"
"os/signal"
"path/filepath"
"regexp"
"syscall"
"videnc-vibe/internal/config"
"videnc-vibe/internal/encoder"
"videnc-vibe/internal/logger"
"videnc-vibe/internal/metadata"
"videnc-vibe/internal/mover"
"videnc-vibe/internal/watcher"
"videnc-vibe/pkg/types"
)
var (
deleteOrigin bool
configPath string
)
func init() {
flag.BoolVar(&deleteOrigin, "d", false, "Delete original after successful encode")
flag.StringVar(&configPath, "c", "", "Config file path (default: ~/.config/videnc-vibe/config.yaml)")
}
func main() {
flag.Parse()
cfg, err := config.Load(configPath)
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to load config: %v\n", err)
os.Exit(1)
}
if err := config.EnsureDirs(cfg); err != nil {
fmt.Fprintf(os.Stderr, "Failed to create directories: %v\n", err)
os.Exit(1)
}
log, err := logger.New(".")
if err != nil {
fmt.Fprintf(os.Stderr, "Failed to initialize logger: %v\n", err)
os.Exit(1)
}
defer log.Close()
enc := encoder.New()
if err := enc.CheckDeps(); err != nil {
log.Error("Dependency check failed", err.Error())
fmt.Fprintf(os.Stderr, "Dependency check failed: %v\n", err)
os.Exit(1)
}
metaClient := metadata.NewClient(cfg.OMDBAPIKey)
done := make(chan struct{})
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigChan
close(done)
}()
w := watcher.New(cfg.Paths.Input, 15)
w.Start(func(inputPath string) error {
return processFile(inputPath, cfg, enc, metaClient, log)
}, done)
log.Info("videnc-vibe started")
}
func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger) error {
log.Info(fmt.Sprintf("Processing: %s", inputPath))
filename := filepath.Base(inputPath)
isSeries, imdbID, tvmazeID, season, episode := metadata.ParseFilename(filename)
width, height, interlaced, err := enc.GetMediaInfo(inputPath)
if err != nil {
log.ErrorFile(inputPath, "Getting media info", err.Error())
mover.MoveToFailed(inputPath, cfg.Paths.Failed)
return err
}
streamLangs, err := enc.GetStreamLanguages(inputPath)
if err != nil {
log.ErrorFile(inputPath, "Getting stream languages", err.Error())
}
mediaType := watcher.DetectMediaType(width, height)
crf := cfg.Encoding.DVD.CRF
preset := cfg.Encoding.DVD.Preset
if mediaType == types.MediaTypeBluRay {
crf = cfg.Encoding.Bluray.CRF
preset = cfg.Encoding.Bluray.Preset
}
var meta *types.Metadata
if isSeries && tvmazeID != "" && season != "" && episode != "" {
meta, err = metaClient.FetchSeriesMetadata(tvmazeID, season, episode)
if err != nil {
log.ErrorFile(inputPath, "Fetching series metadata", err.Error())
}
} else if imdbID != "" {
meta, err = metaClient.FetchMovieMetadata(imdbID)
if err != nil {
log.ErrorFile(inputPath, "Fetching movie metadata", err.Error())
}
}
if meta == nil {
meta = &types.Metadata{
Title: "Unknown",
DateReleased: "",
IMDBID: "",
OriginalMedia: mediaType,
}
}
meta.OriginalMedia = mediaType
job := &types.Job{
InputPath: inputPath,
MediaType: mediaType,
CRF: crf,
Preset: preset,
DeleteOrigin: deleteOrigin,
}
if err := enc.Transcode(inputPath, job, meta, interlaced, streamLangs); err != nil {
log.ErrorFile(inputPath, "Transcoding", err.Error())
mover.MoveToFailed(inputPath, cfg.Paths.Failed)
outputPath := filepath.Join(filepath.Dir(inputPath), "output.mkv")
mover.MoveToFailed(outputPath, cfg.Paths.Failed)
return err
}
outputDir := filepath.Dir(inputPath)
outputPath := filepath.Join(outputDir, "output.mkv")
var outFilename string
switch {
case isSeries && meta.Collection != "":
outFilename = fmt.Sprintf("%s.S%sE%s.mkv", sanitizeFilename(meta.Collection), season, episode)
case !isSeries && meta.IMDBID != "" && meta.Title != "Unknown":
outFilename = fmt.Sprintf("%s.%s.mkv", sanitizeFilename(meta.Title), meta.IMDBID)
default:
outFilename = fmt.Sprintf("%s.nometadata.mkv", generateRandomString(8))
}
finalOutput := filepath.Join(cfg.Paths.Output, outFilename)
if err := mover.Rename(outputPath, finalOutput); err != nil {
log.ErrorFile(inputPath, "Moving output", err.Error())
mover.MoveToFailed(outputPath, cfg.Paths.Failed)
return err
}
if deleteOrigin {
mover.Delete(inputPath)
log.Info(fmt.Sprintf("Deleted original: %s", inputPath))
} else {
mover.MoveToOriginals(inputPath, cfg.Paths.Originals)
log.Info(fmt.Sprintf("Moved original to originals: %s", inputPath))
}
log.Info(fmt.Sprintf("Completed: %s -> %s", inputPath, finalOutput))
return nil
}
func generateRandomString(length int) string {
bytes := make([]byte, length/2+1)
rand.Read(bytes)
return hex.EncodeToString(bytes)[:length]
}
func sanitizeFilename(name string) string {
reg := regexp.MustCompile(`[^a-zA-Z0-9\-äöÄÖ]`)
return reg.ReplaceAllString(name, "")
}
config.example.yaml
+7
View File
@@ -1,5 +1,9 @@
omdb_api_key: "YOUR_API_KEY_HERE" omdb_api_key: "YOUR_API_KEY_HERE"
# Status server listen address. Omit for the default ":8080"; set to "" to disable.
# GET /status returns live encode progress, the input queue, and recent events.
http_addr: ":8080"
encoding: encoding:
dvd: dvd:
crf: 30 crf: 30
@@ -7,6 +11,9 @@ encoding:
bluray: bluray:
crf: 29 crf: 29
preset: 3 preset: 3
# SVT-AV1 logical-processor (thread) cap so a long encode can't pin every core.
# 0 = auto (use all cores). Applies to every profile.
lp: 0
paths: paths:
input: "./input" input: "./input"
docker-compose.yml
+23
View File
@@ -0,0 +1,23 @@
services:
av1dae:
build: .
container_name: av1dae
restart: unless-stopped
# Cap CPU/RAM so a long SVT-AV1 encode can't starve the host. Tune to taste.
# `podman compose` honors deploy.resources; `podman-compose` (python) wants
# the short keys instead — cpus: 4.0 / mem_limit: 4g / cpuset: "0-3".
deploy:
resources:
limits:
cpus: "8.0"
memory: 4g
ports:
- "8080:8080" # status server + dashboard at http://host:8080 (needs http_addr: ":8080" in config)
# By default the queue is HELD on boot — click Start in the UI. Uncomment to auto-start:
# environment:
# - AV1DAE_AUTOSTART=1
volumes:
- ./data/config.yaml:/config/config.yaml:ro # your config — paths inside must point at /data/*
- ./data/media:/data # holds input/ output/ originals/ failed/ work/ + logs
# Uncomment to delete originals after a successful encode (appends -d to the entrypoint):
# command: ["-d"]
go.mod
+3 -1
View File
@@ -1,5 +1,7 @@
module videnc-vibe module av1dae
go 1.26.1 go 1.26.1
require gopkg.in/yaml.v3 v3.0.1 require gopkg.in/yaml.v3 v3.0.1
require github.com/mattn/go-sqlite3 v1.14.44 // indirect
go.sum
+2
View File
@@ -1,3 +1,5 @@
github.com/mattn/go-sqlite3 v1.14.44 h1:3VSe+xafpbzsLbdr2AWlAZk9yRHiBhTBakioXaCKTF8=
github.com/mattn/go-sqlite3 v1.14.44/go.mod h1:pjEuOr8IwzLJP2MfGeTb0A35jauH+C2kbHKBr7yXKVQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
internal/config/config.go
+25 -3
View File
@@ -5,12 +5,12 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"av1dae/pkg/types"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
"videnc-vibe/pkg/types"
) )
const ( const (
appName = "videnc-vibe" appName = "av1dae"
configName = "config.yaml" configName = "config.yaml"
) )
@@ -49,6 +49,9 @@ func Load(configPath string) (*types.Config, error) {
if cfg.Paths.Failed == "" { if cfg.Paths.Failed == "" {
cfg.Paths.Failed = "./failed" cfg.Paths.Failed = "./failed"
} }
if cfg.Paths.Work == "" {
cfg.Paths.Work = "./work"
}
if cfg.Encoding.DVD.CRF == 0 { if cfg.Encoding.DVD.CRF == 0 {
cfg.Encoding.DVD.CRF = 30 cfg.Encoding.DVD.CRF = 30
} }
@@ -61,12 +64,31 @@ func Load(configPath string) (*types.Config, error) {
if cfg.Encoding.Bluray.Preset == 0 { if cfg.Encoding.Bluray.Preset == 0 {
cfg.Encoding.Bluray.Preset = 3 cfg.Encoding.Bluray.Preset = 3
} }
if cfg.Encoding.WebDL.CRF == 0 {
cfg.Encoding.WebDL.CRF = 30
}
if cfg.Encoding.WebDL.Preset == 0 {
cfg.Encoding.WebDL.Preset = 3
}
if cfg.Encoding.TVRip.CRF == 0 {
cfg.Encoding.TVRip.CRF = 32
}
if cfg.Encoding.TVRip.Preset == 0 {
cfg.Encoding.TVRip.Preset = 2
}
if cfg.LogRetentionDays == 0 {
cfg.LogRetentionDays = 7
}
if cfg.HTTPAddr == nil {
def := ":8080"
cfg.HTTPAddr = &def
}
return &cfg, nil return &cfg, nil
} }
func EnsureDirs(cfg *types.Config) error { func EnsureDirs(cfg *types.Config) error {
dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed} dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed, cfg.Paths.Work}
for _, dir := range dirs { for _, dir := range dirs {
if err := os.MkdirAll(dir, 0755); err != nil { if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("creating directory %s: %w", dir, err) return fmt.Errorf("creating directory %s: %w", dir, err)
internal/encoder/encoder.go
+310 -100
View File
@@ -1,16 +1,24 @@
package encoder package encoder
import ( import (
"bytes"
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"regexp" "regexp"
"sort"
"strconv" "strconv"
"strings" "strings"
"sync"
"syscall"
"time"
"videnc-vibe/pkg/types" "av1dae/internal/logger"
"av1dae/internal/status"
"av1dae/pkg/types"
) )
var idetSummaryRegex = regexp.MustCompile(`Multi frame detection:\s+TFF:\s*(\d+)\s+BFF:\s*(\d+)\s+Progressive:\s*(\d+)\s+Undetermined:\s*(\d+)`) var idetSummaryRegex = regexp.MustCompile(`Multi frame detection:\s+TFF:\s*(\d+)\s+BFF:\s*(\d+)\s+Progressive:\s*(\d+)\s+Undetermined:\s*(\d+)`)
@@ -19,6 +27,61 @@ type Encoder struct {
ffmpegPath string ffmpegPath string
ffprobePath string ffprobePath string
opusencPath string opusencPath string
log *logger.Logger
tracker *status.Tracker
procMu sync.Mutex
cur *os.Process // the in-flight video encode, for pause/resume
}
// Pause freezes the active video encode in place via SIGSTOP. libsvtav1 runs in
// the ffmpeg process (no forked children), so one signal suspends all its
// threads. No-op if nothing is encoding. The process keeps its memory and
// partial output and resumes exactly where it left off.
func (e *Encoder) Pause() error {
e.procMu.Lock()
defer e.procMu.Unlock()
if e.cur == nil {
return nil
}
if err := e.cur.Signal(syscall.SIGSTOP); err != nil {
return err
}
if e.tracker != nil {
e.tracker.SetPaused(true)
}
return nil
}
// Resume thaws a paused encode via SIGCONT. No-op if nothing is encoding.
func (e *Encoder) Resume() error {
e.procMu.Lock()
defer e.procMu.Unlock()
if e.cur == nil {
return nil
}
if err := e.cur.Signal(syscall.SIGCONT); err != nil {
return err
}
if e.tracker != nil {
e.tracker.SetPaused(false)
}
return nil
}
func (e *Encoder) setProc(p *os.Process) {
e.procMu.Lock()
e.cur = p
e.procMu.Unlock()
}
func (e *Encoder) clearProc() {
e.procMu.Lock()
e.cur = nil
e.procMu.Unlock()
if e.tracker != nil {
e.tracker.SetPaused(false)
}
} }
type StreamInfo struct { type StreamInfo struct {
@@ -38,13 +101,133 @@ type StreamMetadata struct {
Index int `json:"index"` Index int `json:"index"`
CodecType string `json:"codec_type"` CodecType string `json:"codec_type"`
CodecName string `json:"codec_name"` CodecName string `json:"codec_name"`
Channels int `json:"channels"`
Language string `json:"tags"` Language string `json:"tags"`
Title string `json:"title"` Title string `json:"title"`
} }
func (e *Encoder) GetStreamLanguages(path string) ([]StreamMetadata, error) { func New(log *logger.Logger, tracker *status.Tracker) *Encoder {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path) return &Encoder{
output, err := cmd.CombinedOutput() ffmpegPath: "ffmpeg",
ffprobePath: "ffprobe",
opusencPath: "opusenc",
log: log,
tracker: tracker,
}
}
// runCmd executes the command and emits a debug entry containing the full
// command line and its combined output. file is the source file the command
// is acting on (may be empty).
func (e *Encoder) runCmd(ctx context.Context, label, file, name string, args []string) ([]byte, error) {
cmd := exec.CommandContext(ctx, name, args...)
out, err := cmd.CombinedOutput()
extra, _ := json.Marshal(struct {
Cmd string `json:"cmd"`
Output string `json:"output"`
}{
Cmd: name + " " + strings.Join(args, " "),
Output: string(out),
})
e.log.Debug(label, file, string(extra))
return out, err
}
// runCmdProgress runs ffmpeg with `-progress pipe:1`, streaming progress
// samples to the tracker (instead of buffering all output like runCmd). stdout
// carries only the key=value progress stream; stderr carries real errors and is
// returned for the caller's error message. Used solely for the video encode —
// the one step long enough to be worth watching live.
func (e *Encoder) runCmdProgress(ctx context.Context, label, file, name string, args []string) ([]byte, error) {
cmd := exec.CommandContext(ctx, name, args...)
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
var stderr bytes.Buffer
cmd.Stderr = &stderr
if err := cmd.Start(); err != nil {
return nil, err
}
// Publish the process so Pause/Resume can signal it; clear on exit.
e.setProc(cmd.Process)
defer e.clearProc()
// Reads stdout to EOF (when ffmpeg exits), so Wait below is safe afterwards.
var lastLog time.Time
_ = status.ScanProgress(stdout, func(s status.ProgressSample) {
if e.tracker == nil {
return
}
e.tracker.Update(s.OutTimeSec, s.FPS, s.Speed)
if time.Since(lastLog) >= 5*time.Second {
lastLog = time.Now()
snap := e.tracker.Snapshot()
e.log.Progress(fmt.Sprintf("encoding %s · %.1f%% · %.1ffps · %.2fx · ETA %s",
filepath.Base(file), snap.Percent, snap.FPS, snap.Speed, fmtETA(snap.ETASec)))
}
})
err = cmd.Wait()
extra, _ := json.Marshal(struct {
Cmd string `json:"cmd"`
Output string `json:"output"`
}{
Cmd: name + " " + strings.Join(args, " "),
Output: stderr.String(),
})
e.log.Debug(label, file, string(extra))
return stderr.Bytes(), err
}
// fmtETA renders a seconds count as a compact "11h03m" / "4m12s" / "9s" string.
func fmtETA(sec int) string {
if sec <= 0 {
return "--"
}
d := time.Duration(sec) * time.Second
switch {
case d >= time.Hour:
return fmt.Sprintf("%dh%02dm", int(d.Hours()), int(d.Minutes())%60)
case d >= time.Minute:
return fmt.Sprintf("%dm%02ds", int(d.Minutes()), sec%60)
default:
return fmt.Sprintf("%ds", sec)
}
}
// GetDuration returns the source container duration in seconds via ffprobe.
func (e *Encoder) GetDuration(ctx context.Context, path string) (float64, error) {
args := []string{"-v", "error", "-show_entries", "format=duration", "-of", "default=noprint_wrappers=1:nokey=1", path}
out, err := e.runCmd(ctx, "ffprobe duration", path, e.ffprobePath, args)
if err != nil {
return 0, fmt.Errorf("ffprobe duration: %w", err)
}
d, err := strconv.ParseFloat(strings.TrimSpace(string(out)), 64)
if err != nil {
return 0, fmt.Errorf("parsing duration %q: %w", strings.TrimSpace(string(out)), err)
}
return d, nil
}
func (e *Encoder) CheckDeps() error {
if _, err := exec.LookPath(e.ffmpegPath); err != nil {
return fmt.Errorf("ffmpeg not found")
}
if _, err := exec.LookPath(e.ffprobePath); err != nil {
return fmt.Errorf("ffprobe not found")
}
if _, err := exec.LookPath(e.opusencPath); err != nil {
return fmt.Errorf("opusenc not found")
}
return nil
}
func (e *Encoder) GetStreamLanguages(ctx context.Context, path string) ([]StreamMetadata, error) {
args := []string{"-v", "error", "-show_streams", "-print_format", "json", path}
output, err := e.runCmd(ctx, "ffprobe stream languages", path, e.ffprobePath, args)
if err != nil { if err != nil {
return nil, fmt.Errorf("ffprobe error: %w", err) return nil, fmt.Errorf("ffprobe error: %w", err)
} }
@@ -54,6 +237,7 @@ func (e *Encoder) GetStreamLanguages(path string) ([]StreamMetadata, error) {
Index int `json:"index"` Index int `json:"index"`
CodecType string `json:"codec_type"` CodecType string `json:"codec_type"`
CodecName string `json:"codec_name"` CodecName string `json:"codec_name"`
Channels int `json:"channels"`
Tags map[string]string `json:"tags"` Tags map[string]string `json:"tags"`
} `json:"streams"` } `json:"streams"`
} }
@@ -73,45 +257,24 @@ func (e *Encoder) GetStreamLanguages(path string) ([]StreamMetadata, error) {
Index: s.Index, Index: s.Index,
CodecType: s.CodecType, CodecType: s.CodecType,
CodecName: s.CodecName, CodecName: s.CodecName,
Channels: s.Channels,
Language: lang, Language: lang,
Title: title, Title: title,
}) })
} }
fmt.Printf("DEBUG stream languages: %+v\n", streams) streamsJSON, _ := json.Marshal(streams)
e.log.Debug("parsed stream languages", path, string(streamsJSON))
return streams, nil return streams, nil
} }
func New() *Encoder { func (e *Encoder) GetMediaInfo(ctx context.Context, path string) (width, height int, interlaced bool, err error) {
return &Encoder{ probeArgs := []string{"-v", "error", "-show_streams", "-print_format", "json", path}
ffmpegPath: "ffmpeg", output, err := e.runCmd(ctx, "ffprobe media info", path, e.ffprobePath, probeArgs)
ffprobePath: "ffprobe",
opusencPath: "opusenc",
}
}
func (e *Encoder) CheckDeps() error {
if _, err := exec.LookPath(e.ffmpegPath); err != nil {
return fmt.Errorf("ffmpeg not found")
}
if _, err := exec.LookPath(e.ffprobePath); err != nil {
return fmt.Errorf("ffprobe not found")
}
if _, err := exec.LookPath(e.opusencPath); err != nil {
return fmt.Errorf("opusenc not found")
}
return nil
}
func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool, err error) {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
output, err := cmd.CombinedOutput()
if err != nil { if err != nil {
return 0, 0, false, fmt.Errorf("ffprobe error: %w", err) return 0, 0, false, fmt.Errorf("ffprobe error: %w", err)
} }
fmt.Printf("DEBUG ffprobe output: %s\n", string(output))
var result ProbeResult var result ProbeResult
if err := json.Unmarshal(output, &result); err != nil { if err := json.Unmarshal(output, &result); err != nil {
return 0, 0, false, fmt.Errorf("parsing ffprobe output: %w", err) return 0, 0, false, fmt.Errorf("parsing ffprobe output: %w", err)
@@ -123,7 +286,10 @@ func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool,
width = stream.Width width = stream.Width
height = stream.Height height = stream.Height
foundVideo = true foundVideo = true
fmt.Printf("DEBUG detected video: %dx%d, codec=%s, SAR=%s\n", width, height, stream.CodecName, stream.SampleAspectRatio) info, _ := json.Marshal(map[string]interface{}{
"width": width, "height": height, "codec": stream.CodecName, "sar": stream.SampleAspectRatio,
})
e.log.Debug("detected video stream", path, string(info))
break break
} }
} }
@@ -131,7 +297,7 @@ func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool,
return 0, 0, false, fmt.Errorf("no video stream found") return 0, 0, false, fmt.Errorf("no video stream found")
} }
cmd = exec.Command(e.ffmpegPath, idetArgs := []string{
"-hide_banner", "-hide_banner",
"-nostats", "-nostats",
"-i", path, "-i", path,
@@ -139,11 +305,12 @@ func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool,
"-frames:v", "400", "-frames:v", "400",
"-an", "-sn", "-an", "-sn",
"-f", "null", "-", "-f", "null", "-",
) }
idetOut, _ := cmd.CombinedOutput() idetOut, _ := e.runCmd(ctx, "ffmpeg idet", path, e.ffmpegPath, idetArgs)
interlaced = detectInterlaced(string(idetOut)) interlaced = detectInterlaced(string(idetOut))
fmt.Printf("DEBUG interlaced: %v\n", interlaced) info, _ := json.Marshal(map[string]bool{"interlaced": interlaced})
e.log.Debug("interlace detection", path, string(info))
return width, height, interlaced, nil return width, height, interlaced, nil
} }
@@ -168,15 +335,13 @@ func detectInterlaced(idetOutput string) bool {
return interlaced > prog return interlaced > prog
} }
func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string, int, error) { func (e *Encoder) calculateZscaleWidth(ctx context.Context, path string, originalHeight int) (string, int, error) {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path) args := []string{"-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path}
output, err := cmd.CombinedOutput() output, err := e.runCmd(ctx, "ffprobe zscale info", path, e.ffprobePath, args)
if err != nil { if err != nil {
return "", 0, fmt.Errorf("ffprobe error: %w", err) return "", 0, fmt.Errorf("ffprobe error: %w", err)
} }
fmt.Printf("DEBUG ffprobe for zscale: %s\n", string(output))
var result ProbeResult var result ProbeResult
if err := json.Unmarshal(output, &result); err != nil { if err := json.Unmarshal(output, &result); err != nil {
return "", 0, fmt.Errorf("parsing ffprobe: %w", err) return "", 0, fmt.Errorf("parsing ffprobe: %w", err)
@@ -189,9 +354,7 @@ func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string,
stream := result.Streams[0] stream := result.Streams[0]
width := stream.Width width := stream.Width
height := stream.Height height := stream.Height
sar := stream.SampleAspectRatio sar := stream.SampleAspectRatio
fmt.Printf("DEBUG stream: width=%d, height=%d, sar=%s\n", width, height, sar)
newWidth := width newWidth := width
@@ -211,7 +374,11 @@ func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string,
// Round to nearest, then snap down to even (mod-2 widths preferred by AV1/H.264). // Round to nearest, then snap down to even (mod-2 widths preferred by AV1/H.264).
newWidth = ((width*num + den/2) / den) &^ 1 newWidth = ((width*num + den/2) / den) &^ 1
fmt.Printf("DEBUG calculated new width: %d (sar=%s)\n", newWidth, sar)
info, _ := json.Marshal(map[string]interface{}{
"width": width, "height": height, "sar": sar, "new_width": newWidth,
})
e.log.Debug("zscale calculation", path, string(info))
if newWidth == width { if newWidth == width {
return "", newWidth, nil return "", newWidth, nil
@@ -221,65 +388,113 @@ func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string,
return zscale, newWidth, nil return zscale, newWidth, nil
} }
func (e *Encoder) Transcode(input string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error { func (e *Encoder) Transcode(ctx context.Context, input, workDir string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
dir := filepath.Dir(input) if e.tracker != nil {
audioWavs, err := e.extractAudio(input, dir) e.tracker.SetPhase(status.PhaseAudio)
}
audioWavs, err := e.extractAudio(ctx, input, workDir, streamLangs)
if err != nil { if err != nil {
return fmt.Errorf("extracting audio: %w", err) return fmt.Errorf("extracting audio: %w", err)
} }
defer e.cleanupWavs(audioWavs)
opusFiles, err := e.encodeOpus(audioWavs, dir) opusFiles, err := e.encodeOpus(ctx, audioWavs, workDir)
if err != nil { if err != nil {
return fmt.Errorf("encoding opus: %w", err) return fmt.Errorf("encoding opus: %w", err)
} }
defer e.cleanupOpus(opusFiles)
if err := e.encodeVideo(input, opusFiles, job, metadata, interlaced, streamLangs); err != nil { if err := e.encodeVideo(ctx, input, workDir, opusFiles, job, metadata, interlaced, streamLangs); err != nil {
return fmt.Errorf("encoding video: %w", err) return fmt.Errorf("encoding video: %w", err)
} }
return nil return nil
} }
func (e *Encoder) extractAudio(input, dir string) ([]string, error) { // audioStreamsInSourceOrder returns the audio entries of streamLangs sorted by
cmd := exec.Command(e.ffmpegPath, "-i", input, // their source-side Index. The resulting slice position is the 0-based audio
// stream index used by ffmpeg selectors like `0:a:<n>`.
func audioStreamsInSourceOrder(streamLangs []StreamMetadata) []StreamMetadata {
var audio []StreamMetadata
for _, s := range streamLangs {
if s.CodecType == "audio" {
audio = append(audio, s)
}
}
sort.Slice(audio, func(i, j int) bool { return audio[i].Index < audio[j].Index })
return audio
}
func (e *Encoder) extractAudio(ctx context.Context, input, workDir string, streamLangs []StreamMetadata) ([]string, error) {
audio := audioStreamsInSourceOrder(streamLangs)
if len(audio) == 0 {
return nil, fmt.Errorf("no audio streams found")
}
wavs := make([]string, 0, len(audio))
for srcAudioIndex := range audio {
wavPath := filepath.Join(workDir, fmt.Sprintf("audio.%d.wav", srcAudioIndex))
args := []string{
"-i", input,
"-map", fmt.Sprintf("0:a:%d", srcAudioIndex),
"-vn", "-c:a", "pcm_s16le", "-ar", "48000", "-vn", "-c:a", "pcm_s16le", "-ar", "48000",
"-f", "wav", filepath.Join(dir, "audio.wav")) "-f", "wav", wavPath,
if out, err := cmd.CombinedOutput(); err != nil { }
return nil, fmt.Errorf("ffmpeg extract: %s %w", out, err) out, err := e.runCmd(ctx, "ffmpeg extract audio", input, e.ffmpegPath, args)
if err != nil {
return wavs, fmt.Errorf("ffmpeg extract a:%d: %s %w", srcAudioIndex, out, err)
}
wavs = append(wavs, wavPath)
}
return wavs, nil
} }
matches, _ := filepath.Glob(filepath.Join(dir, "audio.wav")) func (e *Encoder) encodeOpus(ctx context.Context, wavs []string, workDir string) ([]string, error) {
return matches, nil
}
func (e *Encoder) encodeOpus(wavs []string, dir string) ([]string, error) {
var opusFiles []string var opusFiles []string
for _, wav := range wavs { for _, wav := range wavs {
out := strings.Replace(wav, ".wav", ".opus", 1) base := filepath.Base(strings.Replace(wav, ".wav", ".opus", 1))
cmd := exec.Command(e.opusencPath, "--bitrate", "128k", wav, out) out := filepath.Join(workDir, base)
if out, err := cmd.CombinedOutput(); err != nil { args := []string{"--bitrate", "128k", wav, out}
return nil, fmt.Errorf("opusenc: %s %w", out, err) logOut, err := e.runCmd(ctx, "opusenc", wav, e.opusencPath, args)
if err != nil {
return nil, fmt.Errorf("opusenc: %s %w", logOut, err)
} }
opusFiles = append(opusFiles, out) opusFiles = append(opusFiles, out)
} }
return opusFiles, nil return opusFiles, nil
} }
func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error { // svtav1Params builds the -svtav1-params value, appending lp=N (logical
dir := filepath.Dir(input) // processors / encoder thread count) only when lp > 0. lp=0 lets SVT-AV1
outFile := filepath.Join(dir, "output.mkv") // auto-detect, preserving prior behavior.
func svtav1Params(lp int) string {
p := "film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s"
if lp > 0 {
p += fmt.Sprintf(":lp=%d", lp)
}
return p
}
svtParams := fmt.Sprintf("film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s") func (e *Encoder) encodeVideo(ctx context.Context, input, workDir string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
outFile := filepath.Join(workDir, "output.mkv")
zscaleStr, newWidth, err := e.calculateZscaleWidth(input, 0) // Switch the tracker to the encode phase and feed it the source duration so
// progress samples can be turned into a percentage. A failed duration probe
// just means no percent — it must not abort the encode.
if e.tracker != nil {
e.tracker.SetPhase(status.PhaseEncoding)
if dur, derr := e.GetDuration(ctx, input); derr == nil {
e.tracker.SetTotal(dur)
} else {
e.log.Debug("duration probe failed", input, derr.Error())
}
}
svtParams := svtav1Params(job.LP)
zscaleStr, newWidth, err := e.calculateZscaleWidth(ctx, input, 0)
if err != nil { if err != nil {
return fmt.Errorf("calculating zscale: %w", err) return fmt.Errorf("calculating zscale: %w", err)
} }
fmt.Printf("DEBUG zscale: %s (newWidth=%d)\n", zscaleStr, newWidth)
var filters []string var filters []string
if interlaced { if interlaced {
filters = append(filters, "bwdif=mode=0:par=-1:-1") filters = append(filters, "bwdif=mode=0:par=-1:-1")
@@ -288,7 +503,10 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
filters = append(filters, zscaleStr) filters = append(filters, zscaleStr)
} }
fmt.Printf("DEBUG final vf: %s\n", strings.Join(filters, ",")) filterInfo, _ := json.Marshal(map[string]interface{}{
"zscale": zscaleStr, "new_width": newWidth, "interlaced": interlaced, "vf": strings.Join(filters, ","),
})
e.log.Debug("video filter chain", input, string(filterInfo))
args := []string{ args := []string{
"-y", "-y",
@@ -319,17 +537,23 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
args = append(args, "-c:s", "copy") args = append(args, "-c:s", "copy")
args = append(args, "-map_metadata", "-1") args = append(args, "-map_metadata", "-1")
// Walk opusFiles by output audio index (0..N-1) and look up the source
// audio language at the same source-audio-index position. This keeps the
// `-metadata:s:a:i` index aligned with the output stream order, regardless
// of whether some source streams lacked a language tag.
audioStreams := audioStreamsInSourceOrder(streamLangs)
for i := range opusFiles {
if i >= len(audioStreams) {
break
}
lang := audioStreams[i].Language
if lang == "" {
continue
}
args = append(args, fmt.Sprintf("-metadata:s:a:%d", i), fmt.Sprintf("language=%s", lang))
}
for _, stream := range streamLangs { for _, stream := range streamLangs {
if stream.CodecType == "audio" && stream.Language != "" {
idx := 1
for _, s := range streamLangs {
if s.CodecType == "audio" && s.Index < stream.Index {
idx++
}
}
args = append(args, fmt.Sprintf("-metadata:s:a:%d", idx-1), fmt.Sprintf("language=%s", stream.Language))
fmt.Printf("DEBUG setting audio language: stream %d -> language=%s\n", idx-1, stream.Language)
}
if stream.CodecType == "subtitle" && stream.Language != "" { if stream.CodecType == "subtitle" && stream.Language != "" {
idx := 0 idx := 0
for _, s := range streamLangs { for _, s := range streamLangs {
@@ -338,7 +562,6 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
} }
} }
args = append(args, fmt.Sprintf("-metadata:s:s:%d", idx), fmt.Sprintf("language=%s", stream.Language)) args = append(args, fmt.Sprintf("-metadata:s:s:%d", idx), fmt.Sprintf("language=%s", stream.Language))
fmt.Printf("DEBUG setting subtitle language: stream %d -> language=%s\n", idx, stream.Language)
} }
} }
@@ -358,24 +581,11 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
args = append(args, outFile) args = append(args, outFile)
args = append([]string{"-hide_banner", "-v", "error"}, args...) args = append([]string{"-hide_banner", "-v", "error", "-progress", "pipe:1", "-nostats"}, args...)
cmd := exec.Command(e.ffmpegPath, args...) out, err := e.runCmdProgress(ctx, "ffmpeg encode", input, e.ffmpegPath, args)
fmt.Printf("DEBUG FFmpeg command: ffmpeg %s\n", strings.Join(args, " ")) if err != nil {
if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("ffmpeg encode: %s %w", out, err) return fmt.Errorf("ffmpeg encode: %s %w", out, err)
} }
return nil return nil
} }
func (e *Encoder) cleanupWavs(files []string) {
for _, f := range files {
os.Remove(f)
}
}
func (e *Encoder) cleanupOpus(files []string) {
for _, f := range files {
os.Remove(f)
}
}
internal/encoder/encoder_test.go
+23
View File
@@ -0,0 +1,23 @@
package encoder
import (
"strings"
"testing"
)
func TestSvtav1Params(t *testing.T) {
const base = "film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s"
if got := svtav1Params(0); got != base {
t.Errorf("lp=0 should not append lp:\n got %q\nwant %q", got, base)
}
if got := svtav1Params(-1); got != base {
t.Errorf("lp<0 should not append lp: got %q", got)
}
if got := svtav1Params(4); got != base+":lp=4" {
t.Errorf("lp=4: got %q, want suffix :lp=4", got)
}
if got := svtav1Params(4); !strings.HasPrefix(got, base) {
t.Errorf("lp set should keep the base params intact: got %q", got)
}
}
internal/logger/logger.go
+131 -56
View File
@@ -1,94 +1,169 @@
package logger package logger
import ( import (
"encoding/json" "database/sql"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"time" "time"
"videnc-vibe/pkg/types" _ "github.com/mattn/go-sqlite3"
)
const (
LevelDebug = "debug"
LevelInfo = "info"
LevelError = "error"
) )
type Logger struct { type Logger struct {
infoPath string db *sql.DB
errorPath string dbPath string
structPath string retentionDays int
infoFile *os.File
errorFile *os.File
structFile *os.File
} }
func New(logDir string) (*Logger, error) { // New opens (or creates) logs.db inside logDir, applies the schema, and runs
// an initial retention purge. retentionDays <= 0 falls back to 7.
func New(logDir string, retentionDays int) (*Logger, error) {
if retentionDays <= 0 {
retentionDays = 7
}
if err := os.MkdirAll(logDir, 0755); err != nil { if err := os.MkdirAll(logDir, 0755); err != nil {
return nil, fmt.Errorf("creating log directory: %w", err) return nil, fmt.Errorf("creating log directory: %w", err)
} }
now := time.Now().Format("2006-01-02") dbPath := filepath.Join(logDir, "logs.db")
infoPath := filepath.Join(logDir, fmt.Sprintf("info_%s.log", now)) db, err := sql.Open("sqlite3", dbPath+"?_journal_mode=WAL&_busy_timeout=5000")
errorPath := filepath.Join(logDir, fmt.Sprintf("error_%s.log", now))
structPath := filepath.Join(logDir, "structured.json")
infoFile, err := os.OpenFile(infoPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil { if err != nil {
return nil, fmt.Errorf("opening info log: %w", err) return nil, fmt.Errorf("opening log db: %w", err)
} }
errorFile, err := os.OpenFile(errorPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) schema := `
if err != nil { CREATE TABLE IF NOT EXISTS logs (
return nil, fmt.Errorf("opening error log: %w", err) id INTEGER PRIMARY KEY AUTOINCREMENT,
ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
level TEXT NOT NULL,
message TEXT NOT NULL,
file TEXT,
extra TEXT
);
CREATE INDEX IF NOT EXISTS idx_logs_ts_level ON logs(ts, level);
`
if _, err := db.Exec(schema); err != nil {
db.Close()
return nil, fmt.Errorf("applying log schema: %w", err)
} }
structFile, err := os.OpenFile(structPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644) l := &Logger{db: db, dbPath: dbPath, retentionDays: retentionDays}
if err != nil { l.purge()
return nil, fmt.Errorf("opening structured log: %w", err) return l, nil
} }
return &Logger{ func (l *Logger) purge() {
infoPath: infoPath, cutoff := fmt.Sprintf("-%d days", l.retentionDays)
errorPath: errorPath, _, _ = l.db.Exec(`DELETE FROM logs WHERE ts < datetime('now', ?)`, cutoff)
structPath: structPath, }
infoFile: infoFile,
errorFile: errorFile, func (l *Logger) write(level, message, file, extra string) {
structFile: structFile, _, _ = l.db.Exec(
}, nil `INSERT INTO logs(level, message, file, extra) VALUES(?, ?, ?, ?)`,
level, message, nullIfEmpty(file), nullIfEmpty(extra),
)
}
func nullIfEmpty(s string) interface{} {
if s == "" {
return nil
}
return s
}
func stamp() string {
return time.Now().Format("2006-01-02 15:04:05")
} }
func (l *Logger) Info(message string) { func (l *Logger) Info(message string) {
timestamp := time.Now().Format("2006-01-02 15:04:05") fmt.Fprintf(os.Stdout, "[%s] INFO: %s\n", stamp(), message)
entry := fmt.Sprintf("[%s] INFO: %s\n", timestamp, message) l.write(LevelInfo, message, "", "")
l.infoFile.WriteString(entry)
l.writeStructured("info", message, "", "")
} }
func (l *Logger) Error(message, err string) { // Progress prints to stdout only, without a db row. For high-frequency,
timestamp := time.Now().Format("2006-01-02 15:04:05") // ephemeral telemetry (live encode progress) that would otherwise bury real
entry := fmt.Sprintf("[%s] ERROR: %s - %s\n", timestamp, message, err) // events in logs.db and churn until retention purges it.
l.errorFile.WriteString(entry) func (l *Logger) Progress(message string) {
l.writeStructured("error", message, err, "") fmt.Fprintf(os.Stdout, "[%s] INFO: %s\n", stamp(), message)
} }
func (l *Logger) ErrorFile(file string, message, err string) { func (l *Logger) Error(message, errStr string) {
timestamp := time.Now().Format("2006-01-02 15:04:05") msg := fmt.Sprintf("%s: %s", message, errStr)
entry := fmt.Sprintf("[%s] ERROR: %s - %s (file: %s)\n", timestamp, message, err, file) fmt.Fprintf(os.Stderr, "[%s] ERROR: %s\n", stamp(), msg)
l.errorFile.WriteString(entry) l.write(LevelError, msg, "", "")
l.writeStructured("error", message, err, file)
} }
func (l *Logger) writeStructured(level, message, err, file string) { func (l *Logger) ErrorFile(file, message, errStr string) {
entry := types.LogEntry{ msg := fmt.Sprintf("%s: %s", message, errStr)
Timestamp: time.Now(), fmt.Fprintf(os.Stderr, "[%s] ERROR: %s (file: %s)\n", stamp(), msg, file)
Level: level, l.write(LevelError, msg, file, "")
Message: message,
Error: err,
File: file,
} }
data, _ := json.Marshal(entry)
l.structFile.WriteString(string(data) + "\n") // Debug records a debug entry to the database only. extra is an opaque string,
// typically JSON, used for ffprobe output, API response bodies, or command
// stdout/stderr captures. Pass "" if not applicable.
func (l *Logger) Debug(message, file, extra string) {
l.write(LevelDebug, message, file, extra)
}
// DB exposes the underlying connection so the settings store can share the
// same logs.db file rather than opening a second database.
func (l *Logger) DB() *sql.DB { return l.db }
// SetRetention updates the purge horizon at runtime (e.g. from the settings UI).
// Takes effect on the next purge. Ignored for non-positive values.
func (l *Logger) SetRetention(days int) {
if days > 0 {
l.retentionDays = days
}
}
// LogEntry is one row returned by RecentLogs, shaped for the status feed.
type LogEntry struct {
TS string `json:"ts"`
Level string `json:"level"`
Message string `json:"message"`
File string `json:"file,omitempty"`
}
// RecentLogs returns the newest-first non-debug entries, capped at n. Debug
// rows (per-command ffprobe/ffmpeg/API dumps) are excluded — the status feed
// wants real events, not invocation noise.
func (l *Logger) RecentLogs(n int) ([]LogEntry, error) {
if l.db == nil {
return nil, nil
}
rows, err := l.db.Query(
`SELECT ts, level, message, COALESCE(file, '') FROM logs WHERE level != ? ORDER BY id DESC LIMIT ?`,
LevelDebug, n)
if err != nil {
return nil, err
}
defer rows.Close()
var out []LogEntry
for rows.Next() {
var e LogEntry
if err := rows.Scan(&e.TS, &e.Level, &e.Message, &e.File); err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
} }
func (l *Logger) Close() { func (l *Logger) Close() {
l.infoFile.Close() if l.db == nil {
l.errorFile.Close() return
l.structFile.Close() }
l.purge()
l.db.Close()
l.db = nil
} }
internal/metadata/metadata.go
+86 -10
View File
@@ -1,16 +1,19 @@
package metadata package metadata
import ( import (
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"net/url"
"regexp" "regexp"
"strconv" "strconv"
"strings" "strings"
"time" "time"
"videnc-vibe/pkg/types" "av1dae/internal/logger"
"av1dae/pkg/types"
) )
const ( const (
@@ -22,6 +25,7 @@ var (
imdbRegex = regexp.MustCompile(`tt(\d+)`) imdbRegex = regexp.MustCompile(`tt(\d+)`)
tvmRegex = regexp.MustCompile(`(?i)tvm(\d+)`) tvmRegex = regexp.MustCompile(`(?i)tvm(\d+)`)
seRegex = regexp.MustCompile(`(?i)s(\d+)e(\d+)`) seRegex = regexp.MustCompile(`(?i)s(\d+)e(\d+)`)
mediaTypeRegex = regexp.MustCompile(`(?i)\b(dvd|bluray|webdl|tvrip)\b`)
) )
type OMDbResponse struct { type OMDbResponse struct {
@@ -49,15 +53,53 @@ type TVMazeEpisode struct {
type Client struct { type Client struct {
omdbAPIKey string omdbAPIKey string
httpClient *http.Client httpClient *http.Client
log *logger.Logger
} }
func NewClient(apiKey string) *Client { func NewClient(apiKey string, log *logger.Logger) *Client {
return &Client{ return &Client{
omdbAPIKey: apiKey, omdbAPIKey: apiKey,
httpClient: &http.Client{Timeout: 10 * time.Second}, httpClient: &http.Client{Timeout: 10 * time.Second},
log: log,
} }
} }
// SetAPIKey updates the OMDb key (e.g. after a settings change). Called from the
// single-threaded encode loop before a fetch, so no locking is needed.
func (c *Client) SetAPIKey(key string) { c.omdbAPIKey = key }
// redactURL strips secret query parameters (e.g. apikey) before logging.
func redactURL(rawURL string) string {
u, err := url.Parse(rawURL)
if err != nil {
return rawURL
}
q := u.Query()
for _, k := range []string{"apikey", "api_key"} {
if q.Has(k) {
q.Set(k, "REDACTED")
}
}
u.RawQuery = q.Encode()
return u.String()
}
func (c *Client) logHTTP(label, rawURL string, status int, body []byte) {
if c.log == nil {
return
}
extra, _ := json.Marshal(struct {
URL string `json:"url"`
Status int `json:"status"`
Body string `json:"body"`
}{
URL: redactURL(rawURL),
Status: status,
Body: string(body),
})
c.log.Debug(label, "", string(extra))
}
func ParseFilename(filename string) (isSeries bool, imdbID, tvmazeID, season, episode string) { func ParseFilename(filename string) (isSeries bool, imdbID, tvmazeID, season, episode string) {
filename = strings.TrimSuffix(filename, ".mkv") filename = strings.TrimSuffix(filename, ".mkv")
@@ -86,10 +128,36 @@ func parseInt(s string) int {
return n return n
} }
func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) { // ParseMediaType looks for a `.dvd.` / `.bluray.` / `.webdl.` / `.tvrip.`
url := fmt.Sprintf("%s?i=%s&apikey=%s", omdbAPIURL, imdbID, c.omdbAPIKey) // token (case-insensitive) in the filename and returns the matching MediaType.
// Returns an empty MediaType if no token is found, so the caller can fall back
// to the pixel-count heuristic.
func ParseMediaType(filename string) types.MediaType {
m := mediaTypeRegex.FindStringSubmatch(strings.TrimSuffix(filename, ".mkv"))
if m == nil {
return ""
}
switch strings.ToLower(m[1]) {
case "dvd":
return types.MediaTypeDVD
case "bluray":
return types.MediaTypeBluRay
case "webdl":
return types.MediaTypeWebDL
case "tvrip":
return types.MediaTypeTVRip
}
return ""
}
resp, err := c.httpClient.Get(url) func (c *Client) FetchMovieMetadata(ctx context.Context, imdbID string) (*types.Metadata, error) {
reqURL := fmt.Sprintf("%s?i=%s&apikey=%s", omdbAPIURL, imdbID, c.omdbAPIKey)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)
if err != nil {
return nil, fmt.Errorf("building OMDb request: %w", err)
}
resp, err := c.httpClient.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("fetching OMDb: %w", err) return nil, fmt.Errorf("fetching OMDb: %w", err)
} }
@@ -100,6 +168,8 @@ func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) {
return nil, fmt.Errorf("reading OMDb body: %w", err) return nil, fmt.Errorf("reading OMDb body: %w", err)
} }
c.logHTTP("OMDb GET", reqURL, resp.StatusCode, body)
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("OMDb HTTP %d: %s", resp.StatusCode, snippet(body)) return nil, fmt.Errorf("OMDb HTTP %d: %s", resp.StatusCode, snippet(body))
} }
@@ -130,7 +200,7 @@ func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) {
}, nil }, nil
} }
func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.Metadata, error) { func (c *Client) FetchSeriesMetadata(ctx context.Context, tvmazeID, season, episode string) (*types.Metadata, error) {
seasonNum, err := strconv.Atoi(season) seasonNum, err := strconv.Atoi(season)
if err != nil { if err != nil {
return nil, fmt.Errorf("parsing season %q: %w", season, err) return nil, fmt.Errorf("parsing season %q: %w", season, err)
@@ -142,13 +212,13 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
epURL := fmt.Sprintf("%s/shows/%s/episodebynumber?season=%d&number=%d", tvmazeAPIURL, tvmazeID, seasonNum, episodeNum) epURL := fmt.Sprintf("%s/shows/%s/episodebynumber?season=%d&number=%d", tvmazeAPIURL, tvmazeID, seasonNum, episodeNum)
var ep TVMazeEpisode var ep TVMazeEpisode
if err := c.fetchTVMazeJSON(epURL, &ep); err != nil { if err := c.fetchTVMazeJSON(ctx, epURL, &ep); err != nil {
return nil, fmt.Errorf("TVmaze episode %s S%dE%d: %w", tvmazeID, seasonNum, episodeNum, err) return nil, fmt.Errorf("TVmaze episode %s S%dE%d: %w", tvmazeID, seasonNum, episodeNum, err)
} }
showURL := fmt.Sprintf("%s/shows/%s", tvmazeAPIURL, tvmazeID) showURL := fmt.Sprintf("%s/shows/%s", tvmazeAPIURL, tvmazeID)
var show TVMazeShowResponse var show TVMazeShowResponse
if err := c.fetchTVMazeJSON(showURL, &show); err != nil { if err := c.fetchTVMazeJSON(ctx, showURL, &show); err != nil {
return nil, fmt.Errorf("TVmaze show %s: %w", tvmazeID, err) return nil, fmt.Errorf("TVmaze show %s: %w", tvmazeID, err)
} }
@@ -169,8 +239,12 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
}, nil }, nil
} }
func (c *Client) fetchTVMazeJSON(url string, v interface{}) error { func (c *Client) fetchTVMazeJSON(ctx context.Context, reqURL string, v interface{}) error {
resp, err := c.httpClient.Get(url) req, err := http.NewRequestWithContext(ctx, http.MethodGet, reqURL, nil)
if err != nil {
return fmt.Errorf("build request: %w", err)
}
resp, err := c.httpClient.Do(req)
if err != nil { if err != nil {
return fmt.Errorf("GET: %w", err) return fmt.Errorf("GET: %w", err)
} }
@@ -181,6 +255,8 @@ func (c *Client) fetchTVMazeJSON(url string, v interface{}) error {
return fmt.Errorf("read body: %w", err) return fmt.Errorf("read body: %w", err)
} }
c.logHTTP("TVmaze GET", reqURL, resp.StatusCode, body)
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return fmt.Errorf("HTTP %d: %s", resp.StatusCode, snippet(body)) return fmt.Errorf("HTTP %d: %s", resp.StatusCode, snippet(body))
} }
internal/mover/mover.go
+69 -13
View File
@@ -1,36 +1,92 @@
package mover package mover
import ( import (
"errors"
"fmt" "fmt"
"io"
"os" "os"
"path/filepath" "path/filepath"
"syscall"
) )
func MoveToOutput(input, outputDir string) error { // safeRename moves src to dst. Refuses to overwrite an existing dst.
if err := os.MkdirAll(outputDir, 0755); err != nil { // Falls back to copy+remove on EXDEV (cross-device) errors.
return fmt.Errorf("creating output dir: %w", err) func safeRename(src, dst string) error {
if _, err := os.Stat(dst); err == nil {
return fmt.Errorf("destination %s already exists", dst)
} else if !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("stat destination %s: %w", dst, err)
} }
filename := filepath.Base(input)
dest := filepath.Join(outputDir, filename) err := os.Rename(src, dst)
return os.Rename(input, dest) if err == nil {
return nil
}
var linkErr *os.LinkError
if !errors.As(err, &linkErr) || !errors.Is(err, syscall.EXDEV) {
return fmt.Errorf("renaming %s to %s: %w", src, dst, err)
}
return copyAndRemove(src, dst)
}
// copyAndRemove implements the cross-device fallback: copy src to dst, fsync,
// then remove src. Uses O_EXCL to belt-and-suspenders against a race.
func copyAndRemove(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("opening source %s: %w", src, err)
}
defer in.Close()
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0644)
if err != nil {
return fmt.Errorf("creating destination %s: %w", dst, err)
}
if _, err := io.Copy(out, in); err != nil {
out.Close()
os.Remove(dst)
return fmt.Errorf("copying %s to %s: %w", src, dst, err)
}
if err := out.Sync(); err != nil {
out.Close()
os.Remove(dst)
return fmt.Errorf("syncing destination %s: %w", dst, err)
}
if err := out.Close(); err != nil {
os.Remove(dst)
return fmt.Errorf("closing destination %s: %w", dst, err)
}
if err := in.Close(); err != nil {
return fmt.Errorf("closing source %s: %w", src, err)
}
if err := os.Remove(src); err != nil {
return fmt.Errorf("removing source %s: %w", src, err)
}
return nil
} }
func MoveToFailed(input, failedDir string) error { func MoveToFailed(input, failedDir string) error {
if err := os.MkdirAll(failedDir, 0755); err != nil { if err := os.MkdirAll(failedDir, 0755); err != nil {
return fmt.Errorf("creating failed dir: %w", err) return fmt.Errorf("creating failed dir: %w", err)
} }
filename := filepath.Base(input) dest := filepath.Join(failedDir, filepath.Base(input))
dest := filepath.Join(failedDir, filename) return safeRename(input, dest)
return os.Rename(input, dest)
} }
func MoveToOriginals(input, originalsDir string) error { func MoveToOriginals(input, originalsDir string) error {
if err := os.MkdirAll(originalsDir, 0755); err != nil { if err := os.MkdirAll(originalsDir, 0755); err != nil {
return fmt.Errorf("creating originals dir: %w", err) return fmt.Errorf("creating originals dir: %w", err)
} }
filename := filepath.Base(input) dest := filepath.Join(originalsDir, filepath.Base(input))
dest := filepath.Join(originalsDir, filename) return safeRename(input, dest)
return os.Rename(input, dest)
} }
func Delete(path string) error { func Delete(path string) error {
@@ -38,5 +94,5 @@ func Delete(path string) error {
} }
func Rename(source, dest string) error { func Rename(source, dest string) error {
return os.Rename(source, dest) return safeRename(source, dest)
} }
internal/server/index.html
+451
View File
@@ -0,0 +1,451 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>av1dae — status</title>
<style>
:root {
--bg:#0d1117; --panel:#151b23; --panel-2:#1a2230; --line:#26303f;
--text:#cdd5df; --muted:#7d8896; --dim:#5a6573;
--amber:#ffb454; --amber-soft:#ffd9a0; --cyan:#56c7e8; --green:#5cc98b; --red:#f0816a;
--mono:"SF Mono",ui-monospace,"JetBrains Mono","Cascadia Code",Menlo,Consolas,monospace;
--sans:"Inter",system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;
}
* { box-sizing:border-box; }
body {
margin:0; background:var(--bg); color:var(--text);
font-family:var(--sans); font-size:16px; line-height:1.5;
-webkit-font-smoothing:antialiased;
padding:clamp(1.25rem,4vw,2.5rem); max-width:920px; margin:0 auto;
}
a { color:var(--cyan); }
code { font-family:var(--mono); }
/* header */
header { display:flex; align-items:center; gap:1rem; margin-bottom:1.75rem; }
.wordmark { font-family:var(--mono); font-weight:600; font-size:1.4rem; letter-spacing:-.02em; }
.wordmark .prompt { color:var(--dim); }
.wordmark .vibe { color:var(--amber); }
.navlink { font-family:var(--mono); font-size:.78rem; color:var(--muted); text-decoration:none; }
.navlink:hover { color:var(--amber); }
.live { margin-left:auto; display:flex; align-items:center; gap:.5rem; font-family:var(--mono); font-size:.74rem; color:var(--muted); text-transform:uppercase; letter-spacing:.1em; }
.dot { width:9px; height:9px; border-radius:50%; background:var(--dim); }
.dot.ok { background:var(--green); box-shadow:0 0 0 0 rgba(92,201,139,.6); animation:pulse 2s infinite; }
.dot.idle { background:var(--amber); }
.dot.err { background:var(--red); }
@keyframes pulse { 0%{box-shadow:0 0 0 0 rgba(92,201,139,.5);} 70%{box-shadow:0 0 0 7px rgba(92,201,139,0);} 100%{box-shadow:0 0 0 0 rgba(92,201,139,0);} }
.card { background:var(--panel); border:1px solid var(--line); border-radius:12px; padding:1.4rem 1.5rem; margin-bottom:1.25rem; }
.eyebrow { font-family:var(--mono); font-size:.7rem; letter-spacing:.16em; text-transform:uppercase; color:var(--dim); margin:0 0 .9rem; }
/* ---- hero ---- */
.job-head { display:flex; align-items:baseline; gap:.6rem; flex-wrap:wrap; margin-bottom:1rem; }
.job-title { font-size:1.2rem; font-weight:600; color:#e7edf4; word-break:break-word; }
.job-title .se { color:var(--amber); font-family:var(--mono); font-size:1rem; margin-right:.45rem; }
.job-title .ep { color:var(--muted); font-weight:400; }
.badge { font-family:var(--mono); font-size:.66rem; letter-spacing:.08em; text-transform:uppercase; padding:.2rem .5rem; border-radius:5px; border:1px solid var(--line); color:var(--muted); margin-left:auto; white-space:nowrap; }
.badge[data-phase="encoding"] { color:var(--amber); border-color:var(--amber); }
.badge[data-phase="probing"], .badge[data-phase="audio"] { color:var(--cyan); border-color:var(--cyan); }
/* ETA is the hero answer */
.eta-line { display:flex; align-items:baseline; gap:.7rem; flex-wrap:wrap; margin-bottom:.9rem; }
.eta-big { font-family:var(--mono); font-size:2.6rem; font-weight:600; line-height:1; color:var(--amber-soft); }
.eta-big.dim { color:var(--dim); font-size:1.6rem; }
.eta-at { font-family:var(--mono); color:var(--muted); font-size:.95rem; }
.eta-at b { color:var(--text); font-weight:500; }
.bar { position:relative; height:12px; border-radius:6px; background:var(--panel-2); overflow:hidden; border:1px solid var(--line); }
.bar > i { display:block; height:100%; width:0; background:linear-gradient(90deg,var(--amber-soft),var(--amber)); border-radius:6px; transition:width .6s ease; }
.bar.indet > i { width:35%; background:linear-gradient(90deg,transparent,var(--cyan),transparent); animation:slide 1.3s linear infinite; transition:none; }
@keyframes slide { 0%{transform:translateX(-120%);} 100%{transform:translateX(340%);} }
.pct { font-family:var(--mono); font-size:.8rem; color:var(--muted); margin-top:.45rem; }
.pct b { color:var(--text); }
.idle-msg { color:var(--muted); }
/* ---- signature: throughput sparkline ---- */
.pulse { margin-top:1.3rem; padding-top:1.2rem; border-top:1px solid var(--line); display:grid; grid-template-columns:1fr auto; gap:1rem 1.5rem; align-items:center; }
.spark-wrap { grid-column:1; min-width:0; }
.spark-head { display:flex; justify-content:space-between; align-items:baseline; font-family:var(--mono); font-size:.66rem; letter-spacing:.1em; text-transform:uppercase; color:var(--dim); margin-bottom:.35rem; }
svg.spark { width:100%; height:54px; display:block; overflow:visible; }
.ref-label { font-family:var(--mono); font-size:.6rem; fill:var(--dim); }
.gauges { grid-column:2; display:flex; gap:1.6rem; }
.gauge { font-family:var(--mono); text-align:right; }
.gauge .v { font-size:1.5rem; line-height:1; }
.gauge .v.warn { color:var(--amber); }
.gauge .v.bad { color:var(--red); }
.gauge .v.good { color:var(--green); }
.gauge .k { font-size:.62rem; letter-spacing:.12em; text-transform:uppercase; color:var(--dim); margin-top:.35rem; }
.gauge .sub { font-size:.6rem; color:var(--dim); margin-top:.15rem; }
.substats { display:flex; flex-wrap:wrap; gap:1.4rem; margin-top:1.2rem; font-family:var(--mono); }
.substats .s { font-size:.8rem; color:var(--muted); }
.substats .s b { color:var(--text); font-weight:500; }
/* stream chips */
.streams { display:flex; flex-wrap:wrap; gap:.5rem; margin-top:1rem; }
.chip { font-family:var(--mono); font-size:.72rem; padding:.2rem .5rem; border-radius:5px; background:var(--panel-2); border:1px solid var(--line); color:var(--text); }
.chip .lab { color:var(--dim); }
.chip .ch { color:var(--amber); }
/* grid */
.grid { display:grid; grid-template-columns:1fr 1fr; gap:1.25rem; }
@media (max-width:680px){ .grid { grid-template-columns:1fr; } .pulse{ grid-template-columns:1fr; } .gauges{ grid-column:1; justify-content:flex-start; } .gauge{ text-align:left; } }
/* queue — collapsed aggregate */
.qsum { display:flex; align-items:baseline; gap:.5rem; flex-wrap:wrap; margin-bottom:.8rem; }
.qsum .qn { font-family:var(--mono); font-size:1.4rem; color:var(--amber); line-height:1; }
.qsum .ql { color:var(--muted); font-size:.85rem; }
.qsum .qeta { font-family:var(--mono); font-size:.8rem; color:var(--amber-soft); margin-left:auto; }
ul.list { list-style:none; margin:0; padding:0; font-family:var(--mono); font-size:.82rem; }
ul.list li { padding:.35rem 0; border-bottom:1px solid var(--line); color:var(--muted); word-break:break-word; }
ul.list li:last-child { border-bottom:0; }
ul.list li .pre { color:var(--dim); }
ul.list li .key { color:var(--text); }
.qmore { font-family:var(--mono); font-size:.74rem; color:var(--dim); padding-top:.5rem; }
/* events */
.ev { display:flex; gap:.6rem; padding:.4rem 0; border-bottom:1px solid var(--line); font-size:.82rem; }
.ev:last-child { border-bottom:0; }
.ev time { font-family:var(--mono); font-size:.7rem; color:var(--dim); white-space:nowrap; padding-top:.1rem; }
.ev .msg { color:var(--text); word-break:break-word; }
.ev.error .msg { color:var(--red); }
.day { text-align:center; font-family:var(--mono); font-size:.64rem; letter-spacing:.12em; text-transform:uppercase; color:var(--dim); margin:.8rem 0 .2rem; }
.day:first-child { margin-top:0; }
@media (prefers-reduced-motion:reduce){ .dot.ok{animation:none;} .bar.indet>i{animation:none; width:100%; opacity:.4;} }
/* control bar */
.controls { display:flex; align-items:center; gap:.6rem; flex-wrap:wrap; margin-bottom:1.25rem; min-height:1px; }
.ctl { font-family:var(--mono); font-size:.82rem; cursor:pointer; border-radius:7px; padding:.45rem .9rem;
background:var(--panel-2); color:var(--text); border:1px solid var(--line); }
.ctl:hover { border-color:var(--amber); color:var(--amber); }
.ctl.primary { background:var(--amber); color:#1a1200; border-color:var(--amber); font-weight:600; }
.ctl.primary:hover { background:var(--amber-soft); color:#1a1200; }
.ctl.small { font-size:.72rem; padding:.25rem .65rem; }
.ctl-state { font-family:var(--mono); font-size:.72rem; color:var(--muted); margin-left:.3rem; text-transform:uppercase; letter-spacing:.08em; }
.ctl-state.paused { color:var(--amber); }
.count { font-family:var(--mono); color:var(--red); }
/* failed list */
#failed .frow { display:flex; align-items:center; gap:.6rem; padding:.35rem 0; border-bottom:1px solid var(--line); font-family:var(--mono); font-size:.82rem; }
#failed .frow:last-child { border-bottom:0; }
#failed .frow .fn { flex:1; color:var(--text); word-break:break-word; }
</style>
</head>
<body>
<header>
<span class="wordmark"><span class="prompt">$ </span>av<span class="vibe">1</span>dae</span>
<a href="/settings" class="navlink">settings</a>
<span class="live"><span class="dot" id="dot"></span><span id="livetext">connecting</span></span>
</header>
<div class="controls" id="controls"></div>
<section class="card" id="job">
<div id="job-body"></div>
</section>
<section class="card" id="failedCard" hidden>
<p class="eyebrow">Failed <span class="count" id="fcount"></span></p>
<div id="failed"></div>
</section>
<div class="grid">
<section class="card">
<p class="eyebrow">Queue</p>
<div id="queue"></div>
</section>
<section class="card">
<p class="eyebrow">Recent events</p>
<div id="events"></div>
</section>
</div>
<script>
const $ = id => document.getElementById(id);
// Metadata, filenames, and event messages all originate from user-supplied
// filenames — escape every dynamic string before it touches innerHTML.
function esc(s) {
return String(s == null ? "" : s).replace(/[&<>"']/g, c =>
({ "&":"&amp;", "<":"&lt;", ">":"&gt;", '"':"&quot;", "'":"&#39;" }[c]));
}
function base(path) { return (path || "").split("/").pop(); }
function pad2(s) { return String(s || "").padStart(2, "0"); }
// "3h 53m" / "53m 20s" / "20s" — remaining time, spelled for the hero.
function fmtDurLong(sec) {
sec = Math.max(0, Math.floor(sec || 0));
if (sec <= 0) return "—";
const h = Math.floor(sec / 3600), m = Math.floor(sec % 3600 / 60), s = sec % 60;
if (h) return `${h}h ${pad2(m)}m`;
if (m) return `${m}m ${pad2(s)}s`;
return `${s}s`;
}
// wall-clock finish time, "21:35"
function clockAt(sec) {
if (!sec || sec <= 0) return "—";
const d = new Date(Date.now() + sec * 1000), p = n => String(n).padStart(2, "0");
return `${p(d.getHours())}:${p(d.getMinutes())}`;
}
function chLabel(n) { return ({1:"mono",2:"2.0",6:"5.1",8:"7.1"})[n] || (n ? n+"ch" : ""); }
// ---- signature: throughput sparkline, fed by the 1 s poll ----
let speedHist = [], histFile = null;
function pushSpeed(c) {
if (!c || c.file !== histFile) { speedHist = []; histFile = c ? c.file : null; }
if (c && c.phase === "encoding" && c.speed > 0) {
speedHist.push(c.speed);
if (speedHist.length > 90) speedHist.shift(); // ~90 s window
}
}
// Trace of recent speed against a fixed 1.0× realtime reference. The reference
// stays fixed on purpose — autoscaling would hide how far below realtime we run.
function sparkSVG(hist, ref) {
const W = 300, H = 54;
if (hist.length < 2)
return `<svg class="spark" viewBox="0 0 ${W} ${H}" preserveAspectRatio="none" aria-hidden="true"></svg>`;
const max = Math.max(ref * 1.3, ...hist);
const y = v => H - (v / max) * H;
const pts = hist.map((v, i) => `${(i/(hist.length-1)*W).toFixed(1)},${y(v).toFixed(1)}`).join(" ");
const refY = y(ref).toFixed(1);
const below = hist[hist.length-1] < ref;
const col = below ? "var(--red)" : "var(--green)";
return `<svg class="spark" viewBox="0 0 ${W} ${H}" preserveAspectRatio="none" aria-hidden="true">
<line x1="0" y1="${refY}" x2="${W}" y2="${refY}" stroke="var(--dim)" stroke-width="1" stroke-dasharray="3 3"/>
<text class="ref-label" x="2" y="${Math.max(8, refY-3)}">${ref.toFixed(1)}×</text>
<polyline fill="none" stroke="${col}" stroke-width="2" stroke-linejoin="round" points="${pts}"/>
<circle cx="${W}" cy="${y(hist[hist.length-1]).toFixed(1)}" r="2.5" fill="${col}"/>
</svg>`;
}
function trendLabel(hist) {
if (hist.length < 4) return ["—", "var(--dim)"];
const last = hist[hist.length-1], prev = hist[hist.length-3];
if (last < prev - 0.01) return ["slowing", "var(--red)"];
if (last > prev + 0.01) return ["recovering", "var(--green)"];
return ["holding", "var(--dim)"];
}
function speedClass(v) { return v <= 0 ? "" : v < 0.5 ? "bad" : v < 1 ? "warn" : "good"; }
function jobTitleHTML(c) {
const m = c.meta;
if (m && m.is_series && m.collection) {
const se = (m.season && m.episode) ? `S${pad2(m.season)}E${pad2(m.episode)}` : "";
const ep = (m.title && m.title !== "Unknown") ? ` — “${esc(m.title)}”` : "";
return `${se ? `<span class="se">${se}</span>` : ""}${esc(m.collection)}<span class="ep">${ep}</span>`;
}
if (m && m.title && m.title !== "Unknown") return esc(m.title);
return esc(base(c.file));
}
function streamsHTML(streams) {
if (!streams || !streams.length) return "";
const chips = streams.filter(s => s.kind === "audio" || s.kind === "subtitle").map(s => {
const lab = s.kind === "audio" ? "audio" : "subs";
const ch = s.kind === "audio" ? chLabel(s.channels) : "";
const codec = s.kind === "audio" ? esc(s.codec || "") : "";
return `<span class="chip"><span class="lab">${lab}</span> ${esc(s.language || "und")}` +
`${ch ? ` <span class="ch">${ch}</span>` : ""}${codec ? ` ${codec}` : ""}</span>`;
}).join("");
return chips ? `<div class="streams">${chips}</div>` : "";
}
function substatsHTML(c) {
const m = c.meta, parts = [];
if (c.elapsed_sec) parts.push(`elapsed <b>${fmtDurLong(c.elapsed_sec)}</b>`);
if (m && m.media_type) parts.push(`source <b>${esc(m.media_type)}</b>`);
if (m && m.date_released) parts.push(`${m.is_series ? "aired" : "released"} <b>${esc(m.date_released)}</b>`);
return parts.length ? `<div class="substats">${parts.map(p => `<div class="s">${p}</div>`).join("")}</div>` : "";
}
function renderJob(c) {
const body = $("job-body");
if (!c || c.phase === "idle" || !c.file) {
body.innerHTML = `<div class="eta-line"><span class="eta-big dim">Idle</span></div>
<p class="idle-msg">Waiting for files in <code>input/</code>.</p>`;
return;
}
const encoding = c.phase === "encoding" && c.percent > 0;
// Preparing phases (probing/audio) have no progress yet — indeterminate bar.
if (!encoding) {
body.innerHTML = `
<div class="job-head">
<span class="job-title">${jobTitleHTML(c)}</span>
<span class="badge" data-phase="${esc(c.phase)}">${esc(c.phase)}</span>
</div>
<div class="eta-line"><span class="eta-big dim">preparing…</span></div>
<div class="bar indet" role="progressbar" aria-valuetext="preparing"><i></i></div>
${substatsHTML(c)}${streamsHTML(c.streams)}`;
return;
}
const sp = c.speed || 0, [trend, trendCol] = trendLabel(speedHist);
body.innerHTML = `
<div class="job-head">
<span class="job-title">${jobTitleHTML(c)}</span>
<span class="badge" data-phase="encoding">encoding</span>
</div>
<div class="eta-line">
<span class="eta-big">${fmtDurLong(c.eta_sec)}</span>
<span class="eta-at">left · done ~<b>${clockAt(c.eta_sec)}</b></span>
</div>
<div class="bar" role="progressbar" aria-valuenow="${Math.round(c.percent)}" aria-valuemin="0" aria-valuemax="100"><i style="width:${c.percent}%"></i></div>
<div class="pct"><b>${c.percent.toFixed(1)}%</b> encoded</div>
<div class="pulse">
<div class="spark-wrap">
<div class="spark-head"><span>throughput · last 90s</span><span style="color:${trendCol}">${trend}</span></div>
${sparkSVG(speedHist, 1.0)}
</div>
<div class="gauges">
<div class="gauge">
<div class="v ${speedClass(sp)}">${sp.toFixed(2)}×</div>
<div class="k">speed</div>
<div class="sub">${sp > 0 && sp < 1 ? "below realtime" : sp >= 1 ? "realtime+" : ""}</div>
</div>
<div class="gauge">
<div class="v">${(c.fps || 0).toFixed(1)}</div>
<div class="k">fps</div>
</div>
</div>
</div>
${substatsHTML(c)}${streamsHTML(c.streams)}`;
}
// Longest common prefix, so a queue of "Medium S05E02/03/04…" shows the
// shared part once and the differing tail per row.
function commonPrefix(arr) {
if (!arr.length) return "";
let p = arr[0];
for (const s of arr) { while (!s.startsWith(p)) p = p.slice(0, -1); if (!p) break; }
return p;
}
function renderQueue(d) {
const queue = d.queue, current = d.current;
const cur = base(current && current.file);
const pending = (queue || []).filter(f => f !== cur);
const el = $("queue");
if (!pending.length) {
el.innerHTML = `<div class="qsum"><span class="ql">Nothing waiting</span></div>`;
return;
}
const pre = pending.length > 1 ? commonPrefix(pending) : "";
const eta = d.queue_eta_sec > 0
? `<span class="qeta">~${fmtDurLong(d.queue_eta_sec)}${d.queue_eta_partial ? "+" : ""} to clear</span>`
: "";
const head = `<div class="qsum"><span class="qn">${pending.length}</span><span class="ql">${pending.length === 1 ? "file" : "files"} waiting</span>${eta}</div>`;
const rows = pending.slice(0, 3).map(f => {
const tail = pre ? f.slice(pre.length) : f;
return `<li>${pre ? `<span class="pre">${esc(pre)}</span>` : ""}<span class="key">${esc(tail)}</span></li>`;
}).join("");
const more = pending.length > 3 ? `<div class="qmore">+ ${pending.length - 3} more</div>` : "";
el.innerHTML = head + `<ul class="list">${rows}</ul>` + more;
}
// WhatsApp-style: a date divider whenever the day changes, time-only per row.
function dayLabel(dt) {
const t = new Date(), a = new Date(t.getFullYear(), t.getMonth(), t.getDate());
const b = new Date(dt.getFullYear(), dt.getMonth(), dt.getDate());
const diff = Math.round((a - b) / 86400000);
if (diff === 0) return "Today";
if (diff === 1) return "Yesterday";
return dt.toLocaleDateString(undefined, { year:"numeric", month:"short", day:"numeric" });
}
function hms(dt) {
const p = n => String(n).padStart(2, "0");
return `${p(dt.getHours())}:${p(dt.getMinutes())}:${p(dt.getSeconds())}`;
}
function renderEvents(recent) {
if (!recent || !recent.length) {
$("events").innerHTML = `<div class="ev"><span class="msg" style="color:var(--dim)">No events yet</span></div>`;
return;
}
let html = "", lastDay = null;
for (const e of recent) {
const dt = new Date(e.ts);
const day = dayLabel(dt);
if (day !== lastDay) { html += `<div class="day">${esc(day)}</div>`; lastDay = day; }
const cls = e.level === "error" ? "ev error" : "ev";
html += `<div class="${cls}"><time>${hms(dt)}</time><span class="msg">${esc(e.message)}</span></div>`;
}
$("events").innerHTML = html;
}
function setLive(state) {
const dot = $("dot"), txt = $("livetext");
dot.className = "dot " + state;
txt.textContent = state === "ok" ? "live" : state === "idle" ? "idle" : state === "err" ? "offline" : "connecting";
}
// ---- runtime controls: start/hold, pause/resume, retry ----
async function post(path) {
try { await fetch(path, { method: "POST" }); } catch (e) {}
poll(); // reflect the new state immediately
}
function renderControls(d) {
const c = d.current || {};
const encoding = c.file && c.phase !== "idle";
let html = d.running
? `<button class="ctl" data-act="/api/hold">⏸ Hold queue</button>`
: `<button class="ctl primary" data-act="/api/start">▶ Start queue</button>`;
if (encoding) {
html += c.paused
? `<button class="ctl primary" data-act="/api/resume">▶ Resume encode</button>`
: `<button class="ctl" data-act="/api/pause">⏸ Pause encode</button>`;
}
const state = !d.running ? "held" : c.paused ? "running · encode paused" : "running";
html += `<span class="ctl-state${c.paused ? " paused" : ""}">${state}</span>`;
$("controls").innerHTML = html;
}
function renderFailed(failed) {
const card = $("failedCard");
if (!failed || !failed.length) { card.hidden = true; $("failed").innerHTML = ""; return; }
card.hidden = false;
$("fcount").textContent = "(" + failed.length + ")";
$("failed").innerHTML = failed.map(f =>
`<div class="frow"><span class="fn">${esc(f)}</span><button class="ctl small" data-retry="${esc(f)}">retry</button></div>`
).join("");
}
$("controls").addEventListener("click", e => {
const b = e.target.closest("[data-act]");
if (b) post(b.getAttribute("data-act"));
});
$("failed").addEventListener("click", e => {
const b = e.target.closest("[data-retry]");
if (b) post("/api/retry?file=" + encodeURIComponent(b.getAttribute("data-retry")));
});
async function poll() {
try {
const r = await fetch("status", { cache: "no-store" });
if (!r.ok) throw new Error(r.status);
const d = await r.json();
pushSpeed(d.current);
renderControls(d);
renderJob(d.current);
renderFailed(d.failed);
renderQueue(d);
renderEvents(d.recent);
const active = d.current && d.current.file && d.current.phase !== "idle";
setLive(d.current && d.current.paused ? "idle" : active ? "ok" : "idle");
} catch (e) {
setLive("err");
} finally {
setTimeout(poll, 1000);
}
}
poll();
</script>
</body>
</html>
internal/server/server.go
+300
View File
@@ -0,0 +1,300 @@
// Package server exposes a read-only HTTP status endpoint for the daemon:
// the live encode progress, the pending input queue, and recent log events.
package server
import (
_ "embed"
"encoding/json"
"net/http"
"os"
"path/filepath"
"sync"
"time"
"av1dae/internal/logger"
"av1dae/internal/settings"
"av1dae/internal/status"
"av1dae/internal/watcher"
)
//go:embed index.html
var indexHTML []byte
//go:embed settings.html
var settingsHTML []byte
// Controls is the runtime-control surface the UI drives, wired in main from the
// watcher (start/hold gate), encoder (pause/resume), and mover (retry).
type Controls struct {
Running func() bool
SetRunning func(bool)
Pause func() error
Resume func() error
RetryFailed func(name string) error
}
type Server struct {
tracker *status.Tracker
log *logger.Logger
store *settings.Store
controls Controls
durCache *durationCache
inputDir string
failedDir string
}
func New(tracker *status.Tracker, log *logger.Logger, store *settings.Store, controls Controls, probeDuration func(string) (float64, error), inputDir, failedDir string) *Server {
return &Server{
tracker: tracker,
log: log,
store: store,
controls: controls,
durCache: newDurationCache(probeDuration),
inputDir: inputDir,
failedDir: failedDir,
}
}
type durEntry struct {
mtime time.Time
sec float64
}
// durationCache memoizes source durations (keyed by path+mtime) so the queue
// ETA doesn't re-probe every file on every 1s poll. A miss kicks a background
// ffprobe and resolves on a later poll; the file shows as "estimating" until.
type durationCache struct {
probe func(string) (float64, error)
mu sync.Mutex
m map[string]durEntry
inflight map[string]bool
}
func newDurationCache(probe func(string) (float64, error)) *durationCache {
return &durationCache{probe: probe, m: map[string]durEntry{}, inflight: map[string]bool{}}
}
func (c *durationCache) Get(path string, mtime time.Time) (float64, bool) {
c.mu.Lock()
if e, ok := c.m[path]; ok && e.mtime.Equal(mtime) {
c.mu.Unlock()
return e.sec, true
}
if c.inflight[path] {
c.mu.Unlock()
return 0, false
}
c.inflight[path] = true
c.mu.Unlock()
go func() {
sec, err := c.probe(path)
c.mu.Lock()
delete(c.inflight, path)
if err == nil {
c.m[path] = durEntry{mtime, sec}
}
c.mu.Unlock()
}()
return 0, false
}
// Retain drops cached entries for paths no longer present, bounding the map on
// a long-running daemon.
func (c *durationCache) Retain(paths []string) {
keep := make(map[string]bool, len(paths))
for _, p := range paths {
keep[p] = true
}
c.mu.Lock()
for p := range c.m {
if !keep[p] {
delete(c.m, p)
}
}
c.mu.Unlock()
}
// queueETASeconds estimates wall-clock time to clear the queue: the current
// job's remaining time plus each pending file's duration / current speed.
// 0 when there's no speed to extrapolate from (idle/held).
func queueETASeconds(currentETASec int, sumQueuedSec, speed float64) int {
if speed <= 0 {
return 0
}
return currentETASec + int(sumQueuedSec/speed)
}
// Handler returns the mux for all status routes. Phase 3 adds "/" (the HTML
// dashboard) to this same mux.
func (s *Server) Handler() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/status", s.handleStatus)
mux.HandleFunc("/settings", s.handleSettingsPage)
mux.HandleFunc("/api/settings", s.handleAPISettings)
mux.HandleFunc("/api/start", s.gateHandler(true))
mux.HandleFunc("/api/hold", s.gateHandler(false))
mux.HandleFunc("/api/pause", s.actionHandler(func() error { return s.controls.Pause() }, "Encode paused"))
mux.HandleFunc("/api/resume", s.actionHandler(func() error { return s.controls.Resume() }, "Encode resumed"))
mux.HandleFunc("/api/retry", s.handleRetry)
mux.HandleFunc("/", s.handleIndex)
return mux
}
// gateHandler flips the start/hold gate. POST only.
func (s *Server) gateHandler(run bool) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
methodNotAllowed(w)
return
}
s.controls.SetRunning(run)
if run {
s.log.Info("Queue started via web UI")
} else {
s.log.Info("Queue held via web UI")
}
w.WriteHeader(http.StatusNoContent)
}
}
// actionHandler wraps a no-arg control action (pause/resume). POST only.
func (s *Server) actionHandler(fn func() error, logMsg string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
methodNotAllowed(w)
return
}
if err := fn(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
s.log.Info(logMsg + " via web UI")
w.WriteHeader(http.StatusNoContent)
}
}
// handleRetry moves a named file from failed/ back to input/. POST ?file=NAME.
func (s *Server) handleRetry(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
methodNotAllowed(w)
return
}
name := r.URL.Query().Get("file")
if name == "" {
http.Error(w, "missing file", http.StatusBadRequest)
return
}
if err := s.controls.RetryFailed(name); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
s.log.Info("Retry requested via web UI: " + name)
w.WriteHeader(http.StatusNoContent)
}
func methodNotAllowed(w http.ResponseWriter) {
w.Header().Set("Allow", "POST")
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
}
func (s *Server) handleSettingsPage(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write(settingsHTML)
}
// handleAPISettings serves the current settings (GET) and saves new ones (PUT).
// Validation lives in settings.Store.Set; a bad payload returns 400.
func (s *Server) handleAPISettings(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(s.store.Get())
case http.MethodPut:
var in settings.Settings
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
http.Error(w, "invalid JSON: "+err.Error(), http.StatusBadRequest)
return
}
if err := s.store.Set(in); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
s.log.SetRetention(in.LogRetentionDays)
s.log.Info("Settings updated via web UI")
w.WriteHeader(http.StatusNoContent)
default:
w.Header().Set("Allow", "GET, PUT")
http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
}
}
func (s *Server) handleIndex(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write(indexHTML)
}
type statusResponse struct {
Running bool `json:"running"`
Current status.Snapshot `json:"current"`
Queue []string `json:"queue"`
Failed []string `json:"failed"`
QueueETASec int `json:"queue_eta_sec"`
QueueETAPartial bool `json:"queue_eta_partial"`
Recent []logger.LogEntry `json:"recent"`
}
func (s *Server) handleStatus(w http.ResponseWriter, r *http.Request) {
recent, err := s.log.RecentLogs(50)
if err != nil {
http.Error(w, "reading logs", http.StatusInternalServerError)
return
}
snap := s.tracker.Snapshot()
files := watcher.InputFiles(s.inputDir)
// Sum durations of pending files (excluding the current job, whose remaining
// time is already in snap.ETASec). Unprobed files mark the estimate partial.
var sumQueued float64
partial := false
for _, f := range files {
if f == snap.File {
continue
}
info, statErr := os.Stat(f)
if statErr != nil {
partial = true
continue
}
if sec, ok := s.durCache.Get(f, info.ModTime()); ok {
sumQueued += sec
} else {
partial = true
}
}
s.durCache.Retain(files)
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(statusResponse{
Running: s.controls.Running(),
Current: snap,
Queue: baseNames(files),
Failed: baseNames(watcher.InputFiles(s.failedDir)),
QueueETASec: queueETASeconds(snap.ETASec, sumQueued, snap.Speed),
QueueETAPartial: partial,
Recent: recent,
})
}
func baseNames(paths []string) []string {
out := []string{}
for _, p := range paths {
out = append(out, filepath.Base(p))
}
return out
}
internal/server/server_test.go
+62
View File
@@ -0,0 +1,62 @@
package server
import (
"testing"
"time"
)
func TestQueueETASeconds(t *testing.T) {
// No speed → can't extrapolate.
if got := queueETASeconds(100, 3600, 0); got != 0 {
t.Errorf("speed 0: got %d, want 0", got)
}
// current remaining 600s + 1800s of queued source at 0.5x (=3600s) = 4200s.
if got := queueETASeconds(600, 1800, 0.5); got != 4200 {
t.Errorf("got %d, want 4200", got)
}
// At 2x, 1800s of source encodes in 900s; + 600 remaining = 1500.
if got := queueETASeconds(600, 1800, 2); got != 1500 {
t.Errorf("got %d, want 1500", got)
}
// Empty queue → just the current job's remaining.
if got := queueETASeconds(600, 0, 1); got != 600 {
t.Errorf("got %d, want 600", got)
}
}
// waitCached polls Get (the probe resolves on a background goroutine).
func waitCached(c *durationCache, p string, mt time.Time) (float64, bool) {
for i := 0; i < 200; i++ {
if sec, ok := c.Get(p, mt); ok {
return sec, true
}
time.Sleep(2 * time.Millisecond)
}
return 0, false
}
func TestDurationCache(t *testing.T) {
c := newDurationCache(func(p string) (float64, error) { return 42, nil })
mt := time.Unix(1000, 0)
if _, ok := c.Get("/a.mkv", mt); ok {
t.Fatal("first Get should miss")
}
if sec, ok := waitCached(c, "/a.mkv", mt); !ok || sec != 42 {
t.Fatalf("after probe: got %v ok=%v, want 42 true", sec, ok)
}
// A changed mtime invalidates the entry.
if _, ok := c.Get("/a.mkv", time.Unix(2000, 0)); ok {
t.Error("changed mtime should miss")
}
if _, ok := waitCached(c, "/a.mkv", time.Unix(2000, 0)); !ok {
t.Fatal("re-probe under new mtime should cache")
}
// Retain drops entries for paths not in the keep set.
c.Retain([]string{"/b.mkv"})
if _, ok := c.Get("/a.mkv", time.Unix(2000, 0)); ok {
t.Error("Retain should have dropped /a.mkv")
}
}
internal/server/settings.html
+159
View File
@@ -0,0 +1,159 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>av1dae — settings</title>
<style>
:root {
--bg:#0d1117; --panel:#151b23; --panel-2:#1a2230; --line:#26303f;
--text:#cdd5df; --muted:#7d8896; --dim:#5a6573;
--amber:#ffb454; --amber-soft:#ffd9a0; --cyan:#56c7e8; --green:#5cc98b; --red:#f0816a;
--mono:"SF Mono",ui-monospace,"JetBrains Mono","Cascadia Code",Menlo,Consolas,monospace;
--sans:"Inter",system-ui,-apple-system,"Segoe UI",Roboto,sans-serif;
}
* { box-sizing:border-box; }
body {
margin:0 auto; max-width:720px; background:var(--bg); color:var(--text);
font-family:var(--sans); font-size:16px; line-height:1.5;
-webkit-font-smoothing:antialiased; padding:clamp(1.25rem,4vw,2.5rem);
}
a { color:var(--cyan); text-decoration:none; }
a:hover { text-decoration:underline; }
header { display:flex; align-items:baseline; gap:1rem; margin-bottom:1.75rem; }
.wordmark { font-family:var(--mono); font-weight:600; font-size:1.4rem; letter-spacing:-.02em; }
.wordmark .prompt { color:var(--dim); }
.wordmark .vibe { color:var(--amber); }
.crumb { color:var(--muted); font-family:var(--mono); font-size:.85rem; }
.back { margin-left:auto; font-family:var(--mono); font-size:.8rem; }
.card { background:var(--panel); border:1px solid var(--line); border-radius:12px; padding:1.3rem 1.5rem; margin-bottom:1.25rem; }
.eyebrow { font-family:var(--mono); font-size:.7rem; letter-spacing:.16em; text-transform:uppercase; color:var(--amber); margin:0 0 1rem; }
.hint { color:var(--muted); font-size:.85rem; margin:.1rem 0 0; }
.row { display:flex; align-items:center; gap:1rem; padding:.5rem 0; flex-wrap:wrap; }
.row label { flex:1; min-width:12rem; }
.row label .sub { display:block; color:var(--dim); font-size:.78rem; }
input[type=number], input[type=text] {
background:var(--panel-2); border:1px solid var(--line); color:var(--text);
border-radius:6px; padding:.4rem .55rem; font-family:var(--mono); font-size:.9rem; width:6rem;
}
input[type=text] { width:100%; max-width:22rem; }
input:focus { outline:2px solid var(--amber); outline-offset:1px; }
/* profiles grid */
table.prof { width:100%; border-collapse:collapse; }
table.prof th, table.prof td { text-align:left; padding:.4rem .5rem; }
table.prof th { font-family:var(--mono); font-size:.66rem; letter-spacing:.08em; text-transform:uppercase; color:var(--dim); font-weight:500; }
table.prof td.name { font-family:var(--mono); color:var(--amber-soft); }
.toggle { display:flex; align-items:center; gap:.6rem; }
.toggle input { width:1.1rem; height:1.1rem; accent-color:var(--amber); }
.actions { display:flex; align-items:center; gap:1rem; margin-top:.5rem; }
button {
font-family:var(--mono); font-size:.85rem; background:var(--amber); color:#1a1200;
border:0; border-radius:7px; padding:.55rem 1.1rem; cursor:pointer; font-weight:600;
}
button:hover { background:var(--amber-soft); }
button:disabled { opacity:.5; cursor:default; }
#msg { font-family:var(--mono); font-size:.82rem; }
#msg.ok { color:var(--green); }
#msg.err { color:var(--red); }
</style>
</head>
<body>
<header>
<span class="wordmark"><span class="prompt">$ </span>av<span class="vibe">1</span>dae</span>
<span class="crumb">/ settings</span>
<a class="back" href="/">← dashboard</a>
</header>
<form id="form">
<section class="card">
<p class="eyebrow">Encoding profiles</p>
<table class="prof">
<thead><tr><th>Source</th><th>CRF (0–63)</th><th>Preset (0–13)</th></tr></thead>
<tbody>
<tr><td class="name">dvd</td><td><input type="number" id="dvd_crf" min="0" max="63"></td><td><input type="number" id="dvd_preset" min="0" max="13"></td></tr>
<tr><td class="name">bluray</td><td><input type="number" id="bluray_crf" min="0" max="63"></td><td><input type="number" id="bluray_preset" min="0" max="13"></td></tr>
<tr><td class="name">webdl</td><td><input type="number" id="webdl_crf" min="0" max="63"></td><td><input type="number" id="webdl_preset" min="0" max="13"></td></tr>
<tr><td class="name">tvrip</td><td><input type="number" id="tvrip_crf" min="0" max="63"></td><td><input type="number" id="tvrip_preset" min="0" max="13"></td></tr>
</tbody>
</table>
<div class="row" style="margin-top:.6rem">
<label>Thread cap (lp)<span class="sub">SVT-AV1 logical processors · 0 = use all cores</span></label>
<input type="number" id="lp" min="0">
</div>
</section>
<section class="card">
<p class="eyebrow">Metadata & housekeeping</p>
<div class="row">
<label>OMDb API key<span class="sub">Required for movie metadata; TVmaze needs none</span></label>
<input type="text" id="omdb_api_key" autocomplete="off" spellcheck="false">
</div>
<div class="row">
<label>Log retention (days)<span class="sub">Rows older than this are purged from logs.db</span></label>
<input type="number" id="log_retention_days" min="1">
</div>
<div class="row">
<label class="toggle"><input type="checkbox" id="delete_originals"> Delete originals after a successful encode<span class="sub">Off = move them to originals/</span></label>
</div>
</section>
<div class="actions">
<button type="submit" id="save">Save settings</button>
<span id="msg"></span>
</div>
</form>
<script>
const $ = id => document.getElementById(id);
const profiles = ["dvd", "bluray", "webdl", "tvrip"];
function fill(s) {
for (const p of profiles) { $(p + "_crf").value = s[p].crf; $(p + "_preset").value = s[p].preset; }
$("lp").value = s.lp;
$("omdb_api_key").value = s.omdb_api_key || "";
$("log_retention_days").value = s.log_retention_days;
$("delete_originals").checked = !!s.delete_originals;
}
function collect() {
const s = { lp: +$("lp").value, omdb_api_key: $("omdb_api_key").value,
log_retention_days: +$("log_retention_days").value, delete_originals: $("delete_originals").checked };
for (const p of profiles) s[p] = { crf: +$(p + "_crf").value, preset: +$(p + "_preset").value };
return s;
}
function msg(text, cls) { const m = $("msg"); m.textContent = text; m.className = cls || ""; }
async function load() {
try {
const r = await fetch("/api/settings", { cache: "no-store" });
if (!r.ok) throw new Error(r.status);
fill(await r.json());
} catch (e) { msg("Failed to load settings", "err"); }
}
$("form").addEventListener("submit", async e => {
e.preventDefault();
$("save").disabled = true; msg("Saving…");
try {
const r = await fetch("/api/settings", {
method: "PUT", headers: { "Content-Type": "application/json" }, body: JSON.stringify(collect())
});
if (r.ok) msg("Saved — applies to the next encode.", "ok");
else msg("Rejected: " + (await r.text()).trim(), "err");
} catch (e) { msg("Network error", "err"); }
finally { $("save").disabled = false; }
});
load();
</script>
</body>
</html>
internal/settings/settings.go
+130
View File
@@ -0,0 +1,130 @@
// Package settings holds the live-editable configuration: values that can be
// changed from the web UI and persisted, without a config-file edit + restart.
// config.yaml seeds the store on first run; afterwards the DB is the source of
// truth for these values (paths.* and http_addr stay config-only).
package settings
import (
"database/sql"
"encoding/json"
"fmt"
"sync"
"av1dae/pkg/types"
)
// Profile is a per-source-type SVT-AV1 quality pair.
type Profile struct {
CRF int `json:"crf"`
Preset int `json:"preset"`
}
// Settings is the full set of live-editable values.
type Settings struct {
DVD Profile `json:"dvd"`
Bluray Profile `json:"bluray"`
WebDL Profile `json:"webdl"`
TVRip Profile `json:"tvrip"`
LP int `json:"lp"`
OMDBAPIKey string `json:"omdb_api_key"`
LogRetentionDays int `json:"log_retention_days"`
DeleteOriginals bool `json:"delete_originals"`
}
// ProfileFor returns the encoding profile for a media type.
func (s Settings) ProfileFor(mt types.MediaType) Profile {
switch mt {
case types.MediaTypeBluRay:
return s.Bluray
case types.MediaTypeWebDL:
return s.WebDL
case types.MediaTypeTVRip:
return s.TVRip
default:
return s.DVD
}
}
// Validate guards the trust boundary: a bad value from the PUT handler could
// silently break every subsequent encode. Ranges follow SVT-AV1's limits.
func (s Settings) Validate() error {
for name, p := range map[string]Profile{"dvd": s.DVD, "bluray": s.Bluray, "webdl": s.WebDL, "tvrip": s.TVRip} {
if p.CRF < 0 || p.CRF > 63 {
return fmt.Errorf("%s crf %d out of range 0-63", name, p.CRF)
}
if p.Preset < 0 || p.Preset > 13 {
return fmt.Errorf("%s preset %d out of range 0-13", name, p.Preset)
}
}
if s.LP < 0 {
return fmt.Errorf("lp %d must be >= 0", s.LP)
}
if s.LogRetentionDays < 1 {
return fmt.Errorf("log_retention_days %d must be >= 1", s.LogRetentionDays)
}
return nil
}
// Store is the persisted, concurrency-safe settings holder. The encode loop
// reads via Get; the HTTP handler writes via Set.
type Store struct {
mu sync.RWMutex
db *sql.DB
cur Settings
}
// New creates the settings table if needed, loads the persisted row, or seeds
// it from `seed` (the config-derived values) on first run.
func New(db *sql.DB, seed Settings) (*Store, error) {
if _, err := db.Exec(`CREATE TABLE IF NOT EXISTS settings (id INTEGER PRIMARY KEY CHECK (id = 1), data TEXT NOT NULL)`); err != nil {
return nil, fmt.Errorf("settings schema: %w", err)
}
s := &Store{db: db, cur: seed}
var data string
switch err := db.QueryRow(`SELECT data FROM settings WHERE id = 1`).Scan(&data); err {
case sql.ErrNoRows:
if err := s.persist(seed); err != nil { // first run — seed from config
return nil, err
}
case nil:
var loaded Settings
if err := json.Unmarshal([]byte(data), &loaded); err != nil {
return nil, fmt.Errorf("decoding settings: %w", err)
}
s.cur = loaded
default:
return nil, fmt.Errorf("loading settings: %w", err)
}
return s, nil
}
// Get returns a copy of the current settings.
func (s *Store) Get() Settings {
s.mu.RLock()
defer s.mu.RUnlock()
return s.cur
}
// Set validates, persists, and swaps in the new settings.
func (s *Store) Set(n Settings) error {
if err := n.Validate(); err != nil {
return err
}
s.mu.Lock()
defer s.mu.Unlock()
if err := s.persist(n); err != nil {
return err
}
s.cur = n
return nil
}
func (s *Store) persist(n Settings) error {
b, err := json.Marshal(n)
if err != nil {
return err
}
_, err = s.db.Exec(`INSERT INTO settings (id, data) VALUES (1, ?) ON CONFLICT(id) DO UPDATE SET data = excluded.data`, string(b))
return err
}
internal/settings/settings_test.go
+46
View File
@@ -0,0 +1,46 @@
package settings
import (
"testing"
"av1dae/pkg/types"
)
func valid() Settings {
return Settings{
DVD: Profile{30, 2}, Bluray: Profile{29, 3}, WebDL: Profile{30, 3}, TVRip: Profile{32, 2},
LP: 0, LogRetentionDays: 7,
}
}
func TestValidate(t *testing.T) {
if err := valid().Validate(); err != nil {
t.Fatalf("valid settings rejected: %v", err)
}
bad := func(mut func(*Settings)) Settings { s := valid(); mut(&s); return s }
cases := map[string]Settings{
"crf too high": bad(func(s *Settings) { s.DVD.CRF = 64 }),
"crf negative": bad(func(s *Settings) { s.Bluray.CRF = -1 }),
"preset too high": bad(func(s *Settings) { s.WebDL.Preset = 14 }),
"lp negative": bad(func(s *Settings) { s.LP = -1 }),
"retention zero": bad(func(s *Settings) { s.LogRetentionDays = 0 }),
}
for name, s := range cases {
if err := s.Validate(); err == nil {
t.Errorf("%s: expected validation error, got nil", name)
}
}
}
func TestProfileFor(t *testing.T) {
s := valid()
if s.ProfileFor(types.MediaTypeBluRay) != s.Bluray {
t.Error("bluray profile mismatch")
}
if s.ProfileFor(types.MediaTypeWebDL) != s.WebDL {
t.Error("webdl profile mismatch")
}
if s.ProfileFor(types.MediaType("anything-else")) != s.DVD {
t.Error("unknown media type should fall back to DVD")
}
}
internal/status/status.go
+280
View File
@@ -0,0 +1,280 @@
// Package status tracks the live state of the single in-flight encode job.
// The watcher processes one file at a time, so one mutex-guarded value is
// enough — no per-job table, no concurrency design.
package status
import (
"bufio"
"io"
"strconv"
"strings"
"sync"
"time"
)
// Phase labels for the current job.
const (
PhaseIdle = "idle"
PhaseProbing = "probing"
PhaseAudio = "audio"
PhaseEncoding = "encoding"
)
// JobMeta is the fetched metadata for the active job, shaped for display.
type JobMeta struct {
IsSeries bool `json:"is_series"`
Title string `json:"title"` // movie title or episode title
Collection string `json:"collection,omitempty"` // show name (series)
Season string `json:"season,omitempty"`
Episode string `json:"episode,omitempty"`
DateReleased string `json:"date_released,omitempty"` // release date / airdate
MediaType string `json:"media_type,omitempty"`
}
// Stream is one source audio or subtitle stream, for display.
type Stream struct {
Kind string `json:"kind"` // "audio" | "subtitle"
Language string `json:"language,omitempty"`
Codec string `json:"codec,omitempty"`
Channels int `json:"channels,omitempty"` // audio only
Title string `json:"title,omitempty"`
}
// Tracker holds live progress for the active job. Safe for concurrent use:
// the encode goroutine writes, HTTP/log readers call Snapshot.
type Tracker struct {
mu sync.RWMutex
file string
phase string
meta *JobMeta
streams []Stream
totalSec float64 // source duration; 0 until known
outTime float64 // encoded position in seconds
fps float64
speed float64
startedAt time.Time
paused bool
pausedAt time.Time // when the current pause began
pausedTotal time.Duration // accumulated paused time this job
}
// Snapshot is an immutable view of the tracker for readers.
type Snapshot struct {
File string `json:"file"`
Phase string `json:"phase"`
Meta *JobMeta `json:"meta"`
Streams []Stream `json:"streams"`
Percent float64 `json:"percent"`
FPS float64 `json:"fps"`
Speed float64 `json:"speed"`
Paused bool `json:"paused"`
ElapsedSec int `json:"elapsed_sec"`
ETASec int `json:"eta_sec"`
StartedAt time.Time `json:"started_at"`
}
func New() *Tracker {
return &Tracker{phase: PhaseIdle}
}
// Begin marks the start of a new job, resetting all progress fields.
func (t *Tracker) Begin(file string) {
t.mu.Lock()
defer t.mu.Unlock()
t.file = file
t.phase = PhaseProbing
t.meta = nil
t.streams = nil
t.totalSec, t.outTime, t.fps, t.speed = 0, 0, 0, 0
t.startedAt = time.Now()
t.paused = false
t.pausedAt = time.Time{}
t.pausedTotal = 0
}
// SetPaused records pause/resume transitions so elapsed time excludes the
// paused span. Idempotent on repeated same-state calls.
func (t *Tracker) SetPaused(p bool) {
t.mu.Lock()
defer t.mu.Unlock()
if p == t.paused {
return
}
if p {
t.pausedAt = time.Now()
} else if !t.pausedAt.IsZero() {
t.pausedTotal += time.Since(t.pausedAt)
t.pausedAt = time.Time{}
}
t.paused = p
}
func (t *Tracker) SetPhase(p string) {
t.mu.Lock()
defer t.mu.Unlock()
t.phase = p
}
// SetMeta records the fetched metadata for the active job.
func (t *Tracker) SetMeta(m JobMeta) {
t.mu.Lock()
defer t.mu.Unlock()
t.meta = &m
}
// SetStreams records the source audio/subtitle streams for the active job.
func (t *Tracker) SetStreams(s []Stream) {
t.mu.Lock()
defer t.mu.Unlock()
t.streams = s
}
// SetTotal records the source duration in seconds (from ffprobe).
func (t *Tracker) SetTotal(seconds float64) {
t.mu.Lock()
defer t.mu.Unlock()
t.totalSec = seconds
}
// Update records one ffmpeg -progress sample.
func (t *Tracker) Update(outTimeSec, fps, speed float64) {
t.mu.Lock()
defer t.mu.Unlock()
t.outTime, t.fps, t.speed = outTimeSec, fps, speed
}
// Idle clears the tracker when no job is running.
func (t *Tracker) Idle() {
t.mu.Lock()
defer t.mu.Unlock()
t.file = ""
t.phase = PhaseIdle
t.meta = nil
t.streams = nil
t.totalSec, t.outTime, t.fps, t.speed = 0, 0, 0, 0
t.startedAt = time.Time{}
t.paused = false
t.pausedAt = time.Time{}
t.pausedTotal = 0
}
func (t *Tracker) Snapshot() Snapshot {
t.mu.RLock()
defer t.mu.RUnlock()
s := Snapshot{
File: t.file,
Phase: t.phase,
Meta: t.meta,
Streams: t.streams,
FPS: t.fps,
Speed: t.speed,
Paused: t.paused,
StartedAt: t.startedAt,
}
if !t.startedAt.IsZero() {
elapsed := time.Since(t.startedAt) - t.pausedTotal
if t.paused && !t.pausedAt.IsZero() {
elapsed -= time.Since(t.pausedAt)
}
if elapsed < 0 {
elapsed = 0
}
s.ElapsedSec = int(elapsed.Seconds())
}
if t.totalSec > 0 {
s.Percent = t.outTime / t.totalSec * 100
if s.Percent > 100 {
s.Percent = 100
}
if t.speed > 0 {
remaining := t.totalSec - t.outTime
if remaining < 0 {
remaining = 0
}
s.ETASec = int(remaining / t.speed)
}
}
return s
}
// ProgressSample is one completed ffmpeg -progress block.
type ProgressSample struct {
OutTimeSec float64
FPS float64
Speed float64
Done bool // progress=end
}
// ScanProgress reads ffmpeg `-progress` key=value output from r and calls
// onSample once per block (each block is terminated by a "progress=" line).
// Returns when r is exhausted.
func ScanProgress(r io.Reader, onSample func(ProgressSample)) error {
sc := bufio.NewScanner(r)
var cur ProgressSample
for sc.Scan() {
if parseProgressLine(sc.Text(), &cur) {
onSample(cur)
cur = ProgressSample{}
}
}
return sc.Err()
}
// parseProgressLine folds one "key=value" line into s. Returns true when the
// line closes a block (key == "progress").
//
// ponytail: ffmpeg field naming drifts between builds — out_time_us is the
// modern microsecond field; out_time_ms is historically ALSO microseconds (a
// known mislabel); out_time is the "HH:MM:SS.ffffff" string. Prefer out_time_us,
// fall back to the others only if it hasn't set a value this block. Verify
// against the ffmpeg in the Docker image if percentages look off.
func parseProgressLine(line string, s *ProgressSample) (complete bool) {
k, v, ok := strings.Cut(line, "=")
if !ok {
return false
}
k, v = strings.TrimSpace(k), strings.TrimSpace(v)
switch k {
case "out_time_us":
if us, err := strconv.ParseFloat(v, 64); err == nil {
s.OutTimeSec = us / 1e6
}
case "out_time_ms":
if s.OutTimeSec == 0 {
if ms, err := strconv.ParseFloat(v, 64); err == nil {
s.OutTimeSec = ms / 1e6 // really microseconds, see note above
}
}
case "out_time":
if s.OutTimeSec == 0 {
s.OutTimeSec = parseTimecode(v)
}
case "fps":
if f, err := strconv.ParseFloat(v, 64); err == nil {
s.FPS = f
}
case "speed":
if f, err := strconv.ParseFloat(strings.TrimSuffix(v, "x"), 64); err == nil {
s.Speed = f // "N/A" leaves it 0
}
case "progress":
s.Done = v == "end"
return true
}
return false
}
// parseTimecode parses "HH:MM:SS.ffffff" into seconds; 0 on bad input.
func parseTimecode(tc string) float64 {
parts := strings.Split(tc, ":")
if len(parts) != 3 {
return 0
}
h, err1 := strconv.ParseFloat(parts[0], 64)
m, err2 := strconv.ParseFloat(parts[1], 64)
sec, err3 := strconv.ParseFloat(parts[2], 64)
if err1 != nil || err2 != nil || err3 != nil {
return 0
}
return h*3600 + m*60 + sec
}
internal/status/status_test.go
+110
View File
@@ -0,0 +1,110 @@
package status
import (
"strings"
"testing"
)
func TestScanProgress(t *testing.T) {
tests := []struct {
name string
input string
wantOutSec, wantFPS, wantSpd float64
wantDone bool
}{
{
name: "out_time_us preferred over ms and string",
input: "frame=120\nfps=24.00\nout_time_us=5000000\nout_time_ms=9999999\nout_time=00:00:09.000000\nspeed=1.02x\nprogress=continue\n",
wantOutSec: 5, wantFPS: 24, wantSpd: 1.02, wantDone: false,
},
{
name: "out_time string fallback when no us field",
input: "fps=12\nout_time=00:01:30.500000\nspeed=0.09x\nprogress=continue\n",
wantOutSec: 90.5, wantFPS: 12, wantSpd: 0.09, wantDone: false,
},
{
name: "end block",
input: "out_time_us=7200000000\nspeed=2x\nprogress=end\n",
wantOutSec: 7200, wantFPS: 0, wantSpd: 2, wantDone: true,
},
{
name: "speed N/A leaves zero",
input: "out_time_us=1000000\nspeed=N/A\nprogress=continue\n",
wantOutSec: 1, wantFPS: 0, wantSpd: 0, wantDone: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var last ProgressSample
n := 0
if err := ScanProgress(strings.NewReader(tt.input), func(s ProgressSample) { last = s; n++ }); err != nil {
t.Fatalf("ScanProgress: %v", err)
}
if n != 1 {
t.Fatalf("got %d samples, want 1", n)
}
if last.OutTimeSec != tt.wantOutSec || last.FPS != tt.wantFPS || last.Speed != tt.wantSpd || last.Done != tt.wantDone {
t.Errorf("got %+v, want out=%v fps=%v spd=%v done=%v", last, tt.wantOutSec, tt.wantFPS, tt.wantSpd, tt.wantDone)
}
})
}
}
func TestScanProgressMultipleBlocks(t *testing.T) {
in := "out_time_us=1000000\nspeed=1x\nprogress=continue\nout_time_us=2000000\nspeed=1x\nprogress=end\n"
var got []ProgressSample
if err := ScanProgress(strings.NewReader(in), func(s ProgressSample) { got = append(got, s) }); err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d blocks, want 2", len(got))
}
if got[0].OutTimeSec != 1 || got[1].OutTimeSec != 2 || got[0].Done || !got[1].Done {
t.Errorf("blocks = %+v", got)
}
}
func TestSnapshotPercentAndETA(t *testing.T) {
tr := New()
tr.Begin("episode.mkv")
tr.SetTotal(100)
tr.Update(25, 10, 0.5) // 25% done, 75s left at 0.5x -> 150s ETA
s := tr.Snapshot()
if s.Percent != 25 {
t.Errorf("Percent = %v, want 25", s.Percent)
}
if s.ETASec != 150 {
t.Errorf("ETASec = %v, want 150", s.ETASec)
}
if s.File != "episode.mkv" || s.Phase != PhaseProbing {
t.Errorf("File/Phase = %q/%q", s.File, s.Phase)
}
}
func TestSetPausedFlag(t *testing.T) {
tr := New()
tr.Begin("x.mkv")
if tr.Snapshot().Paused {
t.Fatal("should not start paused")
}
tr.SetPaused(true)
if !tr.Snapshot().Paused {
t.Error("expected paused after SetPaused(true)")
}
tr.SetPaused(true) // idempotent — must not double-count
tr.SetPaused(false)
if tr.Snapshot().Paused {
t.Error("expected not paused after SetPaused(false)")
}
tr.Idle()
if tr.Snapshot().Paused {
t.Error("Idle should clear paused")
}
}
func TestSnapshotIdleNoDivByZero(t *testing.T) {
s := New().Snapshot() // no Begin, totalSec 0
if s.Percent != 0 || s.ETASec != 0 || s.Phase != PhaseIdle {
t.Errorf("idle snapshot = %+v", s)
}
}
internal/watcher/watcher.go
+137 -9
View File
@@ -1,19 +1,79 @@
package watcher package watcher
import ( import (
"context"
"fmt" "fmt"
"os"
"path/filepath" "path/filepath"
"sort"
"sync"
"time" "time"
"videnc-vibe/pkg/types" "av1dae/pkg/types"
) )
// failureBackoff is how long we skip a file after processFn returned an error
// for it, provided the file has not been touched since (mtime unchanged). Chosen
// to be much larger than the polling interval so a permanently-broken input
// (e.g. a video-only file producing "no audio streams found") does not spam the
// log every tick, but short enough that an operator who fixes the underlying
// problem by replacing the file sees it picked up promptly on the next stable
// scan.
const failureBackoff = 5 * time.Minute
// inputExtensions lists the source container extensions the watcher will pick
// up. Globbed case-sensitively (matching prior .mkv behavior).
var inputExtensions = []string{
"mkv", "mp4", "m4v", "mov", "avi",
"ts", "m2ts", "mts",
"mpg", "mpeg", "vob",
"webm", "wmv", "flv",
}
// InputFiles returns the source files currently in dir that the watcher would
// consider, sorted. The status server uses this to report the pending queue, so
// it stays in sync with inputExtensions.
func InputFiles(dir string) []string {
var files []string
for _, ext := range inputExtensions {
matches, _ := filepath.Glob(filepath.Join(dir, "*."+ext))
files = append(files, matches...)
}
sort.Strings(files)
return files
}
// fileStat is the (mtime, size) pair used to decide whether a file has settled
// between two consecutive ticks.
type fileStat struct {
mtime time.Time
size int64
}
// failureRecord remembers that processFn failed for a given file, so we can
// back off rather than retrying on every tick.
type failureRecord struct {
failedAt time.Time
fileMtime time.Time
}
type Watcher struct { type Watcher struct {
inputDir string inputDir string
interval time.Duration interval time.Duration
// runMu guards running: the start/hold gate. While held, settled files are
// tracked (so the queue is reported) but not handed to processFn.
runMu sync.RWMutex
running bool
// seen tracks the last-observed (mtime, size) for every file currently in
// the input dir. A file is only handed to processFn once two consecutive
// ticks agree on both fields (partial-write protection).
seen map[string]fileStat
// failed tracks files that recently errored from processFn so we can skip
// them until either failureBackoff elapses or their mtime changes.
failed map[string]failureRecord
} }
func New(inputDir string, intervalSeconds int) *Watcher { func New(inputDir string, intervalSeconds int, autostart bool) *Watcher {
interval := time.Duration(intervalSeconds) * time.Second interval := time.Duration(intervalSeconds) * time.Second
if interval < 10*time.Second { if interval < 10*time.Second {
interval = 10 * time.Second interval = 10 * time.Second
@@ -24,39 +84,107 @@ func New(inputDir string, intervalSeconds int) *Watcher {
return &Watcher{ return &Watcher{
inputDir: inputDir, inputDir: inputDir,
interval: interval, interval: interval,
running: autostart,
seen: make(map[string]fileStat),
failed: make(map[string]failureRecord),
} }
} }
func (w *Watcher) Start(processFn func(string) error, done chan struct{}) { // SetRunning toggles the start/hold gate.
func (w *Watcher) SetRunning(v bool) {
w.runMu.Lock()
w.running = v
w.runMu.Unlock()
}
// Running reports whether the queue is being processed.
func (w *Watcher) Running() bool {
w.runMu.RLock()
defer w.runMu.RUnlock()
return w.running
}
func (w *Watcher) Start(ctx context.Context, processFn func(context.Context, string) error) {
ticker := time.NewTicker(w.interval) ticker := time.NewTicker(w.interval)
defer ticker.Stop() defer ticker.Stop()
w.scanAndProcess(processFn) w.scanAndProcess(ctx, processFn)
for { for {
select { select {
case <-done: case <-ctx.Done():
return return
case <-ticker.C: case <-ticker.C:
w.scanAndProcess(processFn) w.scanAndProcess(ctx, processFn)
} }
} }
} }
func (w *Watcher) scanAndProcess(processFn func(string) error) { func (w *Watcher) scanAndProcess(ctx context.Context, processFn func(context.Context, string) error) {
files, err := filepath.Glob(filepath.Join(w.inputDir, "*.mkv")) var files []string
for _, ext := range inputExtensions {
matches, err := filepath.Glob(filepath.Join(w.inputDir, "*."+ext))
if err != nil { if err != nil {
fmt.Printf("Error scanning input directory: %v\n", err) fmt.Printf("Error scanning input directory: %v\n", err)
return return
} }
files = append(files, matches...)
}
now := time.Now()
nextSeen := make(map[string]fileStat, len(files))
nextFailed := make(map[string]failureRecord, len(w.failed))
for _, file := range files { for _, file := range files {
if err := processFn(file); err != nil { info, err := os.Stat(file)
if err != nil {
// File vanished between glob and stat, or unreadable. Drop any
// state for it by not carrying it forward.
fmt.Printf("Error stating %s: %v\n", file, err)
continue
}
current := fileStat{mtime: info.ModTime(), size: info.Size()}
nextSeen[file] = current
// Carry the failure record forward only if the file hasn't been
// touched since it failed; a changed mtime means the user replaced
// or modified the file and we should give it another chance.
if rec, ok := w.failed[file]; ok && rec.fileMtime.Equal(current.mtime) {
if now.Sub(rec.failedAt) < failureBackoff {
nextFailed[file] = rec
continue
}
// Backoff expired; clear the record and let the file be
// re-processed if it is otherwise stable.
}
prev, ok := w.seen[file]
if !ok || prev.mtime != current.mtime || prev.size != current.size {
// First time we've seen this (mtime, size); wait one more tick
// to make sure the file isn't still being written.
continue
}
if !w.Running() {
// Held: the file stays in the queue (already in nextSeen) and is
// picked up once the user starts processing.
continue
}
if err := processFn(ctx, file); err != nil {
fmt.Printf("Error processing %s: %v\n", file, err) fmt.Printf("Error processing %s: %v\n", file, err)
nextFailed[file] = failureRecord{
failedAt: now,
fileMtime: current.mtime,
} }
} }
} }
w.seen = nextSeen
w.failed = nextFailed
}
func DetectMediaType(width, height int) types.MediaType { func DetectMediaType(width, height int) types.MediaType {
pixels := width * height pixels := width * height
if pixels < 600000 { if pixels < 600000 {
pkg/types/types.go
+13 -12
View File
@@ -1,14 +1,12 @@
package types package types
import (
"time"
)
type MediaType string type MediaType string
const ( const (
MediaTypeDVD MediaType = "DVD" MediaTypeDVD MediaType = "DVD"
MediaTypeBluRay MediaType = "Blu-ray" MediaTypeBluRay MediaType = "Blu-ray"
MediaTypeWebDL MediaType = "WebDL"
MediaTypeTVRip MediaType = "TVRip"
) )
type Job struct { type Job struct {
@@ -18,6 +16,7 @@ type Job struct {
Metadata *Metadata Metadata *Metadata
CRF int CRF int
Preset int Preset int
LP int // SVT-AV1 logical processors; 0 = auto
DeleteOrigin bool DeleteOrigin bool
} }
@@ -35,6 +34,10 @@ type Metadata struct {
type Config struct { type Config struct {
OMDBAPIKey string `yaml:"omdb_api_key"` OMDBAPIKey string `yaml:"omdb_api_key"`
LogRetentionDays int `yaml:"log_retention_days"`
// HTTPAddr is the status server listen address. Absent (nil) defaults to
// ":8080"; an explicit empty string disables the server.
HTTPAddr *string `yaml:"http_addr"`
Encoding EncodingConfig Encoding EncodingConfig
Paths PathsConfig Paths PathsConfig
} }
@@ -42,6 +45,11 @@ type Config struct {
type EncodingConfig struct { type EncodingConfig struct {
DVD EncodingParams `yaml:"dvd"` DVD EncodingParams `yaml:"dvd"`
Bluray EncodingParams `yaml:"bluray"` Bluray EncodingParams `yaml:"bluray"`
WebDL EncodingParams `yaml:"webdl"`
TVRip EncodingParams `yaml:"tvrip"`
// LP caps SVT-AV1's logical-processor (thread) count so a long encode can't
// pin every core. 0 = let SVT-AV1 auto-detect (default). Applies to all profiles.
LP int `yaml:"lp"`
} }
type EncodingParams struct { type EncodingParams struct {
@@ -54,12 +62,5 @@ type PathsConfig struct {
Output string `yaml:"output"` Output string `yaml:"output"`
Originals string `yaml:"originals"` Originals string `yaml:"originals"`
Failed string `yaml:"failed"` Failed string `yaml:"failed"`
} Work string `yaml:"work"`
type LogEntry struct {
Timestamp time.Time `json:"timestamp"`
Level string `json:"level"`
Message string `json:"message"`
File string `json:"file,omitempty"`
Error string `json:"error,omitempty"`
} }