Three lifecycle controls on the dashboard, sharing one /api surface and the
status feed:
- Start/hold gate (#11): the watcher tracks the queue but holds processing
until POST /api/start. Held by default; AV1DAE_AUTOSTART=1 restores start-
on-boot. NOTE: flips the previous auto-start default.
- Pause/resume the active encode (#10): SIGSTOP/SIGCONT on the ffmpeg process
(libsvtav1 is in-process, so one signal suspends all its threads). Resumes
exactly where it left off; the tracker excludes paused time from elapsed.
- Retry a failed file (#3): POST /api/retry?file=NAME moves it from failed/
back to input/, with a base-name guard against path traversal.
/status now reports running + the failed list; snapshots carry a paused flag.
Verified live: held queue, retry move, traversal -> 400, and a real encode
suspending to process state T on pause and S on resume.
Closes#3Closes#10Closes#11
Add a settings page (/settings) and API (/api/settings GET/PUT) to edit
encoding profiles (crf/preset per media type), lp, OMDb key, log retention,
and delete-originals from the browser, persisted to a settings table in
logs.db. config.yaml seeds the store on first run; afterwards the DB is the
source of truth (paths.* and http_addr stay config-only). Each job snapshots
the current settings, so changes apply to the next encode with no restart.
PUT validates ranges (crf 0-63, preset 0-13, lp >=0, retention >=1) before
persisting. No auth, by design for now (isolated network) — see #2.
Closes#1
Add encoding.lp (0 = auto) so a long encode can be capped to N logical
processors instead of pinning every core, as a lighter-weight alternative to
the compose cgroup limit. Threaded config -> types.Job -> -svtav1-params via a
testable svtav1Params helper. Settings-UI surfacing stays with #1.
refs #5
Rename the Go module, the cmd/ entrypoint dir, the binary, the default config
dir (~/.config/av1dae/), the Docker image/compose service, and all docs and
the dashboard wordmark. No behavioral change — import paths and identifiers
only.
Restructure the current-job card around the question the monitor exists to
answer: time remaining is now the hero, with percent riding the bar. Add a
client-side throughput sparkline (90 s rolling buffer of encode speed against
a fixed 1.0x realtime reference) and a health-colored speed gauge so stalls
and sub-realtime encoding are visible at a glance. Collapse the repetitive
input queue into a count plus shared-prefix rows.
Surface the fetched job metadata (show/episode or movie title, season,
episode, release/airdate, media type) and the source audio/subtitle streams
through the tracker and /status, and render them in the dashboard's current-job
card. Audio channel counts are now probed and shown as 5.1/2.0/etc.
ETA now reads as a wall-clock finish time plus remaining duration, e.g.
"17:49 (3h52m)". Recent events drop the per-row date for a WhatsApp-style
Today/Yesterday/date divider with time-only rows. All dynamic strings are
HTML-escaped, since metadata and filenames are user-controlled.
Stream ffmpeg -progress during the video encode into an in-memory tracker
(internal/status) so the long-running step is no longer a black box: percent,
fps, speed, and ETA are derived from the source duration and updated ~1/s.
Progress prints to stdout only (no logs.db row) to avoid burying real events.
Expose it over HTTP (internal/server, default :8080, set http_addr to "" to
disable): GET /status returns the live snapshot, the pending input queue, and
recent non-debug events; GET / serves an embedded dashboard that polls /status
every second. The server shuts down on the same SIGINT/SIGTERM context as the
watcher.
Replace JSON/file logging with a logs.db (WAL) store. Thread the logger
through the encoder and metadata client for debug instrumentation of every
ffmpeg/ffprobe/opusenc invocation and OMDb/TVmaze request. API keys are
redacted before request URLs are logged. Retention defaults to 7 days,
overridable via log_retention_days in config.
The watcher only globbed *.mkv, silently ignoring every other source
container. Expand the glob to a list of common input extensions
(mkv, mp4, m4v, mov, avi, ts, m2ts, mts, mpg, mpeg, vob, webm, wmv,
flv). Downstream is unaffected: workdir naming uses filepath.Ext, the
encoder always emits .mkv output, and the metadata regexes are not
end-anchored.
Thread context.Context from main through watcher, encoder, and metadata
clients so a Ctrl-C during a multi-hour encode immediately kills the
ffmpeg/ffprobe/opusenc children instead of waiting for them to finish.
- main: signal.NotifyContext replaces the manual sigChan + done goroutine
- watcher.Start: takes ctx, exits on ctx.Done(); processFn signature is
now func(context.Context, string) error
- encoder: Transcode and every helper (extractAudio, encodeOpus,
encodeVideo, GetMediaInfo, GetStreamLanguages, calculateZscaleWidth)
take ctx; every exec.Command becomes exec.CommandContext so the child
is SIGKILL'd on cancel
- metadata: FetchMovieMetadata, FetchSeriesMetadata, fetchTVMazeJSON
take ctx and use http.NewRequestWithContext
Mover stays ctx-free intentionally: a rename is fast enough that
mid-cancel cleanup is the next-restart's problem. processFile's
deferred RemoveAll(workDir) and failToFailed still run after cancel,
so partial output dies in the work dir and the source moves to failed/.
Adds a per-job scratch directory under the new `paths.work` config (default
`./work`) so audio.<n>.wav, audio.<n>.opus and output.mkv no longer live
beside the user's sources in paths.input. The work subdir is named after
the input basename and unconditionally removed when processFile returns
(success or failure), which kills bug #4 (intermediates leaking into the
user-owned input folder; output.mkv getting re-picked by the 15-second
watcher tick on rename failure; fixed-name collision risk for any future
concurrency).
Tightens the failure paths in main.processFile too (bug #25):
- mover.MoveToFailed return values are now surfaced via a new
failToFailed helper.
- The helper os.Stats the source first; missing -> log and skip instead
of the previous silent no-op when MoveToFailed was called on
output.mkv before it existed.
- On rename-output failure, the source is now routed to paths.failed
(it was previously left in paths.input, causing an infinite re-encode
loop on the next watcher tick). The old MoveToFailed on the work-dir
output is dropped — the deferred RemoveAll covers it.
Mechanical changes:
- PathsConfig gains `Work string \`yaml:"work"\`` with default ./work,
included in EnsureDirs.
- Encoder.Transcode signature now takes workDir; extractAudio,
encodeOpus and encodeVideo all write into workDir. The internal
cleanupWavs/cleanupOpus defers are gone (RemoveAll in main is the
one cleanup path).
- MANUAL.md updated: example config, field reference, §6 pipeline
step wording, §9 failure handling description, §11 runtime
directories block.
Centralize all move helpers through a single safeRename function that
stats the destination first and refuses to overwrite (bug #14), and
falls back to copy+fsync+remove when os.Rename returns EXDEV across
mounted filesystems (bug #13). Drops the dead MoveToOutput helper.
Two related hardening fixes for the input folder poller (bug #22):
1. Partial-write protection. A freshly-dropped .mkv now has to present
the same (mtime, size) on two consecutive ticks before processFn is
called. A file mid-rsync that grows or has a moving mtime is skipped
until it settles. The per-file (mtime, size) state lives in
Watcher.seen and is rebuilt from the current glob each tick so the
map cannot grow unboundedly.
2. Failure quarantine. When processFn returns an error, the file's
(failedAt, mtime) is recorded in Watcher.failed and subsequent ticks
skip it until either the backoff elapses or its mtime changes (user
replaced or touched it). Previously a permanently-broken input -- e.g.
a video-only mkv that trips the "no audio streams found" path added
in the multi-audio fix -- would be retried and logged every 15 s
forever.
Backoff is 5 minutes: comfortably longer than the 10-30 s polling
interval clamp so we are not effectively retrying every tick, but
short enough that an operator fixing the underlying problem by
replacing the file sees it picked up promptly on the next stable scan.
Source kind is now declared per-file via a `dvd` / `bluray` / `webdl` /
`tvrip` token in the basename (case-insensitive, word-bounded). Matches
the existing `tt…` / `tvm…` filename convention. When no token is
present, falls back to the pixel-count guess in DetectMediaType, which
still only chooses between DVD and Blu-ray.
The parsed media type drives both ORIGINAL_MEDIA_TYPE in the muxed
metadata and the CRF/preset profile used for encoding. EncodingConfig
gains `webdl` and `tvrip` sub-blocks with conservative defaults
(WebDL 30/3, TVRip 32/2); user can override in config.yaml.
Manual updated with the new tokens, config fields, and pipeline
description.
Bug #5: extractAudio invoked ffmpeg without `-map`, so default stream
selection kept only one audio track from sources with multiple audio
streams (e.g. eng/fra/jpn Blu-rays). It now enumerates audio streams
from the streamLangs already fetched by the caller and runs one
`ffmpeg -map 0🅰️<n>` per stream, writing audio.<n>.wav.
Bug #11: the audio language metadata loop in encodeVideo computed the
output index by counting source-side audio streams with a lower Index,
which drifted when some source streams lacked a language tag. It now
walks opusFiles in output order and looks up the language at the
matching source-audio position via a sorted helper.
These ship together because #11 was masked by #5: when only one audio
track survived extraction, the broken index calculation never produced
a visible misalignment. Fixing #5 alone would have caused multi-track
outputs with shuffled language tags; both fixes are required to land
correct multi-track output.
Bug #2: calculateZscaleWidth previously emitted a zscale filter even
when no rescale was needed (SAR 1:1, N/A, empty, or computed width
equal to the source). encodeVideo then unconditionally appended it to
-vf, forcing a pointless colorspace round-trip. Return an empty filter
string in those cases and build the -vf chain conditionally; omit -vf
entirely when no filters apply.
Bug #23: the SAR-to-width math used integer truncation, producing odd
or off-by-one widths (e.g. 853 instead of 854 for 32:27 at 1920), and
the guard accepted SAR 0:N which zeroed the output width. Reject
zero-numerator SARs and round to the nearest integer then mask to an
even width for AV1/H.264 mod-2 alignment.
GetMediaInfo previously only recognized mpeg2video/h264/hevc and silently
returned 0x0 with no error for other codecs (VC-1, MPEG-4 ASP, AV1,
ProRes), causing DetectMediaType to misclassify as DVD and feeding bogus
dimensions to the zscale filter. Pick the first stream with
codec_type=video and return an explicit error if none is found.
exec.Command does not invoke a shell, so the wrapping " characters
were inserted into the muxed tag value verbatim (e.g. TITLE read as
"Snatch" instead of Snatch). Use unquoted Sprintf format strings.
TVmaze: drop trailing slash from base URL (was producing //shows/), send
season/number as plain ints (TVmaze rejects zero-padding), check HTTP
status before decoding so 404 bodies stop being decoded as episodes, and
fetch /shows/{id} separately for show name + IMDb mapping —
episodebynumber does not honor embed=show despite what the docs imply.
OMDb: check HTTP status, and include a body snippet when Response is
non-True with an empty Error field so rate-limit and HTML failure modes
are diagnosable.
Both: assert that the fields we actually depend on come back non-empty
after decode. Catches silent field rename/removal without making us
fragile to TVmaze adding new optional fields.
main.processFile: split the nometadata fallback by branch. Series uses
Collection (legitimate shows can have no IMDb mapping); movies keep the
IMDBID/Title check.
The previous detector substring-matched "TFF"/"BFF" against idet's own
label text, so it returned true on every source, and `cmd.Start()` was
never paired with `Wait()`, leaving a zombie ffmpeg per file.
Run idet bounded with `-frames:v 400 -an -sn -f null -` so it completes,
use CombinedOutput so the child is reaped, and parse the "Multi frame
detection" summary line — interlaced only when TFF+BFF > Progressive.
Esa Kataja