The unanchored 'av1dae' pattern (from the rename) also matched the cmd/av1dae
source dir. Anchor to /av1dae and /av1dae-dev so only the root-level built
binaries are ignored.
Add encoding.lp (0 = auto) so a long encode can be capped to N logical
processors instead of pinning every core, as a lighter-weight alternative to
the compose cgroup limit. Threaded config -> types.Job -> -svtav1-params via a
testable svtav1Params helper. Settings-UI surfacing stays with #1.
refs #5
Rename the Go module, the cmd/ entrypoint dir, the binary, the default config
dir (~/.config/av1dae/), the Docker image/compose service, and all docs and
the dashboard wordmark. No behavioral change — import paths and identifiers
only.
Restructure the current-job card around the question the monitor exists to
answer: time remaining is now the hero, with percent riding the bar. Add a
client-side throughput sparkline (90 s rolling buffer of encode speed against
a fixed 1.0x realtime reference) and a health-colored speed gauge so stalls
and sub-realtime encoding are visible at a glance. Collapse the repetitive
input queue into a count plus shared-prefix rows.
Surface the fetched job metadata (show/episode or movie title, season,
episode, release/airdate, media type) and the source audio/subtitle streams
through the tracker and /status, and render them in the dashboard's current-job
card. Audio channel counts are now probed and shown as 5.1/2.0/etc.
ETA now reads as a wall-clock finish time plus remaining duration, e.g.
"17:49 (3h52m)". Recent events drop the per-row date for a WhatsApp-style
Today/Yesterday/date divider with time-only rows. All dynamic strings are
HTML-escaped, since metadata and filenames are user-controlled.
EXPOSE 8080 in the image, publish 8080 in compose, document http_addr in the
example config, and add the CPU/memory limits so a long encode can't starve
the host.
Stream ffmpeg -progress during the video encode into an in-memory tracker
(internal/status) so the long-running step is no longer a black box: percent,
fps, speed, and ETA are derived from the source duration and updated ~1/s.
Progress prints to stdout only (no logs.db row) to avoid burying real events.
Expose it over HTTP (internal/server, default :8080, set http_addr to "" to
disable): GET /status returns the live snapshot, the pending input queue, and
recent non-debug events; GET / serves an embedded dashboard that polls /status
every second. The server shuts down on the same SIGINT/SIGTERM context as the
watcher.
Multi-stage build (cgo for the SQLite logger) on debian-slim with ffmpeg,
opus-tools, and ca-certificates bundled. Compose mounts ./data/config.yaml
and ./data/media; SIGTERM cancels in-flight encodes cleanly.
Replace JSON/file logging with a logs.db (WAL) store. Thread the logger
through the encoder and metadata client for debug instrumentation of every
ffmpeg/ffprobe/opusenc invocation and OMDb/TVmaze request. API keys are
redacted before request URLs are logged. Retention defaults to 7 days,
overridable via log_retention_days in config.
The watcher only globbed *.mkv, silently ignoring every other source
container. Expand the glob to a list of common input extensions
(mkv, mp4, m4v, mov, avi, ts, m2ts, mts, mpg, mpeg, vob, webm, wmv,
flv). Downstream is unaffected: workdir naming uses filepath.Ext, the
encoder always emits .mkv output, and the metadata regexes are not
end-anchored.
After bug #9's context plumbing, a signal interrupts the current encode
immediately rather than waiting for the poll cycle. Document the new
shutdown semantics: children killed, work dir cleaned via the deferred
RemoveAll, source routed to failed/.
Thread context.Context from main through watcher, encoder, and metadata
clients so a Ctrl-C during a multi-hour encode immediately kills the
ffmpeg/ffprobe/opusenc children instead of waiting for them to finish.
- main: signal.NotifyContext replaces the manual sigChan + done goroutine
- watcher.Start: takes ctx, exits on ctx.Done(); processFn signature is
now func(context.Context, string) error
- encoder: Transcode and every helper (extractAudio, encodeOpus,
encodeVideo, GetMediaInfo, GetStreamLanguages, calculateZscaleWidth)
take ctx; every exec.Command becomes exec.CommandContext so the child
is SIGKILL'd on cancel
- metadata: FetchMovieMetadata, FetchSeriesMetadata, fetchTVMazeJSON
take ctx and use http.NewRequestWithContext
Mover stays ctx-free intentionally: a rename is fast enough that
mid-cancel cleanup is the next-restart's problem. processFile's
deferred RemoveAll(workDir) and failToFailed still run after cancel,
so partial output dies in the work dir and the source moves to failed/.
§9: add notes on destination-collision refusal (silent overwrite was
bug #14) and cross-filesystem move fallback (bug #13). Both are now
implemented but the manual didn't reflect them.
§10: drop the "no atomic-write detection" warning. The watcher now
requires mtime+size stability across two ticks before processing (bug
#22), and quarantines a failing file for 5 minutes — so the
copy-then-mv workaround is no longer a correctness requirement.
Adds a per-job scratch directory under the new `paths.work` config (default
`./work`) so audio.<n>.wav, audio.<n>.opus and output.mkv no longer live
beside the user's sources in paths.input. The work subdir is named after
the input basename and unconditionally removed when processFile returns
(success or failure), which kills bug #4 (intermediates leaking into the
user-owned input folder; output.mkv getting re-picked by the 15-second
watcher tick on rename failure; fixed-name collision risk for any future
concurrency).
Tightens the failure paths in main.processFile too (bug #25):
- mover.MoveToFailed return values are now surfaced via a new
failToFailed helper.
- The helper os.Stats the source first; missing -> log and skip instead
of the previous silent no-op when MoveToFailed was called on
output.mkv before it existed.
- On rename-output failure, the source is now routed to paths.failed
(it was previously left in paths.input, causing an infinite re-encode
loop on the next watcher tick). The old MoveToFailed on the work-dir
output is dropped — the deferred RemoveAll covers it.
Mechanical changes:
- PathsConfig gains `Work string \`yaml:"work"\`` with default ./work,
included in EnsureDirs.
- Encoder.Transcode signature now takes workDir; extractAudio,
encodeOpus and encodeVideo all write into workDir. The internal
cleanupWavs/cleanupOpus defers are gone (RemoveAll in main is the
one cleanup path).
- MANUAL.md updated: example config, field reference, §6 pipeline
step wording, §9 failure handling description, §11 runtime
directories block.
Centralize all move helpers through a single safeRename function that
stats the destination first and refuses to overwrite (bug #14), and
falls back to copy+fsync+remove when os.Rename returns EXDEV across
mounted filesystems (bug #13). Drops the dead MoveToOutput helper.
Two related hardening fixes for the input folder poller (bug #22):
1. Partial-write protection. A freshly-dropped .mkv now has to present
the same (mtime, size) on two consecutive ticks before processFn is
called. A file mid-rsync that grows or has a moving mtime is skipped
until it settles. The per-file (mtime, size) state lives in
Watcher.seen and is rebuilt from the current glob each tick so the
map cannot grow unboundedly.
2. Failure quarantine. When processFn returns an error, the file's
(failedAt, mtime) is recorded in Watcher.failed and subsequent ticks
skip it until either the backoff elapses or its mtime changes (user
replaced or touched it). Previously a permanently-broken input -- e.g.
a video-only mkv that trips the "no audio streams found" path added
in the multi-audio fix -- would be retried and logged every 15 s
forever.
Backoff is 5 minutes: comfortably longer than the 10-30 s polling
interval clamp so we are not effectively retrying every tick, but
short enough that an operator fixing the underlying problem by
replacing the file sees it picked up promptly on the next stable scan.
§6 step 2: drop the codec-name allowlist, describe the bounded idet run
and the Multi-frame summary parse.
§6 step 6/7: per-stream audio.<n>.wav / audio.<n>.opus now.
§6 step 8/9: zscale is conditional and width rounds to mod-2; -vf is
omitted entirely when no filter applies. Audio language tags are indexed
by output position. TITLE/COLLECTION values are unquoted.
§6 step 11: series filename no longer requires an IMDb mapping.
§7: added a token example and notes about what the token does and does
not influence in the output filename.
§8: noted the known log-rotation limitation (filename frozen at start;
restart to rotate; structured.json does not rotate).
Source kind is now declared per-file via a `dvd` / `bluray` / `webdl` /
`tvrip` token in the basename (case-insensitive, word-bounded). Matches
the existing `tt…` / `tvm…` filename convention. When no token is
present, falls back to the pixel-count guess in DetectMediaType, which
still only chooses between DVD and Blu-ray.
The parsed media type drives both ORIGINAL_MEDIA_TYPE in the muxed
metadata and the CRF/preset profile used for encoding. EncodingConfig
gains `webdl` and `tvrip` sub-blocks with conservative defaults
(WebDL 30/3, TVRip 32/2); user can override in config.yaml.
Manual updated with the new tokens, config fields, and pipeline
description.
Bug #5: extractAudio invoked ffmpeg without `-map`, so default stream
selection kept only one audio track from sources with multiple audio
streams (e.g. eng/fra/jpn Blu-rays). It now enumerates audio streams
from the streamLangs already fetched by the caller and runs one
`ffmpeg -map 0🅰️<n>` per stream, writing audio.<n>.wav.
Bug #11: the audio language metadata loop in encodeVideo computed the
output index by counting source-side audio streams with a lower Index,
which drifted when some source streams lacked a language tag. It now
walks opusFiles in output order and looks up the language at the
matching source-audio position via a sorted helper.
These ship together because #11 was masked by #5: when only one audio
track survived extraction, the broken index calculation never produced
a visible misalignment. Fixing #5 alone would have caused multi-track
outputs with shuffled language tags; both fixes are required to land
correct multi-track output.
Bug #2: calculateZscaleWidth previously emitted a zscale filter even
when no rescale was needed (SAR 1:1, N/A, empty, or computed width
equal to the source). encodeVideo then unconditionally appended it to
-vf, forcing a pointless colorspace round-trip. Return an empty filter
string in those cases and build the -vf chain conditionally; omit -vf
entirely when no filters apply.
Bug #23: the SAR-to-width math used integer truncation, producing odd
or off-by-one widths (e.g. 853 instead of 854 for 32:27 at 1920), and
the guard accepted SAR 0:N which zeroed the output width. Reject
zero-numerator SARs and round to the nearest integer then mask to an
even width for AV1/H.264 mod-2 alignment.
GetMediaInfo previously only recognized mpeg2video/h264/hevc and silently
returned 0x0 with no error for other codecs (VC-1, MPEG-4 ASP, AV1,
ProRes), causing DetectMediaType to misclassify as DVD and feeding bogus
dimensions to the zscale filter. Pick the first stream with
codec_type=video and return an explicit error if none is found.
exec.Command does not invoke a shell, so the wrapping " characters
were inserted into the muxed tag value verbatim (e.g. TITLE read as
"Snatch" instead of Snatch). Use unquoted Sprintf format strings.
TVmaze: drop trailing slash from base URL (was producing //shows/), send
season/number as plain ints (TVmaze rejects zero-padding), check HTTP
status before decoding so 404 bodies stop being decoded as episodes, and
fetch /shows/{id} separately for show name + IMDb mapping —
episodebynumber does not honor embed=show despite what the docs imply.
OMDb: check HTTP status, and include a body snippet when Response is
non-True with an empty Error field so rate-limit and HTML failure modes
are diagnosable.
Both: assert that the fields we actually depend on come back non-empty
after decode. Catches silent field rename/removal without making us
fragile to TVmaze adding new optional fields.
main.processFile: split the nometadata fallback by branch. Series uses
Collection (legitimate shows can have no IMDb mapping); movies keep the
IMDBID/Title check.
The previous detector substring-matched "TFF"/"BFF" against idet's own
label text, so it returned true on every source, and `cmd.Start()` was
never paired with `Wait()`, leaving a zombie ffmpeg per file.
Run idet bounded with `-frames:v 400 -an -sn -f null -` so it completes,
use CombinedOutput so the child is reaped, and parse the "Multi frame
detection" summary line — interlaced only when TFF+BFF > Progressive.
Esa Kataja