av1dae/MANUAL.md

av1dae — User Manual

A Go CLI that watches a folder for .mkv rips, transcodes them to SVT-AV1 video + Opus audio, embeds metadata fetched from OMDb (movies) or TVmaze (series), and files the results into output/originals/failed directories.

---

1. Requirements

External binaries must be on $PATH (the program checks them at startup and exits if any are missing):

• ffmpeg
• ffprobe
• opusenc

Go (only to build):

• Go 1.x with module support.

API keys:

• OMDb API key (free at https://www.omdbapi.com/). Required for movie metadata. TVmaze is unauthenticated.

---

2. Build

go build -o av1dae ./cmd/av1dae/

This produces a single static binary ./av1dae.

---

3. Configuration

By default the config is loaded from ~/.config/av1dae/config.yaml. Pass -c /path/to/config.yaml to override.

Example (config.example.yaml):

omdb_api_key: "YOUR_API_KEY_HERE"

encoding:
  dvd:
    crf: 30
    preset: 2
  bluray:
    crf: 29
    preset: 3
  webdl:
    crf: 30
    preset: 3
  tvrip:
    crf: 32
    preset: 2

paths:
  input:     "./input"
  output:    "./output"
  originals: "./originals"
  failed:    "./failed"
  work:      "./work"

Field reference

Field	Meaning	Default
omdb_api_key	OMDb API key for movie metadata lookup	(none — movies won't get metadata without it)
encoding.dvd.crf	SVT-AV1 CRF for SD sources	30
encoding.dvd.preset	SVT-AV1 preset for SD sources	2
encoding.bluray.crf	SVT-AV1 CRF for HD sources	29
encoding.bluray.preset	SVT-AV1 preset for HD sources	3
encoding.webdl.crf	SVT-AV1 CRF for WebDL sources	30
encoding.webdl.preset	SVT-AV1 preset for WebDL sources	3
encoding.tvrip.crf	SVT-AV1 CRF for TVRip sources	32
encoding.tvrip.preset	SVT-AV1 preset for TVRip sources	2
paths.input	Folder polled for new .mkv files	./input
paths.output	Destination for finished encodes	./output
paths.originals	Where source files are moved on success (unless -d)	./originals
paths.failed	Where source files go on failure	./failed
paths.work	Scratch directory for per-job intermediates (wav/opus/output.mkv); deleted after every job	./work

All five directories are created on startup if they don't exist.

---

4. Running

./av1dae                       # default config path, keep originals
./av1dae -d                    # delete originals after successful encode
./av1dae -c /etc/av1dae.yaml   # custom config
./av1dae -c /etc/av1dae.yaml -d

Flags:

• -d — delete the source .mkv after a successful encode instead of moving it to originals/.
• -c PATH — path to config file.

Stop with Ctrl+C (SIGINT) or SIGTERM. The signal cancels any in-flight encode immediately — the ffmpeg / ffprobe / opusenc children are killed, the per-job work directory is removed by its deferred cleanup, and the source .mkv is routed to paths.failed so the next run sees a clean paths.input.

The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter.

---

5. Input filename convention

The base name of each .mkv in paths.input is parsed to decide what metadata to fetch. Three forms are recognized:

5.1 Movies — IMDb ID

Filename must contain an IMDb tag of the form tt<digits>.

Examples:

Heat.tt0113277.mkv
some-rip-tt0114369.mkv

→ OMDb is queried for that IMDb ID. Title, release date, and IMDb ID are embedded.

5.2 Series — TVmaze ID + season/episode

Filename must contain BOTH:

• tvm<digits> — the TVmaze show ID (case-insensitive).
• s<digits>e<digits> — season and episode (case-insensitive).

Examples:

Breaking.Bad.tvm169.S01E01.mkv
the-wire.TVM75.s2e5.mkv

→ TVmaze is queried for that show/season/episode. Show name (Collection), episode title, season, episode, airdate, and the show's IMDb ID are embedded.

5.3 No recognizable tags

If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an .nometadata.mkv suffix.

5.4 Optional: source media type

Any filename can additionally contain a media-type token (case-insensitive, word-bounded):

• dvd
• bluray
• webdl
• tvrip

Examples:

Heat.tt0113277.bluray.mkv
some-rip.tvm169.S01E01.webdl.mkv
old.broadcast.tvrip.tt0066026.mkv

The token controls both the ORIGINAL_MEDIA_TYPE metadata tag written into the output and which encoding.<type>.crf / encoding.<type>.preset pair is used.

If no token is present, the program falls back to guessing from pixel count: width × height < 600,000 → DVD, otherwise Blu-ray. WebDL and TVRip are never auto-detected — they must be declared via the token.

---

6. The processing pipeline

For every .mkv found in paths.input, the program runs these steps in order. A per-job scratch subdirectory under paths.work (named after the input base name without .mkv) holds all intermediates, and is deleted unconditionally at the end of the job. Any error sends the source file to paths.failed; the work subdirectory is wiped regardless of outcome.

1. Parse filename → determines whether this is a movie or series, and what IDs to use.
2. Probe video with ffprobe:
   • Picks the first stream with codec_type=video, regardless of codec name. Errors out if there isn't one.
   • Records width, height, and sample aspect ratio (SAR).
   • Detects interlacing by running ffmpeg -vf idet -frames:v 400 -an -sn -f null - and parsing the Multi frame detection: TFF: a BFF: b Progressive: c Undetermined: d summary line. The source is treated as interlaced only when a+b > c; undetermined frames are ignored, and a missing summary line defaults to progressive.
3. Probe stream languages with ffprobe — collects language tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's -map_metadata -1 strips them otherwise).
4. Detect media type:
   • First, check the filename for a dvd / bluray / webdl / tvrip token (case-insensitive). If present, that wins.
   • Otherwise, fall back to pixel count: width × height < 600,000 → DVD, else Blu-ray. The chosen profile selects which encoding.<type>.crf / encoding.<type>.preset pair from the config to use, and is written into the ORIGINAL_MEDIA_TYPE metadata tag.
5. Fetch metadata from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata.
6. Extract audio — one PCM wav per source audio stream, written to the per-job work directory as audio.0.wav, audio.1.wav, … in source order (PCM s16le, 48 kHz). Errors out if the source has no audio streams.
7. Encode audio — each wav is converted with opusenc --bitrate 128k to a matching audio.<n>.opus in the same work directory.
8. Calculate display width from SAR. The width is rescaled so the output has square pixels only when there's actually work to do — zscale is skipped entirely for square-pixel sources (SAR 1:1, N/A, empty, 0:N), and for any SAR whose calculated width rounds to the source width. When rescaling, the width is rounded to the nearest even number (mod-2, preferred by AV1).
9. Encode video with FFmpeg:
   • Video filter chain is built conditionally. bwdif=mode=0:par=-1:-1 is prepended when the source is interlaced; the zscale step is appended only when a rescale is actually needed (see step 8). If neither applies, -vf is omitted entirely.
   • Codec: libsvtav1, -pix_fmt yuv420p10le.
   • -crf and -preset from the selected profile (step 4).
   • -svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s.
   • Streams mapped: video from input (0:v), subtitles from input (0:s? — optional), and one audio stream per Opus file (1:a, 2:a, …).
   • Audio re-muxed (-c:a copy — copies the already-Opus-encoded streams), subtitles copied (-c:s copy).
   • -map_metadata -1 strips global metadata; per-stream language tags are then re-applied. Audio language tags are indexed by output position (the position in the opus-file list), not by counting source streams, so missing-language streams don't shift the index.
   • Container metadata written: TITLE, DATE_RELEASED, IMDBID, ORIGINAL_MEDIA_TYPE. For series: also COLLECTION, SEASON, EPISODE, TVMAZE_ID. Values are unquoted (literal value, no wrapping "…").
   • Output written to output.mkv in the per-job work directory.
10. Clean up the entire per-job work directory (intermediates and output.mkv together) once the rename/move below succeeds — or, on any failure, when the deferred cleanup runs.
11. Rename and move output.mkv from the work directory to paths.output with a final name:
    • Series with a known show name → <sanitized-show-name>.S<NN>E<NN>.mkv (the series no longer needs a populated IMDb mapping — a TVmaze show with no external IMDb link still gets a useful filename).
    • Movie with a known title and IMDb ID → <sanitized-title>.<imdbID>.mkv.
    • Anything else → <8-hex-chars>.nometadata.mkv. Sanitization keeps a–z A–Z 0–9 - ä ö Ä Ö only.
12. Dispose of the source:
    • -d flag set → delete the original .mkv.
    • Otherwise → move it to paths.originals.

---

7. Output naming examples

Input filename	Result in paths.output	Notes
Heat.tt0113277.mkv	Heat.tt0113277.mkv	Pixel-count fallback → Blu-ray profile + tag
Heat.tt0113277.bluray.mkv	Heat.tt0113277.mkv	Same output filename; muxed ORIGINAL_MEDIA_TYPE=Blu-ray is now from the token, not the guess
Breaking.Bad.tvm169.S01E01.webdl.mkv	BreakingBad.S01E01.mkv	WebDL profile + tag
unrecognized-rip.mkv	a1b2c3d4.nometadata.mkv	No IDs at all

A few things worth knowing:

• The media-type token affects the muxed ORIGINAL_MEDIA_TYPE tag and the CRF/preset profile, but not the output filename.
• The IMDb ID written into the filename is the one returned by the API, not the one in the input filename, so a typo in the source filename will surface in the output name.
• A TVmaze show with no IMDb mapping still gets a <Collection>.S<NN>E<NN>.mkv filename (only the IMDBID metadata tag is left empty).

---

8. Logs

All logs are written to a SQLite database, logs.db, in the current working directory (not the config paths). Run the program from the directory where you want it to land — in Docker that's the mounted /data.

The logs table has columns id, ts, level, message, file, extra. Three levels are recorded:

• info — INFO messages; also printed to stdout.
• error — ERROR messages; also printed to stderr, with the source file recorded.
• debug — verbose diagnostics (ffprobe output, calculated zscale width, the full ffmpeg/opusenc command, OMDb/TVmaze responses with the API key redacted) in the extra column. Database only — not printed.

Live encode progress (encoding … · 47% · 3.2fps · …) prints to stdout only and is deliberately not stored, so it can't flood the database.

Retention: rows older than log_retention_days (default 7) are purged on startup and on shutdown. The value is editable live from the settings page (/settings) and applies at the next purge.

Recent non-debug events are also viewable in the web dashboard and via GET /status (when http_addr is set).

---

9. Failure handling

If any step from probing through encoding through renaming fails:

• The source .mkv is moved to paths.failed (the move itself is os.Stat-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves paths.input on every failure path, so the watcher doesn't retry the same file on the next tick.
• The per-job work directory under paths.work (containing partial wav/opus/output.mkv) is deleted unconditionally.
• The error is logged to logs.db (level error) with the source file path, and printed to stderr.

The watcher continues with the next file; one bad rip won't stop the daemon.

Destination collisions: if a finished encode would land on a name that already exists in paths.output, the move is refused (no silent overwrite) and the source is routed to paths.failed. This is the path you'll hit when two sources sanitize to the same output filename — e.g. two re-rips of the same release, or two episodes that both come out as SxxExx.

Cross-filesystem moves: every move (paths.input → paths.output, … → paths.failed, … → paths.originals) transparently falls back to copy + delete when the source and destination live on different mounts. You can put each path on a different drive without breaking the pipeline.

---

10. Polling behavior

• Input directory is scanned every 15 seconds (clamped to a 10–30 s range).
• Only files matching *.mkv directly in paths.input are picked up — no recursion.
• Files are processed sequentially, one at a time, in the order filepath.Glob returns them (alphabetical on Linux).
• Partial-write protection: the watcher requires a file's mtime and size to be identical on two consecutive scans before processing. A freshly dropped or still-copying file therefore waits at least one full tick (~15 s) before encoding begins. Copying a large file directly into paths.input is now safe; you no longer have to land it under a different name and mv into place (though doing so still works and shaves off the tick delay).
• Failure quarantine: if processFile returns an error, that file is skipped for 5 minutes (or until its mtime changes — e.g. you replace or touch it). Prevents a permanently-broken input from spamming the logs every 15 s. The quarantine is in-memory only; restarting the daemon clears it.

---

11. Project layout

cmd/av1dae/main.go        CLI entrypoint and per-file orchestration
internal/config/          YAML config load + defaults + mkdir
internal/watcher/         Polling loop, media-type detection by pixel count
internal/encoder/         ffprobe/ffmpeg/opusenc wrapper, transcode pipeline
internal/metadata/        Filename parsing, OMDb + TVmaze clients
internal/mover/           File rename/move/delete helpers
internal/logger/          Plain + JSON logging
pkg/types/types.go        Shared structs (Config, Job, Metadata, …)

Runtime directories (from paths.* in the config):

paths.input               User drops new .mkv files here; watcher polls every 15 s
paths.output              Finished encodes land here under their final name
paths.originals           Successfully-encoded sources end up here (unless -d)
paths.failed              Sources of failed jobs end up here
paths.work                Per-job scratch subdir (basename of input); wiped per job