Compare commits

..
10 Commits
Author SHA1 Message Date
Esa Kataja 6a564aabb2 fix: detect mp4 and other common containers in watcher
The watcher only globbed *.mkv, silently ignoring every other source
container. Expand the glob to a list of common input extensions
(mkv, mp4, m4v, mov, avi, ts, m2ts, mts, mpg, mpeg, vob, webm, wmv,
flv). Downstream is unaffected: workdir naming uses filepath.Ext, the
encoder always emits .mkv output, and the metadata regexes are not
end-anchored.
2026-05-18 08:34:55 +03:00
Esa Kataja 321f11dd8f chore: document SIGINT cancellation in §4
After bug #9's context plumbing, a signal interrupts the current encode
immediately rather than waiting for the poll cycle. Document the new
shutdown semantics: children killed, work dir cleaned via the deferred
RemoveAll, source routed to failed/.
2026-05-16 20:39:20 +03:00
Esa Kataja 459ab30d94 fix: cancel in-flight encode on SIGINT/SIGTERM via context plumbing
Thread context.Context from main through watcher, encoder, and metadata
clients so a Ctrl-C during a multi-hour encode immediately kills the
ffmpeg/ffprobe/opusenc children instead of waiting for them to finish.

- main: signal.NotifyContext replaces the manual sigChan + done goroutine
- watcher.Start: takes ctx, exits on ctx.Done(); processFn signature is
  now func(context.Context, string) error
- encoder: Transcode and every helper (extractAudio, encodeOpus,
  encodeVideo, GetMediaInfo, GetStreamLanguages, calculateZscaleWidth)
  take ctx; every exec.Command becomes exec.CommandContext so the child
  is SIGKILL'd on cancel
- metadata: FetchMovieMetadata, FetchSeriesMetadata, fetchTVMazeJSON
  take ctx and use http.NewRequestWithContext

Mover stays ctx-free intentionally: a rename is fast enough that
mid-cancel cleanup is the next-restart's problem. processFile's
deferred RemoveAll(workDir) and failToFailed still run after cancel,
so partial output dies in the work dir and the source moves to failed/.
2026-05-16 20:38:26 +03:00
Esa Kataja 191bc955e4 chore: document watcher stability, collision refusal, EXDEV fallback
§9: add notes on destination-collision refusal (silent overwrite was
bug #14) and cross-filesystem move fallback (bug #13). Both are now
implemented but the manual didn't reflect them.

§10: drop the "no atomic-write detection" warning. The watcher now
requires mtime+size stability across two ticks before processing (bug
#22), and quarantines a failing file for 5 minutes — so the
copy-then-mv workaround is no longer a correctness requirement.
2026-05-16 20:34:11 +03:00
Esa Kataja 706cd22b05 fix: isolate per-job intermediates in paths.work and harden failure cleanup
Adds a per-job scratch directory under the new `paths.work` config (default
`./work`) so audio.<n>.wav, audio.<n>.opus and output.mkv no longer live
beside the user's sources in paths.input. The work subdir is named after
the input basename and unconditionally removed when processFile returns
(success or failure), which kills bug #4 (intermediates leaking into the
user-owned input folder; output.mkv getting re-picked by the 15-second
watcher tick on rename failure; fixed-name collision risk for any future
concurrency).

Tightens the failure paths in main.processFile too (bug #25):
- mover.MoveToFailed return values are now surfaced via a new
  failToFailed helper.
- The helper os.Stats the source first; missing -> log and skip instead
  of the previous silent no-op when MoveToFailed was called on
  output.mkv before it existed.
- On rename-output failure, the source is now routed to paths.failed
  (it was previously left in paths.input, causing an infinite re-encode
  loop on the next watcher tick). The old MoveToFailed on the work-dir
  output is dropped — the deferred RemoveAll covers it.

Mechanical changes:
- PathsConfig gains `Work string \`yaml:"work"\`` with default ./work,
  included in EnsureDirs.
- Encoder.Transcode signature now takes workDir; extractAudio,
  encodeOpus and encodeVideo all write into workDir. The internal
  cleanupWavs/cleanupOpus defers are gone (RemoveAll in main is the
  one cleanup path).
- MANUAL.md updated: example config, field reference, §6 pipeline
  step wording, §9 failure handling description, §11 runtime
  directories block.
2026-05-16 20:32:23 +03:00
Esa Kataja 22b3a73109 fix: refuse destination collisions and fall back on EXDEV in mover
Centralize all move helpers through a single safeRename function that
stats the destination first and refuses to overwrite (bug #14), and
falls back to copy+fsync+remove when os.Rename returns EXDEV across
mounted filesystems (bug #13). Drops the dead MoveToOutput helper.
2026-05-16 20:32:19 +03:00
Esa Kataja 6b72ef03ac fix: add mtime+size stability check and failure quarantine to watcher
Two related hardening fixes for the input folder poller (bug #22):

1. Partial-write protection. A freshly-dropped .mkv now has to present
   the same (mtime, size) on two consecutive ticks before processFn is
   called. A file mid-rsync that grows or has a moving mtime is skipped
   until it settles. The per-file (mtime, size) state lives in
   Watcher.seen and is rebuilt from the current glob each tick so the
   map cannot grow unboundedly.

2. Failure quarantine. When processFn returns an error, the file's
   (failedAt, mtime) is recorded in Watcher.failed and subsequent ticks
   skip it until either the backoff elapses or its mtime changes (user
   replaced or touched it). Previously a permanently-broken input -- e.g.
   a video-only mkv that trips the "no audio streams found" path added
   in the multi-audio fix -- would be retried and logged every 15 s
   forever.

Backoff is 5 minutes: comfortably longer than the 10-30 s polling
interval clamp so we are not effectively retrying every tick, but
short enough that an operator fixing the underlying problem by
replacing the file sees it picked up promptly on the next stable scan.
2026-05-16 20:32:16 +03:00
Esa Kataja 8b38986690 chore: bring MANUAL.md in line with current encoder behavior
§6 step 2: drop the codec-name allowlist, describe the bounded idet run
and the Multi-frame summary parse.

§6 step 6/7: per-stream audio.<n>.wav / audio.<n>.opus now.

§6 step 8/9: zscale is conditional and width rounds to mod-2; -vf is
omitted entirely when no filter applies. Audio language tags are indexed
by output position. TITLE/COLLECTION values are unquoted.

§6 step 11: series filename no longer requires an IMDb mapping.

§7: added a token example and notes about what the token does and does
not influence in the output filename.

§8: noted the known log-rotation limitation (filename frozen at start;
restart to rotate; structured.json does not rotate).
2026-05-16 20:20:05 +03:00
Esa Kataja 244bdce586 add: WebDL / TVRip media types and filename-token override
Source kind is now declared per-file via a `dvd` / `bluray` / `webdl` /
`tvrip` token in the basename (case-insensitive, word-bounded). Matches
the existing `tt…` / `tvm…` filename convention. When no token is
present, falls back to the pixel-count guess in DetectMediaType, which
still only chooses between DVD and Blu-ray.

The parsed media type drives both ORIGINAL_MEDIA_TYPE in the muxed
metadata and the CRF/preset profile used for encoding. EncodingConfig
gains `webdl` and `tvrip` sub-blocks with conservative defaults
(WebDL 30/3, TVRip 32/2); user can override in config.yaml.

Manual updated with the new tokens, config fields, and pipeline
description.
2026-05-16 20:17:10 +03:00
Esa Kataja 022d131cd6 fix: preserve all audio tracks and align language metadata indices
Bug #5: extractAudio invoked ffmpeg without `-map`, so default stream
selection kept only one audio track from sources with multiple audio
streams (e.g. eng/fra/jpn Blu-rays). It now enumerates audio streams
from the streamLangs already fetched by the caller and runs one
`ffmpeg -map 0🅰️<n>` per stream, writing audio.<n>.wav.

Bug #11: the audio language metadata loop in encodeVideo computed the
output index by counting source-side audio streams with a lower Index,
which drifted when some source streams lacked a language tag. It now
walks opusFiles in output order and looks up the language at the
matching source-audio position via a sorted helper.

These ship together because #11 was masked by #5: when only one audio
track survived extraction, the broken index calculation never produced
a visible misalignment. Fixing #5 alone would have caused multi-track
outputs with shuffled language tags; both fixes are required to land
correct multi-track output.
2026-05-16 20:07:59 +03:00
8 changed files with 458 additions and 154 deletions
+89 -34
View File
@@ -48,12 +48,19 @@ encoding:
bluray: bluray:
crf: 29 crf: 29
preset: 3 preset: 3
webdl:
crf: 30
preset: 3
tvrip:
crf: 32
preset: 2
paths: paths:
input: "./input" input: "./input"
output: "./output" output: "./output"
originals: "./originals" originals: "./originals"
failed: "./failed" failed: "./failed"
work: "./work"
``` ```
### Field reference ### Field reference
@@ -65,12 +72,17 @@ paths:
| `encoding.dvd.preset` | SVT-AV1 preset for SD sources | `2` | | `encoding.dvd.preset` | SVT-AV1 preset for SD sources | `2` |
| `encoding.bluray.crf` | SVT-AV1 CRF for HD sources | `29` | | `encoding.bluray.crf` | SVT-AV1 CRF for HD sources | `29` |
| `encoding.bluray.preset` | SVT-AV1 preset for HD sources | `3` | | `encoding.bluray.preset` | SVT-AV1 preset for HD sources | `3` |
| `encoding.webdl.crf` | SVT-AV1 CRF for WebDL sources | `30` |
| `encoding.webdl.preset` | SVT-AV1 preset for WebDL sources | `3` |
| `encoding.tvrip.crf` | SVT-AV1 CRF for TVRip sources | `32` |
| `encoding.tvrip.preset` | SVT-AV1 preset for TVRip sources | `2` |
| `paths.input` | Folder polled for new `.mkv` files | `./input` | | `paths.input` | Folder polled for new `.mkv` files | `./input` |
| `paths.output` | Destination for finished encodes | `./output` | | `paths.output` | Destination for finished encodes | `./output` |
| `paths.originals` | Where source files are moved on success (unless `-d`) | `./originals` | | `paths.originals` | Where source files are moved on success (unless `-d`) | `./originals` |
| `paths.failed` | Where source + partial output go on failure | `./failed` | | `paths.failed` | Where source files go on failure | `./failed` |
| `paths.work` | Scratch directory for per-job intermediates (wav/opus/output.mkv); deleted after every job | `./work` |
All four directories are created on startup if they don't exist. All five directories are created on startup if they don't exist.
--- ---
@@ -88,7 +100,7 @@ Flags:
- `-d` — delete the source `.mkv` after a successful encode instead of moving it to `originals/`. - `-d` — delete the source `.mkv` after a successful encode instead of moving it to `originals/`.
- `-c PATH` — path to config file. - `-c PATH` — path to config file.
Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. A signal triggers a clean shutdown after the current poll cycle. Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. The signal cancels any in-flight encode immediately — the ffmpeg / ffprobe / opusenc children are killed, the per-job work directory is removed by its deferred cleanup, and the source `.mkv` is routed to `paths.failed` so the next run sees a clean `paths.input`.
The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter. The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter.
@@ -131,41 +143,62 @@ the-wire.TVM75.s2e5.mkv
If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an `.nometadata.mkv` suffix. If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an `.nometadata.mkv` suffix.
### 5.4 Optional: source media type
Any filename can additionally contain a media-type token (case-insensitive, word-bounded):
- `dvd`
- `bluray`
- `webdl`
- `tvrip`
Examples:
```
Heat.tt0113277.bluray.mkv
some-rip.tvm169.S01E01.webdl.mkv
old.broadcast.tvrip.tt0066026.mkv
```
The token controls **both** the `ORIGINAL_MEDIA_TYPE` metadata tag written into the output and which `encoding.<type>.crf` / `encoding.<type>.preset` pair is used.
If no token is present, the program falls back to guessing from pixel count: `width × height < 600,000` → DVD, otherwise Blu-ray. WebDL and TVRip are never auto-detected — they must be declared via the token.
--- ---
## 6. The processing pipeline ## 6. The processing pipeline
For every `.mkv` found in `paths.input`, the program runs these steps in order. Any error sends the source file (and the partial `output.mkv`, if any) to `paths.failed`. For every `.mkv` found in `paths.input`, the program runs these steps in order. A per-job scratch subdirectory under `paths.work` (named after the input base name without `.mkv`) holds all intermediates, and is deleted unconditionally at the end of the job. Any error sends the source file to `paths.failed`; the work subdirectory is wiped regardless of outcome.
1. **Parse filename** → determines whether this is a movie or series, and what IDs to use. 1. **Parse filename** → determines whether this is a movie or series, and what IDs to use.
2. **Probe video** with `ffprobe`: 2. **Probe video** with `ffprobe`:
- Picks the first video stream whose codec is `mpeg2video`, `h264`, or `hevc`. - Picks the first stream with `codec_type=video`, regardless of codec name. Errors out if there isn't one.
- Records width, height, and sample aspect ratio (SAR). - Records width, height, and sample aspect ratio (SAR).
- Runs `ffmpeg -vf idet` to detect interlacing (presence of `TFF`/`BFF` in stderr). - Detects interlacing by running `ffmpeg -vf idet -frames:v 400 -an -sn -f null -` and parsing the `Multi frame detection: TFF: a BFF: b Progressive: c Undetermined: d` summary line. The source is treated as interlaced only when `a+b > c`; undetermined frames are ignored, and a missing summary line defaults to progressive.
3. **Probe stream languages** with `ffprobe` — collects `language` tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's `-map_metadata -1` strips them otherwise). 3. **Probe stream languages** with `ffprobe` — collects `language` tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's `-map_metadata -1` strips them otherwise).
4. **Detect media type** from pixel count: 4. **Detect media type**:
- `width × height < 600,000`**DVD** profile. - First, check the filename for a `dvd` / `bluray` / `webdl` / `tvrip` token (case-insensitive). If present, that wins.
- Otherwise**Blu-ray** profile. - Otherwise, fall back to pixel count: `width × height < 600,000` → DVD, else Blu-ray.
The chosen profile selects which CRF/preset pair from the config to use. The chosen profile selects which `encoding.<type>.crf` / `encoding.<type>.preset` pair from the config to use, and is written into the `ORIGINAL_MEDIA_TYPE` metadata tag.
5. **Fetch metadata** from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata. 5. **Fetch metadata** from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata.
6. **Extract audio** to `audio.wav` (PCM s16le, 48 kHz) in the input directory. 6. **Extract audio** — one PCM wav per source audio stream, written to the per-job work directory as `audio.0.wav`, `audio.1.wav`, … in source order (PCM s16le, 48 kHz). Errors out if the source has no audio streams.
7. **Encode audio** with `opusenc --bitrate 128k` `audio.opus`. 7. **Encode audio** — each wav is converted with `opusenc --bitrate 128k` to a matching `audio.<n>.opus` in the same work directory.
8. **Calculate display width** from SAR. If SAR ≠ `1:1`, the width is rescaled so the output has square pixels, using a `zscale` filter (`spline36`). Height is preserved. 8. **Calculate display width** from SAR. The width is rescaled so the output has square pixels only when there's actually work to do — `zscale` is skipped entirely for square-pixel sources (SAR `1:1`, `N/A`, empty, `0:N`), and for any SAR whose calculated width rounds to the source width. When rescaling, the width is rounded to the nearest even number (mod-2, preferred by AV1).
9. **Encode video** with FFmpeg: 9. **Encode video** with FFmpeg:
- Video filter chain: `bwdif=mode=0:par=-1:-1,zscale=w=W:h=H:filter=spline36` if interlaced, else just the `zscale` step. - Video filter chain is built conditionally. `bwdif=mode=0:par=-1:-1` is prepended when the source is interlaced; the `zscale` step is appended only when a rescale is actually needed (see step 8). If neither applies, `-vf` is omitted entirely.
- Codec: `libsvtav1`, `-pix_fmt yuv420p10le`. - Codec: `libsvtav1`, `-pix_fmt yuv420p10le`.
- `-crf` and `-preset` from the selected profile. - `-crf` and `-preset` from the selected profile (step 4).
- `-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s`. - `-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s`.
- Streams mapped: video from input, subtitles from input (`0:s?` — optional), audio from the re-encoded Opus files. - Streams mapped: video from input (`0:v`), subtitles from input (`0:s?` — optional), and one audio stream per Opus file (`1:a`, `2:a`, …).
- Audio copied (`-c:a copy`), subtitles copied (`-c:s copy`). - Audio re-muxed (`-c:a copy` — copies the already-Opus-encoded streams), subtitles copied (`-c:s copy`).
- `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied from the ffprobe pass. - `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied. Audio language tags are indexed by **output position** (the position in the opus-file list), not by counting source streams, so missing-language streams don't shift the index.
- Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`. - Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`. Values are unquoted (literal value, no wrapping `"…"`).
- Output written to `output.mkv` in the input directory. - Output written to `output.mkv` in the per-job work directory.
10. **Clean up** intermediate `.wav` and `.opus` files. 10. **Clean up** the entire per-job work directory (intermediates and `output.mkv` together) once the rename/move below succeeds — or, on any failure, when the deferred cleanup runs.
11. **Rename and move** `output.mkv` to `paths.output` with a final name: 11. **Rename and move** `output.mkv` from the work directory to `paths.output` with a final name:
- Series → `<sanitized-show-name>.S<NN>E<NN>.mkv` - Series with a known show name `<sanitized-show-name>.S<NN>E<NN>.mkv` (the series no longer needs a populated IMDb mapping — a TVmaze show with no external IMDb link still gets a useful filename).
- Movie → `<sanitized-title>.<imdbID>.mkv` - Movie with a known title and IMDb ID `<sanitized-title>.<imdbID>.mkv`.
- No metadata`<8-hex-chars>.nometadata.mkv` - Anything else`<8-hex-chars>.nometadata.mkv`.
Sanitization keeps `az AZ 09 - ä ö Ä Ö` only. Sanitization keeps `az AZ 09 - ä ö Ä Ö` only.
12. **Dispose of the source**: 12. **Dispose of the source**:
- `-d` flag set → delete the original `.mkv`. - `-d` flag set → delete the original `.mkv`.
@@ -175,13 +208,18 @@ For every `.mkv` found in `paths.input`, the program runs these steps in order.
## 7. Output naming examples ## 7. Output naming examples
| Input filename | Result in `paths.output` | | Input filename | Result in `paths.output` | Notes |
|---|---| |---|---|---|
| `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` | | `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` | Pixel-count fallback → Blu-ray profile + tag |
| `Breaking.Bad.tvm169.S01E01.mkv` | `BreakingBad.S01E01.mkv` | | `Heat.tt0113277.bluray.mkv` | `Heat.tt0113277.mkv` | Same output filename; muxed `ORIGINAL_MEDIA_TYPE=Blu-ray` is now from the token, not the guess |
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` | | `Breaking.Bad.tvm169.S01E01.webdl.mkv` | `BreakingBad.S01E01.mkv` | WebDL profile + tag |
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` | No IDs at all |
The IMDb ID returned by the API is used, not the one from the filename, so a typo in the filename would surface there. A few things worth knowing:
- The media-type token affects the muxed `ORIGINAL_MEDIA_TYPE` tag and the CRF/preset profile, but **not** the output filename.
- The IMDb ID written into the filename is the one returned by the API, not the one in the input filename, so a typo in the source filename will surface in the output name.
- A TVmaze show with no IMDb mapping still gets a `<Collection>.S<NN>E<NN>.mkv` filename (only the `IMDBID` metadata tag is left empty).
--- ---
@@ -197,18 +235,24 @@ Run the program from the directory where you want the logs to land.
Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command, etc.). These are intentional but not written to the log files. Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command, etc.). These are intentional but not written to the log files.
**Known limitation:** the date in `info_*.log` / `error_*.log` filenames is computed when the daemon starts and does not roll over at midnight. If the daemon is left running across days, all writes continue into the start-day's file. Restart the daemon to rotate. `structured.json` does not rotate at all.
--- ---
## 9. Failure handling ## 9. Failure handling
If any step from probing through encoding through renaming fails: If any step from probing through encoding through renaming fails:
- The source `.mkv` is moved to `paths.failed`. - The source `.mkv` is moved to `paths.failed` (the move itself is `os.Stat`-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves `paths.input` on every failure path, so the watcher doesn't retry the same file on the next tick.
- Any partial `output.mkv` left in the input directory is also moved to `paths.failed`. - The per-job work directory under `paths.work` (containing partial wav/opus/output.mkv) is deleted unconditionally.
- The error is logged to `error_*.log` and `structured.json` with the source file path. - The error is logged to `error_*.log` and `structured.json` with the source file path.
The watcher continues with the next file; one bad rip won't stop the daemon. The watcher continues with the next file; one bad rip won't stop the daemon.
**Destination collisions:** if a finished encode would land on a name that already exists in `paths.output`, the move is refused (no silent overwrite) and the source is routed to `paths.failed`. This is the path you'll hit when two sources sanitize to the same output filename — e.g. two re-rips of the same release, or two episodes that both come out as `SxxExx`.
**Cross-filesystem moves:** every move (`paths.input → paths.output`, `… → paths.failed`, `… → paths.originals`) transparently falls back to copy + delete when the source and destination live on different mounts. You can put each path on a different drive without breaking the pipeline.
--- ---
## 10. Polling behavior ## 10. Polling behavior
@@ -216,7 +260,8 @@ The watcher continues with the next file; one bad rip won't stop the daemon.
- Input directory is scanned every **15 seconds** (clamped to a 1030 s range). - Input directory is scanned every **15 seconds** (clamped to a 1030 s range).
- Only files matching `*.mkv` directly in `paths.input` are picked up — no recursion. - Only files matching `*.mkv` directly in `paths.input` are picked up — no recursion.
- Files are processed **sequentially**, one at a time, in the order `filepath.Glob` returns them (alphabetical on Linux). - Files are processed **sequentially**, one at a time, in the order `filepath.Glob` returns them (alphabetical on Linux).
- There is no atomic-write detection. If you're copying a large file into `paths.input`, copy it to a different name first and `mv` it into place once complete, otherwise the watcher may try to encode a half-written file. - **Partial-write protection:** the watcher requires a file's `mtime` and `size` to be identical on two consecutive scans before processing. A freshly dropped or still-copying file therefore waits at least one full tick (~15 s) before encoding begins. Copying a large file directly into `paths.input` is now safe; you no longer have to land it under a different name and `mv` into place (though doing so still works and shaves off the tick delay).
- **Failure quarantine:** if `processFile` returns an error, that file is skipped for 5 minutes (or until its `mtime` changes — e.g. you replace or `touch` it). Prevents a permanently-broken input from spamming the logs every 15 s. The quarantine is in-memory only; restarting the daemon clears it.
--- ---
@@ -232,3 +277,13 @@ internal/mover/ File rename/move/delete helpers
internal/logger/ Plain + JSON logging internal/logger/ Plain + JSON logging
pkg/types/types.go Shared structs (Config, Job, Metadata, …) pkg/types/types.go Shared structs (Config, Job, Metadata, …)
``` ```
Runtime directories (from `paths.*` in the config):
```
paths.input User drops new .mkv files here; watcher polls every 15 s
paths.output Finished encodes land here under their final name
paths.originals Successfully-encoded sources end up here (unless -d)
paths.failed Sources of failed jobs end up here
paths.work Per-job scratch subdir (basename of input); wiped per job
```
+67 -30
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"context"
"crypto/rand" "crypto/rand"
"encoding/hex" "encoding/hex"
"flag" "flag"
@@ -9,6 +10,7 @@ import (
"os/signal" "os/signal"
"path/filepath" "path/filepath"
"regexp" "regexp"
"strings"
"syscall" "syscall"
"videnc-vibe/internal/config" "videnc-vibe/internal/config"
@@ -60,57 +62,75 @@ func main() {
metaClient := metadata.NewClient(cfg.OMDBAPIKey) metaClient := metadata.NewClient(cfg.OMDBAPIKey)
done := make(chan struct{}) ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
sigChan := make(chan os.Signal, 1) defer cancel()
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
go func() {
<-sigChan
close(done)
}()
w := watcher.New(cfg.Paths.Input, 15) w := watcher.New(cfg.Paths.Input, 15)
w.Start(func(inputPath string) error { w.Start(ctx, func(ctx context.Context, inputPath string) error {
return processFile(inputPath, cfg, enc, metaClient, log) return processFile(ctx, inputPath, cfg, enc, metaClient, log)
}, done) })
log.Info("videnc-vibe started") log.Info("videnc-vibe started")
} }
func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger) error { func processFile(ctx context.Context, inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger) error {
log.Info(fmt.Sprintf("Processing: %s", inputPath)) log.Info(fmt.Sprintf("Processing: %s", inputPath))
filename := filepath.Base(inputPath) filename := filepath.Base(inputPath)
isSeries, imdbID, tvmazeID, season, episode := metadata.ParseFilename(filename) isSeries, imdbID, tvmazeID, season, episode := metadata.ParseFilename(filename)
width, height, interlaced, err := enc.GetMediaInfo(inputPath) // Per-job work subdirectory under paths.work. Uses the source base name
// (without ".mkv") as the subdir name. MkdirAll is idempotent, so a
// leftover dir from a crashed previous run is harmless to overwrite.
workDirName := strings.TrimSuffix(filename, filepath.Ext(filename))
workDir := filepath.Join(cfg.Paths.Work, workDirName)
if err := os.MkdirAll(workDir, 0755); err != nil {
log.ErrorFile(inputPath, "Creating work directory", err.Error())
failToFailed(inputPath, cfg.Paths.Failed, log)
return err
}
// Cleanup the work directory on every exit path, success or failure.
defer os.RemoveAll(workDir)
width, height, interlaced, err := enc.GetMediaInfo(ctx, inputPath)
if err != nil { if err != nil {
log.ErrorFile(inputPath, "Getting media info", err.Error()) log.ErrorFile(inputPath, "Getting media info", err.Error())
mover.MoveToFailed(inputPath, cfg.Paths.Failed) failToFailed(inputPath, cfg.Paths.Failed, log)
return err return err
} }
streamLangs, err := enc.GetStreamLanguages(inputPath) streamLangs, err := enc.GetStreamLanguages(ctx, inputPath)
if err != nil { if err != nil {
log.ErrorFile(inputPath, "Getting stream languages", err.Error()) log.ErrorFile(inputPath, "Getting stream languages", err.Error())
} }
mediaType := watcher.DetectMediaType(width, height) mediaType := metadata.ParseMediaType(filename)
crf := cfg.Encoding.DVD.CRF if mediaType == "" {
preset := cfg.Encoding.DVD.Preset mediaType = watcher.DetectMediaType(width, height)
if mediaType == types.MediaTypeBluRay {
crf = cfg.Encoding.Bluray.CRF
preset = cfg.Encoding.Bluray.Preset
} }
var profile types.EncodingParams
switch mediaType {
case types.MediaTypeBluRay:
profile = cfg.Encoding.Bluray
case types.MediaTypeWebDL:
profile = cfg.Encoding.WebDL
case types.MediaTypeTVRip:
profile = cfg.Encoding.TVRip
default:
profile = cfg.Encoding.DVD
}
crf := profile.CRF
preset := profile.Preset
var meta *types.Metadata var meta *types.Metadata
if isSeries && tvmazeID != "" && season != "" && episode != "" { if isSeries && tvmazeID != "" && season != "" && episode != "" {
meta, err = metaClient.FetchSeriesMetadata(tvmazeID, season, episode) meta, err = metaClient.FetchSeriesMetadata(ctx, tvmazeID, season, episode)
if err != nil { if err != nil {
log.ErrorFile(inputPath, "Fetching series metadata", err.Error()) log.ErrorFile(inputPath, "Fetching series metadata", err.Error())
} }
} else if imdbID != "" { } else if imdbID != "" {
meta, err = metaClient.FetchMovieMetadata(imdbID) meta, err = metaClient.FetchMovieMetadata(ctx, imdbID)
if err != nil { if err != nil {
log.ErrorFile(inputPath, "Fetching movie metadata", err.Error()) log.ErrorFile(inputPath, "Fetching movie metadata", err.Error())
} }
@@ -134,16 +154,13 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
DeleteOrigin: deleteOrigin, DeleteOrigin: deleteOrigin,
} }
if err := enc.Transcode(inputPath, job, meta, interlaced, streamLangs); err != nil { if err := enc.Transcode(ctx, inputPath, workDir, job, meta, interlaced, streamLangs); err != nil {
log.ErrorFile(inputPath, "Transcoding", err.Error()) log.ErrorFile(inputPath, "Transcoding", err.Error())
mover.MoveToFailed(inputPath, cfg.Paths.Failed) failToFailed(inputPath, cfg.Paths.Failed, log)
outputPath := filepath.Join(filepath.Dir(inputPath), "output.mkv")
mover.MoveToFailed(outputPath, cfg.Paths.Failed)
return err return err
} }
outputDir := filepath.Dir(inputPath) outputPath := filepath.Join(workDir, "output.mkv")
outputPath := filepath.Join(outputDir, "output.mkv")
var outFilename string var outFilename string
switch { switch {
@@ -157,8 +174,11 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
finalOutput := filepath.Join(cfg.Paths.Output, outFilename) finalOutput := filepath.Join(cfg.Paths.Output, outFilename)
if err := mover.Rename(outputPath, finalOutput); err != nil { if err := mover.Rename(outputPath, finalOutput); err != nil {
// Move the source to failed/ so it doesn't get re-encoded on the next
// tick. The deferred RemoveAll(workDir) takes care of the partial
// output.mkv left in the work dir.
log.ErrorFile(inputPath, "Moving output", err.Error()) log.ErrorFile(inputPath, "Moving output", err.Error())
mover.MoveToFailed(outputPath, cfg.Paths.Failed) failToFailed(inputPath, cfg.Paths.Failed, log)
return err return err
} }
@@ -174,6 +194,23 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
return nil return nil
} }
// failToFailed stats path before invoking MoveToFailed: if missing, logs and
// skips; if present, surfaces any move error to the logger. This avoids the
// silent no-op pattern where MoveToFailed was called on a non-existent file.
func failToFailed(path, failedDir string, log *logger.Logger) {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
log.Info(fmt.Sprintf("Skip move to failed; not present: %s", path))
return
}
log.ErrorFile(path, "Stat before move to failed", err.Error())
return
}
if err := mover.MoveToFailed(path, failedDir); err != nil {
log.ErrorFile(path, "Moving to failed", err.Error())
}
}
func generateRandomString(length int) string { func generateRandomString(length int) string {
bytes := make([]byte, length/2+1) bytes := make([]byte, length/2+1)
rand.Read(bytes) rand.Read(bytes)
+16 -1
View File
@@ -49,6 +49,9 @@ func Load(configPath string) (*types.Config, error) {
if cfg.Paths.Failed == "" { if cfg.Paths.Failed == "" {
cfg.Paths.Failed = "./failed" cfg.Paths.Failed = "./failed"
} }
if cfg.Paths.Work == "" {
cfg.Paths.Work = "./work"
}
if cfg.Encoding.DVD.CRF == 0 { if cfg.Encoding.DVD.CRF == 0 {
cfg.Encoding.DVD.CRF = 30 cfg.Encoding.DVD.CRF = 30
} }
@@ -61,12 +64,24 @@ func Load(configPath string) (*types.Config, error) {
if cfg.Encoding.Bluray.Preset == 0 { if cfg.Encoding.Bluray.Preset == 0 {
cfg.Encoding.Bluray.Preset = 3 cfg.Encoding.Bluray.Preset = 3
} }
if cfg.Encoding.WebDL.CRF == 0 {
cfg.Encoding.WebDL.CRF = 30
}
if cfg.Encoding.WebDL.Preset == 0 {
cfg.Encoding.WebDL.Preset = 3
}
if cfg.Encoding.TVRip.CRF == 0 {
cfg.Encoding.TVRip.CRF = 32
}
if cfg.Encoding.TVRip.Preset == 0 {
cfg.Encoding.TVRip.Preset = 2
}
return &cfg, nil return &cfg, nil
} }
func EnsureDirs(cfg *types.Config) error { func EnsureDirs(cfg *types.Config) error {
dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed} dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed, cfg.Paths.Work}
for _, dir := range dirs { for _, dir := range dirs {
if err := os.MkdirAll(dir, 0755); err != nil { if err := os.MkdirAll(dir, 0755); err != nil {
return fmt.Errorf("creating directory %s: %w", dir, err) return fmt.Errorf("creating directory %s: %w", dir, err)
+71 -55
View File
@@ -1,12 +1,13 @@
package encoder package encoder
import ( import (
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"regexp" "regexp"
"sort"
"strconv" "strconv"
"strings" "strings"
@@ -42,8 +43,8 @@ type StreamMetadata struct {
Title string `json:"title"` Title string `json:"title"`
} }
func (e *Encoder) GetStreamLanguages(path string) ([]StreamMetadata, error) { func (e *Encoder) GetStreamLanguages(ctx context.Context, path string) ([]StreamMetadata, error) {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path) cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
output, err := cmd.CombinedOutput() output, err := cmd.CombinedOutput()
if err != nil { if err != nil {
return nil, fmt.Errorf("ffprobe error: %w", err) return nil, fmt.Errorf("ffprobe error: %w", err)
@@ -103,8 +104,8 @@ func (e *Encoder) CheckDeps() error {
return nil return nil
} }
func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool, err error) { func (e *Encoder) GetMediaInfo(ctx context.Context, path string) (width, height int, interlaced bool, err error) {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path) cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
output, err := cmd.CombinedOutput() output, err := cmd.CombinedOutput()
if err != nil { if err != nil {
return 0, 0, false, fmt.Errorf("ffprobe error: %w", err) return 0, 0, false, fmt.Errorf("ffprobe error: %w", err)
@@ -131,7 +132,7 @@ func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool,
return 0, 0, false, fmt.Errorf("no video stream found") return 0, 0, false, fmt.Errorf("no video stream found")
} }
cmd = exec.Command(e.ffmpegPath, cmd = exec.CommandContext(ctx, e.ffmpegPath,
"-hide_banner", "-hide_banner",
"-nostats", "-nostats",
"-i", path, "-i", path,
@@ -168,8 +169,8 @@ func detectInterlaced(idetOutput string) bool {
return interlaced > prog return interlaced > prog
} }
func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string, int, error) { func (e *Encoder) calculateZscaleWidth(ctx context.Context, path string, originalHeight int) (string, int, error) {
cmd := exec.Command(e.ffprobePath, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path) cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path)
output, err := cmd.CombinedOutput() output, err := cmd.CombinedOutput()
if err != nil { if err != nil {
return "", 0, fmt.Errorf("ffprobe error: %w", err) return "", 0, fmt.Errorf("ffprobe error: %w", err)
@@ -221,59 +222,79 @@ func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string,
return zscale, newWidth, nil return zscale, newWidth, nil
} }
func (e *Encoder) Transcode(input string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error { func (e *Encoder) Transcode(ctx context.Context, input, workDir string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
dir := filepath.Dir(input) audioWavs, err := e.extractAudio(ctx, input, workDir, streamLangs)
audioWavs, err := e.extractAudio(input, dir)
if err != nil { if err != nil {
return fmt.Errorf("extracting audio: %w", err) return fmt.Errorf("extracting audio: %w", err)
} }
defer e.cleanupWavs(audioWavs)
opusFiles, err := e.encodeOpus(audioWavs, dir) opusFiles, err := e.encodeOpus(ctx, audioWavs, workDir)
if err != nil { if err != nil {
return fmt.Errorf("encoding opus: %w", err) return fmt.Errorf("encoding opus: %w", err)
} }
defer e.cleanupOpus(opusFiles)
if err := e.encodeVideo(input, opusFiles, job, metadata, interlaced, streamLangs); err != nil { if err := e.encodeVideo(ctx, input, workDir, opusFiles, job, metadata, interlaced, streamLangs); err != nil {
return fmt.Errorf("encoding video: %w", err) return fmt.Errorf("encoding video: %w", err)
} }
return nil return nil
} }
func (e *Encoder) extractAudio(input, dir string) ([]string, error) { // audioStreamsInSourceOrder returns the audio entries of streamLangs sorted by
cmd := exec.Command(e.ffmpegPath, "-i", input, // their source-side Index. The resulting slice position is the 0-based audio
// stream index used by ffmpeg selectors like `0:a:<n>`.
func audioStreamsInSourceOrder(streamLangs []StreamMetadata) []StreamMetadata {
var audio []StreamMetadata
for _, s := range streamLangs {
if s.CodecType == "audio" {
audio = append(audio, s)
}
}
sort.Slice(audio, func(i, j int) bool { return audio[i].Index < audio[j].Index })
return audio
}
func (e *Encoder) extractAudio(ctx context.Context, input, workDir string, streamLangs []StreamMetadata) ([]string, error) {
audio := audioStreamsInSourceOrder(streamLangs)
if len(audio) == 0 {
return nil, fmt.Errorf("no audio streams found")
}
wavs := make([]string, 0, len(audio))
for srcAudioIndex := range audio {
wavPath := filepath.Join(workDir, fmt.Sprintf("audio.%d.wav", srcAudioIndex))
cmd := exec.CommandContext(ctx, e.ffmpegPath, "-i", input,
"-map", fmt.Sprintf("0:a:%d", srcAudioIndex),
"-vn", "-c:a", "pcm_s16le", "-ar", "48000", "-vn", "-c:a", "pcm_s16le", "-ar", "48000",
"-f", "wav", filepath.Join(dir, "audio.wav")) "-f", "wav", wavPath)
if out, err := cmd.CombinedOutput(); err != nil { if out, err := cmd.CombinedOutput(); err != nil {
return nil, fmt.Errorf("ffmpeg extract: %s %w", out, err) return wavs, fmt.Errorf("ffmpeg extract a:%d: %s %w", srcAudioIndex, out, err)
}
wavs = append(wavs, wavPath)
}
return wavs, nil
} }
matches, _ := filepath.Glob(filepath.Join(dir, "audio.wav")) func (e *Encoder) encodeOpus(ctx context.Context, wavs []string, workDir string) ([]string, error) {
return matches, nil
}
func (e *Encoder) encodeOpus(wavs []string, dir string) ([]string, error) {
var opusFiles []string var opusFiles []string
for _, wav := range wavs { for _, wav := range wavs {
out := strings.Replace(wav, ".wav", ".opus", 1) base := filepath.Base(strings.Replace(wav, ".wav", ".opus", 1))
cmd := exec.Command(e.opusencPath, "--bitrate", "128k", wav, out) out := filepath.Join(workDir, base)
if out, err := cmd.CombinedOutput(); err != nil { cmd := exec.CommandContext(ctx, e.opusencPath, "--bitrate", "128k", wav, out)
return nil, fmt.Errorf("opusenc: %s %w", out, err) if logOut, err := cmd.CombinedOutput(); err != nil {
return nil, fmt.Errorf("opusenc: %s %w", logOut, err)
} }
opusFiles = append(opusFiles, out) opusFiles = append(opusFiles, out)
} }
return opusFiles, nil return opusFiles, nil
} }
func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error { func (e *Encoder) encodeVideo(ctx context.Context, input, workDir string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
dir := filepath.Dir(input) outFile := filepath.Join(workDir, "output.mkv")
outFile := filepath.Join(dir, "output.mkv")
svtParams := fmt.Sprintf("film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s") svtParams := fmt.Sprintf("film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s")
zscaleStr, newWidth, err := e.calculateZscaleWidth(input, 0) zscaleStr, newWidth, err := e.calculateZscaleWidth(ctx, input, 0)
if err != nil { if err != nil {
return fmt.Errorf("calculating zscale: %w", err) return fmt.Errorf("calculating zscale: %w", err)
} }
@@ -319,17 +340,24 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
args = append(args, "-c:s", "copy") args = append(args, "-c:s", "copy")
args = append(args, "-map_metadata", "-1") args = append(args, "-map_metadata", "-1")
// Walk opusFiles by output audio index (0..N-1) and look up the source
// audio language at the same source-audio-index position. This keeps the
// `-metadata:s:a:i` index aligned with the output stream order, regardless
// of whether some source streams lacked a language tag.
audioStreams := audioStreamsInSourceOrder(streamLangs)
for i := range opusFiles {
if i >= len(audioStreams) {
break
}
lang := audioStreams[i].Language
if lang == "" {
continue
}
args = append(args, fmt.Sprintf("-metadata:s:a:%d", i), fmt.Sprintf("language=%s", lang))
fmt.Printf("DEBUG setting audio language: stream %d -> language=%s\n", i, lang)
}
for _, stream := range streamLangs { for _, stream := range streamLangs {
if stream.CodecType == "audio" && stream.Language != "" {
idx := 1
for _, s := range streamLangs {
if s.CodecType == "audio" && s.Index < stream.Index {
idx++
}
}
args = append(args, fmt.Sprintf("-metadata:s:a:%d", idx-1), fmt.Sprintf("language=%s", stream.Language))
fmt.Printf("DEBUG setting audio language: stream %d -> language=%s\n", idx-1, stream.Language)
}
if stream.CodecType == "subtitle" && stream.Language != "" { if stream.CodecType == "subtitle" && stream.Language != "" {
idx := 0 idx := 0
for _, s := range streamLangs { for _, s := range streamLangs {
@@ -359,7 +387,7 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
args = append(args, outFile) args = append(args, outFile)
args = append([]string{"-hide_banner", "-v", "error"}, args...) args = append([]string{"-hide_banner", "-v", "error"}, args...)
cmd := exec.Command(e.ffmpegPath, args...) cmd := exec.CommandContext(ctx, e.ffmpegPath, args...)
fmt.Printf("DEBUG FFmpeg command: ffmpeg %s\n", strings.Join(args, " ")) fmt.Printf("DEBUG FFmpeg command: ffmpeg %s\n", strings.Join(args, " "))
if out, err := cmd.CombinedOutput(); err != nil { if out, err := cmd.CombinedOutput(); err != nil {
return fmt.Errorf("ffmpeg encode: %s %w", out, err) return fmt.Errorf("ffmpeg encode: %s %w", out, err)
@@ -367,15 +395,3 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
return nil return nil
} }
func (e *Encoder) cleanupWavs(files []string) {
for _, f := range files {
os.Remove(f)
}
}
func (e *Encoder) cleanupOpus(files []string) {
for _, f := range files {
os.Remove(f)
}
}
+39 -7
View File
@@ -1,6 +1,7 @@
package metadata package metadata
import ( import (
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
@@ -22,6 +23,7 @@ var (
imdbRegex = regexp.MustCompile(`tt(\d+)`) imdbRegex = regexp.MustCompile(`tt(\d+)`)
tvmRegex = regexp.MustCompile(`(?i)tvm(\d+)`) tvmRegex = regexp.MustCompile(`(?i)tvm(\d+)`)
seRegex = regexp.MustCompile(`(?i)s(\d+)e(\d+)`) seRegex = regexp.MustCompile(`(?i)s(\d+)e(\d+)`)
mediaTypeRegex = regexp.MustCompile(`(?i)\b(dvd|bluray|webdl|tvrip)\b`)
) )
type OMDbResponse struct { type OMDbResponse struct {
@@ -86,10 +88,36 @@ func parseInt(s string) int {
return n return n
} }
func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) { // ParseMediaType looks for a `.dvd.` / `.bluray.` / `.webdl.` / `.tvrip.`
// token (case-insensitive) in the filename and returns the matching MediaType.
// Returns an empty MediaType if no token is found, so the caller can fall back
// to the pixel-count heuristic.
func ParseMediaType(filename string) types.MediaType {
m := mediaTypeRegex.FindStringSubmatch(strings.TrimSuffix(filename, ".mkv"))
if m == nil {
return ""
}
switch strings.ToLower(m[1]) {
case "dvd":
return types.MediaTypeDVD
case "bluray":
return types.MediaTypeBluRay
case "webdl":
return types.MediaTypeWebDL
case "tvrip":
return types.MediaTypeTVRip
}
return ""
}
func (c *Client) FetchMovieMetadata(ctx context.Context, imdbID string) (*types.Metadata, error) {
url := fmt.Sprintf("%s?i=%s&apikey=%s", omdbAPIURL, imdbID, c.omdbAPIKey) url := fmt.Sprintf("%s?i=%s&apikey=%s", omdbAPIURL, imdbID, c.omdbAPIKey)
resp, err := c.httpClient.Get(url) req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, fmt.Errorf("building OMDb request: %w", err)
}
resp, err := c.httpClient.Do(req)
if err != nil { if err != nil {
return nil, fmt.Errorf("fetching OMDb: %w", err) return nil, fmt.Errorf("fetching OMDb: %w", err)
} }
@@ -130,7 +158,7 @@ func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) {
}, nil }, nil
} }
func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.Metadata, error) { func (c *Client) FetchSeriesMetadata(ctx context.Context, tvmazeID, season, episode string) (*types.Metadata, error) {
seasonNum, err := strconv.Atoi(season) seasonNum, err := strconv.Atoi(season)
if err != nil { if err != nil {
return nil, fmt.Errorf("parsing season %q: %w", season, err) return nil, fmt.Errorf("parsing season %q: %w", season, err)
@@ -142,13 +170,13 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
epURL := fmt.Sprintf("%s/shows/%s/episodebynumber?season=%d&number=%d", tvmazeAPIURL, tvmazeID, seasonNum, episodeNum) epURL := fmt.Sprintf("%s/shows/%s/episodebynumber?season=%d&number=%d", tvmazeAPIURL, tvmazeID, seasonNum, episodeNum)
var ep TVMazeEpisode var ep TVMazeEpisode
if err := c.fetchTVMazeJSON(epURL, &ep); err != nil { if err := c.fetchTVMazeJSON(ctx, epURL, &ep); err != nil {
return nil, fmt.Errorf("TVmaze episode %s S%dE%d: %w", tvmazeID, seasonNum, episodeNum, err) return nil, fmt.Errorf("TVmaze episode %s S%dE%d: %w", tvmazeID, seasonNum, episodeNum, err)
} }
showURL := fmt.Sprintf("%s/shows/%s", tvmazeAPIURL, tvmazeID) showURL := fmt.Sprintf("%s/shows/%s", tvmazeAPIURL, tvmazeID)
var show TVMazeShowResponse var show TVMazeShowResponse
if err := c.fetchTVMazeJSON(showURL, &show); err != nil { if err := c.fetchTVMazeJSON(ctx, showURL, &show); err != nil {
return nil, fmt.Errorf("TVmaze show %s: %w", tvmazeID, err) return nil, fmt.Errorf("TVmaze show %s: %w", tvmazeID, err)
} }
@@ -169,8 +197,12 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
}, nil }, nil
} }
func (c *Client) fetchTVMazeJSON(url string, v interface{}) error { func (c *Client) fetchTVMazeJSON(ctx context.Context, url string, v interface{}) error {
resp, err := c.httpClient.Get(url) req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return fmt.Errorf("build request: %w", err)
}
resp, err := c.httpClient.Do(req)
if err != nil { if err != nil {
return fmt.Errorf("GET: %w", err) return fmt.Errorf("GET: %w", err)
} }
+69 -13
View File
@@ -1,36 +1,92 @@
package mover package mover
import ( import (
"errors"
"fmt" "fmt"
"io"
"os" "os"
"path/filepath" "path/filepath"
"syscall"
) )
func MoveToOutput(input, outputDir string) error { // safeRename moves src to dst. Refuses to overwrite an existing dst.
if err := os.MkdirAll(outputDir, 0755); err != nil { // Falls back to copy+remove on EXDEV (cross-device) errors.
return fmt.Errorf("creating output dir: %w", err) func safeRename(src, dst string) error {
if _, err := os.Stat(dst); err == nil {
return fmt.Errorf("destination %s already exists", dst)
} else if !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("stat destination %s: %w", dst, err)
} }
filename := filepath.Base(input)
dest := filepath.Join(outputDir, filename) err := os.Rename(src, dst)
return os.Rename(input, dest) if err == nil {
return nil
}
var linkErr *os.LinkError
if !errors.As(err, &linkErr) || !errors.Is(err, syscall.EXDEV) {
return fmt.Errorf("renaming %s to %s: %w", src, dst, err)
}
return copyAndRemove(src, dst)
}
// copyAndRemove implements the cross-device fallback: copy src to dst, fsync,
// then remove src. Uses O_EXCL to belt-and-suspenders against a race.
func copyAndRemove(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("opening source %s: %w", src, err)
}
defer in.Close()
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0644)
if err != nil {
return fmt.Errorf("creating destination %s: %w", dst, err)
}
if _, err := io.Copy(out, in); err != nil {
out.Close()
os.Remove(dst)
return fmt.Errorf("copying %s to %s: %w", src, dst, err)
}
if err := out.Sync(); err != nil {
out.Close()
os.Remove(dst)
return fmt.Errorf("syncing destination %s: %w", dst, err)
}
if err := out.Close(); err != nil {
os.Remove(dst)
return fmt.Errorf("closing destination %s: %w", dst, err)
}
if err := in.Close(); err != nil {
return fmt.Errorf("closing source %s: %w", src, err)
}
if err := os.Remove(src); err != nil {
return fmt.Errorf("removing source %s: %w", src, err)
}
return nil
} }
func MoveToFailed(input, failedDir string) error { func MoveToFailed(input, failedDir string) error {
if err := os.MkdirAll(failedDir, 0755); err != nil { if err := os.MkdirAll(failedDir, 0755); err != nil {
return fmt.Errorf("creating failed dir: %w", err) return fmt.Errorf("creating failed dir: %w", err)
} }
filename := filepath.Base(input) dest := filepath.Join(failedDir, filepath.Base(input))
dest := filepath.Join(failedDir, filename) return safeRename(input, dest)
return os.Rename(input, dest)
} }
func MoveToOriginals(input, originalsDir string) error { func MoveToOriginals(input, originalsDir string) error {
if err := os.MkdirAll(originalsDir, 0755); err != nil { if err := os.MkdirAll(originalsDir, 0755); err != nil {
return fmt.Errorf("creating originals dir: %w", err) return fmt.Errorf("creating originals dir: %w", err)
} }
filename := filepath.Base(input) dest := filepath.Join(originalsDir, filepath.Base(input))
dest := filepath.Join(originalsDir, filename) return safeRename(input, dest)
return os.Rename(input, dest)
} }
func Delete(path string) error { func Delete(path string) error {
@@ -38,5 +94,5 @@ func Delete(path string) error {
} }
func Rename(source, dest string) error { func Rename(source, dest string) error {
return os.Rename(source, dest) return safeRename(source, dest)
} }
+95 -7
View File
@@ -1,16 +1,57 @@
package watcher package watcher
import ( import (
"context"
"fmt" "fmt"
"os"
"path/filepath" "path/filepath"
"time" "time"
"videnc-vibe/pkg/types" "videnc-vibe/pkg/types"
) )
// failureBackoff is how long we skip a file after processFn returned an error
// for it, provided the file has not been touched since (mtime unchanged). Chosen
// to be much larger than the polling interval so a permanently-broken input
// (e.g. a video-only file producing "no audio streams found") does not spam the
// log every tick, but short enough that an operator who fixes the underlying
// problem by replacing the file sees it picked up promptly on the next stable
// scan.
const failureBackoff = 5 * time.Minute
// inputExtensions lists the source container extensions the watcher will pick
// up. Globbed case-sensitively (matching prior .mkv behavior).
var inputExtensions = []string{
"mkv", "mp4", "m4v", "mov", "avi",
"ts", "m2ts", "mts",
"mpg", "mpeg", "vob",
"webm", "wmv", "flv",
}
// fileStat is the (mtime, size) pair used to decide whether a file has settled
// between two consecutive ticks.
type fileStat struct {
mtime time.Time
size int64
}
// failureRecord remembers that processFn failed for a given file, so we can
// back off rather than retrying on every tick.
type failureRecord struct {
failedAt time.Time
fileMtime time.Time
}
type Watcher struct { type Watcher struct {
inputDir string inputDir string
interval time.Duration interval time.Duration
// seen tracks the last-observed (mtime, size) for every file currently in
// the input dir. A file is only handed to processFn once two consecutive
// ticks agree on both fields (partial-write protection).
seen map[string]fileStat
// failed tracks files that recently errored from processFn so we can skip
// them until either failureBackoff elapses or their mtime changes.
failed map[string]failureRecord
} }
func New(inputDir string, intervalSeconds int) *Watcher { func New(inputDir string, intervalSeconds int) *Watcher {
@@ -24,39 +65,86 @@ func New(inputDir string, intervalSeconds int) *Watcher {
return &Watcher{ return &Watcher{
inputDir: inputDir, inputDir: inputDir,
interval: interval, interval: interval,
seen: make(map[string]fileStat),
failed: make(map[string]failureRecord),
} }
} }
func (w *Watcher) Start(processFn func(string) error, done chan struct{}) { func (w *Watcher) Start(ctx context.Context, processFn func(context.Context, string) error) {
ticker := time.NewTicker(w.interval) ticker := time.NewTicker(w.interval)
defer ticker.Stop() defer ticker.Stop()
w.scanAndProcess(processFn) w.scanAndProcess(ctx, processFn)
for { for {
select { select {
case <-done: case <-ctx.Done():
return return
case <-ticker.C: case <-ticker.C:
w.scanAndProcess(processFn) w.scanAndProcess(ctx, processFn)
} }
} }
} }
func (w *Watcher) scanAndProcess(processFn func(string) error) { func (w *Watcher) scanAndProcess(ctx context.Context, processFn func(context.Context, string) error) {
files, err := filepath.Glob(filepath.Join(w.inputDir, "*.mkv")) var files []string
for _, ext := range inputExtensions {
matches, err := filepath.Glob(filepath.Join(w.inputDir, "*."+ext))
if err != nil { if err != nil {
fmt.Printf("Error scanning input directory: %v\n", err) fmt.Printf("Error scanning input directory: %v\n", err)
return return
} }
files = append(files, matches...)
}
now := time.Now()
nextSeen := make(map[string]fileStat, len(files))
nextFailed := make(map[string]failureRecord, len(w.failed))
for _, file := range files { for _, file := range files {
if err := processFn(file); err != nil { info, err := os.Stat(file)
if err != nil {
// File vanished between glob and stat, or unreadable. Drop any
// state for it by not carrying it forward.
fmt.Printf("Error stating %s: %v\n", file, err)
continue
}
current := fileStat{mtime: info.ModTime(), size: info.Size()}
nextSeen[file] = current
// Carry the failure record forward only if the file hasn't been
// touched since it failed; a changed mtime means the user replaced
// or modified the file and we should give it another chance.
if rec, ok := w.failed[file]; ok && rec.fileMtime.Equal(current.mtime) {
if now.Sub(rec.failedAt) < failureBackoff {
nextFailed[file] = rec
continue
}
// Backoff expired; clear the record and let the file be
// re-processed if it is otherwise stable.
}
prev, ok := w.seen[file]
if !ok || prev.mtime != current.mtime || prev.size != current.size {
// First time we've seen this (mtime, size); wait one more tick
// to make sure the file isn't still being written.
continue
}
if err := processFn(ctx, file); err != nil {
fmt.Printf("Error processing %s: %v\n", file, err) fmt.Printf("Error processing %s: %v\n", file, err)
nextFailed[file] = failureRecord{
failedAt: now,
fileMtime: current.mtime,
} }
} }
} }
w.seen = nextSeen
w.failed = nextFailed
}
func DetectMediaType(width, height int) types.MediaType { func DetectMediaType(width, height int) types.MediaType {
pixels := width * height pixels := width * height
if pixels < 600000 { if pixels < 600000 {
+5
View File
@@ -9,6 +9,8 @@ type MediaType string
const ( const (
MediaTypeDVD MediaType = "DVD" MediaTypeDVD MediaType = "DVD"
MediaTypeBluRay MediaType = "Blu-ray" MediaTypeBluRay MediaType = "Blu-ray"
MediaTypeWebDL MediaType = "WebDL"
MediaTypeTVRip MediaType = "TVRip"
) )
type Job struct { type Job struct {
@@ -42,6 +44,8 @@ type Config struct {
type EncodingConfig struct { type EncodingConfig struct {
DVD EncodingParams `yaml:"dvd"` DVD EncodingParams `yaml:"dvd"`
Bluray EncodingParams `yaml:"bluray"` Bluray EncodingParams `yaml:"bluray"`
WebDL EncodingParams `yaml:"webdl"`
TVRip EncodingParams `yaml:"tvrip"`
} }
type EncodingParams struct { type EncodingParams struct {
@@ -54,6 +58,7 @@ type PathsConfig struct {
Output string `yaml:"output"` Output string `yaml:"output"`
Originals string `yaml:"originals"` Originals string `yaml:"originals"`
Failed string `yaml:"failed"` Failed string `yaml:"failed"`
Work string `yaml:"work"`
} }
type LogEntry struct { type LogEntry struct {