Compare commits
10
Commits
71f7ef7bca
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6a564aabb2 | ||
|
|
321f11dd8f | ||
|
|
459ab30d94 | ||
|
|
191bc955e4 | ||
|
|
706cd22b05 | ||
|
|
22b3a73109 | ||
|
|
6b72ef03ac | ||
|
|
8b38986690 | ||
|
|
244bdce586 | ||
|
|
022d131cd6 |
@@ -48,12 +48,19 @@ encoding:
|
||||
bluray:
|
||||
crf: 29
|
||||
preset: 3
|
||||
webdl:
|
||||
crf: 30
|
||||
preset: 3
|
||||
tvrip:
|
||||
crf: 32
|
||||
preset: 2
|
||||
|
||||
paths:
|
||||
input: "./input"
|
||||
output: "./output"
|
||||
originals: "./originals"
|
||||
failed: "./failed"
|
||||
work: "./work"
|
||||
```
|
||||
|
||||
### Field reference
|
||||
@@ -65,12 +72,17 @@ paths:
|
||||
| `encoding.dvd.preset` | SVT-AV1 preset for SD sources | `2` |
|
||||
| `encoding.bluray.crf` | SVT-AV1 CRF for HD sources | `29` |
|
||||
| `encoding.bluray.preset` | SVT-AV1 preset for HD sources | `3` |
|
||||
| `encoding.webdl.crf` | SVT-AV1 CRF for WebDL sources | `30` |
|
||||
| `encoding.webdl.preset` | SVT-AV1 preset for WebDL sources | `3` |
|
||||
| `encoding.tvrip.crf` | SVT-AV1 CRF for TVRip sources | `32` |
|
||||
| `encoding.tvrip.preset` | SVT-AV1 preset for TVRip sources | `2` |
|
||||
| `paths.input` | Folder polled for new `.mkv` files | `./input` |
|
||||
| `paths.output` | Destination for finished encodes | `./output` |
|
||||
| `paths.originals` | Where source files are moved on success (unless `-d`) | `./originals` |
|
||||
| `paths.failed` | Where source + partial output go on failure | `./failed` |
|
||||
| `paths.failed` | Where source files go on failure | `./failed` |
|
||||
| `paths.work` | Scratch directory for per-job intermediates (wav/opus/output.mkv); deleted after every job | `./work` |
|
||||
|
||||
All four directories are created on startup if they don't exist.
|
||||
All five directories are created on startup if they don't exist.
|
||||
|
||||
---
|
||||
|
||||
@@ -88,7 +100,7 @@ Flags:
|
||||
- `-d` — delete the source `.mkv` after a successful encode instead of moving it to `originals/`.
|
||||
- `-c PATH` — path to config file.
|
||||
|
||||
Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. A signal triggers a clean shutdown after the current poll cycle.
|
||||
Stop with `Ctrl+C` (SIGINT) or `SIGTERM`. The signal cancels any in-flight encode immediately — the ffmpeg / ffprobe / opusenc children are killed, the per-job work directory is removed by its deferred cleanup, and the source `.mkv` is routed to `paths.failed` so the next run sees a clean `paths.input`.
|
||||
|
||||
The program runs as a foreground daemon. It scans the input directory on startup and every 15 seconds thereafter.
|
||||
|
||||
@@ -131,41 +143,62 @@ the-wire.TVM75.s2e5.mkv
|
||||
|
||||
If neither pattern matches, encoding still proceeds but the file is treated as having no metadata. The output is named with a random hex string and an `.nometadata.mkv` suffix.
|
||||
|
||||
### 5.4 Optional: source media type
|
||||
|
||||
Any filename can additionally contain a media-type token (case-insensitive, word-bounded):
|
||||
|
||||
- `dvd`
|
||||
- `bluray`
|
||||
- `webdl`
|
||||
- `tvrip`
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
Heat.tt0113277.bluray.mkv
|
||||
some-rip.tvm169.S01E01.webdl.mkv
|
||||
old.broadcast.tvrip.tt0066026.mkv
|
||||
```
|
||||
|
||||
The token controls **both** the `ORIGINAL_MEDIA_TYPE` metadata tag written into the output and which `encoding.<type>.crf` / `encoding.<type>.preset` pair is used.
|
||||
|
||||
If no token is present, the program falls back to guessing from pixel count: `width × height < 600,000` → DVD, otherwise Blu-ray. WebDL and TVRip are never auto-detected — they must be declared via the token.
|
||||
|
||||
---
|
||||
|
||||
## 6. The processing pipeline
|
||||
|
||||
For every `.mkv` found in `paths.input`, the program runs these steps in order. Any error sends the source file (and the partial `output.mkv`, if any) to `paths.failed`.
|
||||
For every `.mkv` found in `paths.input`, the program runs these steps in order. A per-job scratch subdirectory under `paths.work` (named after the input base name without `.mkv`) holds all intermediates, and is deleted unconditionally at the end of the job. Any error sends the source file to `paths.failed`; the work subdirectory is wiped regardless of outcome.
|
||||
|
||||
1. **Parse filename** → determines whether this is a movie or series, and what IDs to use.
|
||||
2. **Probe video** with `ffprobe`:
|
||||
- Picks the first video stream whose codec is `mpeg2video`, `h264`, or `hevc`.
|
||||
- Picks the first stream with `codec_type=video`, regardless of codec name. Errors out if there isn't one.
|
||||
- Records width, height, and sample aspect ratio (SAR).
|
||||
- Runs `ffmpeg -vf idet` to detect interlacing (presence of `TFF`/`BFF` in stderr).
|
||||
- Detects interlacing by running `ffmpeg -vf idet -frames:v 400 -an -sn -f null -` and parsing the `Multi frame detection: TFF: a BFF: b Progressive: c Undetermined: d` summary line. The source is treated as interlaced only when `a+b > c`; undetermined frames are ignored, and a missing summary line defaults to progressive.
|
||||
3. **Probe stream languages** with `ffprobe` — collects `language` tags for every audio/subtitle stream so they can be re-applied after encoding (FFmpeg's `-map_metadata -1` strips them otherwise).
|
||||
4. **Detect media type** from pixel count:
|
||||
- `width × height < 600,000` → **DVD** profile.
|
||||
- Otherwise → **Blu-ray** profile.
|
||||
The chosen profile selects which CRF/preset pair from the config to use.
|
||||
4. **Detect media type**:
|
||||
- First, check the filename for a `dvd` / `bluray` / `webdl` / `tvrip` token (case-insensitive). If present, that wins.
|
||||
- Otherwise, fall back to pixel count: `width × height < 600,000` → DVD, else Blu-ray.
|
||||
The chosen profile selects which `encoding.<type>.crf` / `encoding.<type>.preset` pair from the config to use, and is written into the `ORIGINAL_MEDIA_TYPE` metadata tag.
|
||||
5. **Fetch metadata** from OMDb or TVmaze depending on the parsed filename. Failures here are logged but do not abort the encode — the file is just encoded without metadata.
|
||||
6. **Extract audio** to `audio.wav` (PCM s16le, 48 kHz) in the input directory.
|
||||
7. **Encode audio** with `opusenc --bitrate 128k` → `audio.opus`.
|
||||
8. **Calculate display width** from SAR. If SAR ≠ `1:1`, the width is rescaled so the output has square pixels, using a `zscale` filter (`spline36`). Height is preserved.
|
||||
6. **Extract audio** — one PCM wav per source audio stream, written to the per-job work directory as `audio.0.wav`, `audio.1.wav`, … in source order (PCM s16le, 48 kHz). Errors out if the source has no audio streams.
|
||||
7. **Encode audio** — each wav is converted with `opusenc --bitrate 128k` to a matching `audio.<n>.opus` in the same work directory.
|
||||
8. **Calculate display width** from SAR. The width is rescaled so the output has square pixels only when there's actually work to do — `zscale` is skipped entirely for square-pixel sources (SAR `1:1`, `N/A`, empty, `0:N`), and for any SAR whose calculated width rounds to the source width. When rescaling, the width is rounded to the nearest even number (mod-2, preferred by AV1).
|
||||
9. **Encode video** with FFmpeg:
|
||||
- Video filter chain: `bwdif=mode=0:par=-1:-1,zscale=w=W:h=H:filter=spline36` if interlaced, else just the `zscale` step.
|
||||
- Video filter chain is built conditionally. `bwdif=mode=0:par=-1:-1` is prepended when the source is interlaced; the `zscale` step is appended only when a rescale is actually needed (see step 8). If neither applies, `-vf` is omitted entirely.
|
||||
- Codec: `libsvtav1`, `-pix_fmt yuv420p10le`.
|
||||
- `-crf` and `-preset` from the selected profile.
|
||||
- `-crf` and `-preset` from the selected profile (step 4).
|
||||
- `-svtav1-params film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s`.
|
||||
- Streams mapped: video from input, subtitles from input (`0:s?` — optional), audio from the re-encoded Opus files.
|
||||
- Audio copied (`-c:a copy`), subtitles copied (`-c:s copy`).
|
||||
- `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied from the ffprobe pass.
|
||||
- Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`.
|
||||
- Output written to `output.mkv` in the input directory.
|
||||
10. **Clean up** intermediate `.wav` and `.opus` files.
|
||||
11. **Rename and move** `output.mkv` to `paths.output` with a final name:
|
||||
- Series → `<sanitized-show-name>.S<NN>E<NN>.mkv`
|
||||
- Movie → `<sanitized-title>.<imdbID>.mkv`
|
||||
- No metadata → `<8-hex-chars>.nometadata.mkv`
|
||||
- Streams mapped: video from input (`0:v`), subtitles from input (`0:s?` — optional), and one audio stream per Opus file (`1:a`, `2:a`, …).
|
||||
- Audio re-muxed (`-c:a copy` — copies the already-Opus-encoded streams), subtitles copied (`-c:s copy`).
|
||||
- `-map_metadata -1` strips global metadata; per-stream language tags are then re-applied. Audio language tags are indexed by **output position** (the position in the opus-file list), not by counting source streams, so missing-language streams don't shift the index.
|
||||
- Container metadata written: `TITLE`, `DATE_RELEASED`, `IMDBID`, `ORIGINAL_MEDIA_TYPE`. For series: also `COLLECTION`, `SEASON`, `EPISODE`, `TVMAZE_ID`. Values are unquoted (literal value, no wrapping `"…"`).
|
||||
- Output written to `output.mkv` in the per-job work directory.
|
||||
10. **Clean up** the entire per-job work directory (intermediates and `output.mkv` together) once the rename/move below succeeds — or, on any failure, when the deferred cleanup runs.
|
||||
11. **Rename and move** `output.mkv` from the work directory to `paths.output` with a final name:
|
||||
- Series with a known show name → `<sanitized-show-name>.S<NN>E<NN>.mkv` (the series no longer needs a populated IMDb mapping — a TVmaze show with no external IMDb link still gets a useful filename).
|
||||
- Movie with a known title and IMDb ID → `<sanitized-title>.<imdbID>.mkv`.
|
||||
- Anything else → `<8-hex-chars>.nometadata.mkv`.
|
||||
Sanitization keeps `a–z A–Z 0–9 - ä ö Ä Ö` only.
|
||||
12. **Dispose of the source**:
|
||||
- `-d` flag set → delete the original `.mkv`.
|
||||
@@ -175,13 +208,18 @@ For every `.mkv` found in `paths.input`, the program runs these steps in order.
|
||||
|
||||
## 7. Output naming examples
|
||||
|
||||
| Input filename | Result in `paths.output` |
|
||||
|---|---|
|
||||
| `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` |
|
||||
| `Breaking.Bad.tvm169.S01E01.mkv` | `BreakingBad.S01E01.mkv` |
|
||||
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` |
|
||||
| Input filename | Result in `paths.output` | Notes |
|
||||
|---|---|---|
|
||||
| `Heat.tt0113277.mkv` | `Heat.tt0113277.mkv` | Pixel-count fallback → Blu-ray profile + tag |
|
||||
| `Heat.tt0113277.bluray.mkv` | `Heat.tt0113277.mkv` | Same output filename; muxed `ORIGINAL_MEDIA_TYPE=Blu-ray` is now from the token, not the guess |
|
||||
| `Breaking.Bad.tvm169.S01E01.webdl.mkv` | `BreakingBad.S01E01.mkv` | WebDL profile + tag |
|
||||
| `unrecognized-rip.mkv` | `a1b2c3d4.nometadata.mkv` | No IDs at all |
|
||||
|
||||
The IMDb ID returned by the API is used, not the one from the filename, so a typo in the filename would surface there.
|
||||
A few things worth knowing:
|
||||
|
||||
- The media-type token affects the muxed `ORIGINAL_MEDIA_TYPE` tag and the CRF/preset profile, but **not** the output filename.
|
||||
- The IMDb ID written into the filename is the one returned by the API, not the one in the input filename, so a typo in the source filename will surface in the output name.
|
||||
- A TVmaze show with no IMDb mapping still gets a `<Collection>.S<NN>E<NN>.mkv` filename (only the `IMDBID` metadata tag is left empty).
|
||||
|
||||
---
|
||||
|
||||
@@ -197,18 +235,24 @@ Run the program from the directory where you want the logs to land.
|
||||
|
||||
Note: a number of `DEBUG` lines are printed to stdout/stderr (ffprobe output, calculated zscale width, the full ffmpeg command, etc.). These are intentional but not written to the log files.
|
||||
|
||||
**Known limitation:** the date in `info_*.log` / `error_*.log` filenames is computed when the daemon starts and does not roll over at midnight. If the daemon is left running across days, all writes continue into the start-day's file. Restart the daemon to rotate. `structured.json` does not rotate at all.
|
||||
|
||||
---
|
||||
|
||||
## 9. Failure handling
|
||||
|
||||
If any step from probing through encoding through renaming fails:
|
||||
|
||||
- The source `.mkv` is moved to `paths.failed`.
|
||||
- Any partial `output.mkv` left in the input directory is also moved to `paths.failed`.
|
||||
- The source `.mkv` is moved to `paths.failed` (the move itself is `os.Stat`-guarded — if the source is already gone, the move is skipped and logged; if the move itself errors, that error is logged too). This guarantees the source leaves `paths.input` on every failure path, so the watcher doesn't retry the same file on the next tick.
|
||||
- The per-job work directory under `paths.work` (containing partial wav/opus/output.mkv) is deleted unconditionally.
|
||||
- The error is logged to `error_*.log` and `structured.json` with the source file path.
|
||||
|
||||
The watcher continues with the next file; one bad rip won't stop the daemon.
|
||||
|
||||
**Destination collisions:** if a finished encode would land on a name that already exists in `paths.output`, the move is refused (no silent overwrite) and the source is routed to `paths.failed`. This is the path you'll hit when two sources sanitize to the same output filename — e.g. two re-rips of the same release, or two episodes that both come out as `SxxExx`.
|
||||
|
||||
**Cross-filesystem moves:** every move (`paths.input → paths.output`, `… → paths.failed`, `… → paths.originals`) transparently falls back to copy + delete when the source and destination live on different mounts. You can put each path on a different drive without breaking the pipeline.
|
||||
|
||||
---
|
||||
|
||||
## 10. Polling behavior
|
||||
@@ -216,7 +260,8 @@ The watcher continues with the next file; one bad rip won't stop the daemon.
|
||||
- Input directory is scanned every **15 seconds** (clamped to a 10–30 s range).
|
||||
- Only files matching `*.mkv` directly in `paths.input` are picked up — no recursion.
|
||||
- Files are processed **sequentially**, one at a time, in the order `filepath.Glob` returns them (alphabetical on Linux).
|
||||
- There is no atomic-write detection. If you're copying a large file into `paths.input`, copy it to a different name first and `mv` it into place once complete, otherwise the watcher may try to encode a half-written file.
|
||||
- **Partial-write protection:** the watcher requires a file's `mtime` and `size` to be identical on two consecutive scans before processing. A freshly dropped or still-copying file therefore waits at least one full tick (~15 s) before encoding begins. Copying a large file directly into `paths.input` is now safe; you no longer have to land it under a different name and `mv` into place (though doing so still works and shaves off the tick delay).
|
||||
- **Failure quarantine:** if `processFile` returns an error, that file is skipped for 5 minutes (or until its `mtime` changes — e.g. you replace or `touch` it). Prevents a permanently-broken input from spamming the logs every 15 s. The quarantine is in-memory only; restarting the daemon clears it.
|
||||
|
||||
---
|
||||
|
||||
@@ -232,3 +277,13 @@ internal/mover/ File rename/move/delete helpers
|
||||
internal/logger/ Plain + JSON logging
|
||||
pkg/types/types.go Shared structs (Config, Job, Metadata, …)
|
||||
```
|
||||
|
||||
Runtime directories (from `paths.*` in the config):
|
||||
|
||||
```
|
||||
paths.input User drops new .mkv files here; watcher polls every 15 s
|
||||
paths.output Finished encodes land here under their final name
|
||||
paths.originals Successfully-encoded sources end up here (unless -d)
|
||||
paths.failed Sources of failed jobs end up here
|
||||
paths.work Per-job scratch subdir (basename of input); wiped per job
|
||||
```
|
||||
|
||||
+67
-30
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"flag"
|
||||
@@ -9,6 +10,7 @@ import (
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
"videnc-vibe/internal/config"
|
||||
@@ -60,57 +62,75 @@ func main() {
|
||||
|
||||
metaClient := metadata.NewClient(cfg.OMDBAPIKey)
|
||||
|
||||
done := make(chan struct{})
|
||||
sigChan := make(chan os.Signal, 1)
|
||||
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
|
||||
|
||||
go func() {
|
||||
<-sigChan
|
||||
close(done)
|
||||
}()
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
w := watcher.New(cfg.Paths.Input, 15)
|
||||
w.Start(func(inputPath string) error {
|
||||
return processFile(inputPath, cfg, enc, metaClient, log)
|
||||
}, done)
|
||||
w.Start(ctx, func(ctx context.Context, inputPath string) error {
|
||||
return processFile(ctx, inputPath, cfg, enc, metaClient, log)
|
||||
})
|
||||
|
||||
log.Info("videnc-vibe started")
|
||||
}
|
||||
|
||||
func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger) error {
|
||||
func processFile(ctx context.Context, inputPath string, cfg *types.Config, enc *encoder.Encoder, metaClient *metadata.Client, log *logger.Logger) error {
|
||||
log.Info(fmt.Sprintf("Processing: %s", inputPath))
|
||||
|
||||
filename := filepath.Base(inputPath)
|
||||
isSeries, imdbID, tvmazeID, season, episode := metadata.ParseFilename(filename)
|
||||
|
||||
width, height, interlaced, err := enc.GetMediaInfo(inputPath)
|
||||
// Per-job work subdirectory under paths.work. Uses the source base name
|
||||
// (without ".mkv") as the subdir name. MkdirAll is idempotent, so a
|
||||
// leftover dir from a crashed previous run is harmless to overwrite.
|
||||
workDirName := strings.TrimSuffix(filename, filepath.Ext(filename))
|
||||
workDir := filepath.Join(cfg.Paths.Work, workDirName)
|
||||
if err := os.MkdirAll(workDir, 0755); err != nil {
|
||||
log.ErrorFile(inputPath, "Creating work directory", err.Error())
|
||||
failToFailed(inputPath, cfg.Paths.Failed, log)
|
||||
return err
|
||||
}
|
||||
// Cleanup the work directory on every exit path, success or failure.
|
||||
defer os.RemoveAll(workDir)
|
||||
|
||||
width, height, interlaced, err := enc.GetMediaInfo(ctx, inputPath)
|
||||
if err != nil {
|
||||
log.ErrorFile(inputPath, "Getting media info", err.Error())
|
||||
mover.MoveToFailed(inputPath, cfg.Paths.Failed)
|
||||
failToFailed(inputPath, cfg.Paths.Failed, log)
|
||||
return err
|
||||
}
|
||||
|
||||
streamLangs, err := enc.GetStreamLanguages(inputPath)
|
||||
streamLangs, err := enc.GetStreamLanguages(ctx, inputPath)
|
||||
if err != nil {
|
||||
log.ErrorFile(inputPath, "Getting stream languages", err.Error())
|
||||
}
|
||||
|
||||
mediaType := watcher.DetectMediaType(width, height)
|
||||
crf := cfg.Encoding.DVD.CRF
|
||||
preset := cfg.Encoding.DVD.Preset
|
||||
if mediaType == types.MediaTypeBluRay {
|
||||
crf = cfg.Encoding.Bluray.CRF
|
||||
preset = cfg.Encoding.Bluray.Preset
|
||||
mediaType := metadata.ParseMediaType(filename)
|
||||
if mediaType == "" {
|
||||
mediaType = watcher.DetectMediaType(width, height)
|
||||
}
|
||||
|
||||
var profile types.EncodingParams
|
||||
switch mediaType {
|
||||
case types.MediaTypeBluRay:
|
||||
profile = cfg.Encoding.Bluray
|
||||
case types.MediaTypeWebDL:
|
||||
profile = cfg.Encoding.WebDL
|
||||
case types.MediaTypeTVRip:
|
||||
profile = cfg.Encoding.TVRip
|
||||
default:
|
||||
profile = cfg.Encoding.DVD
|
||||
}
|
||||
crf := profile.CRF
|
||||
preset := profile.Preset
|
||||
|
||||
var meta *types.Metadata
|
||||
if isSeries && tvmazeID != "" && season != "" && episode != "" {
|
||||
meta, err = metaClient.FetchSeriesMetadata(tvmazeID, season, episode)
|
||||
meta, err = metaClient.FetchSeriesMetadata(ctx, tvmazeID, season, episode)
|
||||
if err != nil {
|
||||
log.ErrorFile(inputPath, "Fetching series metadata", err.Error())
|
||||
}
|
||||
} else if imdbID != "" {
|
||||
meta, err = metaClient.FetchMovieMetadata(imdbID)
|
||||
meta, err = metaClient.FetchMovieMetadata(ctx, imdbID)
|
||||
if err != nil {
|
||||
log.ErrorFile(inputPath, "Fetching movie metadata", err.Error())
|
||||
}
|
||||
@@ -134,16 +154,13 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
|
||||
DeleteOrigin: deleteOrigin,
|
||||
}
|
||||
|
||||
if err := enc.Transcode(inputPath, job, meta, interlaced, streamLangs); err != nil {
|
||||
if err := enc.Transcode(ctx, inputPath, workDir, job, meta, interlaced, streamLangs); err != nil {
|
||||
log.ErrorFile(inputPath, "Transcoding", err.Error())
|
||||
mover.MoveToFailed(inputPath, cfg.Paths.Failed)
|
||||
outputPath := filepath.Join(filepath.Dir(inputPath), "output.mkv")
|
||||
mover.MoveToFailed(outputPath, cfg.Paths.Failed)
|
||||
failToFailed(inputPath, cfg.Paths.Failed, log)
|
||||
return err
|
||||
}
|
||||
|
||||
outputDir := filepath.Dir(inputPath)
|
||||
outputPath := filepath.Join(outputDir, "output.mkv")
|
||||
outputPath := filepath.Join(workDir, "output.mkv")
|
||||
|
||||
var outFilename string
|
||||
switch {
|
||||
@@ -157,8 +174,11 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
|
||||
|
||||
finalOutput := filepath.Join(cfg.Paths.Output, outFilename)
|
||||
if err := mover.Rename(outputPath, finalOutput); err != nil {
|
||||
// Move the source to failed/ so it doesn't get re-encoded on the next
|
||||
// tick. The deferred RemoveAll(workDir) takes care of the partial
|
||||
// output.mkv left in the work dir.
|
||||
log.ErrorFile(inputPath, "Moving output", err.Error())
|
||||
mover.MoveToFailed(outputPath, cfg.Paths.Failed)
|
||||
failToFailed(inputPath, cfg.Paths.Failed, log)
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -174,6 +194,23 @@ func processFile(inputPath string, cfg *types.Config, enc *encoder.Encoder, meta
|
||||
return nil
|
||||
}
|
||||
|
||||
// failToFailed stats path before invoking MoveToFailed: if missing, logs and
|
||||
// skips; if present, surfaces any move error to the logger. This avoids the
|
||||
// silent no-op pattern where MoveToFailed was called on a non-existent file.
|
||||
func failToFailed(path, failedDir string, log *logger.Logger) {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
log.Info(fmt.Sprintf("Skip move to failed; not present: %s", path))
|
||||
return
|
||||
}
|
||||
log.ErrorFile(path, "Stat before move to failed", err.Error())
|
||||
return
|
||||
}
|
||||
if err := mover.MoveToFailed(path, failedDir); err != nil {
|
||||
log.ErrorFile(path, "Moving to failed", err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func generateRandomString(length int) string {
|
||||
bytes := make([]byte, length/2+1)
|
||||
rand.Read(bytes)
|
||||
|
||||
@@ -49,6 +49,9 @@ func Load(configPath string) (*types.Config, error) {
|
||||
if cfg.Paths.Failed == "" {
|
||||
cfg.Paths.Failed = "./failed"
|
||||
}
|
||||
if cfg.Paths.Work == "" {
|
||||
cfg.Paths.Work = "./work"
|
||||
}
|
||||
if cfg.Encoding.DVD.CRF == 0 {
|
||||
cfg.Encoding.DVD.CRF = 30
|
||||
}
|
||||
@@ -61,12 +64,24 @@ func Load(configPath string) (*types.Config, error) {
|
||||
if cfg.Encoding.Bluray.Preset == 0 {
|
||||
cfg.Encoding.Bluray.Preset = 3
|
||||
}
|
||||
if cfg.Encoding.WebDL.CRF == 0 {
|
||||
cfg.Encoding.WebDL.CRF = 30
|
||||
}
|
||||
if cfg.Encoding.WebDL.Preset == 0 {
|
||||
cfg.Encoding.WebDL.Preset = 3
|
||||
}
|
||||
if cfg.Encoding.TVRip.CRF == 0 {
|
||||
cfg.Encoding.TVRip.CRF = 32
|
||||
}
|
||||
if cfg.Encoding.TVRip.Preset == 0 {
|
||||
cfg.Encoding.TVRip.Preset = 2
|
||||
}
|
||||
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
func EnsureDirs(cfg *types.Config) error {
|
||||
dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed}
|
||||
dirs := []string{cfg.Paths.Input, cfg.Paths.Output, cfg.Paths.Originals, cfg.Paths.Failed, cfg.Paths.Work}
|
||||
for _, dir := range dirs {
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
return fmt.Errorf("creating directory %s: %w", dir, err)
|
||||
|
||||
+71
-55
@@ -1,12 +1,13 @@
|
||||
package encoder
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
@@ -42,8 +43,8 @@ type StreamMetadata struct {
|
||||
Title string `json:"title"`
|
||||
}
|
||||
|
||||
func (e *Encoder) GetStreamLanguages(path string) ([]StreamMetadata, error) {
|
||||
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
|
||||
func (e *Encoder) GetStreamLanguages(ctx context.Context, path string) ([]StreamMetadata, error) {
|
||||
cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ffprobe error: %w", err)
|
||||
@@ -103,8 +104,8 @@ func (e *Encoder) CheckDeps() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool, err error) {
|
||||
cmd := exec.Command(e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
|
||||
func (e *Encoder) GetMediaInfo(ctx context.Context, path string) (width, height int, interlaced bool, err error) {
|
||||
cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-show_streams", "-print_format", "json", path)
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return 0, 0, false, fmt.Errorf("ffprobe error: %w", err)
|
||||
@@ -131,7 +132,7 @@ func (e *Encoder) GetMediaInfo(path string) (width, height int, interlaced bool,
|
||||
return 0, 0, false, fmt.Errorf("no video stream found")
|
||||
}
|
||||
|
||||
cmd = exec.Command(e.ffmpegPath,
|
||||
cmd = exec.CommandContext(ctx, e.ffmpegPath,
|
||||
"-hide_banner",
|
||||
"-nostats",
|
||||
"-i", path,
|
||||
@@ -168,8 +169,8 @@ func detectInterlaced(idetOutput string) bool {
|
||||
return interlaced > prog
|
||||
}
|
||||
|
||||
func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string, int, error) {
|
||||
cmd := exec.Command(e.ffprobePath, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path)
|
||||
func (e *Encoder) calculateZscaleWidth(ctx context.Context, path string, originalHeight int) (string, int, error) {
|
||||
cmd := exec.CommandContext(ctx, e.ffprobePath, "-v", "error", "-select_streams", "v:0", "-show_entries", "stream=width,height,sample_aspect_ratio", "-print_format", "json", path)
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
return "", 0, fmt.Errorf("ffprobe error: %w", err)
|
||||
@@ -221,59 +222,79 @@ func (e *Encoder) calculateZscaleWidth(path string, originalHeight int) (string,
|
||||
return zscale, newWidth, nil
|
||||
}
|
||||
|
||||
func (e *Encoder) Transcode(input string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
|
||||
dir := filepath.Dir(input)
|
||||
audioWavs, err := e.extractAudio(input, dir)
|
||||
func (e *Encoder) Transcode(ctx context.Context, input, workDir string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
|
||||
audioWavs, err := e.extractAudio(ctx, input, workDir, streamLangs)
|
||||
if err != nil {
|
||||
return fmt.Errorf("extracting audio: %w", err)
|
||||
}
|
||||
defer e.cleanupWavs(audioWavs)
|
||||
|
||||
opusFiles, err := e.encodeOpus(audioWavs, dir)
|
||||
opusFiles, err := e.encodeOpus(ctx, audioWavs, workDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encoding opus: %w", err)
|
||||
}
|
||||
defer e.cleanupOpus(opusFiles)
|
||||
|
||||
if err := e.encodeVideo(input, opusFiles, job, metadata, interlaced, streamLangs); err != nil {
|
||||
if err := e.encodeVideo(ctx, input, workDir, opusFiles, job, metadata, interlaced, streamLangs); err != nil {
|
||||
return fmt.Errorf("encoding video: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *Encoder) extractAudio(input, dir string) ([]string, error) {
|
||||
cmd := exec.Command(e.ffmpegPath, "-i", input,
|
||||
// audioStreamsInSourceOrder returns the audio entries of streamLangs sorted by
|
||||
// their source-side Index. The resulting slice position is the 0-based audio
|
||||
// stream index used by ffmpeg selectors like `0:a:<n>`.
|
||||
func audioStreamsInSourceOrder(streamLangs []StreamMetadata) []StreamMetadata {
|
||||
var audio []StreamMetadata
|
||||
for _, s := range streamLangs {
|
||||
if s.CodecType == "audio" {
|
||||
audio = append(audio, s)
|
||||
}
|
||||
}
|
||||
sort.Slice(audio, func(i, j int) bool { return audio[i].Index < audio[j].Index })
|
||||
return audio
|
||||
}
|
||||
|
||||
func (e *Encoder) extractAudio(ctx context.Context, input, workDir string, streamLangs []StreamMetadata) ([]string, error) {
|
||||
audio := audioStreamsInSourceOrder(streamLangs)
|
||||
if len(audio) == 0 {
|
||||
return nil, fmt.Errorf("no audio streams found")
|
||||
}
|
||||
|
||||
wavs := make([]string, 0, len(audio))
|
||||
for srcAudioIndex := range audio {
|
||||
wavPath := filepath.Join(workDir, fmt.Sprintf("audio.%d.wav", srcAudioIndex))
|
||||
cmd := exec.CommandContext(ctx, e.ffmpegPath, "-i", input,
|
||||
"-map", fmt.Sprintf("0:a:%d", srcAudioIndex),
|
||||
"-vn", "-c:a", "pcm_s16le", "-ar", "48000",
|
||||
"-f", "wav", filepath.Join(dir, "audio.wav"))
|
||||
"-f", "wav", wavPath)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
return nil, fmt.Errorf("ffmpeg extract: %s %w", out, err)
|
||||
return wavs, fmt.Errorf("ffmpeg extract a:%d: %s %w", srcAudioIndex, out, err)
|
||||
}
|
||||
wavs = append(wavs, wavPath)
|
||||
}
|
||||
return wavs, nil
|
||||
}
|
||||
|
||||
matches, _ := filepath.Glob(filepath.Join(dir, "audio.wav"))
|
||||
return matches, nil
|
||||
}
|
||||
|
||||
func (e *Encoder) encodeOpus(wavs []string, dir string) ([]string, error) {
|
||||
func (e *Encoder) encodeOpus(ctx context.Context, wavs []string, workDir string) ([]string, error) {
|
||||
var opusFiles []string
|
||||
for _, wav := range wavs {
|
||||
out := strings.Replace(wav, ".wav", ".opus", 1)
|
||||
cmd := exec.Command(e.opusencPath, "--bitrate", "128k", wav, out)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
return nil, fmt.Errorf("opusenc: %s %w", out, err)
|
||||
base := filepath.Base(strings.Replace(wav, ".wav", ".opus", 1))
|
||||
out := filepath.Join(workDir, base)
|
||||
cmd := exec.CommandContext(ctx, e.opusencPath, "--bitrate", "128k", wav, out)
|
||||
if logOut, err := cmd.CombinedOutput(); err != nil {
|
||||
return nil, fmt.Errorf("opusenc: %s %w", logOut, err)
|
||||
}
|
||||
opusFiles = append(opusFiles, out)
|
||||
}
|
||||
return opusFiles, nil
|
||||
}
|
||||
|
||||
func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
|
||||
dir := filepath.Dir(input)
|
||||
outFile := filepath.Join(dir, "output.mkv")
|
||||
func (e *Encoder) encodeVideo(ctx context.Context, input, workDir string, opusFiles []string, job *types.Job, metadata *types.Metadata, interlaced bool, streamLangs []StreamMetadata) error {
|
||||
outFile := filepath.Join(workDir, "output.mkv")
|
||||
|
||||
svtParams := fmt.Sprintf("film-grain=10:film-grain-denoise=1:scd=1:qm-min=4:qm-max=15:keyint=10s")
|
||||
|
||||
zscaleStr, newWidth, err := e.calculateZscaleWidth(input, 0)
|
||||
zscaleStr, newWidth, err := e.calculateZscaleWidth(ctx, input, 0)
|
||||
if err != nil {
|
||||
return fmt.Errorf("calculating zscale: %w", err)
|
||||
}
|
||||
@@ -319,17 +340,24 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
|
||||
args = append(args, "-c:s", "copy")
|
||||
args = append(args, "-map_metadata", "-1")
|
||||
|
||||
// Walk opusFiles by output audio index (0..N-1) and look up the source
|
||||
// audio language at the same source-audio-index position. This keeps the
|
||||
// `-metadata:s:a:i` index aligned with the output stream order, regardless
|
||||
// of whether some source streams lacked a language tag.
|
||||
audioStreams := audioStreamsInSourceOrder(streamLangs)
|
||||
for i := range opusFiles {
|
||||
if i >= len(audioStreams) {
|
||||
break
|
||||
}
|
||||
lang := audioStreams[i].Language
|
||||
if lang == "" {
|
||||
continue
|
||||
}
|
||||
args = append(args, fmt.Sprintf("-metadata:s:a:%d", i), fmt.Sprintf("language=%s", lang))
|
||||
fmt.Printf("DEBUG setting audio language: stream %d -> language=%s\n", i, lang)
|
||||
}
|
||||
|
||||
for _, stream := range streamLangs {
|
||||
if stream.CodecType == "audio" && stream.Language != "" {
|
||||
idx := 1
|
||||
for _, s := range streamLangs {
|
||||
if s.CodecType == "audio" && s.Index < stream.Index {
|
||||
idx++
|
||||
}
|
||||
}
|
||||
args = append(args, fmt.Sprintf("-metadata:s:a:%d", idx-1), fmt.Sprintf("language=%s", stream.Language))
|
||||
fmt.Printf("DEBUG setting audio language: stream %d -> language=%s\n", idx-1, stream.Language)
|
||||
}
|
||||
if stream.CodecType == "subtitle" && stream.Language != "" {
|
||||
idx := 0
|
||||
for _, s := range streamLangs {
|
||||
@@ -359,7 +387,7 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
|
||||
args = append(args, outFile)
|
||||
|
||||
args = append([]string{"-hide_banner", "-v", "error"}, args...)
|
||||
cmd := exec.Command(e.ffmpegPath, args...)
|
||||
cmd := exec.CommandContext(ctx, e.ffmpegPath, args...)
|
||||
fmt.Printf("DEBUG FFmpeg command: ffmpeg %s\n", strings.Join(args, " "))
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
return fmt.Errorf("ffmpeg encode: %s %w", out, err)
|
||||
@@ -367,15 +395,3 @@ func (e *Encoder) encodeVideo(input string, opusFiles []string, job *types.Job,
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *Encoder) cleanupWavs(files []string) {
|
||||
for _, f := range files {
|
||||
os.Remove(f)
|
||||
}
|
||||
}
|
||||
|
||||
func (e *Encoder) cleanupOpus(files []string) {
|
||||
for _, f := range files {
|
||||
os.Remove(f)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package metadata
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -22,6 +23,7 @@ var (
|
||||
imdbRegex = regexp.MustCompile(`tt(\d+)`)
|
||||
tvmRegex = regexp.MustCompile(`(?i)tvm(\d+)`)
|
||||
seRegex = regexp.MustCompile(`(?i)s(\d+)e(\d+)`)
|
||||
mediaTypeRegex = regexp.MustCompile(`(?i)\b(dvd|bluray|webdl|tvrip)\b`)
|
||||
)
|
||||
|
||||
type OMDbResponse struct {
|
||||
@@ -86,10 +88,36 @@ func parseInt(s string) int {
|
||||
return n
|
||||
}
|
||||
|
||||
func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) {
|
||||
// ParseMediaType looks for a `.dvd.` / `.bluray.` / `.webdl.` / `.tvrip.`
|
||||
// token (case-insensitive) in the filename and returns the matching MediaType.
|
||||
// Returns an empty MediaType if no token is found, so the caller can fall back
|
||||
// to the pixel-count heuristic.
|
||||
func ParseMediaType(filename string) types.MediaType {
|
||||
m := mediaTypeRegex.FindStringSubmatch(strings.TrimSuffix(filename, ".mkv"))
|
||||
if m == nil {
|
||||
return ""
|
||||
}
|
||||
switch strings.ToLower(m[1]) {
|
||||
case "dvd":
|
||||
return types.MediaTypeDVD
|
||||
case "bluray":
|
||||
return types.MediaTypeBluRay
|
||||
case "webdl":
|
||||
return types.MediaTypeWebDL
|
||||
case "tvrip":
|
||||
return types.MediaTypeTVRip
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func (c *Client) FetchMovieMetadata(ctx context.Context, imdbID string) (*types.Metadata, error) {
|
||||
url := fmt.Sprintf("%s?i=%s&apikey=%s", omdbAPIURL, imdbID, c.omdbAPIKey)
|
||||
|
||||
resp, err := c.httpClient.Get(url)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("building OMDb request: %w", err)
|
||||
}
|
||||
resp, err := c.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("fetching OMDb: %w", err)
|
||||
}
|
||||
@@ -130,7 +158,7 @@ func (c *Client) FetchMovieMetadata(imdbID string) (*types.Metadata, error) {
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.Metadata, error) {
|
||||
func (c *Client) FetchSeriesMetadata(ctx context.Context, tvmazeID, season, episode string) (*types.Metadata, error) {
|
||||
seasonNum, err := strconv.Atoi(season)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parsing season %q: %w", season, err)
|
||||
@@ -142,13 +170,13 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
|
||||
|
||||
epURL := fmt.Sprintf("%s/shows/%s/episodebynumber?season=%d&number=%d", tvmazeAPIURL, tvmazeID, seasonNum, episodeNum)
|
||||
var ep TVMazeEpisode
|
||||
if err := c.fetchTVMazeJSON(epURL, &ep); err != nil {
|
||||
if err := c.fetchTVMazeJSON(ctx, epURL, &ep); err != nil {
|
||||
return nil, fmt.Errorf("TVmaze episode %s S%dE%d: %w", tvmazeID, seasonNum, episodeNum, err)
|
||||
}
|
||||
|
||||
showURL := fmt.Sprintf("%s/shows/%s", tvmazeAPIURL, tvmazeID)
|
||||
var show TVMazeShowResponse
|
||||
if err := c.fetchTVMazeJSON(showURL, &show); err != nil {
|
||||
if err := c.fetchTVMazeJSON(ctx, showURL, &show); err != nil {
|
||||
return nil, fmt.Errorf("TVmaze show %s: %w", tvmazeID, err)
|
||||
}
|
||||
|
||||
@@ -169,8 +197,12 @@ func (c *Client) FetchSeriesMetadata(tvmazeID, season, episode string) (*types.M
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) fetchTVMazeJSON(url string, v interface{}) error {
|
||||
resp, err := c.httpClient.Get(url)
|
||||
func (c *Client) fetchTVMazeJSON(ctx context.Context, url string, v interface{}) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return fmt.Errorf("build request: %w", err)
|
||||
}
|
||||
resp, err := c.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("GET: %w", err)
|
||||
}
|
||||
|
||||
+69
-13
@@ -1,36 +1,92 @@
|
||||
package mover
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func MoveToOutput(input, outputDir string) error {
|
||||
if err := os.MkdirAll(outputDir, 0755); err != nil {
|
||||
return fmt.Errorf("creating output dir: %w", err)
|
||||
// safeRename moves src to dst. Refuses to overwrite an existing dst.
|
||||
// Falls back to copy+remove on EXDEV (cross-device) errors.
|
||||
func safeRename(src, dst string) error {
|
||||
if _, err := os.Stat(dst); err == nil {
|
||||
return fmt.Errorf("destination %s already exists", dst)
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("stat destination %s: %w", dst, err)
|
||||
}
|
||||
filename := filepath.Base(input)
|
||||
dest := filepath.Join(outputDir, filename)
|
||||
return os.Rename(input, dest)
|
||||
|
||||
err := os.Rename(src, dst)
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
var linkErr *os.LinkError
|
||||
if !errors.As(err, &linkErr) || !errors.Is(err, syscall.EXDEV) {
|
||||
return fmt.Errorf("renaming %s to %s: %w", src, dst, err)
|
||||
}
|
||||
|
||||
return copyAndRemove(src, dst)
|
||||
}
|
||||
|
||||
// copyAndRemove implements the cross-device fallback: copy src to dst, fsync,
|
||||
// then remove src. Uses O_EXCL to belt-and-suspenders against a race.
|
||||
func copyAndRemove(src, dst string) error {
|
||||
in, err := os.Open(src)
|
||||
if err != nil {
|
||||
return fmt.Errorf("opening source %s: %w", src, err)
|
||||
}
|
||||
defer in.Close()
|
||||
|
||||
out, err := os.OpenFile(dst, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0644)
|
||||
if err != nil {
|
||||
return fmt.Errorf("creating destination %s: %w", dst, err)
|
||||
}
|
||||
|
||||
if _, err := io.Copy(out, in); err != nil {
|
||||
out.Close()
|
||||
os.Remove(dst)
|
||||
return fmt.Errorf("copying %s to %s: %w", src, dst, err)
|
||||
}
|
||||
|
||||
if err := out.Sync(); err != nil {
|
||||
out.Close()
|
||||
os.Remove(dst)
|
||||
return fmt.Errorf("syncing destination %s: %w", dst, err)
|
||||
}
|
||||
|
||||
if err := out.Close(); err != nil {
|
||||
os.Remove(dst)
|
||||
return fmt.Errorf("closing destination %s: %w", dst, err)
|
||||
}
|
||||
|
||||
if err := in.Close(); err != nil {
|
||||
return fmt.Errorf("closing source %s: %w", src, err)
|
||||
}
|
||||
|
||||
if err := os.Remove(src); err != nil {
|
||||
return fmt.Errorf("removing source %s: %w", src, err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func MoveToFailed(input, failedDir string) error {
|
||||
if err := os.MkdirAll(failedDir, 0755); err != nil {
|
||||
return fmt.Errorf("creating failed dir: %w", err)
|
||||
}
|
||||
filename := filepath.Base(input)
|
||||
dest := filepath.Join(failedDir, filename)
|
||||
return os.Rename(input, dest)
|
||||
dest := filepath.Join(failedDir, filepath.Base(input))
|
||||
return safeRename(input, dest)
|
||||
}
|
||||
|
||||
func MoveToOriginals(input, originalsDir string) error {
|
||||
if err := os.MkdirAll(originalsDir, 0755); err != nil {
|
||||
return fmt.Errorf("creating originals dir: %w", err)
|
||||
}
|
||||
filename := filepath.Base(input)
|
||||
dest := filepath.Join(originalsDir, filename)
|
||||
return os.Rename(input, dest)
|
||||
dest := filepath.Join(originalsDir, filepath.Base(input))
|
||||
return safeRename(input, dest)
|
||||
}
|
||||
|
||||
func Delete(path string) error {
|
||||
@@ -38,5 +94,5 @@ func Delete(path string) error {
|
||||
}
|
||||
|
||||
func Rename(source, dest string) error {
|
||||
return os.Rename(source, dest)
|
||||
return safeRename(source, dest)
|
||||
}
|
||||
|
||||
@@ -1,16 +1,57 @@
|
||||
package watcher
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"videnc-vibe/pkg/types"
|
||||
)
|
||||
|
||||
// failureBackoff is how long we skip a file after processFn returned an error
|
||||
// for it, provided the file has not been touched since (mtime unchanged). Chosen
|
||||
// to be much larger than the polling interval so a permanently-broken input
|
||||
// (e.g. a video-only file producing "no audio streams found") does not spam the
|
||||
// log every tick, but short enough that an operator who fixes the underlying
|
||||
// problem by replacing the file sees it picked up promptly on the next stable
|
||||
// scan.
|
||||
const failureBackoff = 5 * time.Minute
|
||||
|
||||
// inputExtensions lists the source container extensions the watcher will pick
|
||||
// up. Globbed case-sensitively (matching prior .mkv behavior).
|
||||
var inputExtensions = []string{
|
||||
"mkv", "mp4", "m4v", "mov", "avi",
|
||||
"ts", "m2ts", "mts",
|
||||
"mpg", "mpeg", "vob",
|
||||
"webm", "wmv", "flv",
|
||||
}
|
||||
|
||||
// fileStat is the (mtime, size) pair used to decide whether a file has settled
|
||||
// between two consecutive ticks.
|
||||
type fileStat struct {
|
||||
mtime time.Time
|
||||
size int64
|
||||
}
|
||||
|
||||
// failureRecord remembers that processFn failed for a given file, so we can
|
||||
// back off rather than retrying on every tick.
|
||||
type failureRecord struct {
|
||||
failedAt time.Time
|
||||
fileMtime time.Time
|
||||
}
|
||||
|
||||
type Watcher struct {
|
||||
inputDir string
|
||||
interval time.Duration
|
||||
// seen tracks the last-observed (mtime, size) for every file currently in
|
||||
// the input dir. A file is only handed to processFn once two consecutive
|
||||
// ticks agree on both fields (partial-write protection).
|
||||
seen map[string]fileStat
|
||||
// failed tracks files that recently errored from processFn so we can skip
|
||||
// them until either failureBackoff elapses or their mtime changes.
|
||||
failed map[string]failureRecord
|
||||
}
|
||||
|
||||
func New(inputDir string, intervalSeconds int) *Watcher {
|
||||
@@ -24,39 +65,86 @@ func New(inputDir string, intervalSeconds int) *Watcher {
|
||||
return &Watcher{
|
||||
inputDir: inputDir,
|
||||
interval: interval,
|
||||
seen: make(map[string]fileStat),
|
||||
failed: make(map[string]failureRecord),
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) Start(processFn func(string) error, done chan struct{}) {
|
||||
func (w *Watcher) Start(ctx context.Context, processFn func(context.Context, string) error) {
|
||||
ticker := time.NewTicker(w.interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
w.scanAndProcess(processFn)
|
||||
w.scanAndProcess(ctx, processFn)
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-done:
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-ticker.C:
|
||||
w.scanAndProcess(processFn)
|
||||
w.scanAndProcess(ctx, processFn)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) scanAndProcess(processFn func(string) error) {
|
||||
files, err := filepath.Glob(filepath.Join(w.inputDir, "*.mkv"))
|
||||
func (w *Watcher) scanAndProcess(ctx context.Context, processFn func(context.Context, string) error) {
|
||||
var files []string
|
||||
for _, ext := range inputExtensions {
|
||||
matches, err := filepath.Glob(filepath.Join(w.inputDir, "*."+ext))
|
||||
if err != nil {
|
||||
fmt.Printf("Error scanning input directory: %v\n", err)
|
||||
return
|
||||
}
|
||||
files = append(files, matches...)
|
||||
}
|
||||
|
||||
now := time.Now()
|
||||
nextSeen := make(map[string]fileStat, len(files))
|
||||
nextFailed := make(map[string]failureRecord, len(w.failed))
|
||||
|
||||
for _, file := range files {
|
||||
if err := processFn(file); err != nil {
|
||||
info, err := os.Stat(file)
|
||||
if err != nil {
|
||||
// File vanished between glob and stat, or unreadable. Drop any
|
||||
// state for it by not carrying it forward.
|
||||
fmt.Printf("Error stating %s: %v\n", file, err)
|
||||
continue
|
||||
}
|
||||
|
||||
current := fileStat{mtime: info.ModTime(), size: info.Size()}
|
||||
nextSeen[file] = current
|
||||
|
||||
// Carry the failure record forward only if the file hasn't been
|
||||
// touched since it failed; a changed mtime means the user replaced
|
||||
// or modified the file and we should give it another chance.
|
||||
if rec, ok := w.failed[file]; ok && rec.fileMtime.Equal(current.mtime) {
|
||||
if now.Sub(rec.failedAt) < failureBackoff {
|
||||
nextFailed[file] = rec
|
||||
continue
|
||||
}
|
||||
// Backoff expired; clear the record and let the file be
|
||||
// re-processed if it is otherwise stable.
|
||||
}
|
||||
|
||||
prev, ok := w.seen[file]
|
||||
if !ok || prev.mtime != current.mtime || prev.size != current.size {
|
||||
// First time we've seen this (mtime, size); wait one more tick
|
||||
// to make sure the file isn't still being written.
|
||||
continue
|
||||
}
|
||||
|
||||
if err := processFn(ctx, file); err != nil {
|
||||
fmt.Printf("Error processing %s: %v\n", file, err)
|
||||
nextFailed[file] = failureRecord{
|
||||
failedAt: now,
|
||||
fileMtime: current.mtime,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
w.seen = nextSeen
|
||||
w.failed = nextFailed
|
||||
}
|
||||
|
||||
func DetectMediaType(width, height int) types.MediaType {
|
||||
pixels := width * height
|
||||
if pixels < 600000 {
|
||||
|
||||
@@ -9,6 +9,8 @@ type MediaType string
|
||||
const (
|
||||
MediaTypeDVD MediaType = "DVD"
|
||||
MediaTypeBluRay MediaType = "Blu-ray"
|
||||
MediaTypeWebDL MediaType = "WebDL"
|
||||
MediaTypeTVRip MediaType = "TVRip"
|
||||
)
|
||||
|
||||
type Job struct {
|
||||
@@ -42,6 +44,8 @@ type Config struct {
|
||||
type EncodingConfig struct {
|
||||
DVD EncodingParams `yaml:"dvd"`
|
||||
Bluray EncodingParams `yaml:"bluray"`
|
||||
WebDL EncodingParams `yaml:"webdl"`
|
||||
TVRip EncodingParams `yaml:"tvrip"`
|
||||
}
|
||||
|
||||
type EncodingParams struct {
|
||||
@@ -54,6 +58,7 @@ type PathsConfig struct {
|
||||
Output string `yaml:"output"`
|
||||
Originals string `yaml:"originals"`
|
||||
Failed string `yaml:"failed"`
|
||||
Work string `yaml:"work"`
|
||||
}
|
||||
|
||||
type LogEntry struct {
|
||||
|
||||
Reference in New Issue
Block a user