Replace the handoff notes with a durable spec and ADRs
The handoff was written as a message to relay information; several decisions lived only there. Split into permanent homes: - docs/spec.md — scope, geometry model, pipeline order, detection, levels, editor, project file, markers, bundle format, noteman's changes, reference values - ADR 0002 — raster only in release 1; SVG slices measured at 40x WebP naive, 2.6x with a bounding-box cull, deferred on risk not size - ADR 0003 — lossless WebP with levels and 16-level alpha; every lossy option and alternative format measured larger for line art - ADR 0004 — detection proposes, the human disposes; no unattended mode - ADR 0005 — PyMuPDF for all PDF access, accepting AGPL - ADR 0006 — systems are found by vertical brackets, not row-darkness gaps, which miscount every page of a 6-voice score Also from testing against the hardest score in the repertoire: scanned PDFs carry their scan as an embedded image and must be extracted at native resolution rather than re-rendered at 600 DPI, and per-page deskew is mandatory (skew varies -2.6 to +1.2 within one PDF).
This commit is contained in:
+5
-4
@@ -32,10 +32,11 @@ pre-sets it on a page's top and bottom slice when they contain no system.
|
|||||||
_Avoid_: delete, skip, exclude
|
_Avoid_: delete, skip, exclude
|
||||||
|
|
||||||
**Slice image**:
|
**Slice image**:
|
||||||
The rendered artifact of a slice. From a raster source: lossless WebP, RGB pure
|
The rendered artifact of a slice: lossless WebP, RGB pure black,
|
||||||
black, `alpha = 255 − luminance`, width capped at 1920px — paper is transparency,
|
`alpha = 255 − luminance`, width capped at 1920px — paper is transparency, ink is
|
||||||
ink is alpha. From a vector source: SVG with text converted to paths. Both are
|
alpha. Display-ready as produced; nothing downstream reprocesses it. An SVG form
|
||||||
display-ready as produced; nothing downstream reprocesses them.
|
for vector sources is designed but deferred, which is why the geometry model is
|
||||||
|
renderer-agnostic.
|
||||||
_Avoid_: PNG, page image, tile
|
_Avoid_: PNG, page image, tile
|
||||||
|
|
||||||
**Marker**:
|
**Marker**:
|
||||||
|
|||||||
@@ -44,6 +44,18 @@ needs a system package.
|
|||||||
| | |
|
| | |
|
||||||
|---|---|
|
|---|---|
|
||||||
| [CONTEXT.md](CONTEXT.md) | Glossary. What a slice, cut, discard, bundle and song scale actually mean here. Start here. |
|
| [CONTEXT.md](CONTEXT.md) | Glossary. What a slice, cut, discard, bundle and song scale actually mean here. Start here. |
|
||||||
| [slicer-handoff.md](slicer-handoff.md) | The design: pipeline, geometry model, detection, bundle format, and what noteman has to change. |
|
| [docs/spec.md](docs/spec.md) | The specification: pipeline, geometry model, detection, editor, bundle format, and what noteman has to change. |
|
||||||
| [docs/adr/0001](docs/adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md) | Why the slicer owns all image processing and the bundle is the only channel. |
|
|
||||||
| [BACKLOG.md](BACKLOG.md) | Deliberately deferred, with the reasoning that got it deferred. |
|
Deferred work is tracked as issues and milestones on the Gitea repo, not in this
|
||||||
|
tree.
|
||||||
|
|
||||||
|
Decisions that were expensive to reach, each with the evidence behind it:
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| [ADR 0001](docs/adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md) | The slicer owns all image processing; the bundle is the only channel to noteman. |
|
||||||
|
| [ADR 0002](docs/adr/0002-raster-only-svg-renderer-deferred.md) | Raster only in release 1 — measured SVG slice sizes and what they showed. |
|
||||||
|
| [ADR 0003](docs/adr/0003-lossless-webp-with-levels-and-alpha-quantisation.md) | Lossless WebP beats every lossy option and every alternative format here. |
|
||||||
|
| [ADR 0004](docs/adr/0004-detection-proposes-the-human-disposes.md) | No unattended mode: detection suggests, a human confirms. |
|
||||||
|
| [ADR 0005](docs/adr/0005-pymupdf-for-all-pdf-access.md) | PyMuPDF for all PDF access, accepting AGPL. |
|
||||||
|
| [ADR 0006](docs/adr/0006-systems-are-found-by-brackets-not-row-gaps.md) | Systems are found by vertical brackets; row-darkness gaps get it wrong. |
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# Raster only in release 1; the SVG renderer is deferred
|
||||||
|
|
||||||
|
Vector PDFs are most of the newer corpus, and keeping them vector all the way to
|
||||||
|
the viewer was an early goal — sheet music is line art, and SVG stays crisp at any
|
||||||
|
tablet zoom. We measured it before building it, and decided to **rasterize vector
|
||||||
|
sources like everything else in release 1** and revisit the SVG renderer once
|
||||||
|
real songs have been cut.
|
||||||
|
|
||||||
|
## The measurement
|
||||||
|
|
||||||
|
One real vector song, 6 pages, 65 systems, rendered both ways:
|
||||||
|
|
||||||
|
| Approach | Total | vs WebP |
|
||||||
|
|---|---|---|
|
||||||
|
| WebP slices (600 DPI → 1920, ink→alpha, lossless) | 1.19 MB | 1× |
|
||||||
|
| SVG, naive `viewBox` + `clipPath` | 26.0 MB | 40× |
|
||||||
|
| SVG, `set_cropbox` per band | 26.5 MB | 41× |
|
||||||
|
| SVG, bounding-box cull + glyph subset | 3.09 MB | 2.6× |
|
||||||
|
|
||||||
|
- **The naive cut is unusable.** A `viewBox` + `clipPath` slice contains the
|
||||||
|
entire page's geometry and merely hides eleven-twelfths of it.
|
||||||
|
- **`set_cropbox` does not help.** MuPDF renders full page content regardless of
|
||||||
|
the crop, so there is no free version of the cull.
|
||||||
|
- **The cull works.** PyMuPDF emits a `<defs>` glyph table (111 KB of a 256 KB
|
||||||
|
page) referenced by `<use transform="matrix(...)">`, plus body `<path>`
|
||||||
|
elements. Filter both by y-extent, then keep only the glyphs the survivors
|
||||||
|
reference. Roughly 50 lines, 15× improvement.
|
||||||
|
|
||||||
|
## Why defer, given the cull works
|
||||||
|
|
||||||
|
**Not size.** At 3.1 MB vs 1.2 MB per song — 225 MB vs 87 MB across a 73-song
|
||||||
|
corpus — both are nothing on a homelab. The measurement killed the lazy
|
||||||
|
implementation, not the idea.
|
||||||
|
|
||||||
|
What defers it is risk and missing evidence:
|
||||||
|
|
||||||
|
- The cull is **heuristic parsing**: glyph extents bounded at baseline ±14pt,
|
||||||
|
path extents read from raw `d` coordinates. It is over-inclusive by design, so
|
||||||
|
it fails safe — but "fails safe" still means a slice quietly carrying a
|
||||||
|
neighbour's slur, or a hairline dropped because the y-window was wrong on some
|
||||||
|
publisher's output. That needs eyeballing per song, a QA loop the raster path
|
||||||
|
doesn't have.
|
||||||
|
- Rendering 65 complex SVGs in a scrolling column may be slower than 65 WebPs.
|
||||||
|
Unmeasured.
|
||||||
|
- **The deciding question is unanswerable from here**: does 1920px WebP actually
|
||||||
|
feel insufficient when pinch-zooming on a tablet? Cutting real songs answers
|
||||||
|
it; more measurement doesn't.
|
||||||
|
|
||||||
|
Vector PDFs are also the *clean* case for the raster path — deskew is a no-op,
|
||||||
|
detection works best, there are no scan artefacts — so rasterizing them is not a
|
||||||
|
degraded fallback.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- The geometry model stays **renderer-agnostic**, in normalised page coordinates,
|
||||||
|
so adding the SVG renderer later is an output stage rather than a redesign.
|
||||||
|
- **Re-export from the project file** regenerates every song's bundle without
|
||||||
|
repeating human work, so songs cut before the SVG renderer exists are not
|
||||||
|
stranded.
|
||||||
|
- noteman needs no SVG support (`image/svg+xml`, `.svg` content type, CSP header
|
||||||
|
on SVG responses) until the renderer ships.
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# Lossless WebP, with levels and alpha quantised to 16 levels
|
||||||
|
|
||||||
|
Slice images are encoded as **lossless WebP**, with the levels adjustment applied
|
||||||
|
and the alpha channel quantised to 16 levels. About 7 KB per slice, ~450 KB for a
|
||||||
|
65-system song. Every lossy option and every alternative format measured
|
||||||
|
*larger* for this content, which is the opposite of the usual intuition — hence
|
||||||
|
this record.
|
||||||
|
|
||||||
|
## The measurement
|
||||||
|
|
||||||
|
20 slices of one real song, levels applied throughout, relative to plain lossless
|
||||||
|
WebP:
|
||||||
|
|
||||||
|
| | vs baseline | |
|
||||||
|
|---|---|---|
|
||||||
|
| **WebP lossless + alpha quantised to 16** | **68%** | chosen |
|
||||||
|
| AVIF q60 | 90% | lossy, for 10% |
|
||||||
|
| WebP lossless | 100% | baseline |
|
||||||
|
| WebP lossy q85 (alpha) | 107% | |
|
||||||
|
| AVIF q85 | 114% | |
|
||||||
|
| JXL lossless | 130–133% | |
|
||||||
|
| WebP lossy q85 (opaque ink-on-white) | 158% | |
|
||||||
|
| PNG grayscale + alpha | 165% | |
|
||||||
|
| AVIF lossless | 188% | |
|
||||||
|
|
||||||
|
Separately, before levels: applying levels alone takes 338 KB → 211 KB, a 38%
|
||||||
|
reduction.
|
||||||
|
|
||||||
|
## Four results that contradict an instinct
|
||||||
|
|
||||||
|
- **Lossy is bigger than lossless here.** Not a quality problem — the measured
|
||||||
|
difference between q85 and lossless is max 12/255, mean 0.33, i.e. invisible.
|
||||||
|
Lossy VP8 simply spends more bits on sharp black/white edges than VP8L's
|
||||||
|
palette and predictor transforms do, and notation is nothing but sharp edges.
|
||||||
|
The "q85 looks fine" intuition comes from photographs and inverts here.
|
||||||
|
- **AVIF and JXL both lose**, AVIF lossless by nearly 2×. Their lossless modes
|
||||||
|
are afterthoughts on photo codecs. WebP's VP8L is close to purpose-built for
|
||||||
|
flat two-tone line art — sheet music is the content type it is best at. JXL
|
||||||
|
additionally has no path forward in Chrome.
|
||||||
|
- **Alpha costs nothing.** Opaque ink-on-white and black-plus-alpha are within
|
||||||
|
0.1% at lossless, so paper-tint removal and future non-rectangular slices are
|
||||||
|
free.
|
||||||
|
- **Levels is the single biggest lever** — 38%, as a side effect of a control
|
||||||
|
that exists for quality reasons anyway. Pushing the white point below the
|
||||||
|
paper's luminance sets vast regions to exactly `alpha = 0`, which costs almost
|
||||||
|
nothing to encode.
|
||||||
|
|
||||||
|
Alpha quantisation to 16 levels is imperceptible: antialiased edges span 2–3 px
|
||||||
|
at 1920, and 16 steps across that is below notice. 8 levels starts to gamble on
|
||||||
|
thin strokes.
|
||||||
|
|
||||||
|
## Rejected as not worth it
|
||||||
|
|
||||||
|
- **Encoder effort tuning** — Pillow's `method=6` buys 3% and a dependency.
|
||||||
|
- **`alpha_quality=60`** — 24%, for less control than quantisation gives.
|
||||||
|
- **Grayscale WebP** — no such mode exists. It wouldn't help anyway: the RGB
|
||||||
|
channels are constant black and compress to nearly nothing, so alpha is the
|
||||||
|
entire payload.
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
# Detection proposes, the human disposes — there is no unattended mode
|
||||||
|
|
||||||
|
Every automatic result the slicer produces — skew angle, cut positions, source
|
||||||
|
type, staff height, ink bounds — is a **suggestion the user confirms or modifies**
|
||||||
|
before it is committed. There is no batch mode, no headless "slice this folder",
|
||||||
|
and no code path that writes a bundle without a human having looked at it.
|
||||||
|
|
||||||
|
This is a constraint on the tool's shape, not a UI preference, which is why it
|
||||||
|
gets an ADR: it deletes an entire phase of the original plan and it will look
|
||||||
|
like a missing feature to anyone who finds the detection code and wonders why it
|
||||||
|
isn't wired to a CLI.
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
The corpus is PDFs from a choir's distribution channel, and quality varies
|
||||||
|
wildly — clean vector engravings at one end, noisy scans with a previous owner's
|
||||||
|
pencil markings at the other. **Testing showed the detection algorithms produce
|
||||||
|
unusable slices on any source with speckles or otherwise poor quality.** Not
|
||||||
|
slightly-off slices: unusable ones.
|
||||||
|
|
||||||
|
But the same testing showed the suggestions land *close* on decent sources —
|
||||||
|
close enough that correcting them is faster than placing cuts from scratch. So
|
||||||
|
detection earns its place as an accelerator, and loses any claim to being
|
||||||
|
load-bearing.
|
||||||
|
|
||||||
|
## What this rejected
|
||||||
|
|
||||||
|
The original plan's **Phase A** was a deliberately non-interactive CLI:
|
||||||
|
rasterize, auto-deskew, auto-detect boundaries, write numbered slices, and fix
|
||||||
|
the misses by hand in GIMP. Its justification was "learn the failure modes before
|
||||||
|
designing the editor," which is a good idea.
|
||||||
|
|
||||||
|
It doesn't survive the premise. A CLI whose output can't be trusted has GIMP as
|
||||||
|
its repair path — routing work back into the manual process the project exists to
|
||||||
|
remove. A diagnostic variant (dump per-page PNGs with proposed cuts drawn in red)
|
||||||
|
was considered and also dropped: it only re-shows a failure already confirmed by
|
||||||
|
testing, and the editor shows the same thing live.
|
||||||
|
|
||||||
|
Release 1 is therefore the editor and detection together. There is no smaller
|
||||||
|
first release that is actually usable.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- **Manual placement is the primary interaction**, not a correction affordance.
|
||||||
|
The editor must be fully usable with detection producing nothing.
|
||||||
|
- **Despeckling targets the detector, not the output.** The known failure mode is
|
||||||
|
specks, so a median blur and a small-component filter clean the row-darkness
|
||||||
|
profile the detector reads; the shipped pixels come from the levels-adjusted
|
||||||
|
image.
|
||||||
|
- Cut placement is deliberately **forgiving** — anywhere in the whitespace gap
|
||||||
|
yields the same output, since trim crops to ink afterwards. Precision is not
|
||||||
|
asked of the human.
|
||||||
|
- The editor should surface **slice edges**, not just cut lines, so trim
|
||||||
|
anomalies (a speck anchoring the bounding box) are visible rather than
|
||||||
|
discovered later in the viewer.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# PyMuPDF for all PDF access, accepting AGPL
|
||||||
|
|
||||||
|
All PDF work — rasterizing at a chosen DPI, exporting SVG, and inspecting page
|
||||||
|
content to classify a source as bitmap or vector — goes through **PyMuPDF**. It
|
||||||
|
is a single wheel with MuPDF bundled, so the tool needs no system packages. Its
|
||||||
|
licence is **AGPL-3.0**, which we accept.
|
||||||
|
|
||||||
|
## Why not the permissive combination
|
||||||
|
|
||||||
|
The obvious permissive stack was `pypdfium2` (Apache/BSD) for rasterizing plus
|
||||||
|
`mutool` or `pdftocairo` shelled out for SVG. Both of those are **system
|
||||||
|
packages** — `mupdf-tools`, `poppler` — and a system package on the vector path
|
||||||
|
is precisely the failure the language choice was made to avoid: the tool is
|
||||||
|
supposed to install once and run from any directory on any machine.
|
||||||
|
|
||||||
|
The SVG step can't simply be skipped, either. Music glyphs come from a notation
|
||||||
|
font (Emmentaler, Bravura, or Sibelius/Finale's). An SVG that *references* a font
|
||||||
|
renders as garbage on a device that lacks it, so text must be converted to paths
|
||||||
|
at export. PyMuPDF does this **by default** — `page.get_svg_image(text_as_path=1)`,
|
||||||
|
verified to emit `<path>` elements and zero `<text>` — so the font risk is closed
|
||||||
|
with no extra tooling.
|
||||||
|
|
||||||
|
Mixing the two (pypdfium2 for raster, PyMuPDF only for SVG) is the worst option:
|
||||||
|
two libraries with overlapping responsibilities, and AGPL linked in anyway.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- **The AGPL propagates only if the slicer is published.** For a local personal
|
||||||
|
tool it costs nothing. A future permissive release would need the rasterizer
|
||||||
|
swapped back to `pypdfium2` — a contained change, since PDF access sits behind
|
||||||
|
the renderer-agnostic geometry model.
|
||||||
|
- **Source-type detection comes free** from the same library: `get_images()` plus
|
||||||
|
a full-page-image area check distinguishes a scan from an engraving.
|
||||||
|
- The SVG export path is present and working even though the SVG *renderer* is
|
||||||
|
deferred — see
|
||||||
|
[ADR 0002](0002-raster-only-svg-renderer-deferred.md).
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
# Systems are found by vertical brackets, not by row-darkness gaps
|
||||||
|
|
||||||
|
System detection anchors on the **vertical bracket / barline** that spans a
|
||||||
|
system's staves, and uses the row-darkness profile only to expand each anchor to
|
||||||
|
its ink extent. The obvious approach — find gaps in the row-darkness profile and
|
||||||
|
cut in the middle of them — does not work on multi-voice choral scores, which is
|
||||||
|
most of the corpus.
|
||||||
|
|
||||||
|
## Why the obvious approach fails
|
||||||
|
|
||||||
|
A row-darkness profile cannot distinguish an **inter-staff** gap from an
|
||||||
|
**inter-system** gap. In a 6-voice closed score, one system is six staves joined
|
||||||
|
by a bracket, and the gaps between those six staves look exactly like the gap
|
||||||
|
between two systems — only smaller, and not reliably so.
|
||||||
|
|
||||||
|
Measured on *Ketun joululaulu*, a 12-page 6-voice arrangement and the hardest
|
||||||
|
score in the repertoire:
|
||||||
|
|
||||||
|
- On page 2's first system, staff gaps run ~47px against a ~211px system gap. A
|
||||||
|
merge threshold tuned there works.
|
||||||
|
- On the same page's second system the lyrics fill the inter-staff gaps, so the
|
||||||
|
ratios invert and the same threshold merges the wrong things.
|
||||||
|
|
||||||
|
Result across all 12 pages, row-profile-only versus bracket-anchored:
|
||||||
|
|
||||||
|
| | bracket-anchored | row-profile only |
|
||||||
|
|---|---|---|
|
||||||
|
| systems per page | 2, 2, 2, 2, 2, 2, 3, 2, 2, 2, 2, 1 | 10, 6, 7, 7, 7, 8, 8, 5, 4, 5, 8, 4 |
|
||||||
|
|
||||||
|
The bracket-anchored counts match the score. The row-profile counts are wrong on
|
||||||
|
every page, and wrong by a different amount each time — so no threshold fixes
|
||||||
|
them.
|
||||||
|
|
||||||
|
## The algorithm
|
||||||
|
|
||||||
|
1. **Deskew per page.** Projection-profile variance sweep over ±5°. Measured skew
|
||||||
|
on this song ranges −2.6° to +1.2° *between pages of the same PDF*, so per-page
|
||||||
|
is not optional.
|
||||||
|
2. **Find anchors.** Binarise, then morphological open with a tall thin kernel
|
||||||
|
(height ≈ 3% of the page) so only long vertical strokes survive. Take
|
||||||
|
connected components taller than 4% of the page; walk them tallest-first,
|
||||||
|
keeping each one whose y-extent doesn't overlap an already-kept anchor. Each
|
||||||
|
surviving stroke is one system.
|
||||||
|
3. **Expand to ink.** Compute the row-darkness profile on a despeckled copy, take
|
||||||
|
its ink runs, and assign each run to the nearest anchor by centre distance. A
|
||||||
|
system's extent is the union of its runs.
|
||||||
|
4. **Place cuts** at the midpoint between consecutive systems' ink extents.
|
||||||
|
|
||||||
|
Step 3 is what makes this work rather than the bracket alone: a bracket stops at
|
||||||
|
the last staff line, but the slice must include the **lyrics below it**. On page
|
||||||
|
2, system 1's bracket spans 177–994 while its true ink extent is 179–1071 — the
|
||||||
|
77px difference is the bottom voice's lyric line, which the bracket misses
|
||||||
|
entirely and the row profile finds.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Detection needs both signals. Neither the column pass nor the row pass is
|
||||||
|
sufficient alone, so `detect.py` computes both.
|
||||||
|
- **Scores without brackets** — single-staff melodies, lead sheets — have no
|
||||||
|
anchors, and fall back to row-profile runs. That fallback is the *only* correct
|
||||||
|
behaviour there, since every ink run genuinely is its own system.
|
||||||
|
- Bar numbers printed above a system (this score uses 11, 16, …) sit in their own
|
||||||
|
ink run and get absorbed into the nearest system by step 3. That is right: they
|
||||||
|
belong to the system they label.
|
||||||
|
- A page number can be absorbed the same way if its darkness clears the profile
|
||||||
|
threshold, inflating the last system's extent. The content rectangle and the
|
||||||
|
bottom discard slice both prevent this; don't rely on the threshold.
|
||||||
+377
@@ -0,0 +1,377 @@
|
|||||||
|
# noteman-slicer — specification
|
||||||
|
|
||||||
|
What the tool does and how it behaves. Vocabulary is in
|
||||||
|
[`CONTEXT.md`](../CONTEXT.md); the reasoning behind the expensive decisions is in
|
||||||
|
[`docs/adr/`](adr/).
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
A local, single-user tool that turns a score PDF into the ordered slice images
|
||||||
|
[noteman](../../noteman) consumes, plus the navigation markers that sit on them.
|
||||||
|
It automates the mechanical part of noteman's ingestion boundary.
|
||||||
|
|
||||||
|
It is **not** a GIMP replacement. Erasing previous-owner pencil marks, chord
|
||||||
|
letters and breath marks stays in GIMP — the irreducible manual part, which GIMP
|
||||||
|
with a stylus already does well.
|
||||||
|
|
||||||
|
**One PDF → one song → one project → one bundle.** Never a many-to-one in any
|
||||||
|
direction. A PDF is either bitmap or vector, never mixed.
|
||||||
|
|
||||||
|
### Why it's separate from noteman
|
||||||
|
|
||||||
|
Splitting it out removed the double-implementation constraint — in-app, every
|
||||||
|
operation needs both a fast browser preview and a real server-side render, and
|
||||||
|
that constraint is what priced dewarp and brush masking out entirely, not the
|
||||||
|
algorithms. It also removed infrastructure noteman doesn't otherwise need (a
|
||||||
|
scratch workspace for multi-MB rasters, an edit-list table, cleanup sweeps for
|
||||||
|
orphaned temp files, poppler in the Docker image, an admin UI surface), and
|
||||||
|
unlocked real image libraries.
|
||||||
|
|
||||||
|
It costs nothing: song creation is admin-only, done at home, once per song.
|
||||||
|
|
||||||
|
## Operating principle
|
||||||
|
|
||||||
|
**Detection proposes, the human disposes.** Every automatic result — skew angle,
|
||||||
|
cut positions, source type, staff height, ink bounds — is a suggestion the user
|
||||||
|
confirms or modifies before it is committed. There is no unattended mode. See
|
||||||
|
[ADR 0004](adr/0004-detection-proposes-the-human-disposes.md).
|
||||||
|
|
||||||
|
## Geometry model
|
||||||
|
|
||||||
|
**One geometry model, two renderers.** Geometry is stored in **normalised page
|
||||||
|
coordinates** (0–1 of page width and height), independent of DPI and of which
|
||||||
|
renderer produces the output. Only the final stage differs.
|
||||||
|
|
||||||
|
| Concept | Raster | Vector |
|
||||||
|
|---|---|---|
|
||||||
|
| Cut | y in pixels | y in PDF user space |
|
||||||
|
| Discard | drop the slice | drop the slice |
|
||||||
|
| Content rectangle | crop before cutting | clip before cutting |
|
||||||
|
| Trim | crop to ink bbox | crop `viewBox` to ink bbox |
|
||||||
|
| Uniform width | transparent right pad | wider `viewBox`, same content |
|
||||||
|
| Staff-height normalise | scale factor | scale factor |
|
||||||
|
| Deskew, levels, ink→alpha, 1920 cap | yes | no |
|
||||||
|
|
||||||
|
Only the raster renderer ships in release 1 — see
|
||||||
|
[ADR 0002](adr/0002-raster-only-svg-renderer-deferred.md). The editor is one
|
||||||
|
editor regardless, since a vector PDF has to be rasterized just to display it on
|
||||||
|
screen.
|
||||||
|
|
||||||
|
### Pipeline order
|
||||||
|
|
||||||
|
```
|
||||||
|
load raster → deskew → levels → content rect → cut → discard
|
||||||
|
→ trim → scale → pad → ink→alpha → encode
|
||||||
|
```
|
||||||
|
|
||||||
|
**Load raster** differs by source type. A scanned PDF carries one full-page image
|
||||||
|
per page, and that image *is* the scan — extract it at its native resolution
|
||||||
|
(`extract_image`) rather than re-rendering the page. Re-rendering at a fixed
|
||||||
|
600 DPI resamples a 200 DPI scan up by 3×, which triples the pixel count and adds
|
||||||
|
no detail. A vector PDF has no embedded raster, so it is rendered — see the DPI
|
||||||
|
note in *Reference values*.
|
||||||
|
|
||||||
|
The rest of the order is not arbitrary:
|
||||||
|
|
||||||
|
- **Levels before anything geometric**, so the trim bounding box is computed on
|
||||||
|
the image that actually ships.
|
||||||
|
- **Content rect before cutting**, so margin junk never enters a slice.
|
||||||
|
- **Trim before scale**, since the scale factor derives from the widest *trimmed*
|
||||||
|
slice.
|
||||||
|
|
||||||
|
### Slices, cuts and discard
|
||||||
|
|
||||||
|
A page starts as a single slice; each cut splits one slice into two. Slices
|
||||||
|
therefore tile the page with no gaps and no overlap.
|
||||||
|
|
||||||
|
Headers, footers and blank regions leave the song via a **discard** flag, not via
|
||||||
|
cuts at the page edges. Modelling a slice as "the region between two cuts" leaks:
|
||||||
|
page 2 has no header, so it would need an invented top cut whose position depends
|
||||||
|
on whether that page happens to have one.
|
||||||
|
|
||||||
|
Cut placement is forgiving — anywhere inside the whitespace gap yields the same
|
||||||
|
output, because trim crops to ink afterwards.
|
||||||
|
|
||||||
|
### Content rectangle
|
||||||
|
|
||||||
|
The region of a page that holds music, set per PDF and adjustable per page,
|
||||||
|
applied before cutting. Everything outside it is dropped.
|
||||||
|
|
||||||
|
This handles margin junk structurally rather than case-by-case, because margin
|
||||||
|
junk is by definition outside the music: scan-edge bands, spine shadows, and page
|
||||||
|
numbers printed in the side margin level with a system. That last one matters
|
||||||
|
more than it looks — see the trim consequences below.
|
||||||
|
|
||||||
|
### Trim, scale, pad
|
||||||
|
|
||||||
|
**Trim** tight on all four sides, per slice. This normalises away the left-margin
|
||||||
|
drift between scanned pages, and flattens the engraved indent of the first
|
||||||
|
system — correct here, since noteman strips the printed header the indent made
|
||||||
|
room for.
|
||||||
|
|
||||||
|
Two consequences:
|
||||||
|
|
||||||
|
- A stray speck at the far left anchors the trim, shifting that slice relative to
|
||||||
|
its neighbours. Mitigate by ignoring connected components under a few hundred
|
||||||
|
pixels (`cv2.connectedComponentsWithStats`) when computing the bounding box.
|
||||||
|
- A page number in the side margin level with a system would set that slice's
|
||||||
|
bounding box, which sets the song's widest slice, which scales the whole song
|
||||||
|
down. One artefact, whole song smaller. Hence the content rectangle.
|
||||||
|
|
||||||
|
**Scale is normalised on staff height, not width.** Width-based scaling assumes
|
||||||
|
every slice comes from the same scan at the same DPI. It breaks for a rescanned
|
||||||
|
page, a PDF mixing scan generations, or a re-engraved replacement system — whose
|
||||||
|
width depends on how much music is in it, not on matching its neighbours. Staff
|
||||||
|
height is the invariant a reader perceives as "the notes are the same size", and
|
||||||
|
it falls out of the same row-darkness profile detection already computes.
|
||||||
|
|
||||||
|
Two steps, both per song: normalise every slice to a common staff height, then
|
||||||
|
scale the song uniformly so its widest slice lands at **1920px**. That is a
|
||||||
|
ceiling, never a target — **never upscale**. A song that comes out narrower stays
|
||||||
|
narrower; enlarging a 600 DPI scan past its real resolution buys softness and
|
||||||
|
bytes and no detail.
|
||||||
|
|
||||||
|
**Pad** narrower slices with transparency on the right, so every slice in a song
|
||||||
|
is the same width, flush left, notes the same size. A short system simply ends
|
||||||
|
earlier.
|
||||||
|
|
||||||
|
### Encoding
|
||||||
|
|
||||||
|
**Lossless WebP, with levels applied and alpha quantised to 16 levels.** Roughly
|
||||||
|
7 KB per slice, about 450 KB for a 65-system song. Lossy encodings and the
|
||||||
|
alternative formats are all *larger* for this content — measured, with the
|
||||||
|
figures, in
|
||||||
|
[ADR 0003](adr/0003-lossless-webp-with-levels-and-alpha-quantisation.md).
|
||||||
|
|
||||||
|
Ink handling is luminance → alpha: ink forced to pure black,
|
||||||
|
`alpha = 255 − luminance`. Not `pixel == white` thresholding — staff lines are
|
||||||
|
antialiased, and binary removal leaves jagged edges.
|
||||||
|
|
||||||
|
## Detection
|
||||||
|
|
||||||
|
All of it is a suggestion, all of it overridable.
|
||||||
|
|
||||||
|
**Deskew** — per page, and not optionally so: measured skew varies from −2.6° to
|
||||||
|
+1.2° *between pages of the same PDF*. Staff lines are by far the strongest
|
||||||
|
horizontal signal in sheet music, so a projection-profile variance sweep over ±5°
|
||||||
|
finds the angle reliably — sum row-darkness for each candidate angle, take the
|
||||||
|
angle of maximum variance. Run on a downscaled copy. Pair with a manual slider.
|
||||||
|
|
||||||
|
**Systems** — anchored on the **vertical bracket** that spans a system's staves,
|
||||||
|
not on gaps in the row-darkness profile. A row profile cannot distinguish an
|
||||||
|
inter-staff gap from an inter-system gap on multi-voice choral scores, and gets
|
||||||
|
the system count wrong on every page. See
|
||||||
|
[ADR 0006](adr/0006-systems-are-found-by-brackets-not-row-gaps.md) for the
|
||||||
|
measurement and the full algorithm. In outline:
|
||||||
|
|
||||||
|
1. Binarise; morphological open with a tall thin kernel so only long vertical
|
||||||
|
strokes survive; keep non-overlapping components taller than 4% of the page.
|
||||||
|
Each is one system.
|
||||||
|
2. Take ink runs from the row-darkness profile and assign each to the nearest
|
||||||
|
anchor. A system's extent is the union of its runs — this is what pulls in the
|
||||||
|
lyrics printed *below* the last staff, which the bracket stops short of.
|
||||||
|
3. Propose cuts at the midpoint between consecutive systems' ink extents, and
|
||||||
|
pre-set the discard flag on a page's top and bottom slice when they contain no
|
||||||
|
system.
|
||||||
|
|
||||||
|
Scores with no bracket — single-staff melodies, lead sheets — have no anchors and
|
||||||
|
fall back to row-profile runs, which is correct there.
|
||||||
|
|
||||||
|
**Staff height** — peak-to-peak spacing in the row profile.
|
||||||
|
|
||||||
|
**Source type** — `get_images(full=True)` / `get_drawings()` proposes bitmap or
|
||||||
|
vector per PDF; the tool asks the user to confirm before routing. (`full=True` is
|
||||||
|
required, or `get_image_bbox` rejects the item.)
|
||||||
|
|
||||||
|
**Despeckle feeds detection only.** A median blur plus dropping tiny connected
|
||||||
|
components denoises the *profile the detector reads*; the shipped pixels come
|
||||||
|
from the levels-adjusted image. The known failure mode is specks, so the fix
|
||||||
|
belongs on the signal, not the output.
|
||||||
|
|
||||||
|
## Levels
|
||||||
|
|
||||||
|
Two sliders per song (black point, white point) applied via `cv2.LUT`, with a
|
||||||
|
per-page override.
|
||||||
|
|
||||||
|
In release 1, not deferred: with `alpha = 255 − luminance`, a scan's greyness
|
||||||
|
*becomes* transparency, so a faint or yellowed source produces washed-out notes
|
||||||
|
on a hazy background and **nothing downstream can rescue it**. Set the white
|
||||||
|
point just under the paper's luminance and the paper vanishes completely; set the
|
||||||
|
black point at the ink's darkest and notes go solid. It is also the single
|
||||||
|
biggest lever on output size.
|
||||||
|
|
||||||
|
Adaptive methods (CLAHE, adaptive thresholding) are the trap — tuned for text,
|
||||||
|
they eat the thin stuff on notation: hairpin tips, slur ends, ledger lines,
|
||||||
|
tapered beams. A global LUT whose effect you can see beats a local algorithm you
|
||||||
|
can't predict.
|
||||||
|
|
||||||
|
## Editor
|
||||||
|
|
||||||
|
**PySide6.** `QGraphicsView` provides the viewport — pan, zoom, screen↔image
|
||||||
|
coordinate mapping, resampling, hit-testing — which would otherwise be ~150 lines
|
||||||
|
of hand-rolled geometry. `cv2.imshow` was rejected: OpenCV's highgui is GTK/X11
|
||||||
|
and lands on XWayland at best, and it has no text input at all.
|
||||||
|
|
||||||
|
What the editor does: pan and zoom the page, drag cut lines, toggle discard,
|
||||||
|
adjust the content rectangle, move the levels sliders, place markers, fill in
|
||||||
|
song metadata, export.
|
||||||
|
|
||||||
|
Marker placement needs a **slice picker** — a `QListView` in icon mode over the
|
||||||
|
slice previews — since every jump source stores an explicit target. One widget
|
||||||
|
serving all six jump types.
|
||||||
|
|
||||||
|
## Project file
|
||||||
|
|
||||||
|
Autosaved JSON beside the source PDF, holding the source path and hash, cuts,
|
||||||
|
discards, content rectangle, skew angles, levels, staff-height overrides, markers
|
||||||
|
and metadata. The bundle is *generated* from it, so export is a pure function of
|
||||||
|
the project file plus the PDF.
|
||||||
|
|
||||||
|
It buys crash safety, resume across sessions (authoring is trickle-in), and
|
||||||
|
**re-export** — change the 1920 cap, fix one cut, or add the SVG renderer later,
|
||||||
|
and every song's bundle regenerates without repeating any human work.
|
||||||
|
|
||||||
|
The project file references the PDF and never contains it; the hash lets the
|
||||||
|
editor warn if the PDF changed underneath.
|
||||||
|
|
||||||
|
## Markers
|
||||||
|
|
||||||
|
Placed here rather than in noteman: at cut time you are already reading the score
|
||||||
|
page by page at full resolution, so the Segno, the Coda sign, the "to coda" text
|
||||||
|
and the rehearsal letters are on screen. Deferring means reading the whole score a
|
||||||
|
second time to find the same symbols.
|
||||||
|
|
||||||
|
noteman's vocabulary, carried verbatim — `rehearsal_letter`, `section_label`,
|
||||||
|
`segno`, `coda`, `fine`, `repeat_start`, `repeat_end`, `volta`, `to_coda`,
|
||||||
|
`ds_al_coda`, `ds_al_fine`, `dc_al_coda`, `dc_al_fine`, `generic_jump`. A small
|
||||||
|
stable enum, but real coupling: adding a type means changing both repos.
|
||||||
|
|
||||||
|
Three shapes among them:
|
||||||
|
|
||||||
|
- **Bare tags:** `segno`, `coda`, `fine`, `repeat_start`, `repeat_end`.
|
||||||
|
- **Tags with free text:** `rehearsal_letter` ("C"), `section_label` ("CHORUS"),
|
||||||
|
`volta` ("1.").
|
||||||
|
- **Jump sources:** `to_coda`, `ds_al_coda`, `ds_al_fine`, `dc_al_coda`,
|
||||||
|
`dc_al_fine`, `generic_jump`.
|
||||||
|
|
||||||
|
**Every jump source stores its target slice explicitly.** noteman's viewer
|
||||||
|
currently resolves by type — a `to_coda` finds the song's unique `coda` at tap
|
||||||
|
time — but that puts an unwritten "exactly one Coda per song" invariant into a
|
||||||
|
contract between two separately-maintained repos, enforced by neither. Authoring
|
||||||
|
the target costs one click on a slice already on screen, and in exchange the
|
||||||
|
bundle is self-describing and a score with two codas simply works.
|
||||||
|
|
||||||
|
## Bundle
|
||||||
|
|
||||||
|
The only channel to noteman. No API, no direct upload — see
|
||||||
|
[ADR 0001](adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md).
|
||||||
|
|
||||||
|
```
|
||||||
|
song.zip
|
||||||
|
song.json
|
||||||
|
original.pdf
|
||||||
|
001.webp 002.webp …
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"v": 1,
|
||||||
|
"title": "…", "composer": "…", "arranger": "…",
|
||||||
|
"slices": [
|
||||||
|
{ "file": "001.webp" },
|
||||||
|
{ "file": "002.webp", "markers": [{ "type": "rehearsal_letter", "label": "A" }] },
|
||||||
|
{ "file": "003.webp", "markers": [{ "type": "to_coda", "destination": 7 }] }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Array order **is** slice order — one ordering, not two. Markers nest inside the
|
||||||
|
slice they sit on, so indices appear in exactly one place: a jump source's
|
||||||
|
`destination`.
|
||||||
|
|
||||||
|
`"v": 1` is eight bytes of insurance. The bundle is the only channel, MIDI and
|
||||||
|
MP3s are planned for a later phase, and bundles are archived artifacts that may be
|
||||||
|
re-imported a year later.
|
||||||
|
|
||||||
|
Otherwise: plain zip, no manifest beyond this, no checksums, hand-fixable.
|
||||||
|
Python's `zipfile` is stdlib; the import side needs one zero-dep library
|
||||||
|
(`fflate`), since Bun has zlib but no zip reader.
|
||||||
|
|
||||||
|
**Contents:** slices, markers, the original PDF, and song-level text metadata
|
||||||
|
(title, subtitle, composer, original artist, arranger, lyricist, translator,
|
||||||
|
voice list). Metadata is included not because the slicer transforms it but
|
||||||
|
because you have to read the title block anyway to mark the header slice
|
||||||
|
discarded — typing eight fields while it's on screen beats reopening the PDF
|
||||||
|
later.
|
||||||
|
|
||||||
|
Rehearsal MIDI and MP3s are deliberately out of the first bundle.
|
||||||
|
|
||||||
|
### One rule for the import side
|
||||||
|
|
||||||
|
**Import creates a new song only; never re-import onto an existing one.** Jump
|
||||||
|
destinations reference slices by ID, so replacing a song's slices silently
|
||||||
|
orphans every marker on it. Re-cutting happens *before* marker authoring in
|
||||||
|
practice, so forbidding it costs nothing and prevents a genuinely nasty data-loss
|
||||||
|
mode. Re-export from the project file is the supported path.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
**Python**, chosen for OpenCV access and iteration speed. Installed as a package
|
||||||
|
via `uv tool install --editable .`, which puts a `noteman-slicer` command on PATH
|
||||||
|
that runs from any directory with no venv to activate. The one cwd trap: load
|
||||||
|
bundled data via `Path(__file__).parent` or `importlib.resources`, never a
|
||||||
|
relative path.
|
||||||
|
|
||||||
|
Dependencies: **PyMuPDF**, **PySide6**, **opencv-python-headless**, **numpy** —
|
||||||
|
all wheels, no system packages. PyMuPDF covers every PDF need; see
|
||||||
|
[ADR 0005](adr/0005-pymupdf-for-all-pdf-access.md).
|
||||||
|
|
||||||
|
Verified: `cv2` 5.0.0 writes 4-channel lossless WebP with alpha preserved
|
||||||
|
byte-exact (`IMWRITE_WEBP_QUALITY, 101`).
|
||||||
|
|
||||||
|
Module boundaries: `pdf.py` (load, source-type detect, rasterize), `detect.py`
|
||||||
|
(deskew, row-darkness profile, system runs, staff height), `bundle.py`,
|
||||||
|
`editor.py`.
|
||||||
|
|
||||||
|
## Changes required in noteman
|
||||||
|
|
||||||
|
On noteman's timeline, not the slicer's — but release 1 produces artifacts
|
||||||
|
nothing consumes until this lands.
|
||||||
|
|
||||||
|
1. **Delete the sharp normalisation pipeline.** The slicer's output is final.
|
||||||
|
2. **Bundle import** — unzip → read `song.json` → create song → insert slices in
|
||||||
|
array order → insert markers, mapping index → new slice UUID → store the PDF.
|
||||||
|
3. **Jump sources carry explicit destinations** — `destinationSliceId` is already
|
||||||
|
nullable on every marker type, so this is viewer logic, not schema.
|
||||||
|
|
||||||
|
SVG support on the noteman side (`image/svg+xml` in the upload path, `.svg` in
|
||||||
|
`CONTENT_TYPES`, and a CSP header on SVG responses) is not needed until the SVG
|
||||||
|
renderer ships.
|
||||||
|
|
||||||
|
## Phasing
|
||||||
|
|
||||||
|
**Release 1 — editor + detection + bundle export, raster only.** Vector PDFs are
|
||||||
|
rasterized like everything else; they're the clean case, where deskew is a no-op
|
||||||
|
and detection works best. Levels, content rectangle, discard, markers, project
|
||||||
|
file.
|
||||||
|
|
||||||
|
Everything else is deferred and tracked as issues on the Gitea repo.
|
||||||
|
|
||||||
|
## Reference values
|
||||||
|
|
||||||
|
- Final slice width cap: **1920px**, matching the viewer sheet's max-width. A
|
||||||
|
ceiling, not a target.
|
||||||
|
- Output format: **lossless WebP**.
|
||||||
|
- Working resolution:
|
||||||
|
- **Scanned sources — the embedded image's native resolution.** Never
|
||||||
|
re-render. Real scans in this corpus run ~200 DPI (1653×2332 for A4), which
|
||||||
|
is *below* the 1920 cap, so those songs ship narrower than 1920 and are never
|
||||||
|
upscaled.
|
||||||
|
- **Vector sources — 600 DPI**, configurable. A4 @ 600 DPI is ~4960×7016 px;
|
||||||
|
the ~2.6× downsample to 1920 is itself a quality win via antialiasing. 300
|
||||||
|
DPI would suffice for the target, but 600 buys headroom for deskew
|
||||||
|
resampling.
|
||||||
|
- A slice = **one system** = one full line of music across all voices, typically
|
||||||
|
4–12 bars, lyrics intact.
|
||||||
|
- Upload/bundle sizes are not constrained by noteman's old 25 MB/file limits —
|
||||||
|
the bundle bypasses that path entirely.
|
||||||
Reference in New Issue
Block a user