Replace the handoff notes with a durable spec and ADRs
The handoff was written as a message to relay information; several decisions lived only there. Split into permanent homes: - docs/spec.md — scope, geometry model, pipeline order, detection, levels, editor, project file, markers, bundle format, noteman's changes, reference values - ADR 0002 — raster only in release 1; SVG slices measured at 40x WebP naive, 2.6x with a bounding-box cull, deferred on risk not size - ADR 0003 — lossless WebP with levels and 16-level alpha; every lossy option and alternative format measured larger for line art - ADR 0004 — detection proposes, the human disposes; no unattended mode - ADR 0005 — PyMuPDF for all PDF access, accepting AGPL - ADR 0006 — systems are found by vertical brackets, not row-darkness gaps, which miscount every page of a 6-voice score Also from testing against the hardest score in the repertoire: scanned PDFs carry their scan as an embedded image and must be extracted at native resolution rather than re-rendered at 600 DPI, and per-page deskew is mandatory (skew varies -2.6 to +1.2 within one PDF).
This commit is contained in:
Esa Kataja
+5
-4
@@ -32,10 +32,11 @@ pre-sets it on a page's top and bottom slice when they contain no system.
|
||||
_Avoid_: delete, skip, exclude
|
||||
|
||||
**Slice image**:
|
||||
The rendered artifact of a slice. From a raster source: lossless WebP, RGB pure
|
||||
black, `alpha = 255 − luminance`, width capped at 1920px — paper is transparency,
|
||||
ink is alpha. From a vector source: SVG with text converted to paths. Both are
|
||||
display-ready as produced; nothing downstream reprocesses them.
|
||||
The rendered artifact of a slice: lossless WebP, RGB pure black,
|
||||
`alpha = 255 − luminance`, width capped at 1920px — paper is transparency, ink is
|
||||
alpha. Display-ready as produced; nothing downstream reprocesses it. An SVG form
|
||||
for vector sources is designed but deferred, which is why the geometry model is
|
||||
renderer-agnostic.
|
||||
_Avoid_: PNG, page image, tile
|
||||
|
||||
**Marker**:
|
||||
|
||||
@@ -44,6 +44,18 @@ needs a system package.
|
||||
| | |
|
||||
|---|---|
|
||||
| [CONTEXT.md](CONTEXT.md) | Glossary. What a slice, cut, discard, bundle and song scale actually mean here. Start here. |
|
||||
| [slicer-handoff.md](slicer-handoff.md) | The design: pipeline, geometry model, detection, bundle format, and what noteman has to change. |
|
||||
| [docs/adr/0001](docs/adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md) | Why the slicer owns all image processing and the bundle is the only channel. |
|
||||
| [BACKLOG.md](BACKLOG.md) | Deliberately deferred, with the reasoning that got it deferred. |
|
||||
| [docs/spec.md](docs/spec.md) | The specification: pipeline, geometry model, detection, editor, bundle format, and what noteman has to change. |
|
||||
|
||||
Deferred work is tracked as issues and milestones on the Gitea repo, not in this
|
||||
tree.
|
||||
|
||||
Decisions that were expensive to reach, each with the evidence behind it:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [ADR 0001](docs/adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md) | The slicer owns all image processing; the bundle is the only channel to noteman. |
|
||||
| [ADR 0002](docs/adr/0002-raster-only-svg-renderer-deferred.md) | Raster only in release 1 — measured SVG slice sizes and what they showed. |
|
||||
| [ADR 0003](docs/adr/0003-lossless-webp-with-levels-and-alpha-quantisation.md) | Lossless WebP beats every lossy option and every alternative format here. |
|
||||
| [ADR 0004](docs/adr/0004-detection-proposes-the-human-disposes.md) | No unattended mode: detection suggests, a human confirms. |
|
||||
| [ADR 0005](docs/adr/0005-pymupdf-for-all-pdf-access.md) | PyMuPDF for all PDF access, accepting AGPL. |
|
||||
| [ADR 0006](docs/adr/0006-systems-are-found-by-brackets-not-row-gaps.md) | Systems are found by vertical brackets; row-darkness gaps get it wrong. |
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# Raster only in release 1; the SVG renderer is deferred
|
||||
|
||||
Vector PDFs are most of the newer corpus, and keeping them vector all the way to
|
||||
the viewer was an early goal — sheet music is line art, and SVG stays crisp at any
|
||||
tablet zoom. We measured it before building it, and decided to **rasterize vector
|
||||
sources like everything else in release 1** and revisit the SVG renderer once
|
||||
real songs have been cut.
|
||||
|
||||
## The measurement
|
||||
|
||||
One real vector song, 6 pages, 65 systems, rendered both ways:
|
||||
|
||||
| Approach | Total | vs WebP |
|
||||
|---|---|---|
|
||||
| WebP slices (600 DPI → 1920, ink→alpha, lossless) | 1.19 MB | 1× |
|
||||
| SVG, naive `viewBox` + `clipPath` | 26.0 MB | 40× |
|
||||
| SVG, `set_cropbox` per band | 26.5 MB | 41× |
|
||||
| SVG, bounding-box cull + glyph subset | 3.09 MB | 2.6× |
|
||||
|
||||
- **The naive cut is unusable.** A `viewBox` + `clipPath` slice contains the
|
||||
entire page's geometry and merely hides eleven-twelfths of it.
|
||||
- **`set_cropbox` does not help.** MuPDF renders full page content regardless of
|
||||
the crop, so there is no free version of the cull.
|
||||
- **The cull works.** PyMuPDF emits a `<defs>` glyph table (111 KB of a 256 KB
|
||||
page) referenced by `<use transform="matrix(...)">`, plus body `<path>`
|
||||
elements. Filter both by y-extent, then keep only the glyphs the survivors
|
||||
reference. Roughly 50 lines, 15× improvement.
|
||||
|
||||
## Why defer, given the cull works
|
||||
|
||||
**Not size.** At 3.1 MB vs 1.2 MB per song — 225 MB vs 87 MB across a 73-song
|
||||
corpus — both are nothing on a homelab. The measurement killed the lazy
|
||||
implementation, not the idea.
|
||||
|
||||
What defers it is risk and missing evidence:
|
||||
|
||||
- The cull is **heuristic parsing**: glyph extents bounded at baseline ±14pt,
|
||||
path extents read from raw `d` coordinates. It is over-inclusive by design, so
|
||||
it fails safe — but "fails safe" still means a slice quietly carrying a
|
||||
neighbour's slur, or a hairline dropped because the y-window was wrong on some
|
||||
publisher's output. That needs eyeballing per song, a QA loop the raster path
|
||||
doesn't have.
|
||||
- Rendering 65 complex SVGs in a scrolling column may be slower than 65 WebPs.
|
||||
Unmeasured.
|
||||
- **The deciding question is unanswerable from here**: does 1920px WebP actually
|
||||
feel insufficient when pinch-zooming on a tablet? Cutting real songs answers
|
||||
it; more measurement doesn't.
|
||||
|
||||
Vector PDFs are also the *clean* case for the raster path — deskew is a no-op,
|
||||
detection works best, there are no scan artefacts — so rasterizing them is not a
|
||||
degraded fallback.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The geometry model stays **renderer-agnostic**, in normalised page coordinates,
|
||||
so adding the SVG renderer later is an output stage rather than a redesign.
|
||||
- **Re-export from the project file** regenerates every song's bundle without
|
||||
repeating human work, so songs cut before the SVG renderer exists are not
|
||||
stranded.
|
||||
- noteman needs no SVG support (`image/svg+xml`, `.svg` content type, CSP header
|
||||
on SVG responses) until the renderer ships.
|
||||
@@ -0,0 +1,58 @@
|
||||
# Lossless WebP, with levels and alpha quantised to 16 levels
|
||||
|
||||
Slice images are encoded as **lossless WebP**, with the levels adjustment applied
|
||||
and the alpha channel quantised to 16 levels. About 7 KB per slice, ~450 KB for a
|
||||
65-system song. Every lossy option and every alternative format measured
|
||||
*larger* for this content, which is the opposite of the usual intuition — hence
|
||||
this record.
|
||||
|
||||
## The measurement
|
||||
|
||||
20 slices of one real song, levels applied throughout, relative to plain lossless
|
||||
WebP:
|
||||
|
||||
| | vs baseline | |
|
||||
|---|---|---|
|
||||
| **WebP lossless + alpha quantised to 16** | **68%** | chosen |
|
||||
| AVIF q60 | 90% | lossy, for 10% |
|
||||
| WebP lossless | 100% | baseline |
|
||||
| WebP lossy q85 (alpha) | 107% | |
|
||||
| AVIF q85 | 114% | |
|
||||
| JXL lossless | 130–133% | |
|
||||
| WebP lossy q85 (opaque ink-on-white) | 158% | |
|
||||
| PNG grayscale + alpha | 165% | |
|
||||
| AVIF lossless | 188% | |
|
||||
|
||||
Separately, before levels: applying levels alone takes 338 KB → 211 KB, a 38%
|
||||
reduction.
|
||||
|
||||
## Four results that contradict an instinct
|
||||
|
||||
- **Lossy is bigger than lossless here.** Not a quality problem — the measured
|
||||
difference between q85 and lossless is max 12/255, mean 0.33, i.e. invisible.
|
||||
Lossy VP8 simply spends more bits on sharp black/white edges than VP8L's
|
||||
palette and predictor transforms do, and notation is nothing but sharp edges.
|
||||
The "q85 looks fine" intuition comes from photographs and inverts here.
|
||||
- **AVIF and JXL both lose**, AVIF lossless by nearly 2×. Their lossless modes
|
||||
are afterthoughts on photo codecs. WebP's VP8L is close to purpose-built for
|
||||
flat two-tone line art — sheet music is the content type it is best at. JXL
|
||||
additionally has no path forward in Chrome.
|
||||
- **Alpha costs nothing.** Opaque ink-on-white and black-plus-alpha are within
|
||||
0.1% at lossless, so paper-tint removal and future non-rectangular slices are
|
||||
free.
|
||||
- **Levels is the single biggest lever** — 38%, as a side effect of a control
|
||||
that exists for quality reasons anyway. Pushing the white point below the
|
||||
paper's luminance sets vast regions to exactly `alpha = 0`, which costs almost
|
||||
nothing to encode.
|
||||
|
||||
Alpha quantisation to 16 levels is imperceptible: antialiased edges span 2–3 px
|
||||
at 1920, and 16 steps across that is below notice. 8 levels starts to gamble on
|
||||
thin strokes.
|
||||
|
||||
## Rejected as not worth it
|
||||
|
||||
- **Encoder effort tuning** — Pillow's `method=6` buys 3% and a dependency.
|
||||
- **`alpha_quality=60`** — 24%, for less control than quantisation gives.
|
||||
- **Grayscale WebP** — no such mode exists. It wouldn't help anyway: the RGB
|
||||
channels are constant black and compress to nearly nothing, so alpha is the
|
||||
entire payload.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Detection proposes, the human disposes — there is no unattended mode
|
||||
|
||||
Every automatic result the slicer produces — skew angle, cut positions, source
|
||||
type, staff height, ink bounds — is a **suggestion the user confirms or modifies**
|
||||
before it is committed. There is no batch mode, no headless "slice this folder",
|
||||
and no code path that writes a bundle without a human having looked at it.
|
||||
|
||||
This is a constraint on the tool's shape, not a UI preference, which is why it
|
||||
gets an ADR: it deletes an entire phase of the original plan and it will look
|
||||
like a missing feature to anyone who finds the detection code and wonders why it
|
||||
isn't wired to a CLI.
|
||||
|
||||
## Why
|
||||
|
||||
The corpus is PDFs from a choir's distribution channel, and quality varies
|
||||
wildly — clean vector engravings at one end, noisy scans with a previous owner's
|
||||
pencil markings at the other. **Testing showed the detection algorithms produce
|
||||
unusable slices on any source with speckles or otherwise poor quality.** Not
|
||||
slightly-off slices: unusable ones.
|
||||
|
||||
But the same testing showed the suggestions land *close* on decent sources —
|
||||
close enough that correcting them is faster than placing cuts from scratch. So
|
||||
detection earns its place as an accelerator, and loses any claim to being
|
||||
load-bearing.
|
||||
|
||||
## What this rejected
|
||||
|
||||
The original plan's **Phase A** was a deliberately non-interactive CLI:
|
||||
rasterize, auto-deskew, auto-detect boundaries, write numbered slices, and fix
|
||||
the misses by hand in GIMP. Its justification was "learn the failure modes before
|
||||
designing the editor," which is a good idea.
|
||||
|
||||
It doesn't survive the premise. A CLI whose output can't be trusted has GIMP as
|
||||
its repair path — routing work back into the manual process the project exists to
|
||||
remove. A diagnostic variant (dump per-page PNGs with proposed cuts drawn in red)
|
||||
was considered and also dropped: it only re-shows a failure already confirmed by
|
||||
testing, and the editor shows the same thing live.
|
||||
|
||||
Release 1 is therefore the editor and detection together. There is no smaller
|
||||
first release that is actually usable.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Manual placement is the primary interaction**, not a correction affordance.
|
||||
The editor must be fully usable with detection producing nothing.
|
||||
- **Despeckling targets the detector, not the output.** The known failure mode is
|
||||
specks, so a median blur and a small-component filter clean the row-darkness
|
||||
profile the detector reads; the shipped pixels come from the levels-adjusted
|
||||
image.
|
||||
- Cut placement is deliberately **forgiving** — anywhere in the whitespace gap
|
||||
yields the same output, since trim crops to ink afterwards. Precision is not
|
||||
asked of the human.
|
||||
- The editor should surface **slice edges**, not just cut lines, so trim
|
||||
anomalies (a speck anchoring the bounding box) are visible rather than
|
||||
discovered later in the viewer.
|
||||
@@ -0,0 +1,36 @@
|
||||
# PyMuPDF for all PDF access, accepting AGPL
|
||||
|
||||
All PDF work — rasterizing at a chosen DPI, exporting SVG, and inspecting page
|
||||
content to classify a source as bitmap or vector — goes through **PyMuPDF**. It
|
||||
is a single wheel with MuPDF bundled, so the tool needs no system packages. Its
|
||||
licence is **AGPL-3.0**, which we accept.
|
||||
|
||||
## Why not the permissive combination
|
||||
|
||||
The obvious permissive stack was `pypdfium2` (Apache/BSD) for rasterizing plus
|
||||
`mutool` or `pdftocairo` shelled out for SVG. Both of those are **system
|
||||
packages** — `mupdf-tools`, `poppler` — and a system package on the vector path
|
||||
is precisely the failure the language choice was made to avoid: the tool is
|
||||
supposed to install once and run from any directory on any machine.
|
||||
|
||||
The SVG step can't simply be skipped, either. Music glyphs come from a notation
|
||||
font (Emmentaler, Bravura, or Sibelius/Finale's). An SVG that *references* a font
|
||||
renders as garbage on a device that lacks it, so text must be converted to paths
|
||||
at export. PyMuPDF does this **by default** — `page.get_svg_image(text_as_path=1)`,
|
||||
verified to emit `<path>` elements and zero `<text>` — so the font risk is closed
|
||||
with no extra tooling.
|
||||
|
||||
Mixing the two (pypdfium2 for raster, PyMuPDF only for SVG) is the worst option:
|
||||
two libraries with overlapping responsibilities, and AGPL linked in anyway.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **The AGPL propagates only if the slicer is published.** For a local personal
|
||||
tool it costs nothing. A future permissive release would need the rasterizer
|
||||
swapped back to `pypdfium2` — a contained change, since PDF access sits behind
|
||||
the renderer-agnostic geometry model.
|
||||
- **Source-type detection comes free** from the same library: `get_images()` plus
|
||||
a full-page-image area check distinguishes a scan from an engraving.
|
||||
- The SVG export path is present and working even though the SVG *renderer* is
|
||||
deferred — see
|
||||
[ADR 0002](0002-raster-only-svg-renderer-deferred.md).
|
||||
@@ -0,0 +1,67 @@
|
||||
# Systems are found by vertical brackets, not by row-darkness gaps
|
||||
|
||||
System detection anchors on the **vertical bracket / barline** that spans a
|
||||
system's staves, and uses the row-darkness profile only to expand each anchor to
|
||||
its ink extent. The obvious approach — find gaps in the row-darkness profile and
|
||||
cut in the middle of them — does not work on multi-voice choral scores, which is
|
||||
most of the corpus.
|
||||
|
||||
## Why the obvious approach fails
|
||||
|
||||
A row-darkness profile cannot distinguish an **inter-staff** gap from an
|
||||
**inter-system** gap. In a 6-voice closed score, one system is six staves joined
|
||||
by a bracket, and the gaps between those six staves look exactly like the gap
|
||||
between two systems — only smaller, and not reliably so.
|
||||
|
||||
Measured on *Ketun joululaulu*, a 12-page 6-voice arrangement and the hardest
|
||||
score in the repertoire:
|
||||
|
||||
- On page 2's first system, staff gaps run ~47px against a ~211px system gap. A
|
||||
merge threshold tuned there works.
|
||||
- On the same page's second system the lyrics fill the inter-staff gaps, so the
|
||||
ratios invert and the same threshold merges the wrong things.
|
||||
|
||||
Result across all 12 pages, row-profile-only versus bracket-anchored:
|
||||
|
||||
| | bracket-anchored | row-profile only |
|
||||
|---|---|---|
|
||||
| systems per page | 2, 2, 2, 2, 2, 2, 3, 2, 2, 2, 2, 1 | 10, 6, 7, 7, 7, 8, 8, 5, 4, 5, 8, 4 |
|
||||
|
||||
The bracket-anchored counts match the score. The row-profile counts are wrong on
|
||||
every page, and wrong by a different amount each time — so no threshold fixes
|
||||
them.
|
||||
|
||||
## The algorithm
|
||||
|
||||
1. **Deskew per page.** Projection-profile variance sweep over ±5°. Measured skew
|
||||
on this song ranges −2.6° to +1.2° *between pages of the same PDF*, so per-page
|
||||
is not optional.
|
||||
2. **Find anchors.** Binarise, then morphological open with a tall thin kernel
|
||||
(height ≈ 3% of the page) so only long vertical strokes survive. Take
|
||||
connected components taller than 4% of the page; walk them tallest-first,
|
||||
keeping each one whose y-extent doesn't overlap an already-kept anchor. Each
|
||||
surviving stroke is one system.
|
||||
3. **Expand to ink.** Compute the row-darkness profile on a despeckled copy, take
|
||||
its ink runs, and assign each run to the nearest anchor by centre distance. A
|
||||
system's extent is the union of its runs.
|
||||
4. **Place cuts** at the midpoint between consecutive systems' ink extents.
|
||||
|
||||
Step 3 is what makes this work rather than the bracket alone: a bracket stops at
|
||||
the last staff line, but the slice must include the **lyrics below it**. On page
|
||||
2, system 1's bracket spans 177–994 while its true ink extent is 179–1071 — the
|
||||
77px difference is the bottom voice's lyric line, which the bracket misses
|
||||
entirely and the row profile finds.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Detection needs both signals. Neither the column pass nor the row pass is
|
||||
sufficient alone, so `detect.py` computes both.
|
||||
- **Scores without brackets** — single-staff melodies, lead sheets — have no
|
||||
anchors, and fall back to row-profile runs. That fallback is the *only* correct
|
||||
behaviour there, since every ink run genuinely is its own system.
|
||||
- Bar numbers printed above a system (this score uses 11, 16, …) sit in their own
|
||||
ink run and get absorbed into the nearest system by step 3. That is right: they
|
||||
belong to the system they label.
|
||||
- A page number can be absorbed the same way if its darkness clears the profile
|
||||
threshold, inflating the last system's extent. The content rectangle and the
|
||||
bottom discard slice both prevent this; don't rely on the threshold.
|
||||
+377
@@ -0,0 +1,377 @@
|
||||
# noteman-slicer — specification
|
||||
|
||||
What the tool does and how it behaves. Vocabulary is in
|
||||
[`CONTEXT.md`](../CONTEXT.md); the reasoning behind the expensive decisions is in
|
||||
[`docs/adr/`](adr/).
|
||||
|
||||
## Scope
|
||||
|
||||
A local, single-user tool that turns a score PDF into the ordered slice images
|
||||
[noteman](../../noteman) consumes, plus the navigation markers that sit on them.
|
||||
It automates the mechanical part of noteman's ingestion boundary.
|
||||
|
||||
It is **not** a GIMP replacement. Erasing previous-owner pencil marks, chord
|
||||
letters and breath marks stays in GIMP — the irreducible manual part, which GIMP
|
||||
with a stylus already does well.
|
||||
|
||||
**One PDF → one song → one project → one bundle.** Never a many-to-one in any
|
||||
direction. A PDF is either bitmap or vector, never mixed.
|
||||
|
||||
### Why it's separate from noteman
|
||||
|
||||
Splitting it out removed the double-implementation constraint — in-app, every
|
||||
operation needs both a fast browser preview and a real server-side render, and
|
||||
that constraint is what priced dewarp and brush masking out entirely, not the
|
||||
algorithms. It also removed infrastructure noteman doesn't otherwise need (a
|
||||
scratch workspace for multi-MB rasters, an edit-list table, cleanup sweeps for
|
||||
orphaned temp files, poppler in the Docker image, an admin UI surface), and
|
||||
unlocked real image libraries.
|
||||
|
||||
It costs nothing: song creation is admin-only, done at home, once per song.
|
||||
|
||||
## Operating principle
|
||||
|
||||
**Detection proposes, the human disposes.** Every automatic result — skew angle,
|
||||
cut positions, source type, staff height, ink bounds — is a suggestion the user
|
||||
confirms or modifies before it is committed. There is no unattended mode. See
|
||||
[ADR 0004](adr/0004-detection-proposes-the-human-disposes.md).
|
||||
|
||||
## Geometry model
|
||||
|
||||
**One geometry model, two renderers.** Geometry is stored in **normalised page
|
||||
coordinates** (0–1 of page width and height), independent of DPI and of which
|
||||
renderer produces the output. Only the final stage differs.
|
||||
|
||||
| Concept | Raster | Vector |
|
||||
|---|---|---|
|
||||
| Cut | y in pixels | y in PDF user space |
|
||||
| Discard | drop the slice | drop the slice |
|
||||
| Content rectangle | crop before cutting | clip before cutting |
|
||||
| Trim | crop to ink bbox | crop `viewBox` to ink bbox |
|
||||
| Uniform width | transparent right pad | wider `viewBox`, same content |
|
||||
| Staff-height normalise | scale factor | scale factor |
|
||||
| Deskew, levels, ink→alpha, 1920 cap | yes | no |
|
||||
|
||||
Only the raster renderer ships in release 1 — see
|
||||
[ADR 0002](adr/0002-raster-only-svg-renderer-deferred.md). The editor is one
|
||||
editor regardless, since a vector PDF has to be rasterized just to display it on
|
||||
screen.
|
||||
|
||||
### Pipeline order
|
||||
|
||||
```
|
||||
load raster → deskew → levels → content rect → cut → discard
|
||||
→ trim → scale → pad → ink→alpha → encode
|
||||
```
|
||||
|
||||
**Load raster** differs by source type. A scanned PDF carries one full-page image
|
||||
per page, and that image *is* the scan — extract it at its native resolution
|
||||
(`extract_image`) rather than re-rendering the page. Re-rendering at a fixed
|
||||
600 DPI resamples a 200 DPI scan up by 3×, which triples the pixel count and adds
|
||||
no detail. A vector PDF has no embedded raster, so it is rendered — see the DPI
|
||||
note in *Reference values*.
|
||||
|
||||
The rest of the order is not arbitrary:
|
||||
|
||||
- **Levels before anything geometric**, so the trim bounding box is computed on
|
||||
the image that actually ships.
|
||||
- **Content rect before cutting**, so margin junk never enters a slice.
|
||||
- **Trim before scale**, since the scale factor derives from the widest *trimmed*
|
||||
slice.
|
||||
|
||||
### Slices, cuts and discard
|
||||
|
||||
A page starts as a single slice; each cut splits one slice into two. Slices
|
||||
therefore tile the page with no gaps and no overlap.
|
||||
|
||||
Headers, footers and blank regions leave the song via a **discard** flag, not via
|
||||
cuts at the page edges. Modelling a slice as "the region between two cuts" leaks:
|
||||
page 2 has no header, so it would need an invented top cut whose position depends
|
||||
on whether that page happens to have one.
|
||||
|
||||
Cut placement is forgiving — anywhere inside the whitespace gap yields the same
|
||||
output, because trim crops to ink afterwards.
|
||||
|
||||
### Content rectangle
|
||||
|
||||
The region of a page that holds music, set per PDF and adjustable per page,
|
||||
applied before cutting. Everything outside it is dropped.
|
||||
|
||||
This handles margin junk structurally rather than case-by-case, because margin
|
||||
junk is by definition outside the music: scan-edge bands, spine shadows, and page
|
||||
numbers printed in the side margin level with a system. That last one matters
|
||||
more than it looks — see the trim consequences below.
|
||||
|
||||
### Trim, scale, pad
|
||||
|
||||
**Trim** tight on all four sides, per slice. This normalises away the left-margin
|
||||
drift between scanned pages, and flattens the engraved indent of the first
|
||||
system — correct here, since noteman strips the printed header the indent made
|
||||
room for.
|
||||
|
||||
Two consequences:
|
||||
|
||||
- A stray speck at the far left anchors the trim, shifting that slice relative to
|
||||
its neighbours. Mitigate by ignoring connected components under a few hundred
|
||||
pixels (`cv2.connectedComponentsWithStats`) when computing the bounding box.
|
||||
- A page number in the side margin level with a system would set that slice's
|
||||
bounding box, which sets the song's widest slice, which scales the whole song
|
||||
down. One artefact, whole song smaller. Hence the content rectangle.
|
||||
|
||||
**Scale is normalised on staff height, not width.** Width-based scaling assumes
|
||||
every slice comes from the same scan at the same DPI. It breaks for a rescanned
|
||||
page, a PDF mixing scan generations, or a re-engraved replacement system — whose
|
||||
width depends on how much music is in it, not on matching its neighbours. Staff
|
||||
height is the invariant a reader perceives as "the notes are the same size", and
|
||||
it falls out of the same row-darkness profile detection already computes.
|
||||
|
||||
Two steps, both per song: normalise every slice to a common staff height, then
|
||||
scale the song uniformly so its widest slice lands at **1920px**. That is a
|
||||
ceiling, never a target — **never upscale**. A song that comes out narrower stays
|
||||
narrower; enlarging a 600 DPI scan past its real resolution buys softness and
|
||||
bytes and no detail.
|
||||
|
||||
**Pad** narrower slices with transparency on the right, so every slice in a song
|
||||
is the same width, flush left, notes the same size. A short system simply ends
|
||||
earlier.
|
||||
|
||||
### Encoding
|
||||
|
||||
**Lossless WebP, with levels applied and alpha quantised to 16 levels.** Roughly
|
||||
7 KB per slice, about 450 KB for a 65-system song. Lossy encodings and the
|
||||
alternative formats are all *larger* for this content — measured, with the
|
||||
figures, in
|
||||
[ADR 0003](adr/0003-lossless-webp-with-levels-and-alpha-quantisation.md).
|
||||
|
||||
Ink handling is luminance → alpha: ink forced to pure black,
|
||||
`alpha = 255 − luminance`. Not `pixel == white` thresholding — staff lines are
|
||||
antialiased, and binary removal leaves jagged edges.
|
||||
|
||||
## Detection
|
||||
|
||||
All of it is a suggestion, all of it overridable.
|
||||
|
||||
**Deskew** — per page, and not optionally so: measured skew varies from −2.6° to
|
||||
+1.2° *between pages of the same PDF*. Staff lines are by far the strongest
|
||||
horizontal signal in sheet music, so a projection-profile variance sweep over ±5°
|
||||
finds the angle reliably — sum row-darkness for each candidate angle, take the
|
||||
angle of maximum variance. Run on a downscaled copy. Pair with a manual slider.
|
||||
|
||||
**Systems** — anchored on the **vertical bracket** that spans a system's staves,
|
||||
not on gaps in the row-darkness profile. A row profile cannot distinguish an
|
||||
inter-staff gap from an inter-system gap on multi-voice choral scores, and gets
|
||||
the system count wrong on every page. See
|
||||
[ADR 0006](adr/0006-systems-are-found-by-brackets-not-row-gaps.md) for the
|
||||
measurement and the full algorithm. In outline:
|
||||
|
||||
1. Binarise; morphological open with a tall thin kernel so only long vertical
|
||||
strokes survive; keep non-overlapping components taller than 4% of the page.
|
||||
Each is one system.
|
||||
2. Take ink runs from the row-darkness profile and assign each to the nearest
|
||||
anchor. A system's extent is the union of its runs — this is what pulls in the
|
||||
lyrics printed *below* the last staff, which the bracket stops short of.
|
||||
3. Propose cuts at the midpoint between consecutive systems' ink extents, and
|
||||
pre-set the discard flag on a page's top and bottom slice when they contain no
|
||||
system.
|
||||
|
||||
Scores with no bracket — single-staff melodies, lead sheets — have no anchors and
|
||||
fall back to row-profile runs, which is correct there.
|
||||
|
||||
**Staff height** — peak-to-peak spacing in the row profile.
|
||||
|
||||
**Source type** — `get_images(full=True)` / `get_drawings()` proposes bitmap or
|
||||
vector per PDF; the tool asks the user to confirm before routing. (`full=True` is
|
||||
required, or `get_image_bbox` rejects the item.)
|
||||
|
||||
**Despeckle feeds detection only.** A median blur plus dropping tiny connected
|
||||
components denoises the *profile the detector reads*; the shipped pixels come
|
||||
from the levels-adjusted image. The known failure mode is specks, so the fix
|
||||
belongs on the signal, not the output.
|
||||
|
||||
## Levels
|
||||
|
||||
Two sliders per song (black point, white point) applied via `cv2.LUT`, with a
|
||||
per-page override.
|
||||
|
||||
In release 1, not deferred: with `alpha = 255 − luminance`, a scan's greyness
|
||||
*becomes* transparency, so a faint or yellowed source produces washed-out notes
|
||||
on a hazy background and **nothing downstream can rescue it**. Set the white
|
||||
point just under the paper's luminance and the paper vanishes completely; set the
|
||||
black point at the ink's darkest and notes go solid. It is also the single
|
||||
biggest lever on output size.
|
||||
|
||||
Adaptive methods (CLAHE, adaptive thresholding) are the trap — tuned for text,
|
||||
they eat the thin stuff on notation: hairpin tips, slur ends, ledger lines,
|
||||
tapered beams. A global LUT whose effect you can see beats a local algorithm you
|
||||
can't predict.
|
||||
|
||||
## Editor
|
||||
|
||||
**PySide6.** `QGraphicsView` provides the viewport — pan, zoom, screen↔image
|
||||
coordinate mapping, resampling, hit-testing — which would otherwise be ~150 lines
|
||||
of hand-rolled geometry. `cv2.imshow` was rejected: OpenCV's highgui is GTK/X11
|
||||
and lands on XWayland at best, and it has no text input at all.
|
||||
|
||||
What the editor does: pan and zoom the page, drag cut lines, toggle discard,
|
||||
adjust the content rectangle, move the levels sliders, place markers, fill in
|
||||
song metadata, export.
|
||||
|
||||
Marker placement needs a **slice picker** — a `QListView` in icon mode over the
|
||||
slice previews — since every jump source stores an explicit target. One widget
|
||||
serving all six jump types.
|
||||
|
||||
## Project file
|
||||
|
||||
Autosaved JSON beside the source PDF, holding the source path and hash, cuts,
|
||||
discards, content rectangle, skew angles, levels, staff-height overrides, markers
|
||||
and metadata. The bundle is *generated* from it, so export is a pure function of
|
||||
the project file plus the PDF.
|
||||
|
||||
It buys crash safety, resume across sessions (authoring is trickle-in), and
|
||||
**re-export** — change the 1920 cap, fix one cut, or add the SVG renderer later,
|
||||
and every song's bundle regenerates without repeating any human work.
|
||||
|
||||
The project file references the PDF and never contains it; the hash lets the
|
||||
editor warn if the PDF changed underneath.
|
||||
|
||||
## Markers
|
||||
|
||||
Placed here rather than in noteman: at cut time you are already reading the score
|
||||
page by page at full resolution, so the Segno, the Coda sign, the "to coda" text
|
||||
and the rehearsal letters are on screen. Deferring means reading the whole score a
|
||||
second time to find the same symbols.
|
||||
|
||||
noteman's vocabulary, carried verbatim — `rehearsal_letter`, `section_label`,
|
||||
`segno`, `coda`, `fine`, `repeat_start`, `repeat_end`, `volta`, `to_coda`,
|
||||
`ds_al_coda`, `ds_al_fine`, `dc_al_coda`, `dc_al_fine`, `generic_jump`. A small
|
||||
stable enum, but real coupling: adding a type means changing both repos.
|
||||
|
||||
Three shapes among them:
|
||||
|
||||
- **Bare tags:** `segno`, `coda`, `fine`, `repeat_start`, `repeat_end`.
|
||||
- **Tags with free text:** `rehearsal_letter` ("C"), `section_label` ("CHORUS"),
|
||||
`volta` ("1.").
|
||||
- **Jump sources:** `to_coda`, `ds_al_coda`, `ds_al_fine`, `dc_al_coda`,
|
||||
`dc_al_fine`, `generic_jump`.
|
||||
|
||||
**Every jump source stores its target slice explicitly.** noteman's viewer
|
||||
currently resolves by type — a `to_coda` finds the song's unique `coda` at tap
|
||||
time — but that puts an unwritten "exactly one Coda per song" invariant into a
|
||||
contract between two separately-maintained repos, enforced by neither. Authoring
|
||||
the target costs one click on a slice already on screen, and in exchange the
|
||||
bundle is self-describing and a score with two codas simply works.
|
||||
|
||||
## Bundle
|
||||
|
||||
The only channel to noteman. No API, no direct upload — see
|
||||
[ADR 0001](adr/0001-slicer-owns-image-processing-bundle-is-the-only-channel.md).
|
||||
|
||||
```
|
||||
song.zip
|
||||
song.json
|
||||
original.pdf
|
||||
001.webp 002.webp …
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"v": 1,
|
||||
"title": "…", "composer": "…", "arranger": "…",
|
||||
"slices": [
|
||||
{ "file": "001.webp" },
|
||||
{ "file": "002.webp", "markers": [{ "type": "rehearsal_letter", "label": "A" }] },
|
||||
{ "file": "003.webp", "markers": [{ "type": "to_coda", "destination": 7 }] }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Array order **is** slice order — one ordering, not two. Markers nest inside the
|
||||
slice they sit on, so indices appear in exactly one place: a jump source's
|
||||
`destination`.
|
||||
|
||||
`"v": 1` is eight bytes of insurance. The bundle is the only channel, MIDI and
|
||||
MP3s are planned for a later phase, and bundles are archived artifacts that may be
|
||||
re-imported a year later.
|
||||
|
||||
Otherwise: plain zip, no manifest beyond this, no checksums, hand-fixable.
|
||||
Python's `zipfile` is stdlib; the import side needs one zero-dep library
|
||||
(`fflate`), since Bun has zlib but no zip reader.
|
||||
|
||||
**Contents:** slices, markers, the original PDF, and song-level text metadata
|
||||
(title, subtitle, composer, original artist, arranger, lyricist, translator,
|
||||
voice list). Metadata is included not because the slicer transforms it but
|
||||
because you have to read the title block anyway to mark the header slice
|
||||
discarded — typing eight fields while it's on screen beats reopening the PDF
|
||||
later.
|
||||
|
||||
Rehearsal MIDI and MP3s are deliberately out of the first bundle.
|
||||
|
||||
### One rule for the import side
|
||||
|
||||
**Import creates a new song only; never re-import onto an existing one.** Jump
|
||||
destinations reference slices by ID, so replacing a song's slices silently
|
||||
orphans every marker on it. Re-cutting happens *before* marker authoring in
|
||||
practice, so forbidding it costs nothing and prevents a genuinely nasty data-loss
|
||||
mode. Re-export from the project file is the supported path.
|
||||
|
||||
## Implementation
|
||||
|
||||
**Python**, chosen for OpenCV access and iteration speed. Installed as a package
|
||||
via `uv tool install --editable .`, which puts a `noteman-slicer` command on PATH
|
||||
that runs from any directory with no venv to activate. The one cwd trap: load
|
||||
bundled data via `Path(__file__).parent` or `importlib.resources`, never a
|
||||
relative path.
|
||||
|
||||
Dependencies: **PyMuPDF**, **PySide6**, **opencv-python-headless**, **numpy** —
|
||||
all wheels, no system packages. PyMuPDF covers every PDF need; see
|
||||
[ADR 0005](adr/0005-pymupdf-for-all-pdf-access.md).
|
||||
|
||||
Verified: `cv2` 5.0.0 writes 4-channel lossless WebP with alpha preserved
|
||||
byte-exact (`IMWRITE_WEBP_QUALITY, 101`).
|
||||
|
||||
Module boundaries: `pdf.py` (load, source-type detect, rasterize), `detect.py`
|
||||
(deskew, row-darkness profile, system runs, staff height), `bundle.py`,
|
||||
`editor.py`.
|
||||
|
||||
## Changes required in noteman
|
||||
|
||||
On noteman's timeline, not the slicer's — but release 1 produces artifacts
|
||||
nothing consumes until this lands.
|
||||
|
||||
1. **Delete the sharp normalisation pipeline.** The slicer's output is final.
|
||||
2. **Bundle import** — unzip → read `song.json` → create song → insert slices in
|
||||
array order → insert markers, mapping index → new slice UUID → store the PDF.
|
||||
3. **Jump sources carry explicit destinations** — `destinationSliceId` is already
|
||||
nullable on every marker type, so this is viewer logic, not schema.
|
||||
|
||||
SVG support on the noteman side (`image/svg+xml` in the upload path, `.svg` in
|
||||
`CONTENT_TYPES`, and a CSP header on SVG responses) is not needed until the SVG
|
||||
renderer ships.
|
||||
|
||||
## Phasing
|
||||
|
||||
**Release 1 — editor + detection + bundle export, raster only.** Vector PDFs are
|
||||
rasterized like everything else; they're the clean case, where deskew is a no-op
|
||||
and detection works best. Levels, content rectangle, discard, markers, project
|
||||
file.
|
||||
|
||||
Everything else is deferred and tracked as issues on the Gitea repo.
|
||||
|
||||
## Reference values
|
||||
|
||||
- Final slice width cap: **1920px**, matching the viewer sheet's max-width. A
|
||||
ceiling, not a target.
|
||||
- Output format: **lossless WebP**.
|
||||
- Working resolution:
|
||||
- **Scanned sources — the embedded image's native resolution.** Never
|
||||
re-render. Real scans in this corpus run ~200 DPI (1653×2332 for A4), which
|
||||
is *below* the 1920 cap, so those songs ship narrower than 1920 and are never
|
||||
upscaled.
|
||||
- **Vector sources — 600 DPI**, configurable. A4 @ 600 DPI is ~4960×7016 px;
|
||||
the ~2.6× downsample to 1920 is itself a quality win via antialiasing. 300
|
||||
DPI would suffice for the target, but 600 buys headroom for deskew
|
||||
resampling.
|
||||
- A slice = **one system** = one full line of music across all voices, typically
|
||||
4–12 bars, lyrics intact.
|
||||
- Upload/bundle sizes are not constrained by noteman's old 25 MB/file limits —
|
||||
the bundle bypasses that path entirely.
|
||||
Reference in New Issue
Block a user