Merge dev: rotated scans, detection and levels fixes, editor tweaks

Three things a first session trips over.

The editor opened at an arbitrary zoom. show_page does fit the page, but
it runs before the window has been laid out, when the viewport is still
its default size; redo it once when the real size arrives.

The bundle was named after the PDF, which is whatever the download was
called. Take the song's title instead — unsafe characters dropped, then
whitespace collapsed to dashes. Accented letters stay, since ä and ö are
not a filesystem's problem, but a leading dot would hide the file.

And the selected slice was drawn as an outline whose top and bottom edges
run under the cut lines painted over them, leaving two thin verticals in
the margins and no way to tell what was selected. Wash the slice, as the
discard and engraved states already do, and keep the outline for the trim
anomalies it exists to show.

docs/guide.md

@@ -44,10 +44,11 @@ off.
    one back.
 4. **Set the content rectangle.** Drag the blue edges so they hold the music and
    nothing else. This is the horizontal crop for every slice on the page.
-5. **Set black and white points.** Pull the *White point* down until the paper
+5. **Check black and white points.** These arrive proposed from the scan, like
-   goes pure white and its texture disappears; pull *Black point* up until the
+   the cuts do. If the paper still shows texture, pull *White point* down; if the
-   notes are solid black rather than grey mush. Scans need this; clean digital
+   notes look grey rather than solid, pull *Black point* up. Getting this wrong
-   PDFs usually don't.
+   is the one mistake you cannot see until the bundle is on the tablet — grey ink
+   becomes half-transparent ink, and nothing downstream can rescue it.
 
 Page Up / Page Down move between pages. Levels carry over from the previous
 page, so a consistent scan only needs setting once.
@@ -77,7 +78,8 @@ and "Andante" cannot.
 
 ## Export
 
-**Export bundle…**, choose where the `.zip` goes, done. Inside are the slice
+**Export bundle…**, choose where the `.zip` goes, done. It is named after the
+song's title — *Bicycle Race* becomes `Bicycle-Race.zip`. Inside are the slice
 images in order, their markers, the song metadata, and the original PDF as the
 archive copy. That zip is the whole interface to noteman; hand it over and open
 it there.

docs/spec.md

@@ -236,6 +236,14 @@ point just under the paper's luminance and the paper vanishes completely; set th
 black point at the ink's darkest and notes go solid. It is also the single
 biggest lever on output size.
 
+**Detection proposes both**, like it proposes cuts and skew, because the default
+0–255 is the one setting whose harm is invisible until the bundle is on a tablet.
+Notation is two-tone, so Otsu's split between ink and paper is the measurement;
+the points sit halfway from it to each end of the range, leaving the ramp between
+them as the antialiasing. A page already scanned bilevel has no interior split —
+Otsu degenerates to 0 there — and is left at 0–255. The proposal is per page and
+the median becomes the song's, so a near-blank page cannot set it.
+
 Adaptive methods (CLAHE, adaptive thresholding) are the trap — tuned for text,
 they eat the thin stuff on notation: hairpin tips, slur ends, ledger lines,
 tapered beams. A global LUT whose effect you can see beats a local algorithm you

noteman_slicer/bundle.py

@@ -13,6 +13,7 @@ a jump source's `destination`.
 from __future__ import annotations
 
 import json
+import re
 import zipfile
 from pathlib import Path
 
@@ -39,6 +40,19 @@ METADATA_FIELDS = (
 NUMERIC_FIELDS = frozenset({"tempo"})
 
 
+def filename(project: Project) -> str:
+    """The bundle's name, from the song's title.
+
+    Spaces become dashes and anything that is not a letter, digit, dash, dot or
+    underscore goes. Letters keep their accents — ä and ö are not a filesystem's
+    problem — but a leading dot would make the bundle invisible.
+    """
+    # Drop the unsafe characters before collapsing whitespace, not after, or
+    # "Sävel & Ääni" keeps the dash the ampersand left behind.
+    title = re.sub(r"[^\w\s.-]", "", (project.metadata.get("title") or ""))
+    return f"{re.sub(r'\s+', '-', title.strip()).lstrip('.-') or 'song'}.zip"
+
+
 def _engraving(project: Project, page: int, slot: int) -> dict | None:
     """The notation behind a re-engraved slice, or None for a scanned one.
 

noteman_slicer/cli.py

@@ -87,7 +87,7 @@ def _export(args: argparse.Namespace) -> int:
     elif project.source_changed():
         print("WARNING: the PDF has changed since these cuts were made")
 
-    out = Path(args.out) if args.out else source.path.with_suffix(".zip")
+    out = Path(args.out) if args.out else source.path.with_name(bundle.filename(project))
     try:
         bundle.write(project, source, out)
     except ValueError as error:

noteman_slicer/detect.py

@@ -26,6 +26,7 @@ _SKEW_WORK_SCALE = 0.25
 _INK = 128  # below this is ink, above is paper
 _ANCHOR_KERNEL = 0.03  # vertical open kernel, as a fraction of page height
 _ANCHOR_MIN = 0.04  # a bracket is at least this tall, as a fraction of page
+_ANCHOR_MAX_RATIO = 2.0  # a stroke this much taller than the typical one is an artefact
 _PROFILE_FLOOR = 0.02  # ink-run threshold, as a fraction of the profile peak
 _EXPAND_REACH = 1.5  # how far past the bracket a system's ink reaches, in staff heights
 _STAFF_KERNEL = 0.05  # horizontal open kernel, as a fraction of page width
@@ -64,6 +65,7 @@ class PageDetection:
     systems: list[System] = field(default_factory=list)
     cuts: list[int] = field(default_factory=list)
     content: tuple[float, float, float, float] = (0.0, 0.0, 1.0, 1.0)
+    levels: tuple[int, int] = (0, 255)
 
     @property
     def bracketless(self) -> bool:
@@ -123,6 +125,17 @@ def system_anchors(gray: np.ndarray) -> list[Anchor]:
         if stats[i, cv2.CC_STAT_HEIGHT] > h * _ANCHOR_MIN
     ]
 
+    # A scanner leaves a dark line down the sheet edge — the binder shadow, the
+    # glass, the page next to it — and it runs the whole height of the scan.
+    # Being the tallest stroke on the page it wins every overlap below and
+    # swallows every system into one. A page's brackets and barlines are all
+    # about one system tall, so anything wildly taller than the typical stroke
+    # is not notation. Relative, not an absolute fraction of the page: a page
+    # holding one big system is legitimate and must survive.
+    if len(tall) > 1:
+        limit = float(np.median([a.bottom - a.top for a in tall])) * _ANCHOR_MAX_RATIO
+        tall = [a for a in tall if a.bottom - a.top <= limit] or tall
+
     # Tallest first, keeping only strokes that don't overlap one already kept:
     # a system's barlines all overlap its bracket, so each system yields one.
     # The kept stroke is the tallest, which is the bracket rather than a barline.
@@ -280,6 +293,26 @@ def staff_height(gray: np.ndarray, top: int, bottom: int) -> float | None:
     return float(np.median(intra) * 4)  # 5 lines, 4 spaces
 
 
+def ink_levels(gray: np.ndarray) -> tuple[int, int]:
+    """Black and white points that put the ink on black and the paper on white.
+
+    Left at 0–255 a slice ships whatever grey the scanner produced, and the
+    downscale to the song's width then blends every stroke edge further, so a
+    fine engraving arrives on the tablet as a wash. Notation is two-tone by
+    nature — ink and paper, nothing in between — so Otsu's split is exactly the
+    measurement wanted, and the points sit halfway to each end of the range from
+    it. Halfway rather than at the split itself: the ramp between them is the
+    antialiasing, and collapsing it would leave the notes jagged.
+    """
+    split = float(cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[0])
+    if split < 1:
+        # A page already scanned bilevel has no interior split to find, and
+        # Otsu degenerates to 0. There is nothing between ink and paper to
+        # stretch, so leave the sliders where they are.
+        return 0, 255
+    return int(split / 2), int(split + (255 - split) / 2)
+
+
 def _gap(run: tuple[int, int], anchor: Anchor) -> int:
     """Vertical distance between an ink run and a bracket; 0 if they overlap."""
     start, end = run
@@ -375,5 +408,9 @@ def detect_page(gray: np.ndarray, skew: float | None = None) -> PageDetection:
     # would risk clipping a tempo mark or a section label above the first staff.
     left, right = content_columns(straight, anchors)
     return PageDetection(
-        skew=angle, systems=systems, cuts=cuts, content=(left, 0.0, right, 1.0)
+        skew=angle,
+        systems=systems,
+        cuts=cuts,
+        content=(left, 0.0, right, 1.0),
+        levels=ink_levels(straight),
     )

noteman_slicer/editor.py

@@ -75,6 +75,8 @@ _CUT = QColor(220, 40, 40)
 _CUT_ACTIVE = QColor(255, 120, 0)
 _VERTEX = QColor(255, 200, 0)
 _DISCARD = QColor(120, 120, 140, 90)
+_SELECT = QColor(0, 170, 0)
+_SELECT_WASH = QColor(0, 200, 60, 40)
 _RECT = QColor(40, 140, 220)
 _MARKER = QColor(150, 60, 190)
 _ENGRAVED = QColor(200, 120, 0)
@@ -104,6 +106,7 @@ class PageView(QGraphicsView):
         self.selected_slice = 0
         self.picking = False
         self._drag: tuple[str, int, int] | None = None
+        self._fitted = False
 
     # -- state ------------------------------------------------------------
 
@@ -137,10 +140,15 @@ class PageView(QGraphicsView):
                     self._slice_polygon(slot, w, h), QPen(Qt.NoPen), QBrush(_DISCARD)
                 )
 
-        # The selected slice, outlined so trim anomalies are visible.
+        # The selected slice. The outline alone is nearly invisible: its top and
-        pen = QPen(QColor(0, 170, 0), 2)
+        # bottom edges run under the cut lines drawn over them, leaving two thin
+        # verticals at the page margins. A wash says which slice is selected at
+        # a glance; the outline stays, because it is what shows trim anomalies.
+        selected = self._slice_polygon(self.selected_slice, w, h)
+        scene.addPolygon(selected, QPen(Qt.NoPen), QBrush(_SELECT_WASH))
+        pen = QPen(_SELECT, 3)
         pen.setCosmetic(True)
-        scene.addPolygon(self._slice_polygon(self.selected_slice, w, h), pen)
+        scene.addPolygon(selected, pen)
 
         x0, y0, x1, y1 = self.project.page_content_rect(self.page_index)
         pen = QPen(_RECT, 2, Qt.DashLine)
@@ -361,6 +369,15 @@ class PageView(QGraphicsView):
         self.redraw()
         self.changed.emit()
 
+    def resizeEvent(self, event) -> None:
+        super().resizeEvent(event)
+        # The fit in show_page runs before the window has been laid out, when
+        # the viewport is still its default size, so the first page opens at
+        # some arbitrary zoom. Redo it once, when the real size arrives.
+        if not self._fitted and self.pixmap is not None:
+            self._fitted = True
+            self.fitInView(self.scene().sceneRect(), Qt.KeepAspectRatio)
+
     def wheelEvent(self, event) -> None:
         factor = 1.15 if event.angleDelta().y() > 0 else 1 / 1.15
         self.scale(factor, factor)
@@ -844,7 +861,10 @@ class Editor(QMainWindow):
             self.metadata["title"].setFocus()
             return
         target, _ = QFileDialog.getSaveFileName(
-            self, "Export bundle", str(self.source.path.with_suffix(".zip")), "Bundle (*.zip)"
+            self,
+            "Export bundle",
+            str(self.source.path.with_name(bundle.filename(self.project))),
+            "Bundle (*.zip)",
         )
         if not target:
             return

noteman_slicer/pdf.py

@@ -90,13 +90,28 @@ def page_raster(source: Source, index: int) -> np.ndarray:
             # Pixmap(doc, xref) rather than decoding extract_image() bytes:
             # MuPDF handles JBIG2 and CCITT, which no image library will.
             pix = pymupdf.Pixmap(source.doc, xref)
-            return _to_gray(pix)
+            # The embedded image is in its own orientation, not the page's: a
+            # scanner that fed the sheet sideways stores it landscape and the
+            # PDF sets /Rotate so viewers turn it upright. Extracting by xref
+            # bypasses that, so apply it here — otherwise every system runs
+            # down the page and detection finds nothing.
+            return _rotate(_to_gray(pix), page.rotation)
         # A scanned PDF whose page has no embedded image (a blank, or a
         # cover typeset in vector). Rendering is the only option left.
 
     return _to_gray(page.get_pixmap(dpi=source.render_dpi, colorspace=pymupdf.csGRAY))
 
 
+def _rotate(gray: np.ndarray, degrees: int) -> np.ndarray:
+    """Turn a page raster clockwise by a multiple of 90°, as /Rotate means it.
+
+    ponytail: quarter turns only. A page rotated by anything else would need
+    resampling, and no scanner produces one.
+    """
+    turns = round(degrees / 90) % 4
+    return np.ascontiguousarray(np.rot90(gray, -turns)) if turns else gray
+
+
 def _to_gray(pix: pymupdf.Pixmap) -> np.ndarray:
     if pix.alpha or pix.colorspace is None or pix.colorspace.n != 1:
         pix = pymupdf.Pixmap(pymupdf.csGRAY, pix)

noteman_slicer/project.py

@@ -15,6 +15,7 @@ import hashlib
 import json
 from dataclasses import dataclass, field
 from pathlib import Path
+from statistics import median
 
 from .detect import PageDetection
 
@@ -296,7 +297,18 @@ class Project:
                     content_rect=detection.content,
                 )
             )
-        return cls(source=source, source_hash=hash_file(source), pages=pages)
+        # Levels per song, not per page: a scanner's contrast does not change
+        # between sheets, and one pair of sliders for the whole song is what a
+        # user actually wants to nudge. The median keeps a near-blank page —
+        # where the ink/paper split is guesswork — from setting them.
+        proposals = [d.levels for d in detections] or [(0, 255)]
+        levels = (
+            int(median(b for b, _ in proposals)),
+            int(median(w for _, w in proposals)),
+        )
+        return cls(
+            source=source, source_hash=hash_file(source), pages=pages, levels=levels
+        )
 
     def save(self, path: Path | None = None) -> Path:
         """Atomic write, so a crash mid-save cannot destroy the previous state."""

noteman_slicer/render.py

@@ -26,7 +26,10 @@ from .project import Cut, Project
 
 MAX_WIDTH = 1920
 ALPHA_LEVELS = 16  # quantising alpha costs nothing visible and ~32% of the bytes
-_SPECK_AREA = 300  # ink blobs smaller than this don't anchor a trim
+# A row or column carrying less ink than this is a fleck, not content: at least
+# this many pixels, and at least this share of the slice's own size.
+_SPECK_INK = 8
+_SPECK_SHARE = 0.005
 
 
 @dataclass
@@ -105,26 +108,24 @@ def _ink_bbox(gray: np.ndarray) -> tuple[int, int, int, int] | None:
 
     One scan fleck at the far left would otherwise anchor the trim and shift
     that slice relative to every other one.
+
+    Measured per row and per column rather than per blob. Judging each blob on
+    its own area throws away a whole line of lyrics — every letter is its own
+    small component, and no single one is big enough to keep — which is how a
+    slice loses its bottom voice's words. A row carrying a line of text carries
+    plenty of ink *in total*, and a fleck's row carries almost none.
     """
-    ink = (gray < 200).astype(np.uint8)
+    ink = gray < 200
-    count, _, stats, _ = cv2.connectedComponentsWithStats(ink, 8)
+    rows, cols = ink.sum(axis=1), ink.sum(axis=0)
-    boxes = [
+    kept_rows = np.where(rows >= max(_SPECK_INK, ink.shape[1] * _SPECK_SHARE))[0]
-        (
+    kept_cols = np.where(cols >= max(_SPECK_INK, ink.shape[0] * _SPECK_SHARE))[0]
-            stats[i, cv2.CC_STAT_LEFT],
+    if not kept_rows.size or not kept_cols.size:
-            stats[i, cv2.CC_STAT_TOP],
-            stats[i, cv2.CC_STAT_LEFT] + stats[i, cv2.CC_STAT_WIDTH],
-            stats[i, cv2.CC_STAT_TOP] + stats[i, cv2.CC_STAT_HEIGHT],
-        )
-        for i in range(1, count)
-        if stats[i, cv2.CC_STAT_AREA] >= _SPECK_AREA
-    ]
-    if not boxes:
         return None
     return (
-        min(b[0] for b in boxes),
+        int(kept_cols[0]),
-        min(b[1] for b in boxes),
+        int(kept_rows[0]),
-        max(b[2] for b in boxes),
+        int(kept_cols[-1]) + 1,
-        max(b[3] for b in boxes),
+        int(kept_rows[-1]) + 1,
     )
 
 

tests/test_detect.py

@@ -14,7 +14,12 @@ import numpy as np
 
 sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
 
-from noteman_slicer.detect import deskew, deskew_angle, detect_page  # noqa: E402
+from noteman_slicer.detect import (  # noqa: E402
+    deskew,
+    deskew_angle,
+    detect_page,
+    ink_levels,
+)
 
 W, H = 1000, 1400
 STAFF_GAP = 15  # → staff height 60, so expansion reaches 90px past a bracket
@@ -71,6 +76,23 @@ def main() -> int:
         found = deskew_angle(deskew(page, angle))
         assert abs(found + angle) <= 0.15, f"skew {angle}: got {found}"
 
+    # Levels are proposed too. A grey scan left at 0–255 ships its wash to the
+    # tablet, and the downscale to the song's width only blends it further.
+    grey = np.full((H, W), 210, np.uint8)  # paper, not white
+    grey[200:400, 100:900] = 70  # ink, not black
+    black, white = ink_levels(grey)
+    assert black < 70 < white < 210, (black, white)
+    # A page already bilevel has nothing between ink and paper to stretch.
+    assert ink_levels(_page()) == (0, 255)
+
+    # A scanner's edge line runs the whole height of the sheet. Being taller
+    # than every bracket it used to win each overlap and swallow the page into
+    # one system — Olukainen juomukainen, where five pages of six came out as a
+    # single slice each.
+    scanned = _page()
+    scanned[10 : H - 10, W - 8 : W - 4] = 0
+    assert len(detect_page(scanned).systems) == 2, "an edge artefact is not a bracket"
+
     # No brackets: every ink run is its own system.
     bare = np.full((H, W), 255, np.uint8)
     for y in (200, 500, 800):

tests/test_editor.py

@@ -19,6 +19,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
 
 from PySide6.QtWidgets import QApplication  # noqa: E402
 
+from noteman_slicer import bundle  # noqa: E402
 from noteman_slicer.detect import detect_page  # noqa: E402
 from noteman_slicer.editor import Editor  # noqa: E402
 from noteman_slicer.pdf import open_source, page_raster  # noqa: E402
@@ -105,6 +106,28 @@ def main() -> int:
     assert reloaded.pages[0].levels == (40, 210)
     assert [c.points for c in reloaded.pages[0].cuts] == [c.points for c in page.cuts]
 
+    # The page fits the viewport once the window has a real size. show_page's
+    # own fit runs before layout, when the viewport is still its default.
+    editor.resize(900, 700)
+    editor.show()
+    app.processEvents()
+    scene = editor.view.sceneRect()
+    scale = editor.view.transform().m11()
+    viewport = editor.view.viewport()
+    fill = max(
+        scale * scene.width() / viewport.width(),
+        scale * scene.height() / viewport.height(),
+    )
+    # Fit means nearly touching one edge — Qt leaves a small margin of its own.
+    # A "≤ 1" check alone would pass a page zoomed down to a dot.
+    assert 0.9 <= fill <= 1.02, f"page is not fitted to the window: {fill:.3f}"
+
+    # The bundle is named after the song, not the PDF.
+    assert bundle.filename(project) == "Ketun-joululaulu.zip"
+    project.metadata["title"] = "AC/DC: T.N.T. (live)"
+    assert bundle.filename(project) == "ACDC-T.N.T.-live.zip"
+    project.metadata["title"] = "Ketun joululaulu"
+
     editor.close()
     source.close()
     for f in (pdf, saved):

tests/test_pdf.py

@@ -28,15 +28,21 @@ def _vector_pdf(path: Path, pages: int = 2) -> None:
     doc.save(path)
 
 
-def _scan_pdf(path: Path, pages: int = 2, w: int = 1653, h: int = 2332) -> None:
+def _scan_pdf(path: Path, pages: int = 2, w: int = 1653, h: int = 2332, rotation: int = 0) -> None:
-    """Each page is one full-page grayscale image — what a real scan looks like."""
+    """Each page is one full-page grayscale image — what a real scan looks like.
+
+    `rotation` reproduces a sheet fed sideways: the image is stored in its own
+    orientation and /Rotate turns it upright for a viewer.
+    """
     art = np.full((h, w), 255, np.uint8)
     art[500:505, 100 : w - 100] = 0  # a staff line, so it isn't uniform
+    art[:60, :60] = 0  # a corner mark, so orientation is checkable
     pix = pymupdf.Pixmap(pymupdf.csGRAY, w, h, bytearray(art.tobytes()), False)
     doc = pymupdf.open()
     for _ in range(pages):
-        page = doc.new_page(width=A4.width, height=A4.height)
+        page = doc.new_page(width=A4.width, height=A4.width * h / w)
         page.insert_image(page.rect, pixmap=pix)
+        page.set_rotation(rotation)
     doc.save(path)
 
 
@@ -64,6 +70,22 @@ def main() -> int:
     assert page.min() == 0 and page.max() == 255, (page.min(), page.max())
     src.close()
 
+    # The corner mark sits top-left in an upright scan.
+    assert page[:60, :60].max() == 0 and page[:60, -60:].min() == 255
+
+    # A sideways scan comes back upright: the page's /Rotate applies to the
+    # image extracted by xref, which bypasses it. Okular gets this right and
+    # the slicer used to not.
+    sideways = tmp / "sideways.pdf"
+    _scan_pdf(sideways, pages=1, w=2332, h=1653, rotation=90)
+    src = open_source(sideways)
+    assert src.type is SourceType.RASTER
+    turned = page_raster(src, 0)
+    assert turned.shape == (2332, 1653), turned.shape
+    # Turned clockwise, so the mark that was top-left is now top-right.
+    assert turned[:60, -60:].max() == 0 and turned[:60, :60].min() == 255
+    src.close()
+
     # An override must win over detection, and say so.
     src = open_source(scan, SourceType.VECTOR)
     assert src.type is SourceType.VECTOR and src.detected is SourceType.RASTER
@@ -71,7 +93,7 @@ def main() -> int:
     assert page_raster(src, 0).shape[1] > 4000, "override must force a render"
     src.close()
 
-    for f in (vec, scan):
+    for f in (vec, scan, sideways):
         f.unlink()
     tmp.rmdir()
     print("ok")

tests/test_render.py

@@ -19,6 +19,7 @@ from noteman_slicer.pdf import open_source, page_raster  # noqa: E402
 from noteman_slicer.project import Cut, Project, default_path  # noqa: E402
 from noteman_slicer.render import (  # noqa: E402
     ALPHA_LEVELS,
+    _ink_bbox,
     apply_levels,
     encode,
     pad_right,
@@ -87,6 +88,19 @@ def main() -> int:
     assert rgba[:, :, 3].min() == 0, "paper must be fully transparent"
     assert len(np.unique(rgba[:, :, 3])) <= ALPHA_LEVELS
 
+    # Trim keeps a line of lyrics and drops a fleck. Each letter is its own
+    # small blob, so judging blobs by area threw the whole line away and the
+    # bottom voice lost its words; a fleck's row carries almost no ink at all.
+    art = np.full((300, 800), 255, np.uint8)
+    art[100:150, 50:750] = 0  # a staff
+    for x in range(60, 700, 30):  # lyrics: many small glyphs, one row
+        art[200:220, x : x + 14] = 0
+    art[5:9, 10:14] = 0  # a fleck in the far corner
+    x0, y0, x1, y1 = _ink_bbox(art)
+    assert (y0, y1) == (100, 220), f"lyrics kept, fleck dropped: {(y0, y1)}"
+    assert (x0, x1) == (50, 750), (x0, x1)
+    assert _ink_bbox(np.full((50, 50), 255, np.uint8)) is None, "blank slice has no box"
+
     # Levels: a white point below the paper value wipes the paper out entirely.
     faint = np.full((10, 10), 200, np.uint8)
     assert apply_levels(faint, 0, 180).max() == 255