Propose the content rectangle from the staff lines
Both scan problems reported from testing came from the same place: the content rectangle defaulted to the whole page, so the mechanism meant to handle margin junk never engaged. Ketun joululaulu has a vertical scan streak down the right margin and Engel a shadow, and because trim is tight and per slice, either one sets that slice's width, which sets the song's widest slice, which scales the whole song down. Detection can propose it. Staff lines are long horizontal runs; scan shadows, spine darkening and glass streaks are vertical, so a wide flat opening keeps one and erases the other. Three corrections were needed against the corpus: - Search only rows inside detected systems. A horizontal artefact above or below the music is itself a long horizontal run reaching the paper edge, which put Engel's left bound at 0. - Take a percentile of the staff-line extents, not the maximum. Where an artefact touches a staff line the two merge into one component: on Ketun p2 the merged line ends at 1575px against 1544px on the clean page. - Take the left bound from the brackets too. A bracket sits left of every staff line, so a staff-line bound crops it off — visible immediately when comparing exported slices. Anchors are now a dataclass carrying their left edge rather than a (top, bottom) tuple. Engel now drops 12-14% of page width and its music fills 1920px instead of leaving the shadow's dead space; Ketun drops 12%. Existing projects keep their saved rectangle; the editor's new Auto-fit and Auto-fit all buttons re-propose it without disturbing cuts.
This commit is contained in:
@@ -194,6 +194,27 @@ fall back to row-profile runs, which is correct there.
|
|||||||
|
|
||||||
**Staff height** — peak-to-peak spacing in the row profile.
|
**Staff height** — peak-to-peak spacing in the row profile.
|
||||||
|
|
||||||
|
**Content rectangle** — proposed per page from the staff lines. Staff lines are
|
||||||
|
long *horizontal* runs, while a scan-edge shadow, a spine darkening and the
|
||||||
|
streak a dirty scanner glass leaves are all *vertical*, so opening with a wide
|
||||||
|
flat kernel keeps the music and erases the artefacts. Three details make it
|
||||||
|
work:
|
||||||
|
|
||||||
|
- Only rows inside detected systems are searched. Otherwise a horizontal scan
|
||||||
|
artefact above or below the music is itself a long horizontal run, and it
|
||||||
|
reaches the paper edge.
|
||||||
|
- The horizontal bounds come from a *percentile* of the staff-line extents, not
|
||||||
|
their maximum. Where an artefact touches the end of a staff line the two merge
|
||||||
|
into one component; a page has dozens of staff lines and only a few are
|
||||||
|
contaminated.
|
||||||
|
- The left bound also considers the **brackets**, which sit left of every staff
|
||||||
|
line. A bound taken from staff lines alone crops the bracket off, and a
|
||||||
|
bracket is notation.
|
||||||
|
|
||||||
|
Only the horizontal bounds are proposed. Vertically the cuts and discard flags
|
||||||
|
already isolate the header and footer, and cropping the top would risk clipping
|
||||||
|
a tempo mark or a section label above the first staff.
|
||||||
|
|
||||||
**Source type** — `get_images(full=True)` / `get_drawings()` proposes bitmap or
|
**Source type** — `get_images(full=True)` / `get_drawings()` proposes bitmap or
|
||||||
vector per PDF; the tool asks the user to confirm before routing. (`full=True` is
|
vector per PDF; the tool asks the user to confirm before routing. (`full=True` is
|
||||||
required, or `get_image_bbox` rejects the item.)
|
required, or `get_image_bbox` rejects the item.)
|
||||||
|
|||||||
+101
-19
@@ -28,6 +28,19 @@ _ANCHOR_KERNEL = 0.03 # vertical open kernel, as a fraction of page height
|
|||||||
_ANCHOR_MIN = 0.04 # a bracket is at least this tall, as a fraction of page
|
_ANCHOR_MIN = 0.04 # a bracket is at least this tall, as a fraction of page
|
||||||
_PROFILE_FLOOR = 0.02 # ink-run threshold, as a fraction of the profile peak
|
_PROFILE_FLOOR = 0.02 # ink-run threshold, as a fraction of the profile peak
|
||||||
_EXPAND_REACH = 1.5 # how far past the bracket a system's ink reaches, in staff heights
|
_EXPAND_REACH = 1.5 # how far past the bracket a system's ink reaches, in staff heights
|
||||||
|
_STAFF_KERNEL = 0.05 # horizontal open kernel, as a fraction of page width
|
||||||
|
_STAFF_MIN_WIDTH = 0.2 # a staff line spans at least this share of the page
|
||||||
|
_CONTENT_MARGIN = 0.01 # slack past the staff ends, for ledger lines and lyrics
|
||||||
|
_EDGE_PERCENTILE = 15 # tolerate this share of staff lines merged into scan artefacts
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Anchor:
|
||||||
|
"""A system's vertical bracket: where it is, and how far left it reaches."""
|
||||||
|
|
||||||
|
top: int
|
||||||
|
bottom: int
|
||||||
|
left: int
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
@@ -48,6 +61,7 @@ class PageDetection:
|
|||||||
skew: float
|
skew: float
|
||||||
systems: list[System] = field(default_factory=list)
|
systems: list[System] = field(default_factory=list)
|
||||||
cuts: list[int] = field(default_factory=list)
|
cuts: list[int] = field(default_factory=list)
|
||||||
|
content: tuple[float, float, float, float] = (0.0, 0.0, 1.0, 1.0)
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def bracketless(self) -> bool:
|
def bracketless(self) -> bool:
|
||||||
@@ -89,8 +103,8 @@ def deskew(gray: np.ndarray, angle: float) -> np.ndarray:
|
|||||||
return _rotate(gray, angle)
|
return _rotate(gray, angle)
|
||||||
|
|
||||||
|
|
||||||
def system_anchors(gray: np.ndarray) -> list[tuple[int, int]]:
|
def system_anchors(gray: np.ndarray) -> list[Anchor]:
|
||||||
"""y-extents of the vertical brackets, one per system."""
|
"""The vertical brackets, one per system."""
|
||||||
h = gray.shape[0]
|
h = gray.shape[0]
|
||||||
binary = (gray < _INK).astype(np.uint8)
|
binary = (gray < _INK).astype(np.uint8)
|
||||||
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, max(3, int(h * _ANCHOR_KERNEL))))
|
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (1, max(3, int(h * _ANCHOR_KERNEL))))
|
||||||
@@ -98,19 +112,82 @@ def system_anchors(gray: np.ndarray) -> list[tuple[int, int]]:
|
|||||||
|
|
||||||
count, _, stats, _ = cv2.connectedComponentsWithStats(strokes, 8)
|
count, _, stats, _ = cv2.connectedComponentsWithStats(strokes, 8)
|
||||||
tall = [
|
tall = [
|
||||||
(stats[i, cv2.CC_STAT_TOP], stats[i, cv2.CC_STAT_TOP] + stats[i, cv2.CC_STAT_HEIGHT])
|
Anchor(
|
||||||
|
int(stats[i, cv2.CC_STAT_TOP]),
|
||||||
|
int(stats[i, cv2.CC_STAT_TOP] + stats[i, cv2.CC_STAT_HEIGHT]),
|
||||||
|
int(stats[i, cv2.CC_STAT_LEFT]),
|
||||||
|
)
|
||||||
for i in range(1, count)
|
for i in range(1, count)
|
||||||
if stats[i, cv2.CC_STAT_HEIGHT] > h * _ANCHOR_MIN
|
if stats[i, cv2.CC_STAT_HEIGHT] > h * _ANCHOR_MIN
|
||||||
]
|
]
|
||||||
|
|
||||||
# Tallest first, keeping only strokes that don't overlap one already kept:
|
# Tallest first, keeping only strokes that don't overlap one already kept:
|
||||||
# a system's barlines all overlap its bracket, so each system yields one.
|
# a system's barlines all overlap its bracket, so each system yields one.
|
||||||
anchors: list[tuple[int, int]] = []
|
# The kept stroke is the tallest, which is the bracket rather than a barline.
|
||||||
for top, bottom in sorted(tall, key=lambda s: s[1] - s[0], reverse=True):
|
anchors: list[Anchor] = []
|
||||||
if any(not (bottom < a[0] or top > a[1]) for a in anchors):
|
for candidate in sorted(tall, key=lambda a: a.bottom - a.top, reverse=True):
|
||||||
|
if any(not (candidate.bottom < a.top or candidate.top > a.bottom) for a in anchors):
|
||||||
continue
|
continue
|
||||||
anchors.append((top, bottom))
|
anchors.append(candidate)
|
||||||
return sorted(anchors)
|
return sorted(anchors, key=lambda a: a.top)
|
||||||
|
|
||||||
|
|
||||||
|
def content_columns(
|
||||||
|
gray: np.ndarray, anchors: list[Anchor] | None = None
|
||||||
|
) -> tuple[float, float]:
|
||||||
|
"""Where the music is horizontally, as normalised x bounds.
|
||||||
|
|
||||||
|
Staff lines are long *horizontal* runs; a scan-edge shadow, a spine
|
||||||
|
darkening and the vertical line a dirty scanner glass leaves are all
|
||||||
|
*vertical*. Opening with a wide flat kernel keeps the first and erases the
|
||||||
|
others, so the staff lines' own bounding box is the music area.
|
||||||
|
|
||||||
|
`anchors` does two jobs. It restricts the search to rows known to hold
|
||||||
|
systems — without that, a horizontal scan artefact above or below the music
|
||||||
|
is itself a long horizontal run reaching the paper edge, which is exactly
|
||||||
|
the measurement being avoided. And its brackets give the true left bound:
|
||||||
|
a bracket sits *left of every staff line*, so a bound taken from staff
|
||||||
|
lines alone crops it off, and a bracket is notation, not artefact.
|
||||||
|
|
||||||
|
This matters more than it looks: trim is tight and per slice, so one dark
|
||||||
|
band down the margin sets that slice's width, which sets the song's widest
|
||||||
|
slice, which scales the whole song down.
|
||||||
|
"""
|
||||||
|
height, width = gray.shape
|
||||||
|
binary = (gray < _INK).astype(np.uint8)
|
||||||
|
if anchors:
|
||||||
|
keep = np.zeros(height, bool)
|
||||||
|
for anchor in anchors:
|
||||||
|
keep[max(0, anchor.top) : min(height, anchor.bottom)] = True
|
||||||
|
binary[~keep] = 0
|
||||||
|
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (max(3, int(width * _STAFF_KERNEL)), 1))
|
||||||
|
lines = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel)
|
||||||
|
|
||||||
|
count, _, stats, _ = cv2.connectedComponentsWithStats(lines, 8)
|
||||||
|
runs = [
|
||||||
|
(stats[i, cv2.CC_STAT_LEFT], stats[i, cv2.CC_STAT_LEFT] + stats[i, cv2.CC_STAT_WIDTH])
|
||||||
|
for i in range(1, count)
|
||||||
|
if stats[i, cv2.CC_STAT_WIDTH] > width * _STAFF_MIN_WIDTH
|
||||||
|
]
|
||||||
|
if not runs:
|
||||||
|
return 0.0, 1.0
|
||||||
|
|
||||||
|
# Percentiles, not the extremes. Where a scan-edge band happens to touch
|
||||||
|
# the end of a staff line the two merge into one component, and that
|
||||||
|
# component then reaches into the artefact — on Ketun joululaulu p2 the
|
||||||
|
# merged line ends at 1575px against 1544px on the clean page. A page has
|
||||||
|
# dozens of staff lines and only a few are contaminated, so a percentile
|
||||||
|
# lands on the true edge while the extreme lands on the worst artefact.
|
||||||
|
lefts = np.array([r[0] for r in runs], float)
|
||||||
|
rights = np.array([r[1] for r in runs], float)
|
||||||
|
margin = width * _CONTENT_MARGIN
|
||||||
|
|
||||||
|
left = float(np.percentile(lefts, _EDGE_PERCENTILE))
|
||||||
|
if anchors:
|
||||||
|
left = min(left, min(a.left for a in anchors))
|
||||||
|
right = float(np.percentile(rights, 100 - _EDGE_PERCENTILE))
|
||||||
|
|
||||||
|
return max(0.0, left - margin) / width, min(float(width), right + margin) / width
|
||||||
|
|
||||||
|
|
||||||
def ink_runs(gray: np.ndarray) -> list[tuple[int, int]]:
|
def ink_runs(gray: np.ndarray) -> list[tuple[int, int]]:
|
||||||
@@ -161,18 +238,17 @@ def staff_height(gray: np.ndarray, top: int, bottom: int) -> float | None:
|
|||||||
return float(np.median(intra) * 4) # 5 lines, 4 spaces
|
return float(np.median(intra) * 4) # 5 lines, 4 spaces
|
||||||
|
|
||||||
|
|
||||||
def _gap(run: tuple[int, int], span: tuple[int, int]) -> int:
|
def _gap(run: tuple[int, int], anchor: Anchor) -> int:
|
||||||
"""Vertical distance between an ink run and a bracket span; 0 if they overlap."""
|
"""Vertical distance between an ink run and a bracket; 0 if they overlap."""
|
||||||
start, end = run
|
start, end = run
|
||||||
top, bottom = span
|
if end > anchor.top and start < anchor.bottom:
|
||||||
if end > top and start < bottom:
|
|
||||||
return 0
|
return 0
|
||||||
return top - end if end <= top else start - bottom
|
return anchor.top - end if end <= anchor.top else start - anchor.bottom
|
||||||
|
|
||||||
|
|
||||||
def _assign(
|
def _assign(
|
||||||
runs: list[tuple[int, int]],
|
runs: list[tuple[int, int]],
|
||||||
anchors: list[tuple[int, int]],
|
anchors: list[Anchor],
|
||||||
reaches: list[float],
|
reaches: list[float],
|
||||||
) -> list[tuple[int, int]]:
|
) -> list[tuple[int, int]]:
|
||||||
"""Give every ink run to one system, and return each system's extent.
|
"""Give every ink run to one system, and return each system's extent.
|
||||||
@@ -196,7 +272,7 @@ def _assign(
|
|||||||
to the system below and the cut lands high. Dragging the cut is the fix;
|
to the system below and the cut lands high. Dragging the cut is the fix;
|
||||||
separating them needs a signal this pass doesn't have.
|
separating them needs a signal this pass doesn't have.
|
||||||
"""
|
"""
|
||||||
bounds = [list(a) for a in anchors]
|
bounds = [[a.top, a.bottom] for a in anchors]
|
||||||
|
|
||||||
def claim(index: int, run: tuple[int, int]) -> None:
|
def claim(index: int, run: tuple[int, int]) -> None:
|
||||||
bounds[index][0] = min(bounds[index][0], run[0])
|
bounds[index][0] = min(bounds[index][0], run[0])
|
||||||
@@ -208,7 +284,7 @@ def _assign(
|
|||||||
# Ink overlapping a bracket belongs to it — to the one it overlaps most,
|
# Ink overlapping a bracket belongs to it — to the one it overlaps most,
|
||||||
# whatever else is in reach.
|
# whatever else is in reach.
|
||||||
inside = [
|
inside = [
|
||||||
(min(run[1], anchors[i][1]) - max(run[0], anchors[i][0]), i)
|
(min(run[1], anchors[i].bottom) - max(run[0], anchors[i].top), i)
|
||||||
for i, g in enumerate(gaps)
|
for i, g in enumerate(gaps)
|
||||||
if g == 0
|
if g == 0
|
||||||
]
|
]
|
||||||
@@ -221,7 +297,7 @@ def _assign(
|
|||||||
continue # a title block or a footer: too far from any system
|
continue # a title block or a footer: too far from any system
|
||||||
|
|
||||||
# Otherwise the system above wins, and only failing that the one below.
|
# Otherwise the system above wins, and only failing that the one below.
|
||||||
above = [i for i in within if anchors[i][1] <= run[0]]
|
above = [i for i in within if anchors[i].bottom <= run[0]]
|
||||||
claim(above[-1] if above else within[0], run)
|
claim(above[-1] if above else within[0], run)
|
||||||
|
|
||||||
return [(lo, hi) for lo, hi in bounds]
|
return [(lo, hi) for lo, hi in bounds]
|
||||||
@@ -238,7 +314,7 @@ def detect_page(gray: np.ndarray, skew: float | None = None) -> PageDetection:
|
|||||||
if anchors:
|
if anchors:
|
||||||
# Staff height is measured on the bracket span, before expansion, so a
|
# Staff height is measured on the bracket span, before expansion, so a
|
||||||
# swallowed title block can't distort it.
|
# swallowed title block can't distort it.
|
||||||
heights = [staff_height(straight, top, bottom) for top, bottom in anchors]
|
heights = [staff_height(straight, a.top, a.bottom) for a in anchors]
|
||||||
reaches = [(h or gray.shape[0] * 0.02) * _EXPAND_REACH for h in heights]
|
reaches = [(h or gray.shape[0] * 0.02) * _EXPAND_REACH for h in heights]
|
||||||
systems = [
|
systems = [
|
||||||
System(top=lo, bottom=hi, staff_height=h)
|
System(top=lo, bottom=hi, staff_height=h)
|
||||||
@@ -252,4 +328,10 @@ def detect_page(gray: np.ndarray, skew: float | None = None) -> PageDetection:
|
|||||||
cuts = [
|
cuts = [
|
||||||
(systems[i].bottom + systems[i + 1].top) // 2 for i in range(len(systems) - 1)
|
(systems[i].bottom + systems[i + 1].top) // 2 for i in range(len(systems) - 1)
|
||||||
]
|
]
|
||||||
return PageDetection(skew=angle, systems=systems, cuts=cuts)
|
# Only the horizontal bounds are proposed. Vertically the cuts and the
|
||||||
|
# discard flags already isolate the header and footer, and cropping the top
|
||||||
|
# would risk clipping a tempo mark or a section label above the first staff.
|
||||||
|
left, right = content_columns(straight, anchors)
|
||||||
|
return PageDetection(
|
||||||
|
skew=angle, systems=systems, cuts=cuts, content=(left, 0.0, right, 1.0)
|
||||||
|
)
|
||||||
|
|||||||
@@ -356,9 +356,17 @@ class Editor(QMainWindow):
|
|||||||
discard = QPushButton("Toggle discard (D)")
|
discard = QPushButton("Toggle discard (D)")
|
||||||
discard.clicked.connect(self.view.toggle_discard)
|
discard.clicked.connect(self.view.toggle_discard)
|
||||||
form.addRow(discard)
|
form.addRow(discard)
|
||||||
reset = QPushButton("Reset content rectangle")
|
rect_row = QHBoxLayout()
|
||||||
|
fit = QPushButton("Auto-fit")
|
||||||
|
fit.setToolTip("Propose the content rectangle from the staff lines on this page")
|
||||||
|
fit.clicked.connect(lambda: self._autofit_rect(False))
|
||||||
|
fit_all = QPushButton("Auto-fit all")
|
||||||
|
fit_all.clicked.connect(lambda: self._autofit_rect(True))
|
||||||
|
reset = QPushButton("Reset")
|
||||||
reset.clicked.connect(self._reset_rect)
|
reset.clicked.connect(self._reset_rect)
|
||||||
form.addRow(reset)
|
for button in (fit, fit_all, reset):
|
||||||
|
rect_row.addWidget(button)
|
||||||
|
form.addRow("Content rect", rect_row)
|
||||||
box.addWidget(page_box)
|
box.addWidget(page_box)
|
||||||
|
|
||||||
meta_box = QGroupBox("Song")
|
meta_box = QGroupBox("Song")
|
||||||
@@ -474,6 +482,23 @@ class Editor(QMainWindow):
|
|||||||
}
|
}
|
||||||
self.autosave.start()
|
self.autosave.start()
|
||||||
|
|
||||||
|
def _autofit_rect(self, every_page: bool) -> None:
|
||||||
|
"""Re-propose the content rectangle from the staff lines.
|
||||||
|
|
||||||
|
Detection does this for new projects; this is how a project made before
|
||||||
|
it existed, or one whose rectangle was dragged wrong, gets it back.
|
||||||
|
"""
|
||||||
|
pages = range(len(self.project.pages)) if every_page else [self.index]
|
||||||
|
for i in pages:
|
||||||
|
# The preview is already deskewed, so the sweep is skipped.
|
||||||
|
proposal = detect_page(self._preview(i), skew=0.0)
|
||||||
|
self.project.pages[i].content_rect = proposal.content
|
||||||
|
self.view.redraw()
|
||||||
|
self._touched()
|
||||||
|
self.statusBar().showMessage(
|
||||||
|
f"content rectangle fitted on {len(list(pages))} page(s)", 2000
|
||||||
|
)
|
||||||
|
|
||||||
def _reset_rect(self) -> None:
|
def _reset_rect(self) -> None:
|
||||||
self.project.pages[self.index].content_rect = None
|
self.project.pages[self.index].content_rect = None
|
||||||
self.view.redraw()
|
self.view.redraw()
|
||||||
|
|||||||
@@ -170,6 +170,9 @@ class Project:
|
|||||||
skew=detection.skew,
|
skew=detection.skew,
|
||||||
cuts=[Cut.straight(y / height) for y in ys],
|
cuts=[Cut.straight(y / height) for y in ys],
|
||||||
discards=discards,
|
discards=discards,
|
||||||
|
# Per page, not per song: scans drift, so the margin junk
|
||||||
|
# sits in a different place on each one.
|
||||||
|
content_rect=detection.content,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return cls(source=source, source_hash=hash_file(source), pages=pages)
|
return cls(source=source, source_hash=hash_file(source), pages=pages)
|
||||||
|
|||||||
Reference in New Issue
Block a user