You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 94086a8
Browse filesBrowse the repository at this point in the historyBrowse files
fix(preprocessing): #78 define loudness contract + metadata trail (does NOT recover Whisper — see #83) (#82)
* fix(preprocessing): #78 restore absolute clip loudness via single-gain target peak (M3c)
M3a per-turn RMS targeting + M3b limiter-only normalization left M3a-shaped
Tier A clips peaking ~6 dB below the −1 dBFS ceiling. Whisper under-
transcribed (WER 0.04 → 0.28, length-ratio 0.76) and UTMOS read the
loosely-limited clips ~0.9 MOS higher, dominating M17 evaluation.
Fix is a *single global gain* post-mix loudness step — preserves per-turn
RMS ratios exactly (M3a's whole point), unlike the legacy per-segment peak
normalizer M3b removed. Configurable target with a sane default.
Changes:
- synthbanshee/augment/preprocessing.py: new peak_normalize_to_target()
helper; preprocess() runs it before the existing −1 dBFS safety limiter
- synthbanshee/config/preprocessing_config.py: target_peak_dbfs field,
default −2.0 dBFS, range [−12.0, −1.0]
- synthbanshee/cli.py: Stage 3b post-augment normalize uses the same
helper + same config so Tier A and Tier B/C exit at the same peak,
fixing a pre-existing asymmetry that hid the regression on Tier A
- tests/unit/test_preprocessing.py: invert the M3b-era "quiet clip not
scaled up" assertion, add direct unit tests for the helper, validate
config range constraints
- tests/integration/test_loudness_regression.py: end-to-end guard
asserting peak lands in [−3, −1] dBFS and per-turn RMS contrast
survives a single global gain
- AGENTS.md, docs/spec.md §3 + §3.1, docs/audio_generation_v3_design.md
§4.7 + tracker M3c row: spec text in sync with the two-stage policy
Verification (sp_neu_a_0001 same-config A/B):
2026-04-15 reference peak −1.00 dBFS rms −20.37 dBFS
2026-05-05 before fix peak −5.62 dBFS rms −26.68 dBFS
2026-05-05 after fix peak −2.00 dBFS rms −23.06 dBFS
Closes#78
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(preprocessing): #78 review pass — reframe to loudness-contract + metadata trail (lever probe killed Whisper hypothesis)
Self-review of the prior commit found the Whisper-fix framing empirically
false. Reframed the PR around the actual deliverables (loudness contract
clarity + metadata trail) and addressed all blocking review points.
Empirical finding (lever probe, 2026-05-05, openai/whisper-large-v3):
across 8 single-global-gain variants of the same source audio at peaks
from −6 to +7 dBFS and RMS from −27 to −14 dBFS, Whisper produced
byte-identical hypotheses in 7/8 cases (WER 0.286, length-ratio 0.762).
Loudness in the spec range is invisible to Whisper's log-mel extractor.
The actual ASR regression is content-driven (likely M15 / #70 prosody
changes) and is now tracked separately in #83.
Review fixes applied:
- #4 Metadata trail (the structural fix that prevents #78 recurring):
GenerationMetadata gains `loudness_target_peak_dbfs: float | None` and
bumps `normalization_strategy` to "per_turn_rms_v2_target_peak".
Without this field, the original regression hid for three weeks
behind unchanged generator_version.
- #5 Pydantic upper bound −1.0 → −1.5. 0.5 dB headroom over the safety
limiter ceiling guarantees the two stages cannot collide under
float-arithmetic noise.
- #6 Step names flattened to literal tokens (peak_normalize, peak_limit,
silence_pad — no embedded numeric parameters). Configured value moves
to PreprocessingResult.target_peak_dbfs as structured data so QA
tooling reads policy via field access, not regex on a step string.
- #7 Renamed misleading `test_loud_clip_clamped_below_ceiling` to
`test_loud_input_lands_at_target_not_at_ceiling` — the limiter is now
a guaranteed no-op in normal flow; the target step is what does the
work.
- #8 Replaced 440 Hz sine fixture with bandpass-filtered Gaussian noise
targeting 18 dB crest, mirroring real Hebrew TTS post-M3a behaviour.
Sine's 3 dB crest let regressions slip past peak-anchored asserts.
- #9 Tightened tolerances 0.5 → 0.1 dB (peak) and 0.5 → 0.2 dB (RMS
contrast). Original #78 deviation was 4.6 dB so 0.5 dB only caught
it by luck; 0.1 dB catches subtle regressions.
- #10 Tier B/C cli.py path coverage: two new tests in
test_tier_b_pipeline.py asserting (a) post-augment peak lands at the
configured target and (b) GenerationMetadata carries the new
normalization_strategy + loudness_target_peak_dbfs fields.
- #11 Dropped fictional "M3c" milestone tag everywhere. Tracker row in
audio_generation_v3_design.md is now indexed by #78. peak_normalize_to_target
docstring expanded to mention dual use (Stage 3a + Stage 3b post-augment).
- #3 Tier B/C −1.0 → −2.0 dBFS behaviour change called out explicitly in
PR body, AGENTS.md, and the §4.7 spec change. This is incidentally
better for Tier B/C (gives 1 dB inter-sample-peak headroom for room IR
/ noise mixing), but it IS a change.
Verification:
- pytest 1720 passed / 1 skipped (added 4 tests, removed 1 obsolete)
- ruff: clean
- mypy: 36 unique errors, identical to main (zero new)
- Live re-render of sp_neu_a_0001 produces peak = -2.000 dBFS exactly,
with metadata fields populated as expected.
Follow-up issue filed: #83 — investigate the actual Whisper WER cause
(prosody/duration, not loudness).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(preprocessing): #78 mirror "5a then 5b" policy on Tier B/C path (PR #82 COPILOT-1)
cli.py Stage 3b previously ran only the target-peak step (5a), not the
safety limiter (5b), giving the documented two-stage policy a paper-vs-
reality gap. The limiter is provably a no-op given Pydantic bounds
(target ≤ −1.5, ceiling = −1.0, PCM_16 quantisation cannot bridge the
0.5 dB margin), but applying it uniformly closes the asymmetry between
Tier A (preprocess()) and Tier B/C (cli.py) and prevents a future
spec/code drift from silently violating §4.7.
PR #82 review thread COPILOT-2 (docs/spec.md §3.1 steps 1-4 stale since
M14) is correct but pre-existing and unrelated to #78 — filed as #84
and resolved out-of-scope here.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
-**Peak limiter ceiling: −1.0 dBFS** — never scale up quiet clips (limiter, not normalizer)
35
+
- **Loudness: peak-normalize to target (−2.0 dBFS default, range [−12.0, −1.5]), then peak-limit at ≤ −1.0 dBFS** — `preprocess()` applies a single global gain so the clip's absolute peak lands at `PreprocessingConfig.target_peak_dbfs`; the −1.0 dBFS limiter is a safety ceiling that is a guaranteed no-op for in-spec target values. The single-gain step preserves per-turn RMS contrast (M3a) — it shifts absolute level but every per-turn RMS *ratio* survives unchanged. Tier B/C augmentation re-applies the same target after room-IR/noise mixing through the shared `peak_normalize_to_target` helper, so all tiers exit at the same absolute peak. **Behaviour change vs. pre-#78:** Tier B/C clips previously normalized to −1.0 dBFS (zero headroom over the limiter); they now exit at the configured target (−2.0 dBFS by default). The `loudness_target_peak_dbfs` field in `GenerationMetadata` records the exact value used so any future loudness regression can be diagnosed from metadata alone — without this trail, #78 hid for three weeks behind unchanged `generator_version`. **Empirical note:** peak/RMS in the spec range does not affect Whisper WER (Whisper's log-mel feature extractor internally normalizes); the M17 ASR regression has a separate prosody-level cause being tracked elsewhere. (#78)
36
36
-**Silence padding: ≥ 0.5 s** at head and tail — `silence_pad_applied_s` is per-side
37
37
- Onset/offset times from `MixedScene` must be shifted by the leading pad only
Copy file name to clipboardExpand all lines: docs/audio_generation_v3_design.md
+12-2Lines changed: 12 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -35,6 +35,7 @@ All P0–P2 work is tracked here. Milestones that span multiple PRs are listed o
35
35
|**M2a**| P0 | ✅ Done | Lowers VIC F0 to adult range; caps AGG pitch escalation | Update `style_map` in speaker YAMLs per §4.2a table — VIC pitch −4→−1 st, AGG pitch capped 0/0/0/+1/+1 across I2–I5 | Small | No |
36
36
|**M3a**| P0 | ✅ Done | Preserves within-scene loudness trajectory |`rms_target_dbfs` on `StyleEntry`; `_apply_rms_gain()` helper in `SceneMixer`; 4-tuple segment API; AGG −28→−15 dBFS (I1→I5), VIC −26→−30 dBFS; FLOAT temp WAV write to avoid PCM_16 hard-clip | Small | No |
37
37
|**M3b**| P0 | ✅ Done | Prevents peak-normalize from erasing inter-scene RMS contrast | Replace `_normalize_peak()` in `preprocess()` with sample-peak limiter (only clips above −1.0 dBFS, no forced scale-up); update `validate_audio()` to check peak ≤ −1.0 dBFS instead of == −1.0 dBFS; update `spec.md §3`| Small | Yes — spec.md §3 |
38
+
| **#78** | P1 | ✅ Done | Defines a clip-level loudness contract (target peak + safety ceiling) and a metadata trail so future loudness regressions are diagnosable from `{clip_id}.json` alone | `peak_normalize_to_target()` helper in `synthbanshee/augment/preprocessing.py` (single global gain — preserves M3a contrast); `PreprocessingConfig.target_peak_dbfs: float = −2.0` (range `[−12.0, −1.5]`, 0.5 dB headroom over the safety limiter); `preprocess()` runs the target-peak step before the −1.0 dBFS safety limiter; `cli.py` Stage 3b post-augment normalize uses the same helper + same config so Tier A and Tier B/C exit at the same absolute peak (Tier B/C target moves from −1.0 → −2.0 dBFS as a side effect — gives 1 dB inter-sample-peak headroom for room-IR/noise mixing); `GenerationMetadata` adds `loudness_target_peak_dbfs` and bumps `normalization_strategy` from `per_turn_rms_v1` → `per_turn_rms_v2_target_peak`; `PreprocessingResult` adds `target_peak_dbfs` field; step-name strings flattened to literal tokens (`peak_normalize`, `peak_limit`, `silence_pad`); integration regression tests with realistic-crest fixture asserting peak deviation < 0.1 dB and per-turn RMS contrast within 0.2 dB; spec updated. **Empirical note (lever probe, 2026-05-05):** peak normalization at this level does not change Whisper WER — Whisper's log-mel extractor internally normalizes — so this is a contract / metadata fix, not an ASR fix. The M17 ASR regression has a separate root cause tracked in the prosody-bisect follow-up. | Small | Yes — spec.md §3, §3.1, §4.7; AGENTS.md |
38
39
|**M4**| P0 | ✅ Done | Fixes silent label corruption in strong-label JSONL | Audit `debug_run_1` emotional states; extend `taxonomy.yaml``emotional_states` to cover all legitimate LLM outputs; replace `_normalize_emotion()` silent fallback with explicit mapping table + logged warning (hard fail for completely unknown states); add clip-level `quality_flags` value `"emotion_downgrade"`| Small | No |
39
40
|**M5**| P1 | ✅ Done | Protects Tier A phonetic quality from over-denoising |`PreprocessingConfig` Pydantic `BaseModel` with `wiener_denoise` flag; update `preprocess()` to accept it (default preserves current Wiener behavior); scene YAMLs gain optional `preprocessing` block; Wiener step skipped when `wiener_denoise=False`; unit tests verify Wiener skip | Small | No |
40
41
|**M6**| P1 | ✅ Done | Replaces mechanical silence gaps with psychologically-motivated turn latencies |`TurnGapController` in `synthbanshee/tts/gap_controller.py`; project-specific gap tables (§4.5); wire into `TTSRenderer.render_scene()` replacing `turn.pause_before_s`; NEG/NEU confusor gap ranges; unit tests per context type and project | Small | No |
@@ -355,11 +356,20 @@ The label generator **must use `audible_onset_s` / `audible_end_s`** for all eve
355
356
356
357
1.**Per-turn RMS targeting at the mix stage:** Each turn is gain-adjusted before being inserted into `SceneMixer` so its RMS matches the intensity-level target from §4.2. This creates the loudness escalation.
357
358
358
-
2.**Replace scene-level peak normalization with a true-peak limiter:** After mixing, apply a limiter at ≤ −1.0 dBFS to ensure technical compliance without collapsing the trajectory. Note: the current `preprocess()` implementation uses sample-peak normalization (a simple divide-by-max). A true-peak limiter accounts for inter-sample peaks (relevant for lossy re-encoding, less critical here but more correct). If implementation uses sample-peak limiting, document it as such in the spec rather than calling it "true-peak."
359
+
2.**Two-stage clip-level loudness normalization (#78, extends the original M3b limiter-only behaviour with an explicit target floor):**
360
+
361
+
-**5a. Peak-normalize to target via single global gain.** A single multiplicative factor is applied to the *entire* mixed scene so its peak lands at `PreprocessingConfig.target_peak_dbfs` (default −2.0 dBFS, range `[−12.0, −1.5]`). Single-gain normalization is the key invariant: it preserves *every* per-turn RMS *ratio* exactly, so the within-scene loudness trajectory created by step 1 survives — only the absolute level shifts. This is *not* the old per-turn divide-by-max normalizer that M3b removed; that one collapsed contrast because it scaled each segment to its own peak. The new step normalizes after mixing, once, globally.
362
+
-**5b. Safety limiter at ≤ −1.0 dBFS.** Defence-in-depth only: attenuates over-range samples without scaling up. The 0.5 dB margin between the target upper bound (−1.5) and the limiter ceiling (−1.0) guarantees this stage is a no-op for all in-spec target values.
363
+
364
+
**History.** PR #25 (M3a) added per-turn RMS targeting; PR #27 (M3b) replaced the legacy peak normalizer with a peak-only limiter to stop it from erasing M3a's per-turn contrast. M3b succeeded at preserving contrast but left the absolute clip peak unspecified — it floated wherever the M3a-targeted RMS produced (typically ~−6 dBFS for Tier A). The M17 Phase A spike noted that quieter clips also showed elevated Whisper WER and shifted UTMOS; issue #78 was filed under the hypothesis that loudness drove the ASR regression. **A lever probe over the same source audio at peaks ranging from −6 to +7 dBFS (and RMS from −27 to −14 dBFS) showed Whisper produces byte-identical hypotheses at every peak level in the spec range** — Whisper's log-mel extractor internally normalizes, so absolute loudness is invisible to it. Step 5a is therefore *not* an ASR fix; it is a **loudness-contract fix**: the spec previously had only an upper bound (`peak ≤ −1.0 dBFS`), no target floor, so two clips could legitimately sit 6 dB apart and both be in-spec. With step 5a in place, target peak is configured policy and `loudness_target_peak_dbfs` in `GenerationMetadata` carries the value used so future loudness drift is diagnosable from clip metadata alone.
365
+
366
+
**Tier B/C augmentation re-applies step 5a** after room-IR + noise mixing, since that pipeline reshapes the peak. Both tiers exit at `target_peak_dbfs`, so the dataset has a consistent loudness regime across tiers (it previously did not — Tier A had no target, Tier B/C hardcoded −1.0).
367
+
368
+
**The Whisper WER regression itself remains open** — it is real but loudness-independent. Lever-probe data and root-cause investigation tracked separately (prosody-bisect follow-up issue): most likely candidates are the M15 prosody tuning (#51) and the inter-word `<break>` tags from #70.
359
369
360
370
3.**LUFS logging in QA:** Log short-term LUFS by turn (via pyloudnorm or equivalent) in addition to RMS. LUFS is more perceptually accurate and is the standard used by broadcast and streaming. Include it in the acoustic QA metrics (§7) for trend monitoring.
361
371
362
-
4.**Spec change required:**`docs/spec.md` §3 currently states "−1.0 dBFS peak normalized." Must be updated to: "Peak-limited to ≤ −1.0 dBFS after mixing; per-turn RMS targeting applied at mix stage to preserve within-scene loudness trajectory."
372
+
4.**Spec text:**`docs/spec.md` §3 row "Amplitude normalization" and §3.1 step 5 are kept in sync with the two-stage policy above.
Copy file name to clipboardExpand all lines: docs/spec.md
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -97,7 +97,7 @@ Project codes: `SP` (She-Proves) · `EL` (Elephant in the Room)
97
97
| Sample rate | 16,000 Hz | Resample before delivery; retain originals at native rate |
98
98
| Bit depth | 16-bit PCM ||
99
99
| Channels | Mono | Downmix to mono before delivery |
100
-
| Amplitude normalization | Peak ≤ −1.0 dBFS (limiter, not normalizer) | Only attenuates; never scales up — preserves within-scene loudness trajectory|
100
+
| Amplitude normalization | Peak-normalize to target (−2.0 dBFS default, range `[−12.0, −1.5]`) via single global gain, then peak-limit at ≤ −1.0 dBFS | Single-gain normalization preserves per-turn RMS contrast (M3a); the 0.5 dB margin between target upper bound and limiter ceiling guarantees the limiter is a no-op in normal flow (#78)|
101
101
| Silence padding | ≥ 0.5 s of ambient baseline before and after target speech ||
102
102
| SNR at acquisition | ≥ 15 dB (Tier A) | Tier B/C may degrade controllably below 15 dB; log actual SNR in metadata |
103
103
| Max clip duration | 300 s (5 min) | Longer source scenes must be segmented |
@@ -111,7 +111,9 @@ All clips must pass through this pipeline before delivery. The "dirty" pre-pipel
111
111
2.**Downmix** — stereo → mono (average channels)
112
112
3.**Spectral filter** — low-pass at 7,500 Hz to remove irrelevant high-frequency noise from budget sensors (Butterworth order 4)
113
113
4.**Denoising** — spectral subtraction (Wiener filtering) to remove electrical hum; parameterize noise profile from silent leading segment
114
-
5.**Peak limit** — attenuate to ≤ −1.0 dBFS if the signal exceeds that ceiling; never scale up. This preserves the within-scene loudness trajectory established by per-turn RMS gain (M3a). A forced scale-up would collapse intensity-level amplitude differences.
114
+
5.**Loudness normalization** (#78) — two stages:
115
+
-**5a. Peak-normalize to target.** Apply a single global gain so the absolute peak lands at `PreprocessingConfig.target_peak_dbfs` (default −2.0 dBFS). A *single* gain preserves per-turn RMS *ratios* exactly, so the within-scene loudness trajectory established by per-turn RMS gain (M3a) survives — only the absolute level shifts. This step replaces the M3b "limiter only, never scale up" behaviour: pre-#78 the spec had only an upper bound on peak, leaving the absolute level unspecified; two clips could legitimately sit 6 dB apart and both be in-spec.
116
+
-**5b. Safety limiter.** Attenuate any sample exceeding −1.0 dBFS. For in-spec target values (`target_peak_dbfs ∈ [−12.0, −1.5]`) this is a guaranteed no-op (0.5 dB margin); it remains as defence-in-depth against upstream over-range samples.
115
117
6.**Silence pad** — verify ≥ 0.5 s ambient baseline at head and tail; add if absent
116
118
7.**Validate** — assert: sample rate == 16000, channels == 1, no NaN/Inf samples, no UTF-8 above U+00A1 in metadata strings
0 commit comments