Skip to content

Commit 51918d0

Browse files
shaypal5shaypal5
authored
docs(readme): improve project presentation (#115)
1 parent fc65a83 commit 51918d0

2 files changed

Lines changed: 104 additions & 67 deletions

File tree

‎README.md‎

Lines changed: 104 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -1,112 +1,149 @@
11
# SynthBanshee
22

3-
**SynthBanshee** is a config-driven pipeline for generating large-scale synthetic Hebrew audio
4-
datasets. It is part of the [AVDP](https://datahack.org.il) (Audio Violence Dataset Project)
5-
initiative run by DataHack, which produces training data for two AI safety products:
3+
[![CI](https://github.com/DataHackIL/SynthBanshee/actions/workflows/ci.yml/badge.svg)](https://github.com/DataHackIL/SynthBanshee/actions/workflows/ci.yml)
4+
[![license: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
5+
![python: 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)
6+
![audio: 16 kHz mono PCM](https://img.shields.io/badge/audio-16kHz%20mono%20PCM-0f766e.svg)
67

7-
- **She-Proves** — a smartphone app that passively monitors for domestic violence incidents and
8-
preserves audio evidence for legal use
9-
- **Elephant in the Room (הפיל שבחדר)** — a Raspberry Pi–class device in social-work offices that
10-
alerts security when a social worker is being physically threatened
8+
Created by [Shay Palachy Affek](http://www.shaypalachy.com/).
119

12-
The goal of SynthBanshee is to supply AI teams with a wide, deliberate distribution of voices,
13-
acoustic conditions, and violence types while real-data (actor recording) pipelines are built in
14-
parallel. Synthetic-to-real gap is expected and documented.
10+
**SynthBanshee** is a config-driven pipeline for generating synthetic Hebrew audio datasets for
11+
DataHack's [AVDP](https://datahack.org.il) (Audio Violence Dataset Project). It turns structured
12+
scene YAML into Hebrew dialogue, rendered speech, acoustic variants, taxonomy labels, QA reports,
13+
and dataset packages that AI teams can use for early model development.
1514

16-
---
15+
The repository supports two sensitive AI-safety product contexts: **She-Proves**, focused on
16+
domestic-violence incident detection research, and **Elephant in the Room** (`הפיל שבחדר`), focused
17+
on threat detection in social-work offices. Synthetic-to-real gap is expected and documented; real
18+
actor and field-data pipelines remain separate.
1719

18-
## How it works
20+
![SynthBanshee audio fixture waveform and spectrogram](docs/assets/synthbanshee-click-fixture-waveform.png)
1921

20-
Every clip is produced by four sequential pipeline stages:
22+
## At a Glance
2123

24+
| Field | Value |
25+
|---|---|
26+
| Role | Synthetic Hebrew audio dataset generation for AVDP |
27+
| Main inputs | `SceneConfig` YAML, speaker profiles, run configs, script templates |
28+
| Main outputs | `.wav`, `.txt`, `.json`, `.jsonl`, manifests, split files, QA reports |
29+
| Language | Hebrew (`he`, rendered with `he-IL` TTS voices) |
30+
| Audio contract | 16 kHz, mono, 16-bit PCM, `-1.0 dBFS` peak ceiling, silence padding |
31+
| Current status | Phase 0 and Phase 1 complete; Tier A baseline generation delivered |
32+
| Packaged examples | 3 example scene configs, 8 run configs, 6 packaged speakers, 4 tracked WAV fixtures |
33+
34+
## Pipeline
35+
36+
```mermaid
37+
flowchart LR
38+
scene["SceneConfig YAML<br/>project, tier, speakers, intensity"] --> script["Script generator<br/>Jinja2 + LLM"]
39+
script --> tts["TTS renderer<br/>Azure he-IL SSML"]
40+
tts --> acoustic["Acoustic augmenter<br/>room, device, background events"]
41+
acoustic --> labels["Label generator<br/>taxonomy + event timings"]
42+
labels --> qa["QA and validation<br/>format, loudness, labels, splits"]
43+
qa --> package["Dataset package<br/>audio + transcript + metadata"]
2244
```
23-
SceneConfig (YAML)
24-
→ [1] Script Generator LLM fills a Jinja2 template → dialogue JSON
25-
→ [2] TTS Renderer Azure he-IL SSML → per-speaker WAV segments
26-
→ [3] Acoustic Augmenter Room IR + device profile + noise → scene WAV
27-
→ [4] Label Generator Script structure + augmentation log → AVDP-schema JSONL
28-
```
29-
30-
Output per clip: `{clip_id}.wav` + `{clip_id}.txt` (Hebrew transcript) + `{clip_id}.json` (metadata).
31-
32-
Audio spec: **16 kHz · mono · 16-bit PCM · −1.0 dBFS peak · ≥ 0.5 s silence pad**.
33-
34-
---
35-
36-
## Dataset tiers
3745

38-
| Tier | Description | Target (per project) |
39-
|---|---|---|
40-
| A | Clean TTS, no acoustic augmentation | 1,000 clips |
41-
| B | Room simulation + device profile + background noise | 2,000 clips |
42-
| C | Hard negatives / confusors (arguments that de-escalate, ambient sounds) | 1,000 clips |
46+
Each generated clip is reproducible from config and seed. Tier A renders clean TTS, Tier B adds room
47+
and device simulation, and Tier C adds hard negatives and confusors for robustness testing.
4348

44-
Two projects — **She-Proves** (3–6 min clips, apartment rooms) and **Elephant in the Room**
45-
(1–4 min clips, clinic/welfare offices) — each receive the full tier stack.
49+
## Output Surface
4650

47-
---
51+
| Artifact | Purpose |
52+
|---|---|
53+
| `{clip_id}.wav` | Rendered 16 kHz mono PCM scene audio |
54+
| `{clip_id}.txt` | Hebrew transcript for the rendered scene |
55+
| `{clip_id}.json` | Clip metadata, taxonomy-derived `has_violence`, speakers, duration, and provenance |
56+
| `{clip_id}.jsonl` | Time-aligned strong labels generated from script structure and augmentation logs |
57+
| Manifest and split files | Dataset inventory, train/validation/test assignment, and batch-level provenance |
58+
| QA reports | Format, loudness, duration, taxonomy, split hygiene, and generated-asset checks |
4859

49-
## Label taxonomy
60+
The preview image above is generated from a tracked 16 kHz test fixture, not from a released dataset
61+
sample. It shows the kind of waveform and spectrogram view used for quick audio QA.
5062

51-
Labels use a three-level hierarchy. `has_violence` (in clip metadata and manifests) is a derived convenience field computed from the taxonomy — not an independent label. The taxonomy is the ground truth; `has_violence` exists for fast filtering and baseline modelling.
63+
## Dataset Tiers
5264

53-
1. **Violence typology** (scene): `SV` · `IT` · `NEG` · `NEU`
54-
2. **Tier 1 category** (event): `PHYS` · `VERB` · `DIST` · `ACOU` · `EMOT` · `NONE`
55-
3. **Tier 2 subtype** (event): e.g. `PHYS_HARD` · `VERB_THREAT` · `DIST_SCREAM`
65+
| Tier | Description | Target per project |
66+
|---|---|---|
67+
| A | Clean TTS with no acoustic augmentation | 1,000 clips |
68+
| B | Room simulation, device profile, and background noise | 2,000 clips |
69+
| C | Hard negatives and confusors, including de-escalating arguments and ambient sounds | 1,000 clips |
5670

57-
Full taxonomy: `synthbanshee/data/taxonomy.yaml`.
71+
Both product contexts receive the full tier stack:
5872

59-
---
73+
- **She-Proves**: 3-6 minute apartment scenes with pre-incident, incident, and aftermath windows.
74+
- **Elephant in the Room**: 1-4 minute clinic or welfare-office scenes for local threat detection.
6075

61-
## Current status
76+
## Label Taxonomy
6277

63-
Phase 0 (pipeline foundation) and Phase 1 (full Tier A pipeline) are complete — AI teams can now use the delivered Tier A dataset to start baseline model development.
78+
Labels use a three-level hierarchy. `has_violence` in clip metadata and manifests is a derived
79+
convenience field computed from the taxonomy, not an independent label.
6480

65-
| Phase | Deliverable | Status |
66-
|---|---|---|
67-
| 0 | Single spec-compliant clip end-to-end | **Done** |
68-
| 1 | Script templates + multi-speaker TTS + LLM generator + batch generation + QA suite | **Done** |
69-
| 2 | 1,000–1,500 Tier B clips/project | Planned |
70-
| 3 | 4,000 clips/project, all tiers | Planned |
81+
| Level | Examples |
82+
|---|---|
83+
| Violence typology | `SV`, `IT`, `NEG`, `NEU` |
84+
| Tier 1 event category | `PHYS`, `VERB`, `DIST`, `ACOU`, `EMOT`, `NONE` |
85+
| Tier 2 event subtype | `PHYS_HARD`, `VERB_THREAT`, `DIST_SCREAM`, `ACOU_BREAK` |
7186

72-
---
87+
Full taxonomy: [`synthbanshee/data/taxonomy.yaml`](synthbanshee/data/taxonomy.yaml).
7388

74-
## Quick start
89+
## Quick Start
7590

7691
```bash
77-
# Install (requires Python ≥ 3.11 and uv)
92+
# Install locally with Python 3.11+ and uv.
7893
uv pip install -e .
7994

80-
# Generate a single clip from a scene config
81-
synthbanshee generate --config configs/scenes/test_scene_001.yaml
95+
# Generate a single clip from a scene config.
96+
synthbanshee generate --config configs/examples/scene_she_proves_IT_example.yaml
8297

83-
# Generate a full Tier A batch from a run config
98+
# Generate a Tier A She-Proves batch from a run config.
8499
synthbanshee generate-batch \
85100
--run-config configs/run_configs/tier_a_500_she_proves.yaml \
86101
--output-dir data/he
87102

88-
# Run the automated QA suite on a dataset directory
103+
# Run automated QA on a dataset directory.
89104
synthbanshee qa-report data/he
90105

91-
# Validate an existing clip
106+
# Validate an existing clip.
92107
synthbanshee validate data/he/clip_001.wav
93108
```
94109

95-
API credentials required (set in environment or `.env`):
110+
Live generation requires provider credentials in your environment or local `.env` file:
96111

97-
```
112+
```bash
98113
AZURE_TTS_KEY=...
99114
AZURE_TTS_REGION=...
100-
ANTHROPIC_API_KEY=... # for script generation
115+
ANTHROPIC_API_KEY=...
101116
```
102117

103-
---
118+
Do not commit provider keys, generated private datasets, or real-data artifacts.
119+
120+
## Safety and Data Boundary
104121

105-
## Docs
122+
SynthBanshee is a synthetic-data generator for research and model-development workflows. Generated
123+
clips are not evidence, user recordings, or a substitute for real validation data. They are designed
124+
to widen scenario coverage while actor-recording and real-data pipelines are handled separately.
125+
126+
Sensitive context should remain explicit in configs and docs, but public-facing materials should not
127+
overstate model readiness, legal usefulness, or field performance.
128+
129+
## Current Status
130+
131+
| Phase | Deliverable | Status |
132+
|---|---|---|
133+
| 0 | Single spec-compliant clip end to end | Done |
134+
| 1 | Script templates, multi-speaker TTS, LLM generation, batch generation, QA suite | Done |
135+
| 2 | 1,000-1,500 Tier B clips per project | Planned |
136+
| 3 | 4,000 clips per project across all tiers | Planned |
137+
138+
## Documentation
106139

107140
| Document | Contents |
108141
|---|---|
109-
| `docs/spec.md` | Audio format, file naming, label schema, IAA protocol |
110-
| `docs/implementation_plan.md` | Phased milestones, module map, API cost estimates |
111-
| `docs/design_approaches.md` | Design decisions and rationale |
112-
| `CLAUDE.md` | Claude Code context guide (pipeline constraints, conventions) |
142+
| [`docs/spec.md`](docs/spec.md) | Audio format, file naming, label schema, and IAA protocol |
143+
| [`docs/implementation_plan.md`](docs/implementation_plan.md) | Phased milestones, module map, and API cost estimates |
144+
| [`docs/design_approaches.md`](docs/design_approaches.md) | Design decisions and rationale |
145+
| [`CLAUDE.md`](CLAUDE.md) | Agent context guide for pipeline constraints and conventions |
146+
147+
## Credits
148+
149+
Created by [Shay Palachy Affek ](http://www.shaypalachy.com/) [[GitHub](https://github.com/shaypal5)]

‎docs/assets/synthbanshee-click-fixture-waveform.png‎

291 KB
Loading

0 commit comments

Comments
 (0)