Bleep

Find where sensitive speech is, without transcribing what was said.

A small frame-tagging model that reads audio and outputs (category, start, end) — a spoken digit string here, a letter-by-letter spelling there — and nothing else. It never produces a transcript, and it has no mechanism that could: the network takes log-mel features and emits nine per-frame channels.

🤗 Model 📊 Dataset

This is a published negative result

At the ASR baseline's own measured false-alarm rate, on a speaker-disjoint test split, Bleep loses badly. That is the finding, and it belongs at the top rather than in a footnote.

Why, specifically

The benchmark was built to separate the reasons, not just report the number.

1 · The failure mode is real, but the corpus put it in the wrong place

…

2 · The text rules were never the bottleneck

An oracle arm — a perfect transcript fed to the same text-PII rules — scores 0.96–1.00 in every condition. So every baseline miss is an ASR word error rather than a detector gap. That one arm turns “the baseline failed” into “the baseline failed here, for this reason”.

3 · The model is badly undertrained

1500 steps, batch 4, ~1.8 epochs over 9.15 hours — 70 minutes on one consumer GPU. Validation loss was still falling monotonically at the final step (0.218 → 0.183): a stopped run, not a converged one.

One directional hint

…

What did hold up

…

Timeline viewer

Real utterances from the test split. Ground truth against what each detector actually found, at the operating points the benchmark reports.

No audio is shipped. These recordings were rendered with the macOS system voices, which are licensed for use and not for republication as a corpus. What you see is a downsampled amplitude envelope — a few hundred numbers per utterance, enough to draw a waveform, not enough to reconstruct speech.

ground truth ASR + text PII Bleep union

Recall by degradation condition

Recall by degradation condition for each arm, training conditions
Training conditions. The subtitle is generated from the data, so the chart cannot assert a conclusion the numbers do not support — it prints the opposite of the project's hypothesis.
Recall by held-out degradation condition
Held-out conditions — the more informative set. The ASR arm's collapse under heavy overlap appears only here.

Recall by category

Recall by category of sensitive speech
★ marks the failure modes reported for transcription-first pipelines.

What this is not

Not a redaction guarantee. It misses most things. An empty result means “nothing was flagged”, never “nothing is there”.

Not a replacement for your ASR-based detector. It is designed for union mode, and the union is proven never to drop a region the existing detector found — zero violations across both splits.

Not trained on real audio. Every recording is synthetic. No real call, no clinical recording, nothing derived from either, at any stage. That is also a limitation: TTS speech is far easier to transcribe than conversation, which is part of why the baseline looks so strong here.

Apache-2.0. Negative results are cheap to hide and expensive to rediscover.