DISENT-KWS — Speech Disentanglement for Robust Custom Word Detection

• Demo Video Link — YouTube Demo Video
• Setup & Result Reproducibility Video Link — YouTube Setup Video

Quick Start

# Install dependencies
uv sync --all-extras

# Run tests (verify 60+ tests pass)
make test

# Option A: Live record 5 utterances from mic, then enroll
python src/demo.py record --model model_final.pt --out enrollment.pt

# Option B: Provide pre-recorded WAV files
python src/demo.py enroll \
    --recordings ./my_recordings/*.wav \
    --model model_final.pt \
    --out enrollment.pt

# Real-time detection
python src/demo.py detect --enrollment enrollment.pt --auto-threshold --vad-threshold 0.02

Project Artifacts

Technical Documentation

All technical documentation is organized in the docs/ directory:

Source Code

The complete source code is organized under src/:

Models Used

Models Published

The model is hosted on Hugging Face’s free tier — no payment required to download or use.

Datasets Used

Datasets Published

No custom datasets were published. All datasets listed above are publicly available.

Architecture Overview

The system uses a dual-head disentangled architecture built on a shared BC-ResNet-2 encoder (total 1.806M parameters):

1. Shared Encoder (BC-ResNet-2) — Broadcasted residual network, 33.8K params
2. Temporal Block (Mamba SSM / Dilated Conv1D) — O(T) temporal context modeling, 10.3K params
3. Phonetic Head (Causal Conformer) — Extracts keyword-discriminative embeddings zₚₕₙ ∈ ℝ¹⁹², 1,673K params
4. Speaker Head (ECAPA-TDNN Lite) — Extracts speaker-discriminative embeddings zₛₚₖ ∈ ℝ¹⁹², 88.8K params
5. Disentanglement Module (GRL + CLUB) — Adversarial gradient reversal + mutual information minimization forces zₚₕₙ ⟂ zₛₚₖ
6. Dual-Gate Scorer — Weighted cosine similarity (w_kw=0.30, w_spk=0.65) with EMA smoothing and DET-calibrated threshold (τ=0.2222)

Three-Layer Defense Against False Accepts

Final Performance Benchmarks

Evaluated on Google Speech Commands v2 test set (11,005 samples, 35 classes) and VoxCeleb1 (1,251 speakers). Scorer weights calibrated via joint verification grid search over 10×10 weight combinations + DET-driven threshold selection.

Detection Error Trade-off (DET) Curve

SNR Robustness Across -5 dB to 30 dB

Ablation Study Results

To isolate each component’s contribution, we systematically disabled modules and re-evaluated:

Key Insight: The temporal block is the single most critical component — removing it degrades keyword EER by 2.4× and speaker EER by 1.5×.

Attribution

This project builds upon and transfers weights from the open-source SpeechBrain repository (ECAPA-TDNN for speaker verification).

Citation

If you use DISENT-KWS in your research, please cite it:

@misc{bt2026disentkws,
  title={DISENT-KWS: Speech Disentanglement for Robust Custom Word Detection},
  author={Banerjee, Sohini and Tripathi, Swarnim},
  year={2026},
  howpublished={\url{https://github.com/tripathiji1312/DISENT_KWS}},
  note={BC-ResNet-2 backbone with Causal Conformer phonetic head, ECAPA-TDNN Lite speaker head, and GRL+CLUB disentanglement. 1.806M parameters.}
}

Novel Contributions Developed for This Solution