Refs

Model Card: DISENT-KWS Model Description DISENT-KWS is a speech disentanglement model for robust custom word detection. It uses a shared BC-ResNet-2 encoder with dual heads — a Causal Conformer (phonetic) and ECAPA-TDNN Lite (speaker) — to produce orthogonal latent embeddings. A Dual-Gate Scorer combines keyword and speaker similarity with calibrated weights for joint verification. Total Parameters: 1.806M ONNX Size: 0.60 MB (INT8 quantized) Input: 80-band log Mel filterbank energies, 200 frames (2 s @ 16 kHz) Output: z_phn ∈ ℝ¹⁹² (phonetic embedding), z_spk ∈ ℝ¹⁹² (speaker embedding) Intended Use Custom keyword spotting with speaker verification for access control, voice assistants, or wake-word systems Real-time streaming inference on CPU (26.4 ms per 2 s window) Few-shot enrollment (5+ utterances) for new users and custom words Architecture Component Type Params Shared Encoder BC-ResNet-2 33.8K Temporal Block Mamba SSM / Dilated Conv1D 10.3K Phonetic Head Causal Conformer (4 heads, kernel 15) 1,673K Speaker Head ECAPA-TDNN Lite (SE ratio 4, scale 4) 88.8K Scorer Dual-Gate (w_kw=0.30, w_spk=0.65, EMA α=0.7) — Training Data Dataset Usage Samples Google Speech Commands v2 Keyword pre-training (35 classes) 105K VoxCeleb1 Speaker verification (1,251 speakers) 153K LibriPhrase Hard-negative triplet pairs 3K triplets MUSAN Noise augmentation (babble, music, environmental) 109 hrs Training Phases Phase 1: AAM-Softmax pre-training on keyword (GSC) and speaker (VoxCeleb) separately Phase 2: Joint training with GRL adversarial reversal + CLUB MI minimization + triplet rejection loss Phase 3a: GE2E speaker head refinement Phase 3b: Hard-negative GE2E with LibriPhrase confusers Performance Metric Value Keyword EER 4.69% Speaker EER 17.86% Joint EER 23.47% Joint AUC 0.8425 CPU Latency 26.43 ms (p95: 28.29 ms) Real-Time Factor (xRT) 0.0132 Optimal Threshold (τ) 0.2222 Hardware & Runtime Training: NVIDIA GPU with 16 GB+ VRAM (tested on Tesla T4) Inference: CPU-only via ONNX Runtime; no GPU required Memory: ~200 MB RAM at runtime Known Limitations Requires clean enrollment recordings (SNR ≥ 10 dB recommended) Speaker EER (17.86%) is higher than keyword EER — joint verification mitigates this Mamba SSM fallback to Dilated Conv1D on platforms without CUDA (no performance loss) License MIT ...

DISENT-KWS — Speech Disentanglement for Robust Custom Word Detection Demo Video Link — YouTube Demo Video Setup & Result Reproducibility Video Link — YouTube Setup Video Quick Start # Install dependencies uv sync --all-extras # Run tests (verify 60+ tests pass) make test # Option A: Live record 5 utterances from mic, then enroll python src/demo.py record --model model_final.pt --out enrollment.pt # Option B: Provide pre-recorded WAV files python src/demo.py enroll \ --recordings ./my_recordings/*.wav \ --model model_final.pt \ --out enrollment.pt # Real-time detection python src/demo.py detect --enrollment enrollment.pt --auto-threshold --vad-threshold 0.02 Project Artifacts Technical Documentation All technical documentation is organized in the docs/ directory: ...

Speech Disentanglement for Robust Custom Word Detection: Solution Architecture and Theoretical Foundations This document serves as a comprehensive technical treatise on the mathematical, architectural, and optimization foundations of the DISENT-KWS system. 1. Problem Formulation and Theoretical Constraints Let $x(t)$ be a monaural, continuous-time acoustic signal recorded in a noisy reverberant environment. The signal is modeled as: $$x(t) = \left( s_T(t) * h_T(t) \right) + \sum_{i=1}^{I} \left( s_i(t) * h_i(t) \right) + n(t)$$Where: ...

Testing Guide for DISENT-KWS Overview This project uses pytest for unit testing, GitHub Actions for CI-CD, and pre-commit hooks for local code quality enforcement. All dependencies are managed via uv (fast Python package installer). Make sure uv is installed before running tests. Quick Start Run All Tests Locally # Option 1: Using make make test # Option 2: Using uv directly uv run pytest tests/ -v # Option 3: Using the Python runner python scripts/test.py Run Tests with Coverage # Using make make test-cov # Using uv directly uv run pytest tests/ -v --cov=data --cov=models --cov=training --cov-report=html # Using the Python runner python scripts/test.py --coverage Run Specific Tests # Run only dataloader tests make test-dataloaders # or uv run pytest tests/test_dataloaders.py -v # Run specific test by name uv run pytest tests/test_dataloaders.py::TestLFBETransform::test_transform_1d_input -v # Run tests matching a pattern uv run pytest tests/ -k "lfbe" -v Test Structure t ├ ├ └ s ├ └ e ─ ─ ─ c ─ ─ s ─ ─ ─ r ─ ─ t i s t t p r t / _ e e t u e i s s s n s n t t / _ t i _ _ t . t d m e p _ a o s y _ t d t . a e s p l l . y o s s a . h d p e y r s . p y # # # # D M S P a o h y t d e t a e l h l l o l n o t s a e c r d s r u e t i n r s p n t e t r e f s o w t r i s t l h o c d a e l t a t i e l s e t d i n o g u t p u t Test Files tests/test_dataloaders.py Tests for data loading and feature extraction: ...