Add VibeVoice-ASR backend

Third ASR backend option alongside faster-whisper and qwen3. Builds
microsoft/VibeVoice from source (pinned to a specific commit, since it's
custom modeling code not in transformers' Auto* registry) into its own venv,
inheriting the base image's torch/CUDA. Comes with built-in speaker
diarization (VibeVoiceASRProcessor.post_process_transcription returns
per-segment speaker ids directly, no separate pyannote pass needed).

Verified end-to-end against a real recording: 200 OK, correct Korean
transcription, speaker labels populated.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
du5t
2026-07-23 16:54:34 +09:00
parent a32b0acb09
commit ce064d6894
7 changed files with 202 additions and 2 deletions

View File

@@ -18,6 +18,7 @@ from asr.config import (
DEFAULT_MODEL,
FASTER_WHISPER_URL,
QWEN3_URL,
VIBEVOICE_URL,
RESULT_DIR,
UPLOAD_DIR,
ensure_runtime_dirs,
@@ -29,6 +30,7 @@ ws_router = APIRouter()
BACKENDS = {
"faster-whisper": FASTER_WHISPER_URL,
"qwen3": QWEN3_URL,
"vibevoice": VIBEVOICE_URL,
}
REALTIME_SAMPLE_RATE = 16000
@@ -50,6 +52,9 @@ def config() -> Dict[str, Any]:
"qwen3": {
"models": ["Qwen/Qwen3-ASR-0.6B-hf", "Qwen/Qwen3-ASR-1.7B-hf"],
},
"vibevoice": {
"models": ["microsoft/VibeVoice-ASR"],
},
},
}
@@ -168,6 +173,10 @@ async def transcribe(
"language": effective_language,
"task": task,
}
elif backend == "vibevoice":
data = {
"model": model,
}
else:
# faster-whisper
data = {