du5t 9472387d1b Auto-unload idle models and evict on model switch across all workers
Every ASR/TTS worker (faster-whisper, qwen3, vibevoice, xtts) kept every
model it ever loaded resident in GPU memory forever, and switching to a
different model (e.g. a different whisper size) just added another one
alongside it rather than freeing the old one. Combined with the shared
24GB GPU, this made memory pressure only ever go up.

Now: only one model stays resident per worker at a time (loading a
different model_id evicts the previous one first), and the whole cache
(plus, for faster-whisper, the diarization pipeline) is dropped after 2
minutes of no requests. Verified end-to-end: the idle timer actually fires
and reclaims memory, and switching qwen3 models evicts the old one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-23 23:29:13 +09:00
2026-06-18 15:25:56 +09:00
2026-07-23 16:54:34 +09:00
Description
ASR v2 — faster-whisper + Qwen3-ASR, multi-venv, GPU
217 KiB
Languages
Python 49.1%
JavaScript 26.2%
HTML 14.7%
CSS 5.9%
Shell 2.2%
Other 1.9%