xavierk/voculinux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

fix(audio): detect and use device-supported channel count (#275)

Browse Source 
* fix(audio): detect and use device-supported channel count

This fixes the "Invalid number of channels" (PyAudio error -9998) that
occurs with certain audio devices (e.g., HD-Audio Generic interfaces) that
require stereo (2 channel) input instead of mono (1 channel).

Changes:
- Add _get_supported_channels() helper function to detect supported channel
  configuration by testing mono first, then falling back to stereo
- Update test_audio_input() to detect channels before opening stream
- Update _record_audio() to detect channels and convert stereo to mono
- Update _attempt_audio_reconnection() to use detected channel count
- Add stereo-to-mono conversion in audio processing loop to ensure speech
  recognition engines receive mono audio as expected

Fixes issues introduced by voice commands feature where audio settings test
would fail with "no speech detected" because the audio stream couldn't be
opened due to channel mismatch.

Closes audio device compatibility issues

* fix(settings): delay callback restoration to prevent race condition in test

The recognition test was restoring text callbacks immediately after
stop_recognition(), but the recognition thread has a 1-second timeout.
If whisper.cpp transcription took longer than 1 second, the callbacks
would be restored before transcription completed, causing the text to
be sent to the wrong callback (the main app callback instead of the
test callback).

Changes:
- Add 500ms delay before restoring callbacks to allow transcription to complete
- Add new _restore_callbacks_and_check_result() helper method
- Increase result check delay from 200ms to 300ms for more reliable detection

This fixes the "No speech detected during test" issue where audio was
being recorded and transcribed, but the result wasn't being captured by
the test callback.

* debug: add diagnostic logging to trace recognition test flow

* fix(recognition): prevent race condition between audio enqueue and recognition thread

- Increase audio thread join timeout from 0.1s to 2.0s to ensure audio thread fully exits
- Remove redundant second audio thread join
- Add comprehensive debug logging to trace recognition thread lifecycle
- Add logging for final buffer enqueue to diagnose race conditions

The race condition occurred because:
1. should_record=False was set
2. Recognition thread could check queue.empty() before audio was enqueued
3. Recognition thread would exit before processing the final audio segment

This fix ensures the audio thread completes before we process the final buffer.

* fix(recognition): use poison pill pattern to prevent premature thread exit

The recognition thread was exiting immediately when should_record=False
and the queue was empty, even if audio was about to be enqueued.

Changes:
- Recognition loop now runs until it receives None signal (poison pill)
- When queue times out and should_record=False, waits 0.5s for final items
- When None is received, drains all remaining queue items before exiting
- Increased recognition thread join timeout from 1s to 5s for transcription

This ensures the recognition thread stays alive long enough to process
the final audio buffer that is enqueued after should_record is set to False.
This commit is contained in:
Jatin K Malik
2026-02-28 16:19:08 -08:00
committed by GitHub
parent 6fc1401ae0
commit 0b044cff6b
3 changed files with 202 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
MagicMock/mock.TemporaryDirectory().name/123922320000400/.config/vocalinux/config.json
+32
View File
@@ -0,0 +1,32 @@
{
"speech_recognition": {
"engine": "vosk",
"language": "auto",
"model_size": "medium",
"vosk_model_size": "medium",
"whisper_model_size": "tiny",
"whisper_cpp_model_size": "tiny",
"vad_sensitivity": 3,
"silence_timeout": 2.0,
"voice_commands_enabled": null
},
"audio": {
"device_index": null,
"device_name": null
},
"shortcuts": {
"toggle_recognition": "ctrl+ctrl"
},
"ui": {
"start_minimized": true,
"show_notifications": true
},
"general": {
"autostart": false,
"first_run": true
},
"advanced": {
"debug_logging": false,
"wayland_mode": false
}
}
src/vocalinux/speech_recognition/recognition_manager.py
+156 -12
View File
@@ -102,6 +102,59 @@ def get_audio_input_devices() -> list:
return devices
def _get_supported_channels(audio, device_index: int = None) -> int:
"""
Detect the supported number of channels for the audio device.
Some audio devices (particularly professional audio interfaces and certain
onboard audio chips) only support specific channel configurations. This
function tests mono (1) and stereo (2) to find a working configuration.
Args:
audio: PyAudio instance
device_index: The device index to test (None for default)
Returns:
int: Number of channels supported (1 or 2), defaults to 1
"""
import pyaudio
FORMAT = pyaudio.paInt16
CHUNK = 1024
RATE = 16000 # Use standard rate for channel testing
# Try mono first (preferred for speech recognition)
for channels in [1, 2]:
try:
stream_kwargs = {
"format": FORMAT,
"channels": channels,
"rate": RATE,
"input": True,
"frames_per_buffer": CHUNK,
}
if device_index is not None:
stream_kwargs["input_device_index"] = device_index
test_stream = audio.open(**stream_kwargs)
test_stream.close()
logger.debug(f"Device supports {channels} channel(s)")
return channels
except (IOError, OSError) as e:
error_str = str(e).lower()
if "invalid number of channels" in error_str or "-9998" in error_str:
logger.debug(f"Device does not support {channels} channel(s)")
continue
else:
# Different error, try next channel count anyway
logger.debug(f"Channel test failed for {channels} channel(s): {e}")
continue
# Default to mono if we couldn't determine
logger.warning("Could not determine supported channel count, defaulting to 1")
return 1
def _get_supported_sample_rate(audio, device_index: int, channels: int = 1) -> int:
"""
Get a supported sample rate for the audio device.
@@ -216,7 +269,6 @@ def test_audio_input(device_index: int = None, duration: float = 1.0) -> dict:
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
audio = pyaudio.PyAudio()
@@ -234,6 +286,10 @@ def test_audio_input(device_index: int = None, duration: float = 1.0) -> dict:
audio.terminate()
return result
# Detect supported channel count first (some devices require stereo)
CHANNELS = _get_supported_channels(audio, device_index)
logger.info(f"Using {CHANNELS} channel(s) for audio test")
# Detect supported sample rate for this device
RATE = _get_supported_sample_rate(audio, device_index, CHANNELS)
result["sample_rate"] = RATE
@@ -1473,9 +1529,11 @@ class SpeechRecognitionManager:
# Stop recording FIRST to prevent capturing the stop sound
self.should_record = False
# Wait briefly for audio thread to stop recording
# Wait for audio thread to finish recording and enqueue any pending audio
# This is critical to prevent race condition where recognition thread exits
# before the final audio segment is enqueued
if self.audio_thread and self.audio_thread.is_alive():
self.audio_thread.join(timeout=0.1)
self.audio_thread.join(timeout=2.0)
# Discard the last ~1 second of audio to avoid transcribing the stop sound
# Audio is recorded in 1024-sample chunks at 16000 Hz = ~64ms per chunk
@@ -1493,19 +1551,20 @@ class SpeechRecognitionManager:
self.audio_buffer = []
if self.audio_buffer:
logger.info(f"DEBUG: Enqueuing final buffer with {len(self.audio_buffer)} chunks")
self._enqueue_audio_segment(self.audio_buffer)
self.audio_buffer = []
# Now play the stop sound (after recording has stopped)
play_stop_sound()
# Wait for threads to finish
if self.audio_thread and self.audio_thread.is_alive():
self.audio_thread.join(timeout=1.0)
# Wake up recognition thread so it can drain queued segments and stop
self._signal_recognition_stop()
if self.recognition_thread and self.recognition_thread.is_alive():
self.recognition_thread.join(timeout=5.0) # Increased timeout for transcription
self._signal_recognition_stop()
if self.recognition_thread and self.recognition_thread.is_alive():
self.recognition_thread.join(timeout=1.0)
@@ -1530,7 +1589,6 @@ class SpeechRecognitionManager:
# PyAudio configuration
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
# Initialize PyAudio with reconnection support
self._pyaudio_instance = pyaudio.PyAudio()
@@ -1548,6 +1606,10 @@ class SpeechRecognitionManager:
except (IOError, OSError):
continue
# Detect supported channel count first (some devices require stereo)
CHANNELS = _get_supported_channels(audio, self.audio_device_index)
logger.info(f"Using {CHANNELS} channel(s) for recording")
# Detect supported sample rate for the selected device
RATE = _get_supported_sample_rate(audio, self.audio_device_index, CHANNELS)
self._capture_sample_rate = RATE
@@ -1620,6 +1682,15 @@ class SpeechRecognitionManager:
data = stream.read(CHUNK, exception_on_overflow=False)
# Convert stereo to mono if necessary
# Speech recognition engines expect mono (1 channel) audio
if CHANNELS == 2:
audio_array = np.frombuffer(data, dtype=np.int16)
# Reshape to (n_samples, 2) and average channels
stereo_samples = audio_array.reshape(-1, 2)
mono_samples = stereo_samples.mean(axis=1).astype(np.int16)
data = mono_samples.tobytes()
# Resample to 16kHz if capturing at non-16kHz for Vosk/Whisper compatibility
if self._capture_sample_rate != 16000:
audio_array = np.frombuffer(data, dtype=np.int16)
@@ -1787,6 +1858,9 @@ class SpeechRecognitionManager:
return
# Process text - either with voice commands or pass through directly
logger.info(
f"DEBUG: _process_audio_buffer got text='{text[:50] if text else '(empty)'}...'"
)
if text:
if self._voice_commands_enabled:
# Process with voice commands (original behavior)
@@ -1797,8 +1871,14 @@ class SpeechRecognitionManager:
actions = []
# Call text callbacks with processed text
logger.info(
f"DEBUG: processed_text='{processed_text[:50] if processed_text else '(empty)'}...', callbacks={len(self.text_callbacks)}"
)
if processed_text:
for callback in self.text_callbacks:
logger.info(
f"DEBUG: invoking text callback: {callback.__name__ if hasattr(callback, '__name__') else callback}"
)
callback(processed_text)
# Call action callbacks for each action
@@ -1808,28 +1888,89 @@ class SpeechRecognitionManager:
def _perform_recognition(self):
"""Perform speech recognition in real-time."""
while self.should_record or not self._segment_queue.empty():
logger.info("DEBUG: _perform_recognition thread started")
while True:
logger.debug(
f"DEBUG: Recognition loop - should_record={self.should_record}, queue_empty={self._segment_queue.empty()}"
)
try:
segment = self._segment_queue.get(timeout=0.1)
except queue.Empty:
continue
# Only exit if we're not recording AND queue is empty
if not self.should_record and self._segment_queue.empty():
logger.info(
"DEBUG: Recognition loop - not recording and queue empty, checking for final items..."
)
# Give a brief moment for any final items to be enqueued
try:
segment = self._segment_queue.get(timeout=0.5)
except queue.Empty:
logger.info("DEBUG: Recognition loop - no more items, exiting")
break
else:
logger.debug("DEBUG: Recognition loop - queue timeout, continuing")
continue
if segment is None:
continue
logger.info(
"DEBUG: Recognition loop - got None signal, draining remaining items..."
)
# Drain any remaining items before exiting
while not self._segment_queue.empty():
try:
remaining = self._segment_queue.get_nowait()
if remaining is not None:
logger.info(
f"DEBUG: Recognition loop - processing remaining segment with {len(remaining)} chunks"
)
self._update_state(RecognitionState.PROCESSING)
self._process_audio_buffer(remaining)
except queue.Empty:
break
logger.info("DEBUG: Recognition loop - exiting after None signal")
break
logger.info(f"DEBUG: Recognition loop - processing segment with {len(segment)} chunks")
self._update_state(RecognitionState.PROCESSING)
self._process_audio_buffer(segment)
if self.should_record:
self._update_state(RecognitionState.LISTENING)
logger.info("DEBUG: _perform_recognition thread exiting")
"""Perform speech recognition in real-time."""
logger.info("DEBUG: _perform_recognition thread started")
while self.should_record or not self._segment_queue.empty():
logger.debug(
f"DEBUG: Recognition loop - should_record={self.should_record}, queue_empty={self._segment_queue.empty()}"
)
try:
segment = self._segment_queue.get(timeout=0.1)
except queue.Empty:
logger.debug("DEBUG: Recognition loop - queue timeout, continuing")
continue
if segment is None:
logger.info("DEBUG: Recognition loop - got None signal, continuing")
continue
logger.info(f"DEBUG: Recognition loop - processing segment with {len(segment)} chunks")
self._update_state(RecognitionState.PROCESSING)
self._process_audio_buffer(segment)
if self.should_record:
self._update_state(RecognitionState.LISTENING)
logger.info("DEBUG: _perform_recognition thread exiting")
def _enqueue_audio_segment(self, audio_buffer: List[bytes]):
"""Queue an audio segment for asynchronous transcription."""
segment = audio_buffer.copy()
if not segment:
logger.warning("DEBUG: _enqueue_audio_segment called with empty buffer")
return
logger.info(f"DEBUG: _enqueue_audio_segment called with {len(segment)} chunks")
try:
self._segment_queue.put_nowait(segment)
logger.info("DEBUG: Enqueued segment successfully")
except queue.Full:
logger.warning("Transcription queue is full, dropping oldest pending segment")
try:
@@ -1984,7 +2125,10 @@ class SpeechRecognitionManager:
# Stream configuration
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
# Detect supported channel count first (some devices require stereo)
CHANNELS = _get_supported_channels(audio_instance, self.audio_device_index)
logger.debug(f"Reconnecting with {CHANNELS} channel(s)")
# Detect supported sample rate for the device
RATE = _get_supported_sample_rate(audio_instance, self.audio_device_index, CHANNELS)
src/vocalinux/ui/settings_dialog.py
+14 -4
View File
@@ -1852,19 +1852,29 @@ class SettingsDialog(Gtk.Dialog):
self.speech_engine.stop_recognition()
if hasattr(self, "_saved_text_callbacks"):
self.speech_engine.set_text_callbacks(self._saved_text_callbacks)
del self._saved_text_callbacks
# Wait a bit for any pending transcription to complete before restoring callbacks
# This ensures the test callback receives the transcription result
GLib.timeout_add(500, self._restore_callbacks_and_check_result)
self._test_active = False
self.test_button.set_sensitive(True)
self.test_button.set_label("Start Test (3 seconds)")
self.update_recognition_progress("Idle")
GLib.timeout_add(200, self._check_test_result)
return False
def _restore_callbacks_and_check_result(self):
"""Restore callbacks and check test result after delay."""
# Restore original text callbacks
if hasattr(self, "_saved_text_callbacks"):
self.speech_engine.set_text_callbacks(self._saved_text_callbacks)
del self._saved_text_callbacks
# Check result after giving time for final callbacks to complete
GLib.timeout_add(300, self._check_test_result)
return False
def _check_test_result(self):
"""Check if any text was captured after all callbacks have run."""
final_text = self.test_buffer.get_text(