xavierk/voculinux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

feat: add remote API speech recognition engine with installation and configuration support. (#335)

Browse Source 
* feat: add remote API speech recognition engine with installation and configuration support.

* fix: reformatted by black

* fix: unit test in test_main_initializes_components

* fix typo

* Apply suggestion from @jatinkrmalik

Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>

* feat: Validate remote API URL format, enhance connection testing, update default remote API endpoint.

* fix typo

* feat: add remote API recognition manager test suite

* fix: add Python 3.14 support and lxml>=6.1.0 (fixes #404)

- Add Python 3.14 to classifiers
- Add lxml>=6.1.0 to dependencies to fix install on Python 3.14 (CachyOS, Arch)
- Update black target-version to include py314

Fixes #404
Closes #405

* fix(install): validate pyproject.toml/setup.py content before entering local repo mode (fixes #396)

Fix false local repo detection when users run `uv init` before installation. Changed detection from naive file existence check to content validation:
- For pyproject.toml: check for `name = "vocalinux"`
- For setup.py: check for "vocalinux" string
- Falls through to remote clone mode if neither matches

Fixes #396

* fix(recognition): preserve final speech on stop (fixes #401)

Replace hard-coded stop tail trim with configurable stop_sound_guard_ms
(default 200ms) so quick key release no longer clips the final spoken
word in push-to-talk mode.

Co-authored-by: Fakhareddine <fe.sabbari@claranet.com>

* fix(ibus): handle engine instance destruction on layout switch (fixes #388, closes #389)

Override do_destroy() to clear _active_instance when IBus destroys the
engine on keyboard layout switch (Super+Space). Add retry logic in
inject_text() — on NO_ENGINE, re-activates via switch_engine() and
retries once to recover cleanly.

Co-authored-by: Sbenazar <127324689+Sbenazar@users.noreply.github.com>

* chore: ignore opentree worktree dir and config (#412)

.opentree/ contains git worktrees created by the opentree CLI tool
(https://github.com/axelgar/opentree). These are local developer
artifacts and should not be tracked.

opentree.toml is the per-repo opentree config - also local/personal,
not something contributors need to share.

* test(ibus): fix do_destroy CI failure (issubclass TypeError without IBus) (#410)

* test(ibus): fix do_destroy super() test failing in CI without IBus installed

Patch builtins.super instead of relying on mock_ibus.Engine.do_destroy,
which fails with issubclass() TypeError when VocalinuxEngine is built
with object as its base (IBus unavailable in CI).

* test(ibus): fix MagicMock.__new__ TypeError on Python < 3.13

VocalinuxEngine inherits from MagicMock at import time (IBus is mocked
via sys.modules at test-file level). Calling VocalinuxEngine.__new__(VocalinuxEngine)
delegates to MagicMock.__new__ which runs issubclass(cls, AsyncMockMixin).
On Python < 3.13 that raises TypeError when the module-cached class is a
MagicMock spec object rather than a real type.

Fix: introduce _make_engine() helper that allocates via a thin concrete
subclass whose __new__ calls object.__new__ directly, bypassing the
MagicMock allocation path entirely. Verified across all three
TestVocalinuxEngineDestroy test methods.

* test(ibus): use unbound do_destroy call to avoid MagicMock interference

The previous fix (thin _ConcreteEngine subclass) still failed because
VocalinuxEngine inherits MagicMock.__getattribute__, which intercepts
.do_destroy on instances and returns a mock attribute rather than the
real method - regardless of Python version.

Call VocalinuxEngine.do_destroy(engine) directly with a plain object()
as the self argument instead. do_destroy only uses self for identity
comparison (is VocalinuxEngine._active_instance), so any object works.
This bypasses MagicMock's MRO entirely.

* test(ibus): reload module and instantiate real engine class for do_destroy tests

* refactor(ibus): extract destroy handler for stable cross-version tests

* test(ibus): cover destroy helper branches and do_destroy delegation

* test(ibus): stabilize do_destroy CI behavior across environments

Avoid brittle super() mocking in no-IBus environments and gate the IBus-only parent destroy path test behind runtime capability checks to prevent false CI failures on Python matrix runners.

* test(ibus): remove flaky do_destroy class-path assertions

Drop brittle class-method destroy tests that fail under mocked GI/IBus on older Python runners and keep deterministic helper-level coverage for destroy behavior.

* fix(text-injection): recover from IBus runtime failures (#411)

* fix(ibus): retry and recover from transient socket failures

Harden IBus injection against connection-refused, timeout, and engine process races so dictation is more resilient in real-world desktop sessions.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* fix(text-injection): fallback to non-IBus backend on runtime IBus failures

Allow dictation to continue when IBus runtime injection fails by automatically switching to xdotool/wtype/ydotool fallbacks instead of returning an immediate failure.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* fix(ibus): preserve live socket injection during recovery

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* test(coverage): add runtime fallback and ibus retry coverage

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* fix(ibus): probe engine readiness at startup and harden NO_ENGINE retries (#391)

* fix(ibus): wait for engine readiness before first injection

Add a startup readiness probe that pings the IBus engine after activation so first-use dictation no longer races do_enable(). Also switch NO_ENGINE retry behavior to exponential backoff and cover the new flow in tests.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* test(ibus): cover readiness probe edge cases

Add focused tests for the new startup readiness path, including direct PING success, retry exhaustion, unexpected responses, and engine re-activation before injection. This raises coverage on vocalinux.text_injection.ibus_engine from 68% to 73% in the targeted suite.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* fix(text-injection): avoid blocking IBus and clipboard fallbacks

Warm IBus in the background so startup no longer stalls while readiness probing, and make clipboard helpers fail fast with per-session health caching so non-ASCII ydotool fallback does not lock dictation.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* test(ibus): update retry tests for merged NO_ENGINE recovery logic

---------

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>

* chore(deps-dev): bump postcss from 8.4.39 to 8.5.10 in /web

Bump postcss dev dependency (CSS processing). No runtime impact.

* fix(logging): clean up runtime log noise and cache hardware detection

Clean up duplicated code artifacts from recent merges and cache hardware detection to eliminate ~1.3s UI lag when opening settings dialog.

* test: cover recognition internals and suppress desktop notifications (#414)

* test(recognition): cover _perform_recognition and _enqueue_audio_segment

The previous commit demoted logger.info('DEBUG: ...') to logger.debug(...)
and removed a dead loop body in _perform_recognition. Codecov flagged 12
uncovered lines — all inside _perform_recognition and _enqueue_audio_segment,
which existing tests always mocked out.

Add direct (non-mocked) tests for both methods:

- _enqueue_audio_segment:
  * empty buffer → early return
  * normal buffer → enqueues correctly
  * full queue → drops oldest and retries

- _perform_recognition:
  * processes a segment then exits on None signal
  * drains remaining items on None signal
  * exits cleanly when idle with empty queue
  * continues looping on queue timeout while recording

These tests exercise the actual threading/queue logic and cover the
logger.debug / logger.warning calls as a side effect.

* test(conftest): suppress real desktop notifications during test runs

_show_notification shells out to notify-send, which pops up real system
notifications when tests exercise the model-not-ready path. Add an autouse
fixture that patches _show_notification for all tests EXCEPT those whose
name contains 'show_notification' (so direct tests of the helper still
verify subprocess.Popen calls).

* chore(gitignore): ignore .sisyphus directory

* fix(install): improve openSUSE dependency handling with fallbacks (#418)

The suse case in install_system_dependencies() previously installed all
packages blindly, causing the entire install to fail if a single package
was unavailable (e.g., glslang on some openSUSE repos).

Changes:
- Check each package individually with rpm -q before installing
- Add graceful fallback for glslang -> glslang-devel
- Add graceful fallback for libappindicator-gtk3 -> typelib-1_0-AppIndicator3-0_1 -> libayatana-appindicator3-1
- Continue installation with warning instead of hard failure

Fixes #407

* fix(remote_api): re-init engine on resume from suspend

Add a remote_api branch to reinitialize_after_resume mirroring the
dispatch chain in __init__, so the engine is recreated after a
suspend/resume cycle. Null self._http_session alongside the existing
self.model and self.recognizer resets so a stale pooled-connection
session is not reused after resume.

* perf(remote_api): pool HTTP connections via requests.Session

Introduces a manager-owned requests.Session (_http_session) so all
remote_api engine HTTP traffic reuses a single connection pool instead
of opening a new connection for every request.

Lifecycle:
- __init__: self._http_session = None (attribute always present)
- _init_remote_api: close prior session if it exists, then create a
  new requests.Session(); use session.get for the connection test.
  Prevents connection leaks on back-to-back inits (re-init guard).
- _try_openai_api / _try_whispercpp_server_api: replaced requests.post
  with self._http_session.post; local 'import requests' retained for
  the requests.exceptions.ConnectionError catch.
- reconfigure: captures old engine before self.engine is updated;
  closes and nulls _http_session when switching away from remote_api.
- reinitialize_after_resume: nulls _http_session before _init_remote_api
  recreates it.

Tests: added TestHTTPSession class (4 cases) in
tests/test_recognition_manager_remote_api.py; all 40 tests pass.
Updated _setup_requests_get_*/post_* helpers to configure the session
instance (mock_requests.Session.return_value) instead of the
module-level mock, keeping all 36 pre-existing tests green.

* docs: add remote HTTP transcription setup guide

* fix(install): avoid asking for remote server URL twice

run_interactive_install collects the URL upfront in Step 3, but
install_python_package re-declared REMOTE_API_URL as local and prompted
again, ignoring the wizard value. Inherit the wizard's value instead.

* ui(settings): hide remote API behind an Advanced tab

Addresses upstream PR #335 review feedback from @jatinkrmalik that
remote API is not plug-and-play and should not be a peer of the local
engines for casual users.

- Remove 'Remote API' from the Speech Engine tab's engine combo; the
  combo now shows local engines only (vosk / whisper / whisper_cpp).
- Add a new 'Advanced' notebook tab with a 'Remote Server' group
  containing a 'Use remote server' switch, the existing URL/Key/
  Endpoint fields, and the Test Connection button + status label.
- The switch overrides the local engine selection. get_selected_settings
  returns engine='remote_api' iff the switch is on, otherwise the combo
  selection. URL/Key/Endpoint persist independently of the toggle state.
- _load_and_apply_settings: if config has engine='remote_api', the
  switch is turned on and the combo is set to a sensible local fallback
  (whisper_cpp, or first available local engine) so toggling off has
  somewhere to land.
- _update_engine_specific_ui no longer special-cases the combo for
  remote_api (the combo never has it); it hides the model row, info
  card and legend whenever the remote switch is on.
- Update HTTP_REMOTE.md to reflect the new tab location.

* ui(remote-api): mark remote API as advanced and default toggle off

- install.sh: relabel option 4 as 'REMOTE API (ADVANCED)' to signal
  it as a non-default, advanced engine choice.
- settings_dialog.py: explicitly initialise the 'Use remote server'
  switch to inactive at widget creation, making the default state
  unambiguous (saved-config load logic still toggles it on for
  existing remote_api installs).

* fix(remote-api): preserve trailing newline from transcription responses

The remote-API transcription path stripped all whitespace from the
upstream response before injection, silently discarding a trailing
newline that some pipelines (post-processing proxies, fixup layers)
emit to signal "submit" / Enter. Whisper itself doesn't emit trailing
newlines, so for the common case this is a no-op; for users with a
post-processing layer it stops their submit signal from being eaten.

_filter_non_speech now uses a whitespace-stripped probe for its
non-speech pattern checks but returns probe + "\n" when the original
input ended with "\n" or "\r" (CRLF normalises to LF). The redundant
.strip() at the call site is removed so the function receives the raw
API text. Existing filter behaviour (BLANK_AUDIO, music notes, low
speech-content ratio, etc.) is unchanged. Four new unit tests cover
the trailing-newline cases.

* test(remote-api): cover transcribe exception handler and unknown-engine reinit branch

Adds two unit tests buttoning up previously-uncovered defensive branches:

- _transcribe_with_remote_api swallows unexpected exceptions and returns
  "" — verified by raising RuntimeError from _try_whispercpp_server_api.
- reinitialize_after_resume short-circuits without dispatching when the
  engine attribute is unknown — verified by mutating engine to a bogus
  value and asserting no _init_* method is called.

* feat(settings): add Advanced tab with whisper.cpp anti-hallucination parameters (#415)

* feat(settings): add Advanced tab with whisper.cpp anti-hallucination parameters

Add a new Advanced settings tab that exposes whisper.cpp decoding
parameters to help users reduce hallucinations and tune recognition
behavior.

Parameters exposed:
- No Timestamps (default: on)
- Suppress Non-Speech Tokens (default: on)
- No Context (default: on)
- Initial Prompt (default: empty)
- Temperature (default: 0.0)
- Temperature Increment (default: -1.0, disables fallback)
- Entropy Threshold (default: 2.4)
- Logprob Threshold (default: -1.0)
- No-Speech Threshold (default: 0.6)

All parameters are saved under the advanced config section and are
only active when the whisper.cpp engine is selected. The UI grays out
these settings and shows an explanatory note when another engine is
active.

Changes:
- config_manager.py: add whisper.cpp keys to DEFAULT_CONFIG[advanced]
- recognition_manager.py: read params from kwargs/config, pass to Model()
- main.py: pass advanced settings from config to SpeechRecognitionManager
- settings_dialog.py: new Advanced tab with switches, spin buttons, entry
- tests: update test_main.py assertions, add whispercpp_advanced_defaults
  and whispercpp_advanced_persistence tests

Fixes #363

* fix(whispercpp): gracefully filter unsupported params for pywhispercpp 1.4.1

pywhispercpp 1.4.1 does not expose no_timestamps or suppress_non_speech_tokens
to Python even though they exist in the underlying whisper.cpp C struct.
This causes AttributeError on startup.

Add _load_model_with_compatible_params() which catches AttributeError,
identifies the unsupported param from the error message, removes it, and
retries until the model loads successfully or a non-AttributeError occurs.

Also refactor _build_whispercpp_model_kwargs() from _load_whispercpp_model()
for testability.

Fixes local startup crash reported during PR testing.

* fix(whispercpp): persist advanced settings correctly

* chore(config): add advanced power-user default

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* feat(settings): gate advanced controls behind power-user opt-in

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

* fix(whispercpp): prefilter unsupported native params

* fix(settings): polish advanced whispercpp controls

* fix(installer): reuse existing whispercpp builds (#421)

* docs: clarify PyPI installation requirements (#423)

* chore(deps): bump next (#429)

Bumps the npm_and_yarn group with 1 update in the /web directory: [next](https://github.com/vercel/next.js).


Updates `next` from 15.5.15 to 15.5.18
- [Release notes](https://github.com/vercel/next.js/releases)
- [Changelog](https://github.com/vercel/next.js/blob/canary/release.js)
- [Commits](https://github.com/vercel/next.js/compare/v15.5.15...v15.5.18)

---
updated-dependencies:
- dependency-name: next
  dependency-version: 15.5.18
  dependency-type: direct:production
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(installer): repair pywhispercpp library loading (#433)

* fix: play stop sound immediately on release (#426)

* fix(recognition): play stop sound after audio thread joins to prevent capture on short recordings (#436)

Moves play_stop_sound() to after audio_thread.join() but before the
final buffer processing. This closes the race where the audio thread's
in-flight chunk could capture the stop cue for short (≤ ~192 ms) recordings,
while still feeling immediate compared to the original placement after all
recognition cleanup.

* fix(install): correct openSUSE Tumbleweed dependencies (#420)

* fix(install): correct openSUSE Tumbleweed dependencies

* style: format openSUSE installer changes

---------

Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>

* fix(installer): harden Debian compatibility layer (#437)

Add missing pywhispercpp build prerequisites (libssl-dev, autoconf,
automake, libtool, patchelf) to both Debian and Ubuntu apt package lists.
These are not pulled in transitively on clean Debian installs and cause
CMake bootstrap failures during source compilation.

Add runtime smoke test for pywhispercpp to verify_installation() so
"installs but does not run" failures surface at install time with
actionable diagnostic steps instead of silently at first launch.

Treat Debian as a first-class target — remove the "primarily designed for
Ubuntu" warning for Debian systems. Remain conservative for non-Debian,
non-Ubuntu distros.

Add DEBIAN_FRONTEND=noninteractive to all apt install commands to prevent
interactive debconf prompts from blocking unattended installs.

Improve ydotool failure messaging on Debian with compile-from-source
instructions since ydotool is not packaged in Debian standard repos.

Add Debian-specific troubleshooting section to DISTRO_COMPATIBILITY.md
covering build deps, ydotool compilation, scoped source builds, and
libwhisper.so resolution diagnostics.

References: https://medium.com/@cslev/talking-to-my-linux-box-without-talking-to-the-cloud-vocalinux-on-debian-without-the-tears-10bf053ea21b

* fix(install): refresh ldconfig after openSUSE typelib install, clarify python3XY placeholder convention (#438)

- Add 'sudo ldconfig' after zypper install in suse_install_appindicator_runtime()
  to ensure GI typelib cache is refreshed so newly installed typelib packages
  are immediately discoverable by GObject introspection
- Add section-level comment in distro-package-map.yaml explaining that
  python3XY is a placeholder convention resolved at install time by
  suse_python_package_prefix() / suse_python_package_candidates()

Follow-up polish from review of #420.

* fix(whispercpp): reduce CPU threads and ensure GPU backend builds in dev mode (#439)

* fix(whispercpp): reduce CPU threads and ensure GPU backend builds in dev mode

- Reduce default n_threads from cpu_count() (16) to max 8 (CPU) / max 4 (GPU)
- Add runtime pywhispercpp GPU capability detection (_detect_pywhispercpp_gpu_backend)
- Make n_threads configurable via whispercpp_n_threads config option
- Fix install.sh regression: GPU source build was skipped in --dev mode
- Extract GPU install logic into install_whispercpp_with_gpu_support()
- Add is_pywhispercpp_gpu_capable() to detect GPU libs in installed pywhispercpp
- Force rebuild pywhispercpp when GPU is detected but lacks GPU support

Fixes CPU spiking to 100% on GPU systems due to excessive thread count
and ensures dev installs get GPU acceleration.

* fix(audio): skip output-only HDMI devices when selecting input device

Add _resolve_valid_input_device() that filters devices by
maxInputChannels > 0, falling back preferred → default → first valid.

Previously, HDMI output devices (0 input channels) could be selected
as the default audio input, causing 'Invalid number of channels' errors.
Now both _record_audio() and _attempt_audio_reconnection() validate the
device has input channels before probing or opening the stream.

* fix(whispercpp): treat n_threads=0 as auto-detect, not literal zero

Config default whispercpp_n_threads=0 was being passed directly
to whisper.cpp as n_threads=0, causing 'cannot create std::vector
larger than max_size()' transcription errors. Now 0 and negative
values are treated as auto-detect, falling through to the smart
thread count logic (max 4 with GPU, max 8 on CPU).

* test: add whispercpp_n_threads to mock assertions and config defaults

Update test_main.py mock to include the new whispercpp_n_threads=0 kwarg
and test_config_manager.py to assert the default config value.

* fix(audio): harden device enumeration against test mocks

The new HDMI-filter helper and GPU-backend detector crashed unit tests
that pass MagicMock() for the PyAudio/pathlib surfaces:

- _find_pywhispercpp_shared_library_dirs hit TypeError when tests
  monkey-patch os.stat (Path.is_dir internally calls S_ISDIR on a
  non-int st_mode). Widen the catch to OSError/TypeError/ValueError so
  unconfigured stat results just skip the candidate dir.

- _resolve_valid_input_device crashed with 'MagicMock > int' on
  audio.get_device_count() / info.get('maxInputChannels'). Coerce the
  count, treat zero/unknown as 'use system default', and include
  non-dict info objects rather than filtering everything out.

No production behavior changes: real PyAudio always returns int counts
and dict device info, so the HDMI filter still works as designed.

* test(recognition): cover new HDMI filter and GPU backend detector

- TestResolveValidInputDevice: 9 cases for the HDMI-filter resolver
  (preferred passthrough, HDMI fallback, no-default fallback, no-input
  None return, and three mock-tolerance escape hatches).
- TestDetectPywhispercppGpuBackend: 4 cases for runtime GPU library
  detection (cpu/vulkan/cuda/empty dirs).
- TestFindPywhispercppSharedLibraryDirsResilience: proves the widened
  exception catch survives a TypeError from pathlib.Path.is_dir.

* test(text-injection): wait for IBus init thread to avoid race on slow runners

The IBus environment promotion runs in a background daemon thread spawned
by TextInjector.__init__. The two TestIBusRuntimeFallback cases asserted
on injector.environment immediately after construction, which raced the
thread on Python 3.10 GHA runners and produced intermittent failures.

Join the init thread (with a generous timeout) before asserting, so the
test exercises the post-warmup state deterministically.

* chore(release): prepare v0.11.0-beta (#440)

- Bump version from 0.10.2-beta to 0.11.0-beta
- Update all version references across codebase
- Add What's New section for v0.11.0-beta in README and docs/UPDATE.md
- Add v0.11.0-beta entry to website changelog
- Update SECURITY.md supported versions to 0.11.x
- Update website schema, badge, and package version
- Update RELEASE_PROCESS.md version history table

* fix(startup): allow launch without pynput backend (#448)

* feat(vad): add Silero VAD and drop silence-only buffers (#447)

* feat(vad): replace amplitude-based VAD with Silero neural network

Replace the simple amplitude threshold VAD with Silero VAD, a lightweight
neural network model that classifies audio as speech/non-speech. This
addresses the root cause of #302 (silence not stopping recording) and

- Add silero_vad.py wrapper using ONNX Runtime with bundled Silero v5 model
- Accumulate 512-sample chunks with 64-sample context window per official API
- Map existing vad_sensitivity (1-5) to Silero thresholds (0.8-0.3)
- Use max() across sub-chunks to avoid missing speech onsets
- Fall back to amplitude VAD when onnxruntime is unavailable
- Add onnxruntime as optional dependency (pip install vocalinux[vad])

Closes #344

* address review: add tests, LICENSE, logging, validation fixes

- Add unit tests for SileroVAD (process, reset, fallback, sensitivity mapping)
- Add MIT LICENSE for bundled Silero VAD model in data/
- Log which VAD backend is active on init
- Validate chunk length in SileroVAD.process()
- Skip is_speech check until a full Silero chunk is processed
- Add thread safety note to SileroVAD docstring
- Document resampling rate effect on VAD resolution
- Comment explaining why load_silero_vad() catches broad Exception

* address review: bundle license, document [vad] extra, surface backend in UI

- Pack data/LICENSE into the wheel (pyproject package-data) so attribution
  travels with the model
- README: short note on enabling neural VAD via vocalinux[vad]
- Settings: VAD Sensitivity row subtitle shows the active backend; if
  onnxruntime is missing, an info-row beneath the group spells out the
  install command and the restart requirement
- silero_vad.is_silero_available() probes onnxruntime + model presence
  w/o instantiating an InferenceSession
- CI installs .[dev,vad] so Silero paths get exercised
- Integration test for _record_audio Silero branch (mocked pyaudio +
  scripted SileroVAD), covering silence flush, speech reset, and
  sensitivity threshold mapping at sens=1 vs sens=5
- Tests no longer skip when other modules globally mock numpy: pop the
  top-level numpy ref so a fresh import gives the real package, and
  swap sys.modules["numpy"] inside the _record_audio call so the lazy
  import inside the function picks up real numpy too

* fix(vad): drop silence-only recording buffers

* test(vad): align stop buffer expectations

* test(vad): preserve real numpy module

* fix(install): install vad support opportunistically

* test(vad): cover recognition manager vad edges

---------

Co-authored-by: Sbenazar <127324689+Sbenazar@users.noreply.github.com>

* fix(web): clarify speech demo browser support (#449)

* fix black format check

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: jay <sevenjay@users.noreply.github.com>
Co-authored-by: Jatin K Malik <jatinkrmalik@gmail.com>
Co-authored-by: Fakhareddine <fe.sabbari@claranet.com>
Co-authored-by: Sbenazar <127324689+Sbenazar@users.noreply.github.com>
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parko636 <alex@alexparkinson.net>
Co-authored-by: juanfradb <94815906+juanfradb@users.noreply.github.com>
Co-authored-by: Pep Locanto <peplocanto@gmail.com>
This commit is contained in:
sevenjay
2026-06-06 15:52:46 -07:00
committed by GitHub
co-authored by Sbenazar jay Jatin K Malik Fakhareddine Sisyphus dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> parko636 juanfradb Pep Locanto
parent b0572a36dc
commit 84d2e1a929
9 changed files with 1702 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/vocalinux/main.py
+4 -1
View File
@@ -58,7 +58,7 @@ def parse_arguments():
parser.add_argument(
"--engine",
type=str,
choices=["vosk", "whisper", "whisper_cpp"],
choices=["vosk", "whisper", "whisper_cpp", "remote_api"],
help="Speech recognition engine to use (whisper_cpp recommended for best performance)",
)
parser.add_argument("--wayland", action="store_true", help="Force Wayland compatibility mode")
@@ -382,6 +382,9 @@ def main():
whispercpp_logprob_thold=advanced_settings.get("whispercpp_logprob_thold", -1.0),
whispercpp_no_speech_thold=advanced_settings.get("whispercpp_no_speech_thold", 0.6),
whispercpp_n_threads=advanced_settings.get("whispercpp_n_threads", 0),
remote_api_url=saved_settings.get("remote_api_url", ""),
remote_api_key=saved_settings.get("remote_api_key", ""),
remote_api_endpoint=saved_settings.get("remote_api_endpoint", "/inference"),
)
# Initialize text injection system
src/vocalinux/speech_recognition/recognition_manager.py
+302 -7
View File
@@ -569,6 +569,14 @@ def _filter_non_speech(text: str) -> str:
[BLANK_AUDIO], music notes, or other non-speech artifacts when
transcribing silent or ambiguous audio.
A trailing newline returned by the upstream transcription service is
preserved in the output. Some pipelines (post-processing proxies,
fixup layers) emit a meaningful trailing '\\n' to signal that the
typed text should be submitted (Enter key); stripping it silently
discards that signal. Pattern matching still runs against a
whitespace-stripped probe so the existing non-speech checks are
unaffected.
Args:
text: The transcribed text to filter
@@ -580,7 +588,8 @@ def _filter_non_speech(text: str) -> str:
if not text or not text.strip():
return ""
text = text.strip()
# Probe is used for non-speech-pattern matching only.
probe = text.strip()
# Non-speech patterns to filter out
non_speech_patterns = [
@@ -596,20 +605,24 @@ def _filter_non_speech(text: str) -> str:
]
for pattern in non_speech_patterns:
if re.match(pattern, text, re.IGNORECASE):
logger.debug(f"Filtered non-speech token: '{text}'")
if re.match(pattern, probe, re.IGNORECASE):
logger.debug(f"Filtered non-speech token: '{probe}'")
return ""
# Check if text has enough actual speech content
# At least 30% of characters should be alphanumeric or common speech punctuation
speech_chars = sum(1 for c in text if c.isalnum() or c in ".,!?-'\"")
total_chars = len(text)
speech_chars = sum(1 for c in probe if c.isalnum() or c in ".,!?-'\"")
total_chars = len(probe)
if total_chars > 0 and speech_chars / total_chars < 0.3:
logger.debug(f"Filtered low-speech-content text: '{text}'")
logger.debug(f"Filtered low-speech-content text: '{probe}'")
return ""
return text
# Strip whitespace as before, but preserve a trailing newline if the
# upstream API sent one — it carries an explicit submit/Enter signal
# from post-processing proxies.
trailing_newline = text.endswith(("\n", "\r"))
return probe + "\n" if trailing_newline else probe
def _show_notification(title: str, message: str, icon: str = "dialog-warning"):
@@ -764,6 +777,12 @@ class SpeechRecognitionManager:
self.whispercpp_no_speech_thold = kwargs.get("whispercpp_no_speech_thold", 0.6)
self.whispercpp_n_threads = kwargs.get("whispercpp_n_threads", None)
# Remote API settings
self.remote_api_url = kwargs.get("remote_api_url", "")
self.remote_api_key = kwargs.get("remote_api_key", "")
self.remote_api_endpoint = kwargs.get("remote_api_endpoint", "/inference")
self._http_session = None
# Audio diagnostics tracking
self._last_audio_level = 0.0
self._audio_level_callbacks: list[Callable[[float], None]] = []
@@ -801,6 +820,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif engine == "whisper_cpp":
self._init_whispercpp()
elif engine == "remote_api":
self._init_remote_api()
else:
raise ValueError(f"Unsupported speech recognition engine: {engine}")
@@ -1339,6 +1360,252 @@ class SpeechRecognitionManager:
logger.error(f"Error in whisper.cpp transcription: {e} ({audio_info})", exc_info=True)
return ""
def _init_remote_api(self):
"""Initialize remote API speech recognition engine.
Verify URL settings and try connection test. No need to load local model.
"""
if not self.remote_api_url:
logger.warning("Remote API URL not set. Please enter the server URL in settings.")
self._model_initialized = False
return
if not self.remote_api_url.startswith(("http://", "https://")):
logger.error(
f"Remote API URL must start with http:// or https://, "
f"got: '{self.remote_api_url}'"
)
self._model_initialized = False
return
# Clean trailing slash from URL
self.remote_api_url = self.remote_api_url.rstrip("/")
logger.info(f"Initialize remote API engine, server: {self.remote_api_url}")
# Replace any existing session (prevents leak on back-to-back inits)
import requests
if self._http_session:
self._http_session.close()
self._http_session = requests.Session()
# Try connection test
try:
test_url = self.remote_api_url
headers = {}
if self.remote_api_key:
headers["Authorization"] = f"Bearer {self.remote_api_key}"
response = self._http_session.get(test_url, headers=headers, timeout=5)
if response.ok:
logger.info(
f"Remote server connection test successful (status={response.status_code})"
)
else:
logger.warning(
f"Remote server connection test returned non-success status: "
f"{response.status_code}. Will try again during recognition."
)
except Exception as e:
logger.warning(
f"Remote server connection test failed: {e}. "
"Will try to connect again during recognition."
)
# Remote API does not need local models, directly mark as ready
self._model_initialized = True
logger.info("Remote API engine setup complete.")
def _transcribe_with_remote_api(self, audio_buffer: list[bytes]) -> str:
"""Transcribe audio via remote API.
Package audio buffer into WAV format and send to remote server via HTTP POST.
Supports OpenAI compatible format (/v1/audio/transcriptions) and
whisper.cpp server format (/inference).
Args:
audio_buffer: Audio data chunk list (16-bit PCM at 16kHz)
Returns:
Transcribed text
"""
import io
import time
import wave
try:
if not audio_buffer:
return ""
if not self.remote_api_url:
logger.error("Remote API URL not set")
return ""
# Convert audio buffer to WAV format
audio_data = b"".join(audio_buffer)
wav_buffer = io.BytesIO()
with wave.open(wav_buffer, "wb") as wav_file:
wav_file.setnchannels(1) # Mono
wav_file.setsampwidth(2) # 16-bit
wav_file.setframerate(16000) # 16kHz
wav_file.writeframes(audio_data)
wav_buffer.seek(0)
wav_bytes = wav_buffer.read()
duration = len(audio_data) / (2 * 16000) # 16-bit = 2 bytes/sample
logger.debug(
f"Remote API transcription: {duration:.2f} seconds audio, "
f"{len(wav_bytes)} bytes WAV"
)
# Prepare language parameters
lang = self.language
if lang == "en-us":
lang = "en"
elif lang == "auto":
lang = None
# Prepare HTTP request headers
headers = {}
if self.remote_api_key:
headers["Authorization"] = f"Bearer {self.remote_api_key}"
transcribe_start = time.time()
text = None
if self.remote_api_endpoint == "/inference":
text = self._try_whispercpp_server_api(wav_bytes, lang, headers)
else:
text = self._try_openai_api(wav_bytes, lang, headers)
# If both formats fail
if text is None:
logger.error(
"Remote API transcription failed: Cannot connect to server or API format not supported"
)
return ""
transcribe_duration = time.time() - transcribe_start
rtf = transcribe_duration / duration if duration > 0 else 0
# Filter non-speech content. Do NOT pre-strip — _filter_non_speech
# preserves a trailing '\n' from the upstream API (used by
# post-processing proxies to signal Enter), and a pre-strip here
# would silently discard it.
text = _filter_non_speech(text) if text else ""
if text:
logger.info(f"Remote API transcription result: '{text}'")
logger.info(
f"Remote API transcription took {transcribe_duration:.3f}s "
f"({duration:.2f}s audio, RTF: {rtf:.2f}x)"
)
else:
logger.debug(
f"Remote API returned blank transcription result ({transcribe_duration:.3f}s)"
)
return text
except Exception as e:
audio_info = (
f"audio buffer: {len(audio_buffer)} chunks"
if audio_buffer
else "empty audio buffer"
)
logger.error(f"Remote API transcription error: {e} ({audio_info})", exc_info=True)
return ""
def _try_openai_api(self, wav_bytes: bytes, lang, headers: dict):
"""Try to transcribe using OpenAI compatible API format.
Args:
wav_bytes: Audio data in WAV format
lang: Language core (e.g. "en", None for auto detect)
headers: HTTP request headers
Returns:
Transcribed text, or None if format is not supported
"""
import requests
url = f"{self.remote_api_url}{self.remote_api_endpoint}"
files = {"file": ("audio.wav", wav_bytes, "audio/wav")}
data = {"model": "whisper-1"}
if lang:
data["language"] = lang
try:
response = self._http_session.post(
url, headers=headers, files=files, data=data, timeout=30
)
if response.status_code == 404:
logger.debug("OpenAI API endpoint does not exist, try other formats")
return None
response.raise_for_status()
result = response.json()
# OpenAI format returns {"text": "..."}
return result.get("text", "")
except requests.exceptions.ConnectionError as e:
logger.error(f"Cannot connect to remote server {url}: {e}")
return None
except Exception as e:
logger.debug(f"OpenAI API format attempt failed: {e}")
return None
def _try_whispercpp_server_api(self, wav_bytes: bytes, lang, headers: dict):
"""Try to transcribe using whisper.cpp server API format.
Args:
wav_bytes: Audio data in WAV format
lang: Language core (e.g. "en", None for auto detect)
headers: HTTP request headers
Returns:
Transcribed text, or None if format is not supported
"""
import requests
url = f"{self.remote_api_url}{self.remote_api_endpoint}"
files = {"file": ("audio.wav", wav_bytes, "audio/wav")}
data = {
"temperature": "0.0",
"temperature_inc": "0.2",
"response_format": "json",
}
if lang:
data["language"] = lang
try:
response = self._http_session.post(
url, headers=headers, files=files, data=data, timeout=30
)
if response.status_code == 404:
logger.debug("whisper.cpp server endpoint does not exist")
return None
response.raise_for_status()
result = response.json()
# whisper.cpp server format returns {"text": "..."}
return result.get("text", "")
except requests.exceptions.ConnectionError as e:
logger.error(f"Cannot connect to remote server {url}: {e}")
return None
except Exception as e:
logger.debug(f"whisper.cpp server API format attempt failed: {e}")
return None
def _download_whispercpp_model(self):
"""Download a whisper.cpp model with progress tracking."""
import requests
@@ -1816,6 +2083,9 @@ class SpeechRecognitionManager:
@property
def model_ready(self) -> bool:
"""Check if the model is initialized and ready for recognition."""
# Remote API does not need local models
if self.engine == "remote_api":
return self._model_initialized
return self._model_initialized and self.model is not None
def _get_stop_sound_guard_chunks(self) -> int:
@@ -2279,6 +2549,9 @@ class SpeechRecognitionManager:
elif self.engine == "whisper_cpp":
text = self._transcribe_with_whispercpp(audio_buffer)
elif self.engine == "remote_api":
text = self._transcribe_with_remote_api(audio_buffer)
else:
logger.error(f"Unknown engine: {self.engine}")
return
@@ -2418,6 +2691,7 @@ class SpeechRecognitionManager:
)
restart_needed = False
old_engine = self.engine
if engine is not None and engine != self.engine:
self.engine = engine
restart_needed = True
@@ -2467,6 +2741,18 @@ class SpeechRecognitionManager:
setattr(self, param_name, kwargs[param_name])
restart_needed = True
# Handle Remote API settings
if "remote_api_url" in kwargs:
new_url = kwargs.get("remote_api_url", "")
if new_url != self.remote_api_url:
self.remote_api_url = new_url
if self.engine == "remote_api":
restart_needed = True
if "remote_api_key" in kwargs:
self.remote_api_key = kwargs.get("remote_api_key", "")
if "remote_api_endpoint" in kwargs:
self.remote_api_endpoint = kwargs.get("remote_api_endpoint", "/inference")
self._voice_commands_enabled = self._resolve_voice_commands_enabled()
if restart_needed:
@@ -2490,6 +2776,10 @@ class SpeechRecognitionManager:
# Release old resources explicitly if necessary (Python's GC might handle it)
self.model = None
self.recognizer = None
if old_engine == "remote_api" and self.engine != "remote_api":
if self._http_session is not None:
self._http_session.close()
self._http_session = None
try:
if self.engine == "vosk":
self._init_vosk()
@@ -2497,6 +2787,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif self.engine == "whisper_cpp":
self._init_whispercpp()
elif self.engine == "remote_api":
self._init_remote_api()
else:
raise ValueError(f"Unsupported engine during reconfigure: {self.engine}")
logger.info("Speech engine re-initialized successfully.")
@@ -2628,6 +2920,7 @@ class SpeechRecognitionManager:
with self._model_lock:
self.model = None
self.recognizer = None
self._http_session = None
self._model_initialized = False
try:
@@ -2637,6 +2930,8 @@ class SpeechRecognitionManager:
self._init_whisper()
elif self.engine == "whisper_cpp":
self._init_whispercpp()
elif self.engine == "remote_api":
self._init_remote_api()
else:
logger.error("Cannot reinitialize: unknown engine '%s'", self.engine)
return
src/vocalinux/ui/config_manager.py
+3
View File
@@ -29,6 +29,9 @@ DEFAULT_CONFIG = {
"silence_timeout": 2.0, # Seconds of silence before stopping
"stop_sound_guard_ms": 200, # Small tail trim to avoid the stop sound without clipping speech
"voice_commands_enabled": None, # None = auto (enabled for VOSK, disabled for Whisper)
"remote_api_url": "", # Remote speech recognition server URL (e.g. http://192.168.1.100:8080)
"remote_api_key": "", # Remote server API key (optional)
"remote_api_endpoint": "/inference", # Remote server API endpoint format (/v1/audio/transcriptions or /inference)
},
"audio": {
"device_index": None, # Audio input device index (None for system default)
src/vocalinux/ui/settings_dialog.py
+343 -42
View File
@@ -73,8 +73,31 @@ ENGINE_MODELS = {
"medium",
"large",
], # whisper.cpp models (ggml format)
"remote_api": [], # Remote API does not need local models
}
# Engine display name mapping
ENGINE_DISPLAY_NAMES = {
"vosk": "Vosk",
"whisper": "Whisper",
"whisper_cpp": "Whisper_cpp",
"remote_api": "Remote API",
}
def _engine_display_name(engine: str) -> str:
"""Get the display name of the engine."""
return ENGINE_DISPLAY_NAMES.get(engine, engine.capitalize())
def _engine_from_display(display_name: str) -> str:
"""Reverse lookup engine ID from display name."""
for engine_id, name in ENGINE_DISPLAY_NAMES.items():
if name == display_name:
return engine_id
return display_name.lower()
# Whisper model metadata for display
WHISPER_MODEL_INFO = {
"tiny": {"size_mb": 75, "desc": "Fastest, lowest accuracy", "params": "39M"},
@@ -90,7 +113,7 @@ def get_available_engines():
Detect which speech recognition engines are available/installed.
Returns a dictionary of engine_name -> availability (bool).
"""
engines = {"vosk": False, "whisper": False, "whisper_cpp": False}
engines = {"vosk": False, "whisper": False, "whisper_cpp": False, "remote_api": False}
# Check VOSK
try:
@@ -116,6 +139,14 @@ def get_available_engines():
except ImportError:
pass
# Remote API is always available (only requires requests package)
try:
import requests # noqa: F401
engines["remote_api"] = True
except ImportError:
pass
logger.debug(f"Available engines: {engines}")
return engines
@@ -858,8 +889,11 @@ class SettingsDialog(Gtk.Dialog):
general_label.set_tooltip_text("General settings")
notebook.append_page(self.general_tab, general_label)
# Advanced tab - whisper.cpp parameters and power-user features (remote API, etc.)
advanced_label = Gtk.Label(label="Advanced")
advanced_label.set_tooltip_text("Advanced whisper.cpp parameters")
advanced_label.set_tooltip_text(
"Advanced whisper.cpp parameters and settings for power users"
)
self.advanced_page_num = notebook.append_page(self.advanced_tab, advanced_label)
notebook.connect("switch-page", self._on_settings_page_switched)
@@ -1086,24 +1120,24 @@ class SettingsDialog(Gtk.Dialog):
self.model_combo = Gtk.ComboBoxText()
self.model_combo.set_size_request(180, -1)
_prevent_scroll_on_hover(self.model_combo)
model_row = PreferenceRow(
self.model_row = PreferenceRow(
title="Model Size",
subtitle="Larger models are more accurate but slower",
widget=self.model_combo,
)
group.add_row(model_row)
group.add_row(self.model_row)
# Language selection
self.language_combo = Gtk.ComboBoxText()
self.language_combo.set_size_request(180, -1)
self.language_combo.set_tooltip_text("Primary language for speech recognition")
_prevent_scroll_on_hover(self.language_combo)
language_row = PreferenceRow(
self.language_row = PreferenceRow(
title="Language",
subtitle="Primary language for recognition",
widget=self.language_combo,
)
group.add_row(language_row)
group.add_row(self.language_row)
self.content_box.pack_start(group, False, False, 0)
@@ -1134,10 +1168,10 @@ class SettingsDialog(Gtk.Dialog):
self.content_box.pack_start(self.language_warning, False, False, 0)
# Legend
legend_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL, spacing=16)
legend_box.set_halign(Gtk.Align.CENTER)
legend_box.set_margin_top(4)
legend_box.set_margin_bottom(4)
self.legend_box = Gtk.Box(orientation=Gtk.Orientation.HORIZONTAL, spacing=16)
self.legend_box.set_halign(Gtk.Align.CENTER)
self.legend_box.set_margin_top(4)
self.legend_box.set_margin_bottom(4)
for symbol, text in [
("✓", "Downloaded"),
@@ -1146,15 +1180,102 @@ class SettingsDialog(Gtk.Dialog):
]:
item = Gtk.Label(label=f"{symbol} {text}")
item.get_style_context().add_class("status-info")
legend_box.pack_start(item, False, False, 0)
self.legend_box.pack_start(item, False, False, 0)
self.content_box.pack_start(legend_box, False, False, 0)
self.content_box.pack_start(self.legend_box, False, False, 0)
# Connect signals
self.engine_combo.connect("changed", self._on_engine_changed)
self.model_combo.connect("changed", self._on_model_changed)
self.language_combo.connect("changed", self._on_language_changed)
def _on_remote_api_settings_changed(self, widget):
"""Handle remote API URL/Key/endpoint changes (Advanced tab)."""
if self._initializing or self._applying_settings:
return
url = self.remote_api_url_entry.get_text().strip()
key = self.remote_api_key_entry.get_text().strip()
endpoint = self.remote_api_endpoint_combo.get_active_id() or "/inference"
self.config_manager.set("speech_recognition", "remote_api_url", url)
self.config_manager.set("speech_recognition", "remote_api_key", key)
self.config_manager.set("speech_recognition", "remote_api_endpoint", endpoint)
self.config_manager.save_settings()
# Auto apply only if remote engine is currently active
if self.use_remote_switch.get_active():
self._auto_apply_settings()
def _on_test_remote_connection(self, widget):
"""Test remote server connection."""
url = self.remote_api_url_entry.get_text().strip()
if not url:
self.remote_status_label.set_markup(
"<span foreground='#c01c28'>✗ Please enter a server URL</span>"
)
return
self.remote_test_btn.set_sensitive(False)
self.remote_test_btn.set_label("Testing...")
self.remote_status_label.set_markup("<i>Connecting...</i>")
key = self.remote_api_key_entry.get_text().strip()
endpoint = self.remote_api_endpoint_combo.get_active_id() or "/inference"
def test_connection(url=url, key=key, endpoint=endpoint):
try:
import requests
headers = {}
if key:
headers["Authorization"] = f"Bearer {key}"
clean_url = url.rstrip("/")
# Try connecting to the server
response = requests.get(clean_url, headers=headers, timeout=5)
# Determine server type
server_info = ""
try:
if endpoint == "/v1/audio/transcriptions":
# Try OpenAI endpoint
openai_resp = requests.get(
f"{clean_url}/v1/models", headers=headers, timeout=5
)
if openai_resp.status_code == 200:
server_info = " (OpenAI compatible)"
elif endpoint == "/inference":
# Try whisper.cpp server endpoint
whispercpp_resp = requests.get(
f"{clean_url}/inference", headers=headers, timeout=5
)
if whispercpp_resp.status_code != 404:
server_info = " (whisper.cpp server)"
except Exception:
pass
GLib.idle_add(
self.remote_status_label.set_markup,
f"<span foreground='#26a269'>✓ Connected! "
f"(status={response.status_code}){server_info}</span>",
)
except Exception as e:
error_msg = str(e)[:80]
GLib.idle_add(
self.remote_status_label.set_markup,
f"<span foreground='#c01c28'>✗ Connection failed: {error_msg}</span>",
)
# Restore button state
GLib.idle_add(self.remote_test_btn.set_sensitive, True)
GLib.idle_add(self.remote_test_btn.set_label, "Test Connection")
threading.Thread(target=test_connection, daemon=True).start()
def _build_recognition_section(self):
"""Build the Recognition Settings section."""
group = PreferencesGroup(title="Recognition Settings")
@@ -1677,6 +1798,113 @@ class SettingsDialog(Gtk.Dialog):
self.power_user_switch.connect("state-set", self._on_power_user_toggled)
"""Build the Advanced tab (currently: Remote Server settings)."""
self.remote_api_group = PreferencesGroup(
title="Remote Server",
description=(
"Offload speech recognition to a server on your network. "
"Supports whisper.cpp and OpenAI-compatible APIs."
),
)
# Toggle: enable remote engine (overrides Speech Engine tab selection)
self.use_remote_switch = Gtk.Switch()
self.use_remote_switch.set_active(False)
self.use_remote_switch.set_tooltip_text(
"When enabled, transcription is performed by the configured remote "
"server instead of the local engine selected on the Speech Engine tab."
)
use_remote_row = PreferenceRow(
title="Use remote server",
subtitle="Override the local engine and send audio to a remote server",
widget=self.use_remote_switch,
)
self.remote_api_group.add_row(use_remote_row)
# Server URL
self.remote_api_url_entry = Gtk.Entry()
self.remote_api_url_entry.set_placeholder_text("http://192.168.1.100:8080")
self.remote_api_url_entry.set_tooltip_text(
"URL of the remote speech recognition server\n"
"Supports OpenAI compatible API and whisper.cpp server"
)
self.remote_api_url_entry.set_size_request(280, -1)
remote_url_row = PreferenceRow(
title="Server URL",
subtitle="Remote speech recognition server address",
widget=self.remote_api_url_entry,
)
self.remote_api_group.add_row(remote_url_row)
# API Key
self.remote_api_key_entry = Gtk.Entry()
self.remote_api_key_entry.set_placeholder_text("(optional)")
self.remote_api_key_entry.set_visibility(False)
self.remote_api_key_entry.set_tooltip_text("API Key for authentication (optional)")
self.remote_api_key_entry.set_size_request(280, -1)
remote_key_row = PreferenceRow(
title="API Key",
subtitle="Authentication key (optional)",
widget=self.remote_api_key_entry,
)
self.remote_api_group.add_row(remote_key_row)
# API Endpoint
self.remote_api_endpoint_combo = Gtk.ComboBoxText()
self.remote_api_endpoint_combo.set_size_request(280, -1)
self.remote_api_endpoint_combo.set_tooltip_text(
"Select the API format of the remote server (API Endpoint Format)"
)
self.remote_api_endpoint_combo.append(
"/v1/audio/transcriptions", "OpenAI (/v1/audio/transcriptions)"
)
self.remote_api_endpoint_combo.append("/inference", "Whisper.cpp (/inference)")
_prevent_scroll_on_hover(self.remote_api_endpoint_combo)
remote_endpoint_row = PreferenceRow(
title="API Endpoint",
subtitle="API format for the remote server",
widget=self.remote_api_endpoint_combo,
)
self.remote_api_group.add_row(remote_endpoint_row)
# Connection test
self.remote_test_btn = Gtk.Button(label="Test Connection")
self.remote_test_btn.set_tooltip_text("Test connection to remote server")
self.remote_test_btn.connect("clicked", self._on_test_remote_connection)
remote_test_row = PreferenceRow(
title="Connection Test",
subtitle="Verify remote server is reachable",
widget=self.remote_test_btn,
)
self.remote_api_group.add_row(remote_test_row)
self.advanced_tab.pack_start(self.remote_api_group, False, False, 0)
# Status label below the group
self.remote_status_label = Gtk.Label(label="", use_markup=True, xalign=0)
self.remote_status_label.set_margin_start(16)
self.remote_status_label.set_margin_top(4)
self.remote_status_label.get_style_context().add_class("status-info")
self.advanced_tab.pack_start(self.remote_status_label, False, False, 0)
# Load saved values into the widgets
saved_url = self.config_manager.get("speech_recognition", "remote_api_url", "")
saved_key = self.config_manager.get("speech_recognition", "remote_api_key", "")
saved_endpoint = self.config_manager.get(
"speech_recognition", "remote_api_endpoint", "/inference"
)
if saved_url:
self.remote_api_url_entry.set_text(saved_url)
if saved_key:
self.remote_api_key_entry.set_text(saved_key)
self.remote_api_endpoint_combo.set_active_id(saved_endpoint)
# Connect signals (toggle and field changes both trigger re-apply)
self.use_remote_switch.connect("notify::active", self._on_use_remote_toggled)
self.remote_api_url_entry.connect("changed", self._on_remote_api_settings_changed)
self.remote_api_key_entry.connect("changed", self._on_remote_api_settings_changed)
self.remote_api_endpoint_combo.connect("changed", self._on_remote_api_settings_changed)
def _on_power_user_toggled(self, widget, state):
"""Handle the power-user opt-in toggle."""
if self._initializing or self._applying_settings:
@@ -1771,6 +1999,13 @@ class SettingsDialog(Gtk.Dialog):
self._auto_apply_settings()
def _on_use_remote_toggled(self, switch, _gparam):
"""Toggle remote-engine override: forces engine to remote_api when on."""
if self._initializing or self._applying_settings:
return
self._update_engine_specific_ui()
self._auto_apply_settings()
def _load_and_apply_settings(self):
"""Load current settings and populate the UI."""
settings = self._get_current_settings()
@@ -1797,44 +2032,63 @@ class SettingsDialog(Gtk.Dialog):
self.copy_to_clipboard_switch.set_active(copy_to_clipboard)
self.sound_effects_switch.set_active(self.config_manager.is_sound_effects_enabled())
# Populate engine combo with only available engines
# Populate engine combo with only available LOCAL engines.
# Remote API is not a peer engine in the combo — it lives on the
# Advanced tab as a toggle that overrides the local engine.
available_engines = get_available_engines()
available_count = 0
for engine in ENGINE_MODELS.keys():
if engine == "remote_api":
continue
if available_engines.get(engine, False):
capitalized_engine = engine.capitalize()
self.engine_combo.append(capitalized_engine, capitalized_engine)
display_name = _engine_display_name(engine)
self.engine_combo.append(display_name, display_name)
available_count += 1
if available_count == 0:
logger.error("No speech recognition engines available!")
logger.error("No local speech recognition engines available!")
# Still add them so the UI works, but log the error
for engine in ENGINE_MODELS.keys():
capitalized_engine = engine.capitalize()
self.engine_combo.append(capitalized_engine, capitalized_engine)
if engine == "remote_api":
continue
display_name = _engine_display_name(engine)
self.engine_combo.append(display_name, display_name)
else:
logger.info(f"Populated {available_count} available engines: {available_engines}")
logger.info(f"Populated {available_count} available local engines: {available_engines}")
# Set engine active - check if current engine is available
if not available_engines.get(self.current_engine, False):
logger.warning(
f"Current engine '{self.current_engine}' is not available, selecting first available"
)
# Find first available engine
for engine in ENGINE_MODELS.keys():
if available_engines.get(engine, False):
self.current_engine = engine
break
# If config says remote_api, turn the Advanced toggle on and pick a
# local fallback for the combo (so toggling off has somewhere to land).
remote_active = self.current_engine == "remote_api"
if remote_active:
self.use_remote_switch.set_active(True)
local_fallback = "whisper_cpp"
if not available_engines.get(local_fallback, False):
for engine in ENGINE_MODELS.keys():
if engine != "remote_api" and available_engines.get(engine, False):
local_fallback = engine
break
self.current_engine = local_fallback
else:
self.use_remote_switch.set_active(False)
if not available_engines.get(self.current_engine, False):
logger.warning(
f"Current engine '{self.current_engine}' is not available, "
"selecting first available"
)
for engine in ENGINE_MODELS.keys():
if engine != "remote_api" and available_engines.get(engine, False):
self.current_engine = engine
break
engine_text = self.current_engine.capitalize()
engine_text = _engine_display_name(self.current_engine)
logger.info(f"Setting active engine to: {engine_text}")
if not self.engine_combo.set_active_id(engine_text):
logger.warning("Could not set engine by ID, trying by index")
# Find index of current engine
model = self.engine_combo.get_model()
for i, row in enumerate(model):
if row[0].lower() == self.current_engine.lower():
if _engine_from_display(row[0]) == self.current_engine:
self.engine_combo.set_active(i)
break
else:
@@ -1926,9 +2180,14 @@ class SettingsDialog(Gtk.Dialog):
logger.warning("No engine selected during model options population")
return
engine = engine_text.lower()
engine = _engine_from_display(engine_text)
logger.info(f"Populating model options for engine: {engine}")
# Remote API does not need model options
if engine == "remote_api":
logger.info("Remote API engine selected, no model options needed")
return
saved_model_for_engine = self.config_manager.get_model_size_for_engine(engine)
logger.info(f"Saved model for {engine}: {saved_model_for_engine}")
@@ -2001,7 +2260,7 @@ class SettingsDialog(Gtk.Dialog):
if not engine_text:
return
engine = engine_text.lower()
engine = _engine_from_display(engine_text)
current_lang = self.language_combo.get_active_id()
if current_lang:
@@ -2009,7 +2268,7 @@ class SettingsDialog(Gtk.Dialog):
current_lang == "auto" or not SUPPORTED_LANGUAGES.get(current_lang, {}).get("vosk")
):
self.language = "en-us"
elif engine in ["whisper", "whisper_cpp"] and not current_lang:
elif engine in ["whisper", "whisper_cpp", "remote_api"] and not current_lang:
self.language = "auto"
self._populate_model_options()
@@ -2026,7 +2285,7 @@ class SettingsDialog(Gtk.Dialog):
if voice_commands_enabled is None:
engine_text = self.engine_combo.get_active_text()
engine = engine_text.lower() if engine_text else sr_config.get("engine", "whisper_cpp")
engine = _engine_from_display(engine_text) if engine_text else "whisper_cpp"
auto_enabled = engine == "vosk"
self.voice_commands_switch.set_active(auto_enabled)
@@ -2070,7 +2329,7 @@ class SettingsDialog(Gtk.Dialog):
if not engine:
return
engine = engine.lower()
engine = _engine_from_display(engine)
for lang_code, lang_info in SUPPORTED_LANGUAGES.items():
display_text = lang_info["name"]
@@ -2081,7 +2340,7 @@ class SettingsDialog(Gtk.Dialog):
continue
is_downloaded = _is_vosk_model_downloaded("small", lang_code)
display_text += " ✓" if is_downloaded else " ↓"
elif engine in ["whisper", "whisper_cpp"]:
elif engine in ["whisper", "whisper_cpp", "remote_api"]:
# Both Whisper and whisper.cpp support auto-detect
if lang_code == "auto":
display_text += " ⚠"
@@ -2105,7 +2364,7 @@ class SettingsDialog(Gtk.Dialog):
self._processing_language_change = True
try:
engine = engine.lower()
engine = _engine_from_display(engine)
lang_info = SUPPORTED_LANGUAGES.get(lang_code, {})
if lang_info.get("warning"):
@@ -2124,7 +2383,26 @@ class SettingsDialog(Gtk.Dialog):
self._processing_language_change = False
def _update_engine_specific_ui(self):
"""Show/hide UI elements specific to the selected engine."""
"""Show/hide UI elements driven by the active engine.
The engine combo is local-only (vosk/whisper/whisper_cpp). The remote
engine is enabled via the Advanced tab's "Use remote server" switch,
which overrides the local selection.
"""
is_remote = (
getattr(self, "use_remote_switch", None) is not None
and self.use_remote_switch.get_active()
)
# When remote is active, the local model/legend UI is irrelevant
if is_remote:
self.model_row.hide()
self.model_info_card.hide()
self.legend_box.hide()
else:
self.model_row.show_all()
self.legend_box.show_all()
self._update_model_info()
self._update_advanced_tab_sensitivity()
@@ -2161,7 +2439,16 @@ class SettingsDialog(Gtk.Dialog):
self.model_info_card.hide()
return
engine = engine_text.lower()
# Remote engine override (Advanced tab) hides the local model info card.
if (
getattr(self, "use_remote_switch", None) is not None
and self.use_remote_switch.get_active()
):
self.model_info_card.hide()
return
engine = _engine_from_display(engine_text)
model_id = self.model_combo.get_active_id()
if not model_id:
self.model_info_card.hide()
@@ -2341,14 +2628,18 @@ class SettingsDialog(Gtk.Dialog):
model_id = self.model_combo.get_active_id()
language_id = self.language_combo.get_active_id()
engine = engine_text.lower() if engine_text else "vosk"
# Advanced "Use remote server" toggle overrides the local engine combo.
if self.use_remote_switch.get_active():
engine = "remote_api"
else:
engine = _engine_from_display(engine_text) if engine_text else "vosk"
model_size = model_id.lower() if model_id else "small"
language = language_id if language_id else "auto"
vad = int(self.vad_spin.get_value())
silence = self.silence_spin.get_value()
return {
settings = {
"engine": engine,
"model_size": model_size,
"language": language,
@@ -2368,6 +2659,16 @@ class SettingsDialog(Gtk.Dialog):
"whispercpp_no_speech_thold": self.advanced_no_speech_thold_spin.get_value(),
}
# Remote API additional settings
if engine == "remote_api":
settings["remote_api_url"] = self.remote_api_url_entry.get_text().strip()
settings["remote_api_key"] = self.remote_api_key_entry.get_text().strip()
settings["remote_api_endpoint"] = (
self.remote_api_endpoint_combo.get_active_id() or "/inference"
)
return settings
def _on_test_clicked(self, widget):
"""Handle click on the test button."""
if self._test_active: