feat(asr): FunASR/SenseVoice support for the Remote API engine (#468)
* feat(asr): add chat audio remote benchmarking * style(asr): apply black formatting * test(main): include remote API model kwarg * test(asr): add coverage for chat-completions error handling and parsing Add 10 new tests to cover uncovered lines in PR #468: - ConnectionError handler in _try_chat_completions_audio_api (lines 1601-1602) - _parse_chat_completion_transcription edge cases: None, list content, empty/whitespace strings, JSON without text field, plain text, and single-line language metadata (lines 1610, 1613-1621, 1625, 1629-1640) Coverage for recognition_manager.py improved from 85% to 86%. * feat(asr): support FunASR remote responses * refactor(asr): scope remote FunASR support to the #469 path Drop chat-completions/Qwen client, evaluation notes, and the remote ASR benchmark harness. Keep model field, tolerant OpenAI response parsing, SenseVoice label stripping, FunASR docs, and tests.
This commit is contained in:
Jatin K Malik
@@ -48,6 +48,7 @@ No internet required. No data leaves your machine. Just speak and type.
|
||||
### ✨ New Features
|
||||
|
||||
- **Guided whisper.cpp model variants** — The Settings dialog now splits whisper.cpp selection into **Model Size** and **Specialization**, exposing English-only, quantized (Q5/Q8), Large v3 Turbo, and legacy large models with language-aware recommendations and hover guidance. Exact model IDs (e.g. `medium.en-q5_0`, `large-v3-turbo`) can also be passed to `--model` (#465)
|
||||
- **Remote API FunASR/SenseVoice support** — OpenAI-compatible remote servers can send a model name (e.g. `sensevoice`) and return richer response shapes; SenseVoice metadata labels are stripped before text injection (#468, #469)
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
|
||||
+78
-12
@@ -4,12 +4,12 @@ Vocalinux can offload speech recognition to a remote HTTP server instead of runn
|
||||
|
||||
Vocalinux speaks two wire formats out of the box:
|
||||
|
||||
- **OpenAI-compatible** — `POST /v1/audio/transcriptions` (e.g. OpenAI, [Speaches](https://github.com/speaches-ai/speaches), LocalAI, vLLM with Whisper)
|
||||
- **OpenAI-compatible** — `POST /v1/audio/transcriptions` (e.g. OpenAI, [Speaches](https://github.com/speaches-ai/speaches), LocalAI, FunASR/SenseVoice)
|
||||
- **whisper.cpp server** — `POST /inference` (the binary shipped with [whisper.cpp](https://github.com/ggerganov/whisper.cpp))
|
||||
|
||||
Pick whichever your server exposes — the rest of this guide applies to both.
|
||||
|
||||
> **Tip — share the server with your phone.** Both formats are open standards, so the same self-hosted server can also back mobile dictation apps. On Android, apps like *Dictate* and *Transcribro* speak OpenAI-compatible Whisper; on iOS, Shortcuts-based dictation clients can hit the same endpoint. Run one Whisper server, point your laptop and phone at it, and you get consistent dictation everywhere without uploading audio to a third party.
|
||||
> **Tip — share the server with your phone.** These HTTP formats are open enough that the same self-hosted server can also back mobile dictation apps. On Android, apps like *Dictate* and *Transcribro* speak OpenAI-compatible Whisper; on iOS, Shortcuts-based dictation clients can hit the same endpoint. Run one Whisper server, point your laptop and phone at it, and you get consistent dictation everywhere without uploading audio to a third party.
|
||||
|
||||
## How It Works
|
||||
|
||||
@@ -17,14 +17,14 @@ When the **Remote API** engine is active, Vocalinux:
|
||||
|
||||
1. Captures microphone audio locally (16 kHz, mono, 16-bit PCM).
|
||||
2. On end-of-utterance, packages the buffer as a WAV file in memory.
|
||||
3. Uploads it via `multipart/form-data` to the configured server.
|
||||
4. Reads the transcribed text from the JSON response (`{"text": "..."}`) and types it into the focused window.
|
||||
3. Uploads it using the configured HTTP format.
|
||||
4. Reads the transcribed text from the JSON response and types it into the focused window.
|
||||
|
||||
No audio is ever written to disk. The local machine still runs the VAD/segmentation logic — only the heavy ASR step is remote.
|
||||
|
||||
## Server-Side Setup
|
||||
|
||||
You need a server that accepts audio uploads and returns `{"text": "..."}`. Two common options:
|
||||
You need a server that accepts audio uploads and returns text. Three common setups:
|
||||
|
||||
### Option A: whisper.cpp server
|
||||
|
||||
@@ -53,6 +53,29 @@ docker run --rm -p 8000:8000 ghcr.io/speaches-ai/speaches:latest
|
||||
|
||||
The endpoint is `/v1/audio/transcriptions`.
|
||||
|
||||
### Option C: FunASR/SenseVoice local server
|
||||
|
||||
[FunASR](https://github.com/modelscope/FunASR) can run a local
|
||||
OpenAI-compatible transcription service, which keeps audio on your own machine
|
||||
or LAN while letting Vocalinux use SenseVoice or Paraformer-family models:
|
||||
|
||||
```bash
|
||||
pip install torch torchaudio
|
||||
pip install funasr vllm fastapi uvicorn python-multipart
|
||||
funasr-server --device cuda
|
||||
```
|
||||
|
||||
The server listens on `http://localhost:8000` by default and exposes
|
||||
`POST /v1/audio/transcriptions`. In Vocalinux, set:
|
||||
|
||||
- **Server URL**: `http://localhost:8000`
|
||||
- **API Endpoint**: `OpenAI/FunASR (/v1/audio/transcriptions)`
|
||||
- **Model**: `sensevoice`
|
||||
|
||||
Use `--device cpu` for CPU-only testing if your FunASR install supports it. A
|
||||
remote GPU host works too: bind the server to the LAN interface, open the port,
|
||||
and point Vocalinux at that host.
|
||||
|
||||
### Network checklist
|
||||
|
||||
- The server must be reachable from the client over HTTP or HTTPS.
|
||||
@@ -64,14 +87,15 @@ The endpoint is `/v1/audio/transcriptions`.
|
||||
|
||||
### Via the Settings dialog
|
||||
|
||||
The remote engine is a power-user override and lives on the **Advanced** tab so casual users aren't confronted with it.
|
||||
The remote engine is a power-user option in the **Speech Engine** settings.
|
||||
|
||||
1. Launch Vocalinux and open **Settings → Advanced → Remote Server**.
|
||||
2. Toggle **Use remote server** on. This overrides whatever local engine is selected on the Speech Engine tab.
|
||||
1. Launch Vocalinux and open **Settings → Speech Engine → Remote Server**.
|
||||
2. Toggle **Use remote server** on. This overrides whatever local engine is selected above it.
|
||||
3. Fill in the fields:
|
||||
- **Server URL**: base URL of the server, e.g. `http://192.168.1.100:8080` (no trailing slash needed; one is stripped automatically).
|
||||
- **API Key** (optional): sent as `Authorization: Bearer <key>`. Leave blank if your server doesn't require auth.
|
||||
- **API Endpoint**: pick **Whisper.cpp (`/inference`)** or **OpenAI (`/v1/audio/transcriptions`)** to match your server.
|
||||
- **API Endpoint**: pick **Whisper.cpp (`/inference`)** or **OpenAI/FunASR (`/v1/audio/transcriptions`)** to match your server.
|
||||
- **Model**: model identifier sent to OpenAI-compatible servers. Use `whisper-1` for classic Whisper servers, or `sensevoice` for FunASR/SenseVoice.
|
||||
4. Click **Test Connection** — a successful test means the URL is reachable and credentials (if any) are accepted. A failure here is just a warning; Vocalinux will still try again on the first transcription.
|
||||
|
||||
Settings auto-save and re-initialise the engine immediately, so you can start dictating as soon as the test passes. Toggling the switch off restores the local engine selected on the Speech Engine tab.
|
||||
@@ -86,13 +110,28 @@ The same options live in `~/.config/vocalinux/config.json`:
|
||||
"engine": "remote_api",
|
||||
"remote_api_url": "http://192.168.1.100:8080",
|
||||
"remote_api_key": "",
|
||||
"remote_api_endpoint": "/inference"
|
||||
"remote_api_endpoint": "/inference",
|
||||
"remote_api_model": "whisper-1"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Restart Vocalinux after editing the file by hand.
|
||||
|
||||
FunASR/SenseVoice example:
|
||||
|
||||
```json
|
||||
{
|
||||
"speech_recognition": {
|
||||
"engine": "remote_api",
|
||||
"remote_api_url": "http://localhost:8000",
|
||||
"remote_api_key": "",
|
||||
"remote_api_endpoint": "/v1/audio/transcriptions",
|
||||
"remote_api_model": "sensevoice"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Wire Protocol Reference
|
||||
|
||||
If you're writing a custom server, here's exactly what the client sends.
|
||||
@@ -151,12 +190,31 @@ en
|
||||
--boundary--
|
||||
```
|
||||
|
||||
### Response (both formats)
|
||||
### Response (`/inference` and `/v1/audio/transcriptions`)
|
||||
|
||||
```json
|
||||
{ "text": "the transcribed utterance" }
|
||||
```
|
||||
|
||||
For OpenAI-compatible FunASR/SenseVoice wrappers, extra metadata is allowed and
|
||||
ignored by Vocalinux:
|
||||
|
||||
```json
|
||||
{
|
||||
"text": "the transcribed utterance",
|
||||
"language": "en",
|
||||
"emotion": "neutral",
|
||||
"segments": [
|
||||
{ "text": "the transcribed utterance", "start": 0.0, "end": 1.4 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
If `text` is not present, Vocalinux also accepts `transcript`, `transcription`,
|
||||
or a `segments` list with per-segment `text` fields. If a SenseVoice wrapper
|
||||
returns rich labels inline, for example `<|en|><|NEUTRAL|><|Speech|>hello`,
|
||||
Vocalinux strips the leading labels before text injection.
|
||||
|
||||
Empty utterances should return `{"text": ""}`. HTTP 4xx/5xx responses are surfaced as transcription errors in the Vocalinux log.
|
||||
|
||||
### Language handling
|
||||
@@ -188,9 +246,15 @@ curl -H "Authorization: Bearer $API_KEY" \
|
||||
-F file=@sample.wav \
|
||||
-F model=whisper-1 \
|
||||
http://192.168.1.100:8000/v1/audio/transcriptions
|
||||
|
||||
# FunASR/SenseVoice
|
||||
curl -F file=@sample.wav \
|
||||
-F model=sensevoice \
|
||||
http://localhost:8000/v1/audio/transcriptions
|
||||
```
|
||||
|
||||
Both should return JSON with a `text` field.
|
||||
The multipart endpoints should return JSON with a text field or one of the
|
||||
compatible response shapes above.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -199,6 +263,8 @@ Both should return JSON with a `text` field.
|
||||
| Connection test reports "failed" | Wrong host/port, firewall, or server not running | Check `curl <url>` from the client; check server logs |
|
||||
| HTTP 401/403 | API key missing or wrong | Set the **API Key** field; ensure the server is configured for that key |
|
||||
| HTTP 404 on first send | Endpoint format mismatch | Switch the **API Endpoint** dropdown to the format your server actually exposes |
|
||||
| FunASR/SenseVoice server returns model errors | Model field does not match the served model | Try `sensevoice`, `paraformer`, or the exact model configured on the server |
|
||||
| SenseVoice labels appear in injected text | Wrapper returned labels somewhere other than the leading transcript prefix | Return plain text in `text` and put language/emotion/speech labels in JSON metadata |
|
||||
| Transcription always empty | Server rejected the WAV (sample rate, channels) | Confirm server accepts 16 kHz mono 16-bit WAV; check server logs |
|
||||
| First transcription is slow | Cold model load on the server | Warm the server before dictating, or increase the client timeout server-side |
|
||||
| Garbled text in foreign language | Wrong `language` setting | Set the language explicitly in Settings instead of `auto` |
|
||||
|
||||
@@ -3,7 +3,8 @@
|
||||
Vocalinux Remote API Test Server
|
||||
|
||||
A simple mock server for testing Vocalinux's remote API feature.
|
||||
Supports both whisper.cpp (/inference) and OpenAI-compatible (/v1/audio/transcriptions) formats.
|
||||
Supports whisper.cpp (/inference) and OpenAI/FunASR-compatible
|
||||
(/v1/audio/transcriptions) formats.
|
||||
|
||||
Usage:
|
||||
python test_remote_server.py [--port PORT] [--delay SECONDS]
|
||||
@@ -17,8 +18,7 @@ Examples:
|
||||
import argparse
|
||||
import json
|
||||
import time
|
||||
from http.server import HTTPServer, BaseHTTPRequestHandler
|
||||
from urllib.parse import parse_qs
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
|
||||
|
||||
class RemoteAPITestHandler(BaseHTTPRequestHandler):
|
||||
@@ -40,7 +40,7 @@ class RemoteAPITestHandler(BaseHTTPRequestHandler):
|
||||
"message": "Vocalinux Remote API Test Server",
|
||||
"endpoints": [
|
||||
"/inference (whisper.cpp format)",
|
||||
"/v1/audio/transcriptions (OpenAI format)",
|
||||
"/v1/audio/transcriptions (OpenAI/FunASR format)",
|
||||
"/v1/models (OpenAI models list)",
|
||||
],
|
||||
}
|
||||
@@ -58,7 +58,13 @@ class RemoteAPITestHandler(BaseHTTPRequestHandler):
|
||||
"object": "model",
|
||||
"created": 1234567890,
|
||||
"owned_by": "openai",
|
||||
}
|
||||
},
|
||||
{
|
||||
"id": "sensevoice",
|
||||
"object": "model",
|
||||
"created": 1767225600,
|
||||
"owned_by": "mock",
|
||||
},
|
||||
],
|
||||
"object": "list",
|
||||
}
|
||||
@@ -113,7 +119,7 @@ class RemoteAPITestHandler(BaseHTTPRequestHandler):
|
||||
# whisper.cpp format
|
||||
response = {"text": transcription}
|
||||
else:
|
||||
# OpenAI format
|
||||
# OpenAI/FunASR-compatible format
|
||||
response = {"text": transcription}
|
||||
|
||||
self.wfile.write(json.dumps(response, indent=2).encode())
|
||||
@@ -187,7 +193,7 @@ def main():
|
||||
print(" GET /v1/models - OpenAI models list")
|
||||
print(" GET /inference - whisper.cpp health check")
|
||||
print(" POST /inference - whisper.cpp transcription")
|
||||
print(" POST /v1/audio/transcriptions - OpenAI transcription")
|
||||
print(" POST /v1/audio/transcriptions - OpenAI/FunASR transcription")
|
||||
print()
|
||||
print("Press Ctrl+C to stop")
|
||||
print("=" * 60)
|
||||
|
||||
@@ -392,6 +392,7 @@ def main():
|
||||
remote_api_url=saved_settings.get("remote_api_url", ""),
|
||||
remote_api_key=saved_settings.get("remote_api_key", ""),
|
||||
remote_api_endpoint=saved_settings.get("remote_api_endpoint", "/inference"),
|
||||
remote_api_model=saved_settings.get("remote_api_model", "whisper-1"),
|
||||
)
|
||||
|
||||
# Initialize text injection system
|
||||
|
||||
@@ -11,6 +11,7 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import queue
|
||||
import re
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
@@ -844,6 +845,7 @@ class SpeechRecognitionManager:
|
||||
self.remote_api_url = kwargs.get("remote_api_url", "")
|
||||
self.remote_api_key = kwargs.get("remote_api_key", "")
|
||||
self.remote_api_endpoint = kwargs.get("remote_api_endpoint", "/inference")
|
||||
self.remote_api_model = kwargs.get("remote_api_model", "whisper-1")
|
||||
self._http_session = None
|
||||
|
||||
# Audio diagnostics tracking
|
||||
@@ -1576,7 +1578,7 @@ class SpeechRecognitionManager:
|
||||
url = f"{self.remote_api_url}{self.remote_api_endpoint}"
|
||||
|
||||
files = {"file": ("audio.wav", wav_bytes, "audio/wav")}
|
||||
data = {"model": "whisper-1"}
|
||||
data = {"model": self.remote_api_model or "whisper-1"}
|
||||
if lang:
|
||||
data["language"] = lang
|
||||
|
||||
@@ -1590,8 +1592,7 @@ class SpeechRecognitionManager:
|
||||
response.raise_for_status()
|
||||
result = response.json()
|
||||
|
||||
# OpenAI format returns {"text": "..."}
|
||||
return result.get("text", "")
|
||||
return self._extract_remote_transcription_text(result)
|
||||
|
||||
except requests.exceptions.ConnectionError as e:
|
||||
logger.error(f"Cannot connect to remote server {url}: {e}")
|
||||
@@ -1600,6 +1601,43 @@ class SpeechRecognitionManager:
|
||||
logger.debug(f"OpenAI API format attempt failed: {e}")
|
||||
return None
|
||||
|
||||
def _extract_remote_transcription_text(self, payload: object) -> str:
|
||||
"""Extract transcript text from OpenAI-compatible ASR responses.
|
||||
|
||||
FunASR/SenseVoice wrappers may include extra metadata or segment arrays.
|
||||
Vocalinux only injects the transcript text and ignores the metadata.
|
||||
"""
|
||||
if payload is None:
|
||||
return ""
|
||||
|
||||
if isinstance(payload, str):
|
||||
return self._strip_remote_transcription_metadata(payload)
|
||||
|
||||
if isinstance(payload, list):
|
||||
parts = [self._extract_remote_transcription_text(item).strip() for item in payload]
|
||||
return "\n".join(part for part in parts if part)
|
||||
|
||||
if not isinstance(payload, dict):
|
||||
return self._strip_remote_transcription_metadata(str(payload))
|
||||
|
||||
for key in ("text", "transcript", "transcription"):
|
||||
value = payload.get(key)
|
||||
if isinstance(value, str):
|
||||
return self._strip_remote_transcription_metadata(value)
|
||||
|
||||
segments = payload.get("segments")
|
||||
if isinstance(segments, list):
|
||||
return self._extract_remote_transcription_text(segments)
|
||||
|
||||
return ""
|
||||
|
||||
def _strip_remote_transcription_metadata(self, text: str) -> str:
|
||||
"""Remove inline ASR metadata tags while preserving ordinary text."""
|
||||
if not text.strip():
|
||||
return ""
|
||||
|
||||
return re.sub(r"^\s*(?:<\|[^|>\n]+\|>)+\s*", "", text)
|
||||
|
||||
def _try_whispercpp_server_api(self, wav_bytes: bytes, lang, headers: dict, session):
|
||||
"""Try to transcribe using whisper.cpp server API format.
|
||||
|
||||
@@ -2825,6 +2863,8 @@ class SpeechRecognitionManager:
|
||||
self.remote_api_key = kwargs.get("remote_api_key", "")
|
||||
if "remote_api_endpoint" in kwargs:
|
||||
self.remote_api_endpoint = kwargs.get("remote_api_endpoint", "/inference")
|
||||
if "remote_api_model" in kwargs:
|
||||
self.remote_api_model = kwargs.get("remote_api_model", "whisper-1")
|
||||
|
||||
self._voice_commands_enabled = self._resolve_voice_commands_enabled()
|
||||
|
||||
|
||||
@@ -31,7 +31,8 @@ DEFAULT_CONFIG = {
|
||||
"voice_commands_enabled": None, # None = auto (enabled for VOSK, disabled for Whisper)
|
||||
"remote_api_url": "", # Remote speech recognition server URL (e.g. http://192.168.1.100:8080)
|
||||
"remote_api_key": "", # Remote server API key (optional)
|
||||
"remote_api_endpoint": "/inference", # Remote server API endpoint format (/v1/audio/transcriptions or /inference)
|
||||
"remote_api_endpoint": "/inference", # Remote server API endpoint format
|
||||
"remote_api_model": "whisper-1", # Model name sent to compatible remote APIs
|
||||
},
|
||||
"audio": {
|
||||
"device_index": None, # Audio input device index (None for system default)
|
||||
|
||||
@@ -1437,17 +1437,19 @@ class SettingsDialog(Gtk.Dialog):
|
||||
self.language_combo.connect("changed", self._on_language_changed)
|
||||
|
||||
def _on_remote_api_settings_changed(self, widget):
|
||||
"""Handle remote API URL/Key/endpoint changes (Advanced tab)."""
|
||||
"""Handle remote API URL/Key/endpoint changes."""
|
||||
if self._initializing or self._applying_settings:
|
||||
return
|
||||
|
||||
url = self.remote_api_url_entry.get_text().strip()
|
||||
key = self.remote_api_key_entry.get_text().strip()
|
||||
endpoint = self.remote_api_endpoint_combo.get_active_id() or "/inference"
|
||||
model = self.remote_api_model_entry.get_text().strip() or "whisper-1"
|
||||
|
||||
self.config_manager.set("speech_recognition", "remote_api_url", url)
|
||||
self.config_manager.set("speech_recognition", "remote_api_key", key)
|
||||
self.config_manager.set("speech_recognition", "remote_api_endpoint", endpoint)
|
||||
self.config_manager.set("speech_recognition", "remote_api_model", model)
|
||||
self.config_manager.save_settings()
|
||||
|
||||
# Auto apply only if remote engine is currently active
|
||||
@@ -2252,7 +2254,7 @@ class SettingsDialog(Gtk.Dialog):
|
||||
"Select the API format of the remote server (API Endpoint Format)"
|
||||
)
|
||||
self.remote_api_endpoint_combo.append(
|
||||
"/v1/audio/transcriptions", "OpenAI (/v1/audio/transcriptions)"
|
||||
"/v1/audio/transcriptions", "OpenAI/FunASR (/v1/audio/transcriptions)"
|
||||
)
|
||||
self.remote_api_endpoint_combo.append("/inference", "Whisper.cpp (/inference)")
|
||||
_prevent_scroll_on_hover(self.remote_api_endpoint_combo)
|
||||
@@ -2263,6 +2265,20 @@ class SettingsDialog(Gtk.Dialog):
|
||||
)
|
||||
self.remote_server_group.add_row(remote_endpoint_row)
|
||||
|
||||
# Model name
|
||||
self.remote_api_model_entry = Gtk.Entry()
|
||||
self.remote_api_model_entry.set_placeholder_text("whisper-1")
|
||||
self.remote_api_model_entry.set_tooltip_text(
|
||||
"Model identifier sent to OpenAI-compatible and FunASR servers"
|
||||
)
|
||||
self.remote_api_model_entry.set_size_request(280, -1)
|
||||
remote_model_row = PreferenceRow(
|
||||
title="Model",
|
||||
subtitle="Remote model name, for example whisper-1 or sensevoice",
|
||||
widget=self.remote_api_model_entry,
|
||||
)
|
||||
self.remote_server_group.add_row(remote_model_row)
|
||||
|
||||
# Connection test
|
||||
self.remote_test_btn = Gtk.Button(label="Test Connection")
|
||||
self.remote_test_btn.set_tooltip_text("Test connection to remote server")
|
||||
@@ -2289,15 +2305,18 @@ class SettingsDialog(Gtk.Dialog):
|
||||
saved_endpoint = self.config_manager.get(
|
||||
"speech_recognition", "remote_api_endpoint", "/inference"
|
||||
)
|
||||
saved_model = self.config_manager.get("speech_recognition", "remote_api_model", "whisper-1")
|
||||
if saved_url:
|
||||
self.remote_api_url_entry.set_text(saved_url)
|
||||
if saved_key:
|
||||
self.remote_api_key_entry.set_text(saved_key)
|
||||
self.remote_api_endpoint_combo.set_active_id(saved_endpoint)
|
||||
self.remote_api_model_entry.set_text(saved_model or "whisper-1")
|
||||
|
||||
self.remote_api_url_entry.connect("changed", self._on_remote_api_settings_changed)
|
||||
self.remote_api_key_entry.connect("changed", self._on_remote_api_settings_changed)
|
||||
self.remote_api_endpoint_combo.connect("changed", self._on_remote_api_settings_changed)
|
||||
self.remote_api_model_entry.connect("changed", self._on_remote_api_settings_changed)
|
||||
|
||||
self.remote_server_group.hide()
|
||||
self.remote_status_label.hide()
|
||||
@@ -3217,6 +3236,9 @@ class SettingsDialog(Gtk.Dialog):
|
||||
settings["remote_api_endpoint"] = (
|
||||
self.remote_api_endpoint_combo.get_active_id() or "/inference"
|
||||
)
|
||||
settings["remote_api_model"] = (
|
||||
self.remote_api_model_entry.get_text().strip() or "whisper-1"
|
||||
)
|
||||
|
||||
return settings
|
||||
|
||||
|
||||
@@ -175,6 +175,10 @@ class TestConfigManager(unittest.TestCase):
|
||||
config_manager = ConfigManager()
|
||||
value = config_manager.get("speech_recognition", "engine")
|
||||
self.assertIn(value, ["vosk", "whisper", "whisper_cpp"]) # Should be one of valid engines
|
||||
self.assertEqual(
|
||||
config_manager.get("speech_recognition", "remote_api_model"),
|
||||
"whisper-1",
|
||||
)
|
||||
|
||||
def test_get_nonexistent_value(self):
|
||||
"""Test getting a nonexistent configuration value."""
|
||||
|
||||
@@ -244,6 +244,7 @@ class TestMainModule(unittest.TestCase):
|
||||
remote_api_url="",
|
||||
remote_api_key="",
|
||||
remote_api_endpoint="/inference",
|
||||
remote_api_model="whisper-1",
|
||||
)
|
||||
mock_text.assert_called_once_with(wayland_mode=True)
|
||||
mock_action_handler.assert_called_once_with(mock_text_instance)
|
||||
|
||||
@@ -232,6 +232,19 @@ class TestRemoteAPIEngine(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(manager.remote_api_endpoint, "/v1/audio/transcriptions")
|
||||
|
||||
def test_remote_api_init_with_model_name(self):
|
||||
"""Test remote API initialization stores a configurable model name."""
|
||||
SpeechRecognitionManager = _import_manager()
|
||||
_setup_requests_get_ok()
|
||||
|
||||
manager = SpeechRecognitionManager(
|
||||
engine="remote_api",
|
||||
remote_api_url="http://localhost:8000",
|
||||
remote_api_endpoint="/v1/audio/transcriptions",
|
||||
remote_api_model="sensevoice",
|
||||
)
|
||||
self.assertEqual(manager.remote_api_model, "sensevoice")
|
||||
|
||||
def test_model_ready_remote_api_initialized(self):
|
||||
"""Test model_ready returns True for remote API when initialized."""
|
||||
SpeechRecognitionManager = _import_manager()
|
||||
@@ -432,6 +445,55 @@ class TestOpenAIAPIFormat(unittest.TestCase):
|
||||
call_kwargs = mock_post.call_args[1]
|
||||
self.assertEqual(call_kwargs["data"]["language"], "fr")
|
||||
|
||||
def test_try_openai_api_uses_configured_model(self):
|
||||
"""Test OpenAI API sends the configured model name."""
|
||||
mock_post = _setup_requests_post_ok({"text": "result"})
|
||||
self.manager.remote_api_model = "custom-asr-model"
|
||||
|
||||
self.manager._try_openai_api(b"wav-bytes", "en", {}, self.manager._http_session)
|
||||
|
||||
call_kwargs = mock_post.call_args[1]
|
||||
self.assertEqual(call_kwargs["data"]["model"], "custom-asr-model")
|
||||
|
||||
def test_try_openai_api_accepts_funasr_metadata_response(self):
|
||||
"""Test OpenAI-compatible FunASR/SenseVoice responses with metadata."""
|
||||
_setup_requests_post_ok(
|
||||
{
|
||||
"text": "hello from sensevoice",
|
||||
"language": "en",
|
||||
"emotion": "neutral",
|
||||
"duration": 1.2,
|
||||
}
|
||||
)
|
||||
|
||||
result = self.manager._try_openai_api(b"wav-bytes", "en", {}, self.manager._http_session)
|
||||
|
||||
self.assertEqual(result, "hello from sensevoice")
|
||||
|
||||
def test_try_openai_api_strips_sensevoice_inline_labels(self):
|
||||
"""Test SenseVoice rich-text labels are not injected into the desktop."""
|
||||
_setup_requests_post_ok({"text": "<|en|><|NEUTRAL|><|Speech|><|withitn|>hello"})
|
||||
|
||||
result = self.manager._try_openai_api(b"wav-bytes", "en", {}, self.manager._http_session)
|
||||
|
||||
self.assertEqual(result, "hello")
|
||||
|
||||
def test_try_openai_api_accepts_segmented_response(self):
|
||||
"""Test OpenAI-compatible responses that include transcript segments."""
|
||||
_setup_requests_post_ok(
|
||||
{
|
||||
"segments": [
|
||||
{"text": "first part"},
|
||||
{"text": "second part"},
|
||||
],
|
||||
"metadata": {"model": "sensevoice"},
|
||||
}
|
||||
)
|
||||
|
||||
result = self.manager._try_openai_api(b"wav-bytes", "en", {}, self.manager._http_session)
|
||||
|
||||
self.assertEqual(result, "first part\nsecond part")
|
||||
|
||||
def test_try_openai_api_server_error(self):
|
||||
"""Test OpenAI API handles 500 server error via raise_for_status."""
|
||||
_setup_requests_post_status(500, Exception("500 Server Error"))
|
||||
@@ -549,6 +611,11 @@ class TestRemoteAPIReconfiguration(unittest.TestCase):
|
||||
self.manager.reconfigure(remote_api_endpoint="/v1/audio/transcriptions")
|
||||
self.assertEqual(self.manager.remote_api_endpoint, "/v1/audio/transcriptions")
|
||||
|
||||
def test_update_remote_api_model(self):
|
||||
"""Test updating remote API model via reconfigure."""
|
||||
self.manager.reconfigure(remote_api_model="sensevoice")
|
||||
self.assertEqual(self.manager.remote_api_model, "sensevoice")
|
||||
|
||||
|
||||
class TestRemoteAPIReinitializeAfterResume(unittest.TestCase):
|
||||
"""Test cases for reinitialize_after_resume with remote_api engine."""
|
||||
|
||||
@@ -512,6 +512,8 @@ class TestSettingsDialogInstantApply(unittest.TestCase):
|
||||
|
||||
self.assertIn("self.content_box.pack_start(self.remote_server_group", source_code)
|
||||
self.assertIn("self.content_box.pack_start(self.remote_status_label", source_code)
|
||||
self.assertIn("self.remote_api_model_entry", source_code)
|
||||
self.assertIn("OpenAI/FunASR", source_code)
|
||||
self.assertNotIn("advanced_tab.pack_start(self.remote_server_group", source_code)
|
||||
self.assertNotIn("advanced_tab.pack_start(self.remote_status_label", source_code)
|
||||
self.assertNotIn("self.use_remote_switch", source_code)
|
||||
|
||||
Reference in New Issue
Block a user