AnySpeechRecognizer and AnyDecisionModel Bring Open-Source Speech and Decision Models to Apple Platforms

The Hugging Face blog has introduced AnySpeechRecognizer and AnyDecisionModel, two open-source Swift packages that bring flexible, on-device speech recognition and decision-making to Apple platforms. These tools let developers choose from multiple recognition engines — including NVIDIA's Parakeet and Alibaba's Qwen3-ASR — and run them locally on Core ML, MLX, or llama.cpp, without sending audio or data to a server.

What AnySpeechRecognizer and AnyDecisionModel Do

AnySpeechRecognizer implements Apple's new speech-to-text API (SpeechAnalyzer, introduced in macOS 26 and iOS 26) but lets developers swap the underlying engine. By default it uses Apple's Speech framework, but changing a single import lets you switch to NVIDIA's Parakeet or Alibaba's Qwen3-ASR. The package mirrors Apple's SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector, and AssetInventory classes, so existing code that uses Apple's API needs only an import change.

AnyDecisionModel answers yes-or-no, multiple-choice, and rating-scale questions with typed results and probabilities. Instead of generating text, it derives each answer from the model's probabilities, running on device with MLX, Core AI, or llama.cpp, or connecting to decision models from Jev or Ollama. Both packages can download open models from the Hugging Face Hub and run them locally.

Supported Models and Performance

AnySpeechRecognizer supports three variants of NVIDIA's Parakeet: Parakeet v3 (TDT 0.6B v3), which recognizes 25 European languages and auto-detects the language; Parakeet Ultra, a post-trained version that was more accurate in most measurements; and Parakeet Unified, an English-only model. All three run on the package's own Core ML runtime, supporting file and live input, volatile results, word timing, and word confidence.

Qwen3-ASR from Alibaba comes in 0.6B and 1.7B sizes, recognizing 30 languages and 22 Chinese dialects including Mandarin, Cantonese, Japanese, and Korean — coverage Parakeet does not offer. It auto-detects the language or lets you set one, but returns only one final result per segment (up to 40 seconds) without volatile results, word timing, or confidence. It runs on the GPU and is slower than Parakeet, making it better suited to transcribing files than live input.

Performance tests on an M4 Max Mac Studio with macOS 27.0.1 showed Parakeet transcribing an hour of audio in about 20 to 40 seconds from a file, or about 1.5 to 2.5 minutes live, using about 100 MB of memory. Qwen3-ASR is much slower and needs a few gigabytes of memory. On AMI meeting recordings where people talk over each other, every engine tested had a word error rate above 23%.

Speaker Diarization and Extensibility

AnySpeechRecognizer adds speaker diarization — tracking who said what — which Apple's Speech framework does not offer. It runs NVIDIA's Streaming Sortformer on Core ML alongside an Apple or Parakeet transcriber on the same timeline, tracking up to four speakers even when they talk over each other. A SpeakerTimeline collects results and attributes each timed word to a speaker, so you can print a transcript by speaker turn.

To add another engine, developers implement SpeechRecognitionModel for its configuration and SpeechRecognitionExecutor for one module's session. The README includes a small provider that reports placeholder text to help get started.

Model Availability and Integration

All models are converted to Core ML and published on the Hugging Face Hub. Each repository records its source checkpoint and a checksum for every file, and the package pins to a fixed revision. Downloads happen only when your app asks for them: about 600 MB for each Parakeet, plus about 590 MB for the live encoder, 0.9 to 2.4 GB for Qwen3-ASR, and about 250 MB for Sortformer. Files go in the standard Hugging Face cache, so a Mac app outside the sandbox shares them with Python and other Hugging Face tools.

Why It Matters

These tools give developers the flexibility to choose the best engine for their use case — speed, accuracy, or language coverage — while keeping everything on device for privacy and low latency. For end users, this means future apps on Apple platforms will likely offer more accurate and versatile speech recognition and decision support, from transcribing meetings to routing customer support calls, all without sending data to the cloud.