NVIDIA Nemotron 3.5 Transcribe: Open-Weight Model Tops Multilingual Speech Benchmarks

NVIDIA has released Nemotron 3.5 Transcribe, an open-weight AI model that transcribes recorded speech, translates it, and identifies speakers — all from a single checkpoint. Available on Hugging Face and ModelScope, the model handles transcription in 40 languages, translation to or from English in 31 languages, and speaker attribution (diarization) in 13 languages. It can process recordings up to an hour long and label up to eight speakers. Developers guide the output with natural-language prompts, choose where their audio is processed, and adapt recognition to domain-specific vocabulary and accents.

Benchmark Performance: First Among Open-Weight Models

Nemotron 3.5 Transcribe ranks first among open-weight models on the Hugging Face Open ASR Leaderboard for English with a 5.33% average Word Error Rate (WER). On the multilingual track it scores 4.62% mean WER across seven languages, and on the long-form track it achieves 10.98% mean WER — the lowest among evaluated open-weight models in each case.

On the Artificial Analysis Speech-to-Text Leaderboard, Nemotron 3.5 Transcribe ties for first among open-weight models at 2.8% AA-WER with about 4 billion active parameters, versus 24 billion for the model it ties. On the held-out AA-AgentTalk voice-agent speech set, it scores 1.8% WER versus 2.3% for the next-best open-weight model. The model pairs its speech encoder with a hybrid Mamba–Transformer mixture-of-experts language model, so only about 4 billion of its 30 billion parameters are active per token. This places it on Artificial Analysis's open-weight Pareto frontier of accuracy versus active parameters, ahead of several widely used proprietary speech APIs.

Speaker Attribution: Lowest Error Rate Across 21 Languages

For speaker attribution, Nemotron 3.5 Transcribe posts the lowest diarization error rate on David AI's 21-language DAI-ASR-I18N leaderboard among the 14 systems tested. It also ranks first in VoiceArena's 14-system comparison. The model uses Sortformer speaker cues that reach the language model alongside acoustic features, allowing it to tag each turn during generation rather than relying solely on a later step to attach speakers to words.

Granary v2: Extending Prompt-Level Control to Training

Granary v2 is a multilingual data pipeline and fine-tuning recipe set, also available on Hugging Face and ModelScope, that extends prompt-level control to training. Developers can define the written form they need and adapt the model to the speech their applications encounter. In a medical case study, fine-tuning cut the word error rate (WER) on held-out speakers by 45.8% relative to the best prompt. The NeMo Speech fine-tuning tools support this workflow.

Practical Applications and Use Cases

The same checkpoint can turn one meeting recording into different outputs depending on the instruction: "Transcribe this recording and label each speaker," or "What decisions were made in this meeting?" For unfamiliar terms, developers can provide context before training. In a healthcare setting, for example, a prompt can specify: "Transcribe this medical discussion in the original language. Possible terms: HbA1c, metformin. Use these terms only when supported by the audio. Preserve standard medical spelling and write numbers in conventional written form."

Agora, a real-time voice and video platform, used word-level timestamps from Nemotron 3.5 Transcribe in a classroom demo: the model turns a recorded class into a transcript shown alongside the replay, so students can see what was said and when. Agora's blog walks through how they built it.

Availability and Deployment

Quantized versions of Nemotron 3.5 Transcribe are coming soon, allowing the model to run locally on NVIDIA RTX Spark, DGX Spark, or Jetson AGX Thor hardware, or scale up on Blackwell data-center GPUs. The model card includes recommended presets, including reasoning for audio questions, a full prompt guide, and a task-by-language matrix. Word-level timestamps are supported. Audio question answering is trained in English only.