NVIDIA Nemotron 3.5 Transcribe and Granary v2: Intelligent Multilingual Transcription You Can Prompt and Adapt

Introduction

NVIDIA has released Nemotron 3.5 Transcribe, an open‑weight, prompt‑driven speech model that unifies transcription, speaker attribution (diarization), translation, and audio question answering in a single checkpoint. Available on Hugging Face and ModelScope, the model supports 40 languages for transcription, 31 for translation, and speaker labeling for up to eight speakers in 13 languages. Coupled with the Granary v2 multilingual data pipeline and NeMo‑Speech fine‑tuning tools, developers can adapt the system to specialized vocabularies, accents, and domain conventions without building a new model from scratch.

---

1. What the Model Does Out‑of‑the‑Box

• Transcription – Generates word‑level timestamps and supports three transcript styles (verbatim, cleaned, and inverse‑text‑normalized) that can be selected via natural‑language prompts.
• Speaker Attribution – Integrated diarization tags each utterance with a speaker label during generation, handling overlapping speech and up to eight speakers.
• Translation – One‑step translation between any of the 31 supported language pairs and English, useful for multilingual meetings or cross‑border calls.
• Audio Q&A – The same checkpoint can answer questions about the audio content (e.g., “What decisions were made?”) when prompted in English.

Developers can steer the output with simple instructions such as: ``` Transcribe this recording and label each speaker. Translate this audio into English. What were the key action items? ``` The model also respects domain‑specific term lists supplied in the prompt, making it practical for medical, legal, or technical use cases.

---

2. Performance Highlights

• Open‑ASR Leaderboard (English) – 5.33 % average WER, the best among open‑weight models.
• Multilingual WER – 4.62 % mean WER across seven languages, again the top open‑weight result.
• Long‑Form Audio – 10.98 % mean WER for recordings up to an hour.
• Diarization – Lowest diarization error rate (18.26 %) on David AI’s 21‑language benchmark, and best cpWER on VoiceArena’s full‑session test.
• Efficiency – Approximately 4 B active parameters per token (out of 30 B total) thanks to a hybrid Mamba‑Transformer MoE, placing the model on the accuracy‑vs‑active‑parameter Pareto frontier.

These numbers demonstrate that Nemotron 3.5 Transcribe delivers high accuracy while remaining lightweight enough for deployment on RTX 6000/8000 GPUs, DGX systems, or Jetson AGX devices (quantized versions are forthcoming).

---

3. Granary v2: An Open Data Pipeline for Customization

Granary v2 supplies a curated, multilingual speech corpus (463 k hours across 48 languages) and a reusable four‑stage pipeline:

1. Audio Preparation – Segments raw audio, tracks speakers, and annotates language and acoustic events.
2. Transcription – Routes each segment to primary and fallback ASR models, applying hallucination checks.
3. Task Label Generation – Produces multiple transcript variants and optional translations, biasing terms, and Q&A pairs.
4. High‑Fidelity Filtering – Removes noisy or multi‑speaker segments, yielding a high‑quality subset for fine‑tuning.

The public manifests expose three transcript forms (`tnraw`, `itnraw`, `an`) that match the model’s promptable output styles. Developers can run the pipeline on their own data using NeMo Curator with Ray or NVIDIA Dynamo backends, then fine‑tune Nemotron 3.5 Transcribe to improve word‑error‑rate on domain‑specific speakers—e.g., a medical case study reported a 45.8 % relative WER reduction on held‑out speakers.

---

4. Practical Deployment Tips

• Prompt Design – Start with clear, concise instructions; include any required term lists or formatting preferences directly in the prompt.
• Batch Processing – For large archives, use the vLLM server or NeMo‑Speech.cpp (when quantized) to achieve throughput of ~100 hours of audio in under three minutes on a single Blackwell GPU.
• Local vs. Cloud – Quantized checkpoints enable edge deployment on RTX Spark or Jetson devices; cloud users can leverage NVIDIA’s inference endpoints for scalable batch jobs.
• Quality Assurance – Spot‑check speaker changes, proper handling of names, and answer relevance for audio Q&A; the model’s timestamps make verification straightforward.

---

Conclusion

Nemotron 3.5 Transcribe, together with the Granary v2 data pipeline, provides a versatile, open‑weight solution for multilingual speech processing that rivals proprietary APIs in accuracy while offering full promptability and fine‑tuning control. Whether you need fast transcription of global meetings, speaker‑aware captions for educational content, or domain‑specific medical dictation, the model’s flexible interface and robust data foundation make it a compelling choice for developers looking to build adaptable speech‑centric applications.