NVIDIA Debuts Open Nemotron 3.5 Models for Embedding, Reranking, Parsing – Unite.AI

NVIDIA on October 21, 2026, expanded its Nemotron Retriever collection with new open, commercially available models for embedding, reranking, and document extraction: Nemotron 3.5 Embed in 8B BF16, 1B BF16, and 1B NVFP4 variants, Nemotron 3.5 Rerank 8B BF16, and Nemotron 3.5 Parse at 631 million parameters.

In an NVIDIA-authored announcement published on Hugging Face, the company said the release targets enterprise agent workflows, adding multimodal retrieval, relevance ranking, and document processing capabilities. NVIDIA framed retrieval quality as a factor in both the answers an agent produces and the LLM inference cost of a task, stating that when evidence comes back incomplete or irrelevant, agents run more searches and review more documents, burning additional inference tokens before finishing.

The four Embed and Rerank variants cover distinct deployment roles. Nemotron 3.5 Embed 8B BF16 is positioned for quality-focused search across text and visually rich documents. The 1B BF16 model condenses the 8B model’s multimodal retrieval into a smaller footprint for multimodal indexing and query serving. The 1B NVFP4 model applies NVFP4 quantization to both the text and image encoders, targeting large-scale document indexing and high-throughput retrieval on supported hardware. Nemotron 3.5 Rerank 8B BF16 scores query–document pairs jointly to sharpen relevance ordering across text and image candidates, selecting evidence for retrieval-augmented generation and agentic workflows. Retrieval spans document text as well as images such as tables, charts, infographics, datasheets, and schematics. NVIDIA evaluated text retrieval in 34 languages and multilingual visual retrieval on MIRACL Vision, which spans 18 languages, and KoViDoRe, noting the models may handle additional languages beyond those tested.

Agentic Retrieval Experiments

To measure how retrieval affects agent behavior, NVIDIA evaluated Nemotron 3.5 Embed and Rerank on ViDoRe v3, BRIGHT, and BrowseComp-Plus. ViDoRe v3 uses combined text and image inputs, while BRIGHT and BrowseComp-Plus use text-only inputs. The tested pipeline has an agent search a corpus with a search tool, review the returned evidence, and adjust its subsequent queries. The test matrix swapped out the search tool’s implementation across embedding-only and embedding-plus-reranking setups using different models, while the agent’s LLM, tools, task set, corpus, and execution policy stayed the same. The evaluation tracked whether the agent found the relevant information for each task, along with total agent inference tokens per task across all turns, counting both input and output tokens.

According to NVIDIA, inserting Nemotron 3.5 Rerank into the pipeline cut agentic token cost per query by roughly 14% alongside the 8B embed model and 13% alongside the 1B model, and retrieval accuracy improved in both configurations. Paired with the 8B embed model, the reranker posted about 0.9 nDCG@10 points more than Qwen3-VL-Reranker-8B at a comparable agentic token cost, NVIDIA reports.

Leaderboard Results

On ViDoRe v3.1, NVIDIA reports a 61.17 nDCG@10 for Nemotron 3.5 Embed 8B BF16, placing it first among dense retrieval models as of October 20, 2026, which the company describes as an 8.94-point improvement over the 52.23 scored by its Llama Nemotron Embed VL 1B model. The distilled 1B BF16 model posted 58.38 nDCG@10, holding 95.4% of the 8B teacher’s score by NVIDIA’s account. On MIRACL Vision, NVIDIA reports 72.6 nDCG@10 for the 8B model, 4.3 points above Qwen3-VL-Embedding-8B, and says both embed sizes beat the Qwen baselines on KoViDoRe for combined image-and-text inputs.

NVIDIA, together with the MTEB team, is launching a ViDoRe v3.1 reranking leaderboard covering image-only, text-only, and combined image-and-text inputs. NVIDIA reports the reranker scored 60.49 nDCG@10 on combined inputs against 59.17 for Qwen3-VL-Reranker-8B, topping that leaderboard as of October 20, 2026.

Throughput and Domain Fine-Tuning

For production deployment, NVIDIA reports the NVFP4 variant delivers 27.1% higher throughput than the 1B BF16 model on NVIDIA GB200, reaching 128.8 documents per second versus 101.3, while holding 99.85% of the BF16 model’s retrieval accuracy on ViDoRe v3 for combined image-and-text inputs. In the company’s testing on an RTX PRO 6000 Blackwell Server Edition, NVIDIA NIM served 6,031 query requests per second, about 2.9 times the 2,099 requests per second of a comparable vLLM configuration, with passage-processing throughput described as comparable.

NVIDIA also published a multimodal embedding fine-tuning recipe that walks through synthetic data generation, hard-negative mining, fine-tuning with NVIDIA NeMo AutoModel, and evaluation as a single workflow, so teams can build queries and relevance labels from their own document collections. NVIDIA’s own test of the recipe used 8,202 chart and infographic images with 8,591 synthetic queries generated by Qwen3.8-Flash-Next, focused on enumerative and open-ended queries with multiple relevant images. After one epoch of fine-tuning on 6,872 queries, the 1B BF16 model’s nDCG@10 rose from 37.34 to 44.12 on 1,719 held-out synthetic queries, which NVIDIA describes as an 18% relative gain; the 8B BF16 model moved from 41.44 to 42.75, a 3% relative gain.

Model Architecture and Training

For the 8B embed model, NVIDIA built on Ministral-3-8B-Instruct and initialized the language component from Nemotron-3-Embed-8B-BF16, then applied contrastive learning over image and text corpora to produce multimodal retrieval representations. The 1B models were distilled from the 8B model with cosine and mean squared error losses aligning the student’s embeddings to the teacher’s, and their language component was initialized from Nemotron-3-Embed-1B-BF16. Since the language component stays frozen during embedding training, NVIDIA says text-only embeddings should match those of the corresponding Nemotron 3 Embed models, so existing text embeddings and vector indexes can carry over.

Nemotron 3.5 Rerank shares the 8B embed model’s backbone architecture but operates as a cross-encoder, jointly encoding each query–document pair instead of embedding queries and documents separately for similarity scoring; a multilayer perceptron scoring head then outputs a scalar relevance logit. All of the models were trained on a mixture of publicly available and synthetic multilingual, multimodal data, with final released weights produced through checkpoint ensembling. NVIDIA says further training-data details are available in each model’s Hugging Face model card.

Nemotron 3.5 Parse

Nemotron 3.5 Parse converts document images into machine-readable text while preserving the structure that gives the content meaning. The model detects document elements, their positions, and reading order, then outputs tables and charts as Markdown, HTML tables, and structured blocks that developers can index, search, and process downstream. NVIDIA reports a ParseBench score of 75.20 for Parse at 10.5 pages per second on a single NVIDIA H100, placing it first among models under 1 billion parameters as of October 20, 2026. ParseBench tests table and chart extraction, content faithfulness, semantic formatting, and visual grounding.

Three inference modes let developers trade throughput for accuracy with the same model. A single pass reads each page once and posts 72.74 at 16.1 pages per second. A double pass revisits tables and charts flagged in the initial page parse, reaching 74.92 at 11.2 pages per second. A triple pass adds a detection step that finds additional regions for refinement, reaching 75.20 at 10.5 pages per second. NVIDIA describes all three operating points as Pareto optimal within its comparison, since none of the other compared configurations improves accuracy and throughput at the same time.

Training data for Parse spanned enterprise documents, scientific papers, forms, tables, charts, and multilingual text, broadened by a synthetic data engine, and NVIDIA reports strong performance across evaluations spanning 163 language and script combinations. For post-training, NVIDIA applied Group Relative Policy Optimization with custom rewards for correct structure and penalties for failures, which it says reduced repetitive generation loops and unclosed blocks, made tables more consistent, and improved adherence to chart schemas. NVIDIA also thanked LightOn, saying the shared open-source techniques were pivotal to improving the model.

Partner Evaluations and Availability

The announcement includes early evaluations from enterprise partners. For Elastic, the post says the Inference API in Elasticsearch exposes Nemotron Retriever embedding and reranking models through NVIDIA inference endpoints, with Nemotron 3 to be added through Elastic Inference Service soon and on-premises deployment support on the roadmap. Vectorize tested Nemotron 3.5 Embed inside its Hindsight Agent Memory and, according to the post, it took first place on two long-conversation memory benchmarks, posting 93.8% recall@10 on LoCoMo and 65.9% on BEAM 100K against seven competing embedding models, along with the highest MRR and recall@5. Perplexity’s evaluation put Nemotron 3.5 Rerank 8B at the highest average score on Perplexity’s private 72-task reranking benchmark, with strong results reported across all six domains. Turbopuffer, which tested the 8B embed model on an enterprise PDF search benchmark, reports about 5% more relevant results at Recall@20 than the next-best model, and now offers the models as a native embedding option.

Open weights, data, and training recipes are available on Hugging Face, with production deployment through NVIDIA NIM and through AI cloud and inference partners Baseten, DeepInfra, and FriendliAI. Embedding and reranking fine-tuning recipes are published in the NVIDIA NeMo Nemotron GitHub repository.