NVIDIA Nemotron 3.5 Super VL is now available in Amazon SageMaker JumpStart

NVIDIA Nemotron 3.5 Super VL Joins Amazon SageMaker JumpStart

Amazon SageMaker JumpStart now hosts NVIDIA’s latest vision‑language model, Nemotron 3.5 Super VL. The offering gives developers a turnkey path to deploy a 120‑billion‑parameter multimodal model that can ingest images, video, and text and generate natural‑language responses. Two model checkpoints are provided: a quant‑aware distilled NVFP4 version optimized for latency‑focused inference, and a BF16 full‑precision reference for fine‑tuning and research.

---

What makes Super VL unique?

• Hybrid architecture – A 120 B‑parameter language backbone (with 12 B active parameters per token) interleaves Mamba‑2 and Mixture‑of‑Experts (MoE) layers, while a dedicated C‑RADIO vision encoder processes visual inputs.
• Massive context – Supports up to 1 million tokens of context, enabling long‑document analysis and multi‑turn conversations.
• Multi‑Token Prediction (MTP) – Built‑in speculative decoding reduces latency for long‑form generation.
• Configurable reasoning – The `enable_thinking` flag lets you toggle on‑demand chain‑of‑thought style reasoning per request.
• Tool‑calling & JSON output – Responses can include tool calls or structured JSON, facilitating multimodal agent pipelines.

These capabilities target agentic AI scenarios where the model must see and understand visual data before acting, such as document intelligence, GUI grounding for computer‑use agents, and short‑video question answering.

---

JumpStart deployment options

| Variant | Precision | Checkpoint size | Default instance | Typical throughput | |---------|-----------|----------------|------------------|---------------------| | NVFP4 | 4‑bit (W4A4) quantized | ~75 GB | `ml.g7.24xlarge` (G7 GPU) | 63 tokens / sec per request | | BF16* | Full‑precision | ~249 GB | `ml.p5en.48xlarge` (P5e GPU) | 141 tokens / sec per request |

Measured with a 250‑token prompt, 1,500‑token generation, and five concurrent streaming requests. Time‑to‑first‑token stayed around 0.5 s* across all configurations.

• NVFP4 is ideal for production workloads that prioritize low latency and lower GPU memory footprints.
• BF16 is suited for research, fine‑tuning, or custom quantization pipelines, offering the full fidelity of the original model.

Both variants expose the same API surface in SageMaker Studio and the SageMaker Python SDK, so switching between them is a matter of selecting the appropriate endpoint configuration.

---

Core use cases

1. Document intelligence – OCR, table extraction, and multi‑page PDF analysis. A single 1,536 × 1,024 image consumes roughly 1,536 tokens of context, enabling end‑to‑end summarization or extraction.
2. GUI understanding – Screen‑grounding for agents that navigate desktop or web applications, interpreting UI elements and issuing tool calls.
3. Video understanding – Short MP4 clips can be supplied (base64‑encoded) for summarization, captioning, or visual Q&A. The default endpoint accepts one video per request.
4. Multimodal agents – Combine vision, language, and tool use in a single workflow, leveraging JSON‑structured responses for downstream processing.
5. Customization – The BF16 checkpoint can be downloaded for supervised fine‑tuning (SFT), reinforcement learning from human feedback (RLHF), or domain‑specific distillation. LoRA adapters are not loaded by the default JumpStart endpoint, but you can spin up a custom SageMaker training job to apply them.

---

Getting started

1. Open SageMaker Studio and navigate to JumpStart → Models.
2. Search for Nemotron 3.5 Super VL and select the desired variant (NVFP4 or BF16).
3. Click Deploy – the wizard provisions an endpoint on the recommended instance type.
4. Use the SageMaker Python SDK:

```python import sagemaker from sagemaker import Model

model = Model( imageuri="public.ecr.aws/sagemaker‑jumpstart‑model‑registry/nemotron‑3‑5‑super‑vl", modeldata="s3://path/to/checkpoint", role="SageMakerExecutionRole", ) predictor = model.deploy(initialinstancecount=1, instance_type="ml.g7.24xlarge")

response = predictor.predict({"inputs": {"text": "Describe this image", "image": img_bytes}}) print(response) ```

1. Adjust `enable_thinking=True` in the request payload to activate chain‑of‑thought reasoning when needed.

---

Practical considerations

• Cost – The NVFP4 variant runs on G7 GPUs, which are cheaper per hour than the P5e GPUs required for BF16. Estimate your workload with the AWS Pricing Calculator.
• Payload limits – Video payloads must respect the endpoint’s request size (typically 6 MiB). For larger clips, pre‑process on an EC2 instance before sending frames.
• Latency vs. quality – NVFP4’s speculative decoding yields comparable quality to BF16 on most multimodal benchmarks while halving memory usage. If you need the absolute best scores on niche benchmarks, prefer BF16.
• Security – Both variants are covered under NVIDIA’s OpenMDW License (v1.1). Ensure your usage complies with the license terms, especially for commercial deployments.

---

Closing thoughts

The integration of NVIDIA Nemotron 3.5 Super VL into SageMaker JumpStart lowers the barrier for building sophisticated multimodal agents on AWS. By offering a quantized, inference‑ready checkpoint alongside a full‑precision reference, Amazon gives teams the flexibility to start with production‑grade performance and later dive into custom fine‑tuning if the use case demands it. Whether you’re automating document processing, building a visual assistant for software tools, or exploring video‑centric AI, Super VL provides a powerful, scalable foundation that can be launched with just a few clicks.