Close Menu
NCIJ Network NCIJ Network
    What's Hot

    Latvian leader calls for closure of EU borders to all Russians – POLITICO

    November 20, 2026

    What is the EU-UK youth experience scheme and when will it start?

    November 20, 2026

    Andy Burnham gets his first taste of EU dealmaking and seems hungry for more

    November 20, 2026
    Facebook X (Twitter) Instagram
    Trending
    • Latvian leader calls for closure of EU borders to all Russians – POLITICO
    • What is the EU-UK youth experience scheme and when will it start?
    • Andy Burnham gets his first taste of EU dealmaking and seems hungry for more
    • The Book Lover’s Gift Guide
    • Texas Law Enforcement Agencies Get Financial Boost From 287(g) Program — ProPublica
    • Governing African Economies as They Are by Geci Karuri-Sebina & Stacey-Leigh Joseph
    • Work and pensions secretary corrects comparison of Labour and Conservative records on employment – Full Fact
    • Reform selects Robert Jenrick to fight to keep Newark seat
    • About
      • Our Team
      • Editorial Policy
      • Editorial Independence
      • International Support
    • Trust & Standards
      • AI Usage Policy
      • Conflict of Interest Policy
      • Corrections Policy
      • Ethics Policy
      • Fact-Checking Policy
      • Source Protection
    • Get Involved
      • Guide for Sources
      • Support Independent Journalism
    • Legal
      • Cookie Policy
      • Privacy Policy
      • Terms of Use
    Facebook X (Twitter) Instagram
    NCIJ Network NCIJ Network
    Friday, November 20
    • Home
    • World
    • Ai
    • Business
    • Politics
    • Health
    • Crypto
    • Science
    • Technology
    • Cybersecurity
    • Defense & Security
    • Economy
    • Energy
    • Europe
    • More
      • Fact Check
      • Investigations
      • Opinion & Analysis
      • Environment
    NCIJ Network NCIJ Network
    Home»Artificial Intelligence

    NVIDIA AI Releases Nemotron-Labs-Sigma, a Continuous Diffusion Language Model with 14 checkpoints at 3B/8B

    NCIJ NETWNCIJ NETWORKBy NCIJ NETWNCIJ NETWORKNovember 20, 2026 Artificial Intelligence No Comments4 Mins Read
    Share
    Facebook Twitter LinkedIn Pinterest Email

    NVIDIA has released Nemotron-Labs-Sigma, a collection of 14 open checkpoints at 3B and 8B scale. Nemotron-Labs-Sigma is a continuous diffusion language model family that denoises text in a 16-D embedding space. Most diffusion LLMs, such as LLaDA and Dream, work on discrete tokens. Sigma works in a smooth latent space instead. NVIDIA team states that this makes few-step distillation and trajectory steering practical for text.

    TL;DR

    • Size: 3B and 8B parameters, 14 checkpoints. Context is 4,096 tokens (8,192 for Sigma-8B-instruct).
    • Runs on: 1 NVIDIA GPU in bfloat16 (Ampere, Lovelace, Hopper, Blackwell). About 120 to 140 tokens/s on 1 H100. No CPU support, no quantized variants listed.
    • Performance: Sigma-8B base averages 71.16 on NVIDIA’s 6-task suite, ahead of NLD-8B (70.22).
    • Best: 91.96 on GSM8K (Sigma-8B-instruct, 256 DDPM steps).
    • Worst: 17.00 on AIME25 (same model and sampler).
    • Bottom line: Best: the 8-step distilled 8B keeps a 70.34 average. Worst: the instruct average (59.14) trails NLD-8B (59.92).

    What is Nemotron-Labs-Sigma?

    The research team calls Sigma the first large-scale family of continuous diffusion language models. The collection has 4 variant types at each size:

    • Base: Sigma-3B and Sigma-8B.
    • Instruct: Sigma-3B-instruct and Sigma-8B-instruct.
    • Few-step distilled: distill-3B and distill-8B at NFE 2, 4, 8 and 16 (8 checkpoints).
    • Self-speculative: distill-spec-3B and distill-spec-8B.

    How does continuous diffusion for language work?

    Each token maps to a point in a learned 16-D embedding space. Training adds Gaussian noise to these embeddings and teaches the model to remove it. The embedding geometry is learned jointly with the denoiser.

    Generation runs blockwise. Each denoised block is committed to a KV cache before the next starts. Blocks default to 64 tokens.

    The backbone is a Ministral-style decoder-only Transformer with grouped-query attention, SwiGLU and rotary embeddings. Training starts from an autoregressive warm start of about 1T tokens. About 0.3T tokens of diffusion training follow.

    At inference, classifier-free guidance and score temperature matter most. The model card page recommends guidance 3 and temperature 0.3 at 32 DDIM steps.

    Why does a continuous space matter?

    Discrete diffusion jumps between token states, so its trajectories are hard to steer. Sigma follows smooth ODE/SDE paths instead. Per the research paper, these degrade gracefully at low step counts and distill efficiently. Embedding-space steering also tunes the quality-diversity trade-off for pass@k sampling.

    How does Sigma perform on benchmarks?

    On the research’s 6-task base suite, Sigma-8B with 256 DDPM steps averages 71.16. NLD-8B in diffusion mode scores 70.22, Dream-7B 58.74 and LLaDA-8B 41.78. Qwen3-8B in AR mode scores 68.91. A cheaper 32-step DDIM sampler still gives Sigma-8B a 70.18 average.

    Coding is the standout. Sigma-8B scores 65.12 on HumanEval versus 62.80 for NLD-8B. MBPP is weaker, at 65.74 versus 67.00.

    After SFT, Sigma-8B-instruct scores 91.96 on GSM8K, 78.40 on MATH-500 and 82.50 on HumanEval. Hard reasoning stays limited, at 23.33 on AIME24 and 17.00 on AIME25. Its 10-task average of 59.14 sits just below NLD-8B at 59.92. All scores are NVIDIA’s own evaluations.

    3B8B checkpoints Continuous Diffusion language model NemotronLabsSigma Nvidia Releases
    NCIJ NETWNCIJ NETWORK
    • Website

    Keep Reading

    Perplexity Launches Orchestrator API for Failover and Model Routing

    Meta FAIR Self-Improving Memory Hits 56.0 on Continual Learning Bench

    13 Best Agent Gateways in 2026 for MCP, A2A and LLM Traffic: Kong, Bifrost, AgentCore, Arcade

    After the Nvidia Scandal: The Gulf Is America’s Next AI Security Test

    Not If, But How: Responding to Security Incidents in the Age of AI Attackers

    Google DeepMind Releases Gemini 4 Neon: 1M Context, #1 on AA Coding Agent Index

    Add A Comment
    Leave A Reply Cancel Reply

    Editors Picks

    Latvian leader calls for closure of EU borders to all Russians – POLITICO

    November 20, 2026

    What is the EU-UK youth experience scheme and when will it start?

    November 20, 2026

    Andy Burnham gets his first taste of EU dealmaking and seems hungry for more

    November 20, 2026

    The Book Lover’s Gift Guide

    November 20, 2026
    Latest Posts

    How ‘gentle power’ can preserve traditional biodiversity in Tokyo

    August 21, 2026

    This weight-loss hormone may protect the liver even without weight loss

    August 21, 2026

    Bitcoin Rally Lifts Crypto Stocks as Canaan Jumps 25%

    August 21, 2026

    Subscribe to News

    Get the latest sports news from NewsSite about world, sports and politics.

    NCIJ Network is an independent digital news platform delivering trusted investigative journalism, European and global news, in-depth analysis, and fact-based reporting with accuracy, transparency, and integrity.

    Facebook X (Twitter) Instagram Pinterest YouTube

    Latvian leader calls for closure of EU borders to all Russians – POLITICO

    November 20, 2026

    What is the EU-UK youth experience scheme and when will it start?

    November 20, 2026

    Andy Burnham gets his first taste of EU dealmaking and seems hungry for more

    November 20, 2026

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    Type above and press Enter to search. Press Esc to cancel.