Tech Tips
NVIDIA cuOpt: Benchmark the Workflow, Not Just the Solver
A practical guide to evaluating cuOpt as an end-to-end optimization service rather than a benchmark number.
Topic · 11 stories
#nvidia
Reporting, analysis and practical guides connected to nvidia.
Latest in #nvidia
Newest first
AI News
Nemotron 3.5 Lightning: Nvidia's 3B-Active Agent Workhorse
Nvidia released Nemotron 3.5 Lightning on August 11, 2026: an open 30B mixture-of-experts model with 3B active parameters, licensed under OpenMDW-1.1 with weights, training data and recipes included. It targets the execution layer of long-running agents rather than frontier reasoning, reaching 86% accuracy on PinchBench while completing 10,000 tasks 30% faster than Qwen3.6 35B, and up to 4x the output speed of similar-sized models. Speed comes from baked-in multi-token prediction plus DSpark and DFlash draft models, with NVFP4 and BF16 checkpoints. It runs on Jetson, RTX 5090 and DGX Spark via LM Studio, llama.cpp, Ollama and Unsloth, and ships alongside NeMo Switchyard for routing planning to frontier models and execution to Lightning.
Deep Dives
FlashAttention: The IO-Aware Trick That Made Long Context Cheap
FlashAttention is an IO-aware, exact attention algorithm from 2022 that avoids writing the full N-by-N attention matrix to slow GPU HBM. Using tiling, an online-softmax running-statistics trick, kernel fusion, and recomputation, it cuts memory from O(N^2) to O(N) and delivered up to 7.6x speedups. FlashAttention-2 reached ~70% of A100 peak FLOPs; FlashAttention-3 (2024) exploits Hopper asynchrony and FP8 to hit ~840 TFLOPs BF16 (~75% H100 utilization). It now powers PyTorch, vLLM, and long-context serving.
AI News
Etched: The $5B Sohu Chip Betting the Transformer Never Dies
Etched, a startup building the transformer-only Sohu inference ASIC, has booked over $1 billion in contracts and reached a $5 billion valuation, with reports of new rounds valuing it up to $20 billion. Sohu hard-wires the transformer graph into silicon on TSMC N4P with 144GB HBM3E, and Etched claims an 8-chip server exceeds 500,000 Llama 70B tokens/sec. No independent benchmarks exist yet.
AI News
OpenAI Jalapeño: Custom AI Chip Aims to Beat Nvidia
OpenAI unveiled Jalapeño, its first custom inference chip co-designed with Broadcom on TSMC 3nm. Built in a nine-month design cycle, the reticle-sized ASIC targets roughly 50% lower inference cost than current Nvidia GPUs, with deployment starting late 2026 and Microsoft reportedly taking 40% of the first run for Azure.
Open Source
NVIDIA GR00T N1.7: The Open Robot Brain Trained on Human Video
NVIDIA GR00T N1.7 is an open robot brain, trained on human video, showing real dexterity scaling.
Open Source
Nemotron 3 Nano Omni: NVIDIA's 30B Open Model Sees and Hears
NVIDIA's Nemotron 3 Nano Omni is a 30B open multimodal model, processing diverse data with high throughput.
Open Source
NVIDIA Ising: Open-Source AI Models That Make Quantum Computing Actually Work
NVIDIA Ising offers open-source AI models to improve quantum error correction, making quantum computing more effective.
AI News
LillyPod: Eli Lilly's 9,000-Petaflop Supercomputer Bets Big on AI Drug Discovery
Eli Lilly's LillyPod, a 9,000-petaflop AI supercomputer, is making big bets on drug discovery.
AI News
NVIDIA Nemotron 3 Super: The Hybrid Architecture That Rewrites the Agent Playbook
NVIDIA's Nemotron 3 Super, a hybrid architecture, delivers 5x throughput and top agentic benchmarks.
Deep Dives
Meta MTIA: Four Custom AI Chips in Two Years to Challenge Nvidia
Meta's MTIA custom AI chips, with 25x compute improvement, are rapidly challenging Nvidia's market position.
