Reporting, analysis and practical guides connected to nvidia.
Encoder-prefill-decode disaggregation can reduce multimodal time to first token and increase same-SLO throughput, but only when vision encoding is a large enough share of request time. Teams should trace each serving stage, benchmark a workload matrix, optimize media preprocessing and caching, and promote only the traffic buckets that clear a reversible latency and cost gate.
A practical four-test framework for tracking whether Hugging Face remains open, multi-provider, and hardware-neutral under NVIDIA.
A practical guide to evaluating cuOpt as an end-to-end optimization service rather than a benchmark number.
Nvidia released Nemotron 3.5 Lightning on August 11, 2026: an open 30B mixture-of-experts model with 3B active parameters, licensed under OpenMDW-1.1 with weights, training data and recipes included. It targets the execution layer of long-running agents rather than frontier reasoning, reaching 86% accuracy on PinchBench while completing 10,000 tasks 30% faster than Qwen3.6 35B, and up to 4x the output speed of similar-sized models. Speed comes from baked-in multi-token prediction plus DSpark and DFlash draft models, with NVFP4 and BF16 checkpoints. It runs on Jetson, RTX 5090 and DGX Spark via LM Studio, llama.cpp, Ollama and Unsloth, and ships alongside NeMo Switchyard for routing planning to frontier models and execution to Lightning.
FlashAttention is an IO-aware, exact attention algorithm from 2022 that avoids writing the full N-by-N attention matrix to slow GPU HBM. Using tiling, an online-softmax running-statistics trick, kernel fusion, and recomputation, it cuts memory from O(N^2) to O(N) and delivered up to 7.6x speedups. FlashAttention-2 reached ~70% of A100 peak FLOPs; FlashAttention-3 (2024) exploits Hopper asynchrony and FP8 to hit ~840 TFLOPs BF16 (~75% H100 utilization). It now powers PyTorch, vLLM, and long-context serving.
Etched, a startup building the transformer-only Sohu inference ASIC, has booked over $1 billion in contracts and reached a $5 billion valuation, with reports of new rounds valuing it up to $20 billion. Sohu hard-wires the transformer graph into silicon on TSMC N4P with 144GB HBM3E, and Etched claims an 8-chip server exceeds 500,000 Llama 70B tokens/sec. No independent benchmarks exist yet.
OpenAI unveiled Jalapeño, its first custom inference chip co-designed with Broadcom on TSMC 3nm. Built in a nine-month design cycle, the reticle-sized ASIC targets roughly 50% lower inference cost than current Nvidia GPUs, with deployment starting late 2026 and Microsoft reportedly taking 40% of the first run for Azure.
NVIDIA GR00T N1.7 is an open robot brain, trained on human video, showing real dexterity scaling.
NVIDIA's Nemotron 3 Nano Omni is a 30B open multimodal model, processing diverse data with high throughput.
NVIDIA Ising offers open-source AI models to improve quantum error correction, making quantum computing more effective.
Eli Lilly's LillyPod, a 9,000-petaflop AI supercomputer, is making big bets on drug discovery.
NVIDIA's Nemotron 3 Super, a hybrid architecture, delivers 5x throughput and top agentic benchmarks.
Meta's MTIA custom AI chips, with 25x compute improvement, are rapidly challenging Nvidia's market position.
