Tag
benchmarks
16 articles
GPT-5.6: OpenAI's Sol, Terra, and Luna Go Public
OpenAI made its three-tier GPT-5.6 family (Sol, Terra, Luna) generally available on July 9, 2026 after government safety review. Pricing runs from Luna at $1/$6 to Sol at $5/$30 per 1M tokens, with a Sol Fast option at $12.50/$75 on Cerebras. The release adds Programmatic Tool Calling in the Responses API (63.5% fewer tokens, 50.1% fewer turns) and longer prompt caching, but Sol's 64.6% on SWE-Bench Pro still trails Claude Mythos 5 (80.3%).
By Sarah Chen · 5 min · Jul 11, 2026
Gemini 3.5 Flash: The Flash Model That Beats Google's Own Pro Tier
Google released Gemini 3.5 Flash on May 19, 2026, at Google I/O. The Flash-tier model beats Gemini 3.1 Pro on coding and agentic benchmarks (76.2% Terminal-Bench 2.1, 83.6% MCP Atlas, 1656 GDPval-AA Elo) while running 4x faster and costing $1.50/$9 per 1M tokens, 40% below 3.1 Pro. It trails Pro on academic reasoning (Humanity's Last Exam, ARC-AGI-2) and dense long-context recall. It powers Gemini Spark, Antigravity 2.0, and is now the default model for the Gemini app and AI Mode in Search.
By Sarah Chen · 5 min · Jul 7, 2026
Claude Fable 5: The Coding Crown Returns After 19 Days Offline
Claude Fable 5, Anthropic's Mythos-class model, was suspended June 12, 2026 under a U.S. export-control directive and restored July 1 after Anthropic made security commitments. It leads coding benchmarks at 80.3% SWE-Bench Pro (vs 69.2% for Opus 4.8) and 29.3% FrontierCode. A grace window counts it toward 50% of weekly usage through July 7; credits billing follows.
By Sarah Chen · 5 min · Jul 3, 2026
Claude Sonnet 5: Anthropic's Most Agentic Mid-Tier Model Yet
Claude Sonnet 5, released June 30, 2026, is Anthropic's most agentic mid-tier model. It beats Sonnet 4.6 on every published benchmark (63.2% SWE-bench Pro, 80.4% Terminal-Bench 2.1, 81.2% OSWorld) and edges Opus 4.8 on GDPval-AA v2 knowledge work. Intro pricing is /0 per million tokens through Aug 31, 2026, then /5. A new tokenizer can raise token counts up to 1.35x, and xhigh effort can cost more than Opus 4.8.
By Sarah Chen · 5 min · Jul 1, 2026
Grok 4.3: xAI's Frontier Model Hits Amazon Bedrock
Grok 4.3 is generally available on Amazon Bedrock with a 1M-token context window, $1.25/$2.50 pricing, and a top hallucination-rate score.
By Sarah Chen · 4 min · Jun 29, 2026
GLM-5.2: Zhipu's Open-Weight Model Beats GPT-5.5 at 1/6 the Cost
Z.AI released GLM-5.2 on June 16, 2026: a 753B-parameter MoE model under an MIT license with a 1M-token context. It tops open-weight coding benchmarks, beating GPT-5.5 on SWE-bench Pro, FrontierSWE and PostTrainBench at roughly one-sixth the cost.
By Sarah Chen · 5 min · Jun 26, 2026
Test-Time Compute: Why Reasoning Models Think Before Answering
Test-time compute spends extra computation during inference, not training, to improve answers. It powers reasoning models like OpenAI o1 and DeepSeek-R1. Two strategies exist: sequential scaling (longer chains of thought, e.g. the s1 paper's budget forcing) and parallel scaling (Best-of-N, majority voting). More thinking is not always better, overthinking degrades accuracy, and hidden reasoning tokens are billable. Match compute to task difficulty.
By Aisha Patel · 8 min · Jun 17, 2026
MiniMax M3: Open-Weight Frontier Coding Model With 1M Context
MiniMax M3 is an open-weight model pairing a 1M-token context and revived sparse attention with frontier coding benchmarks at 15x lower cost than Claude Opus 4.7.
By Sarah Chen · 6 min · Jun 16, 2026
Speculative Decoding: How a Tiny Draft Model Doubles LLM Speed
Speculative decoding speeds up LLM inference 2-6x by having a small draft model propose tokens that the target model verifies in parallel via rejection sampling, guaranteeing lossless output. EAGLE-3 and Medusa reduce or remove the separate draft model. Gains are largest at low batch sizes.
By Aisha Patel · 7 min · Jun 15, 2026
Kimi K2.7-Code: A 30% Token Cut With a Benchmark Asterisk
Moonshot AI's Kimi K2.7-Code is an open-weights, OpenAI-compatible coding model (1T-param MoE, 32B active, 256K context) claiming a 30% cut in reasoning tokens and a narrow win over Claude Opus 4.8. But all published benchmarks are Moonshot's own proprietary suites, with no independent results yet, so the efficiency claims remain unverified.
By Sarah Chen · 5 min · Jun 14, 2026
Diffusion LLMs: How Text Diffusion Is Challenging Autoregression
Diffusion language models (dLLMs) abandon left-to-right autoregressive generation, instead refining masked noise into text over a few parallel denoising steps. Inception Labs' Mercury Coder runs at 1,100+ tokens per second on H100s versus 50-200 for autoregressive models, and LLaDA 8B's bidirectional design breaks the reversal curse. They still trail the best models on hard reasoning benchmarks, but the one-token-at-a-time assumption is no longer a law of nature.
By Aisha Patel · 8 min · Jun 12, 2026
MAI-Code-1-Flash: Microsoft's Lean Coding Model Hits Copilot
Microsoft launched MAI-Code-1-Flash on June 2, 2026, a lightweight, agentic coding model built end-to-end in-house and rolling out to GitHub Copilot users in VS Code. It outperforms Claude Haiku 4.5 across four coding benchmarks (including 51.2% vs 35.2% on SWE-Bench Pro) while using up to 60% fewer tokens, signaling Microsoft's push for AI independence from OpenAI.
By Sarah Chen · 5 min · Jun 6, 2026
DeepSeek V4-Pro: 75% Price Cut Becomes Permanent
On May 22, 2026, DeepSeek made its 75% promotional discount on V4-Pro permanent rather than letting it expire May 31. New permanent rates: $0.435/M input, $0.87/M output, $0.003625/M cache hit. That puts V4-Pro output roughly 34x cheaper than GPT-5.5 and 17x cheaper than Claude Opus 4.7, while landing within 3-7 points on coding and reasoning benchmarks. The underrated detail is the cache-hit price, which can cut input cost ~88% for agents with stable prefixes. Teams should re-run their build math and route the easy majority of traffic to V4-Pro.
By Sarah Chen · 5 min · Jun 1, 2026
Claude Opus 4.8: Anthropic's Honest, Parallel-Agent Flagship
Anthropic released Claude Opus 4.8 on May 28, 2026, 41 days after Opus 4.7. It scores 69.2% on SWE-Bench Pro, emphasizes calibrated honesty and longer autonomy, adds Dynamic Workflows for hundreds of parallel subagents, runs fast mode ~2.5x quicker, and holds pricing flat from 4.7.
By Sarah Chen · 4 min · May 30, 2026
Gemini 3.5 Flash: Google's Flash Tier Eats Pro on Agent Benchmarks
Gemini 3.5 Flash outperforms the Pro tier on agent benchmarks with superior speed and efficiency.
By Sarah Chen · 5 min · May 28, 2026
Meta Muse Spark: The First Model From Superintelligence Labs Is a Strategic Reset
Meta Muse Spark, from Superintelligence Labs, marks a strategic AI reset with top benchmarks and medical reasoning.
By Sarah Chen · 5 min · Apr 9, 2026