Tag
open-weights
11 articles
DeepSeek V4: 1.6T Open Weights and 1M Context, Now the Default
DeepSeek released V4 as two open-weight mixture-of-experts models: V4-Pro (1.6T total / 49B active) and V4-Flash (284B / 13B active), both with a 1M-token default context and 384K max output. A novel token-wise compression plus DeepSeek Sparse Attention (DSA) makes the long window affordable. API pricing is aggressive (V4-Flash $0.14/M input, $0.28/M output; V4-Pro $0.435/$0.87), and the old deepseek-chat and deepseek-reasoner endpoints were retired after July 24, 2026. Reported ~80.6% on SWE-bench Verified.
By Sarah Chen · 5 min · Aug 1, 2026
Laguna S 2.1: Poolside's 118B Open-Weight Coding Model
Poolside released Laguna S 2.1 on July 21, 2026, a 118B-parameter Mixture-of-Experts coding model activating ~8B params per token, with a 1M-token context and a permissive OpenMDW-1.1 license. First-party benchmarks show 78.5% on SWE-Bench Multilingual, but independent verification is still pending. Day-one FP8/NVFP4/INT4 and GGUF builds make it genuinely self-hostable.
By Sarah Chen · 5 min · Jul 31, 2026
FLUX 3: Black Forest Labs' One Model for Video, Audio & Action
FLUX 3, released July 23 2026, is Black Forest Labs' first multimodal model to generate video, audio, and robot actions from one set of weights, built on the Self-Flow method. FLUX 3 Video produces up to 20-second clips with native audio and led human-preference tests over Luma Ray 3.2 (93%) and Runway Gen-4.5 (77%), tying Seedance 2.0 and Gemini Omni Flash at 52%. Access is gated: video and action first, image next, open weights last.
By Sarah Chen · 5 min · Jul 28, 2026
Inkling: Mira Murati's Thinking Machines Ships Its First Open Model
Thinking Machines Lab, founded by ex-OpenAI CTO Mira Murati, released Inkling on July 15, 2026 — an open-weight mixture-of-experts model with 975B total parameters (41B active), trained on 45 trillion multimodal tokens. The company openly says it isn't the strongest model available; instead it's a customizable foundation enterprises fine-tune via the Tinker platform. The release doubles as an argument that owned, adaptable models beat rented one-size-fits-all APIs.
By Sarah Chen · 5 min · Jul 18, 2026
Kimi K3: Moonshot's 2.8T Open Model Nears the Frontier
Moonshot AI released Kimi K3 on July 16, 2026, a 2.8-trillion-parameter open Mixture-of-Experts model that activates 16 of 896 experts, ships native vision and a 1M-token context, and leads benchmarks like SWE Marathon, BrowseComp, and OmniDocBench while trailing Fable 5 and GPT-5.6 Sol overall. Weights release July 27 under a Modified MIT license.
By Sarah Chen · 5 min · Jul 17, 2026
Mixture of Experts: How Sparse Models Beat Dense LLMs
Mixture of Experts (MoE) replaces a transformer's single feed-forward network with many smaller expert networks plus a learned router that sends each token to only its top-k experts (sparse activation). This decouples total parameters (which set memory) from active parameters (which set compute). Mixtral 8x7B has 46.7B total but 12.9B active via top-2 routing; DeepSeek-V3 has 671B total but 37B active (5.5%) using 256 routed experts plus one shared expert and top-8 routing. The design traces to Shazeer et al. (2017) and Google's Switch Transformer (2021, top-1 routing, 1.6T params). Trade-offs include memory footprint, load-balancing difficulty, training instability, communication overhead, and harder fine-tuning.
By Aisha Patel · 6 min · Jul 10, 2026
GLM-5.2: Zhipu's Open-Weight Model Beats GPT-5.5 at 1/6 the Cost
Z.AI released GLM-5.2 on June 16, 2026: a 753B-parameter MoE model under an MIT license with a 1M-token context. It tops open-weight coding benchmarks, beating GPT-5.5 on SWE-bench Pro, FrontierSWE and PostTrainBench at roughly one-sixth the cost.
By Sarah Chen · 5 min · Jun 26, 2026
LLM Quantization: GGUF vs AWQ vs GPTQ in 2026
A practical breakdown of the three dominant LLM quantization formats in 2026. GGUF is the portable, CPU-friendly default (use Q4_K_M); AWQ wins on 4-bit quality for GPU serving via activation-aware precision; GPTQ remains a solid NVIDIA-focused option. Quantization is lossy, so test on your real workload.
By Aisha Patel · 7 min · Jun 25, 2026
MiniMax M3: Open-Weight Frontier Coding Model With 1M Context
MiniMax M3 is an open-weight model pairing a 1M-token context and revived sparse attention with frontier coding benchmarks at 15x lower cost than Claude Opus 4.7.
By Sarah Chen · 6 min · Jun 16, 2026
Kimi K2.7-Code: A 30% Token Cut With a Benchmark Asterisk
Moonshot AI's Kimi K2.7-Code is an open-weights, OpenAI-compatible coding model (1T-param MoE, 32B active, 256K context) claiming a 30% cut in reasoning tokens and a narrow win over Claude Opus 4.8. But all published benchmarks are Moonshot's own proprietary suites, with no independent results yet, so the efficiency claims remain unverified.
By Sarah Chen · 5 min · Jun 14, 2026
Mistral Medium 3.5: 128B Open-Weight Model That Opens PRs
Mistral Medium 3.5 is a powerful 128B open-weight model capable of opening GitHub pull requests.
By Sarah Chen · 7 min · May 4, 2026