Tech Tips
LFM2.5 QAD vs DSpark: Two Levers for Faster Local AI
A decision guide for choosing LFM2.5 QAD or DSpark based on memory, quality, runtime support, and decode latency.
Topic · 4 stories
#Quantization
Reporting, analysis and practical guides connected to Quantization.
Latest in #Quantization
Newest first
Deep Dives
LoRA and QLoRA: Fine-Tune Massive LLMs on a Single GPU
LoRA (2021) freezes a model's weights and trains tiny low-rank matrices, cutting GPT-3's trainable parameters 10,000x with no inference latency. QLoRA (2023) quantizes the frozen base to 4-bit NF4, fitting a 65B model on one 48GB GPU at ~33% less memory but ~39% more training time. Rank sets capacity; alpha (via alpha/r) sets scale. Adapt attention projections first and raise rank only when quality demands it.
Deep Dives
LLM Quantization: GGUF vs AWQ vs GPTQ in 2026
A practical breakdown of the three dominant LLM quantization formats in 2026. GGUF is the portable, CPU-friendly default (use Q4_K_M); AWQ wins on 4-bit quality for GPU serving via activation-aware precision; GPTQ remains a solid NVIDIA-focused option. Quantization is lossy, so test on your real workload.
Deep Dives
TurboQuant: Google's 6x KV Cache Compression Hits 3-Bit With Zero Loss
Google's TurboQuant compresses KV cache 6x at 3 bits with zero loss, speeding up attention.
