GGUF Quant Decoder: What Does Q4_K_M Mean?
Type any GGUF quant name, like Q4_K_M or IQ4_XS, for a plain-English decode, plus a picker for which quant fits your memory.
Which quant should I download?
Enter the model's parameter count and how much RAM or VRAM you have. This sizes weights plus a small runtime overhead only, not a running conversation's context, so leave some headroom above the memory you actually have.
| Quant | Estimated size | Fits? |
|---|---|---|
| FP16 | 18.30 GB | No |
| Q8_0 | 10.43 GB | Fits |
| Q6_K | 8.42 GB | Fits |
| Q5_K_M | 7.47 GB | Fits |
| Q4_K_M | 6.59 GB | Fits |
| IQ4_XS | 5.96 GB | Fits |
| Q3_K_M | 5.61 GB | Fits |
| Q2_K | 5.02 GB | Fits |
This tool runs entirely in your browser. The quant name you type and the numbers you enter are never sent anywhere, there is nothing to upload.
Overview
Q4_K_M means 4-bit precision using llama.cpp's k-quant format, tier M. K-quants group weights into blocks with per-block scales, keeping sensitive tensors like attention and output layers at higher precision than the S tier. At 4.85 bits per weight, Q4_K_M is about 30 percent the size of the original FP16 weights, with only mild quality loss, making it the most commonly recommended default download.
How it works
- 1Type any GGUF quant name into the decoder, like Q4_K_M, IQ4_XS, Q8_0, or FP16.
- 2Read the plain-English breakdown: bit width, quantization scheme (legacy, k-quant, i-quant, or float), tier, and size versus FP16.
- 3Scroll to the picker below and enter the model's parameter count in billions.
- 4Enter how much RAM or VRAM you have available.
- 5Read the table, sorted biggest quant first, to see which ones fit and which don't.
Worked example
Decoding Q4_K_M and IQ4_XS
Q4_K_M decodes to 4-bit precision, k-quant scheme, tier M, at 30 percent the size of FP16 (4.85 of 16 bits per weight). The M tier means attention and output layers are kept at higher precision than the S tier, at a modest size cost. IQ4_XS, a different 4-bit family, comes in smaller still at 27 percent of FP16 size (4.25 bits per weight), since i-quants use importance-matrix calibration instead of a flat per-block scale.
Picking a quant for an 8B model with 16GB of memory
For an 8B-parameter model and 16 GB of memory, the picker marks Q8_0 (8.5 bits per weight) as the biggest quant that fits, at an estimated 10.43 GB (8.5 GB weights plus 1.93 GB overhead). FP16 needs an estimated 18.30 GB and does not fit. Everything from Q6_K down through Q2_K fits comfortably, leaving room for a running conversation's context on top.
Methodology & privacy
The decoder parses the quant name's structure rather than looking up a fixed list, so it can explain any bit width and tier the grammar allows. Legacy names match Q<bits>_0 or Q<bits>_1, llama.cpp's original per-block round-to-nearest format. K-quants match Q<bits>_K, optionally followed by _S, _M, or _L, llama.cpp's per-block scale-and-minimum format, where the suffix controls how many sensitive tensors get bumped to higher precision. I-quants match IQ<bits>_<tier> (XXS, XS, S, M, or NL for the non-linear 4-bit variant), llama.cpp's importance-matrix-calibrated format. Float types match F16, FP16, BF16, or F32. Size versus FP16 uses the exact bits-per-weight measured for a quant in the shared quant table when it is one of the eight commonly cited quants from llama.cpp's own quantize documentation, or bits divided by 16 otherwise. The download picker estimates each quant's footprint as parameter count times bits-per-weight divided by 8, plus a flat 1.5 GB plus 5 percent of weights for runtime overhead, the same weights-plus-overhead formula the LLM RAM Calculator uses, but without KV cache, since no context length is asked here: it answers whether the download and its weights fit, not whether a long conversation will too.
- Source: llama.cpp quantize tool, bits-per-weight table
- Source: r/LocalLLaMA: "Qwen3 local 14B Q4_K_M or 30B A3B Q2_K_L" quant confusion thread
- Source: r/LocalLLaMA: "Llama 4 Scout best quantization resource" thread
- Source: r/ollama: Mistral 7B Q4_0 quantization question
Every quant name you type and every number you enter in the picker stays in this page's own memory. Nothing is uploaded, there is no file to drop and no server involved.
FAQ
What does Q4_K_M mean?
+
Q4_K_M means 4-bit precision using the k-quant scheme, tier M. K-quants group weights into small blocks with a learned scale and minimum per block rather than one flat scale, and the M tier keeps a handful of sensitive tensors, like attention and output layers, at higher precision than the S tier. At 4.85 bits per weight it comes out to about 30 percent the size of the original FP16 weights.
Q4_K_M vs Q5_K_M, which should I pick?
+
Q5_K_M uses more bits per weight (5.69 vs 4.85), so it is a somewhat larger download with noticeably less quality loss. Q4_K_M is the more common default because it is smaller and the quality difference is usually small for most tasks; step up to Q5_K_M or Q6_K if you have memory to spare and want to be safe.
IQ4_XS vs Q4_K_M, what's the difference?
+
Both are roughly 4-bit, but IQ4_XS (4.25 bits per weight) uses importance-matrix calibration and non-uniform bit allocation to come in smaller than Q4_K_M (4.85 bits per weight) at similar quality. The tradeoff is that i-quants can run slower on CPU-only or older-hardware backends than k-quants, so Q4_K_M is often still the safer default outside of GPU-heavy setups.
Which GGUF quant should I download?
+
Use the picker above: enter the model's parameter count and how much RAM or VRAM you have, and it lists every quant sized against that budget, biggest first. As a rule of thumb, Q4_K_M is the most commonly recommended starting point, moving up to Q5_K_M, Q6_K, or Q8_0 if your memory allows it.
What's the difference between K_S, K_M, and K_L?
+
All three are the same base bit width, but differ in how many of the model's more sensitive tensors, like attention and output layers, are kept at higher precision. K_S keeps most tensors at the base bit width for the smallest file, K_M raises a handful of tensors, and K_L raises more still, trading some size for quality within the same nominal bit count.
How do I compare GGUF quant quality without downloading every option?
+
Compare bits per weight as a rough proxy for quality: higher generally means closer to the original FP16 weights. This decoder shows each quant's size versus FP16, and the picker above sizes every option in the shared quant table so you can compare downloads before committing to one.
Related tools
Can Your Mac Run It? LLM RAM Calculator
Check whether a model and quant will fit your Mac unified memory or GPU VRAM, with KV cache math, a max context solver, and a tokens per second estimate.
Local vs Cloud LLM Break-Even Calculator
Calculate whether a local LLM or a cloud API costs less per 1M tokens, including amortized hardware, power, and the break-even point.
Token Counter & Context-Window Fit (with boundary view)
Paste text for an exact GPT-4o/GPT-4 token count, rough Claude, Llama 3, and Gemma estimates, and a colored boundary view. Nothing you type leaves your browser.
Downloads Folder Analyzer: Clean Up Downloads on Mac
See what is actually clogging your Downloads folder: type breakdown, biggest files, screenshot piles, old files, and duplicates. Runs entirely in your browser.
This tool fixes one moment. recal handles the whole day.
recal is a private brain for your Mac. It watches how you work, on device, and starts doing your busywork itself. You approve every action.
Get early access