Can I run this LLM? VRAM & speed calculator
Paste any Hugging Face model. Vramwise reads its config live and shows the memory each quantization needs, how much context fits, and a tokens-per-second estimate on your GPU or Mac.
1 · Model
Model details (edit if a value is missing)
2 · Hardware
| Quantization | Weights | KV cache | Total | Fits? | Max context | Speed (tok/s) |
|---|
One-pass decision model: it reads your input once and returns scores, so there is no tokens-per-second figure. "Per call" is the memory-bandwidth floor (every call reads the weights once at batch 1); long inputs add compute time on top. These models usually run in PyTorch with the publisher's own runtime rather than llama.cpp, so check the model card for which precisions actually exist: the rows below the published one show what a quantized build would need.
Estimates, not guarantees: real use differs by runtime (llama.cpp, Ollama, LM Studio, MLX, vLLM), batch size and drivers. Total includes ~1 GB runtime overhead. Speed is 50–80% of the memory-bandwidth limit for one user.
How Vramwise calculates memory
Weights = parameters × bits per weight ÷ 8. Parameter counts come straight from the model's safetensors or GGUF metadata on Hugging Face, so new models work the day they are published. Bits per weight are llama.cpp's averages for each quant mix (Q4_K_M ≈ 4.85, Q8_0 = 8.5).
KV cache = 2 × layers × KV heads × head size × bytes × context tokens. Vramwise reads layers, KV heads and head size from config.json, handles grouped-query attention, sliding-window layers (Gemma 3, gpt-oss) and DeepSeek's compressed MLA cache.
Mixture-of-experts models (Qwen3-30B-A3B, gpt-oss, DeepSeek) must fit all experts in memory, but each token only reads the active ones, so they generate much faster than a dense model of the same size.
How much of a Mac's memory can the GPU use?
By default macOS lets the GPU wire about two-thirds of unified memory up to 36 GB and about three-quarters above that. You can raise it (for example sudo sysctl iogpu.wired_limit_mb=57344 on a 64 GB Mac) at your own risk; tick the box to see what fits then.
What if it doesn't fit?
Try a smaller quantization (Q4_K_M is the usual sweet spot), a shorter context, or a quantized KV cache. On a PC, llama.cpp can keep some layers in system RAM ("offload"): it works, but generation drops to a few tokens per second.
Can I run MATILDA-jev locally?
Usually yes, with enough memory. Maincode/matilda-jev-v1 is a one-pass decision model: it scores the options you give it (choice, yes/no or a rating) in a single forward pass and generates no text. Its Hugging Face files list 26.1B parameters in BF16, which is 52.2 GB of weights; with an 8K-token input Vramwise puts the total at about 54 GB. That fits an 80 GB A100 or H100, a 96 GB RTX PRO 6000, or a Mac, DGX Spark or Ryzen AI Max with 96 GB or more of unified memory, but not a single 24–32 GB card or two 24 GB cards.
Maincode also publishes an FP4 build, Maincode/matilda-jev-fp4, with 17.2 GB of weights (about 19 GB in total at 8K input), which is within a 24 GB RTX 3090 or 4090 on paper. It runs through Maincode's own runtime and Triton kernel, and Maincode says it tested only on AMD Instinct MI355X; NVIDIA and other hardware are untested by the publisher. No GGUF build was published when we checked (15 Oct 2026).
Pick MATILDA-jev, Matilda Decision, JEV or Decider under "Decision & scoring models" above to check your own hardware: every size is read live from the model's Hugging Face files. Hybrid models like these keep a fixed state in their linear-attention layers, so only the full-attention layers' cache grows with input length. Open MATILDA-jev on an A100 80GB.
Gated models
Meta Llama and Google Gemma configs need a login to download. Vramwise then reads the same architecture from a public mirror or asks you to fill in the model details.
Is anything uploaded?
No. Your browser asks the public Hugging Face API for the model's metadata; your hardware choices stay on your device.
Privacy policy
Vramwise (this page and the Vramwise Chrome extension) collects no personal data. To read a model's size and architecture, your browser asks the public Hugging Face API for that model's metadata; your hardware choices are stored only in your browser. There are no accounts, analytics cookies or ads in the tool. The Chrome extension uses activeTab only to read the address of the tab you are on when you click its button (to recognise a Hugging Face model page), and contextMenus to add "Can I run this model?" to the right-click menu on Hugging Face links. It never reads page content, keeps no browsing history and sends nothing to CyberMax. This web page counts anonymous page views; the extension does not.
Questions: cybermax.tools@gmail.com. Last updated 28 Sep 2026. © 2026 CyberMax.