Can I run Flint 40B-A2.5B locally? Requirements and speed

Check your PC

Memory it needs

The model’s weights plus the conversation it is holding, at each compression. For it to run at full speed, all of it has to fit in the graphics card’s memory with about 1.2 GB to spare. Slow although it should fit? See where it is really running. Which compression to download.

How fast it runs on common PCs

With an 8,192-token conversation, at the best compression that fits each PC. Writing speed is how fast an answer appears; most people read at about 5 to 8 tokens a second.

About Flint 40B-A2.5B

From its page on Hugging Face, read on 8 November 2026. LM Studio, Ollama and Jan load it from GGUF files such as unsloth/flint-GGUF. Bottleneck is not connected with Recursive.

How it is worked out

Writing an answer is limited by memory rather than arithmetic: for every word-piece (token) a model writes, the computer reads every weight that token uses once. So the speed is how fast the memory holding the model can be read, divided by how much is read for each token: the model’s weights at the chosen compression, plus the conversation so far. How much of its memory speed each graphics card reaches is measured, not assumed: it comes from the card’s own result in llama.cpp’s benchmark, which LM Studio, Ollama and Jan are built on. A model that does not fit in the graphics card’s memory spills into system memory, five to fifteen times slower, and one that does not fit in memory at all can be read from the drive as it runs, slower again; the estimate counts each part at its own speed, and a processor with few threads cannot use all of fast memory. Graphics built into a processor use system memory, at its speed. Reading a prompt is limited by arithmetic, so it follows the measured prompt speed.

• Graphics cards’ speeds: llama.cpp’s community scoreboards on GitHub for NVIDIA (CUDA), Vulkan (AMD and Intel cards, and graphics built into processors), AMD (ROCm) and Apple Silicon: Llama 2 7B at 4 bits on each, writing and reading a prompt. Parts without a result there are estimated from their nearest relative.
• Checks against larger and newer models: Qwen3.5 27B on an RTX 3090 and gpt-oss 20B, Qwen3 14B and 32B on an RTX PRO 4500 (the same scoreboards), Meta’s own Muse Glimmer 30B figure on an RTX 5090, a published log of 51 models on a Ryzen AI Max+ 395, processors from an Intel N100 mini PC to EPYC servers running DeepSeek R1, and DeepSeek R1 read partly from an NVMe drive. The estimates fall within about a fifth of those measurements, which is why each is shown as a range.
• The models: each one’s own Hugging Face page and configuration, linked from its page here.