Ollama using the CPU instead of the GPU on Windows: how to tell, and how to fix it

First: see where the model is running

Ask the model something, then, within a few minutes — before Ollama unloads it — open PowerShell or a Command Prompt and type:

ollama ps

The PROCESSOR column says where each loaded model is:

NAME           ID              SIZE     PROCESSOR          CONTEXT    UNTIL
gpt-oss:20b    17052f91a42e    14 GB    30%/70% CPU/GPU    32768      4 minutes from now

• 100% GPU: all of it is on the graphics card. That is as fast as it gets — unless it has overflowed, which the last section covers.
• 100% CPU: none of it is. If the PC has a graphics card, Ollama is not using it; see Ollama isn’t using the card at all.
• Two figures, such as 30%/70% CPU/GPU: part of it did not fit on the card; see The model doesn’t fit on the card.

Why it matters so much: to write each word, a model reads its whole weights from memory. A graphics card such as the RTX 4070 reads its own memory at about 500 GB a second; dual-channel DDR5-6000 system memory manages about 96. And the card waits for the processor on every word, so a fifth of a model left on the processor can more than halve how fast the whole of it writes.

Ollama isn’t using the card at all

If ollama ps says 100% CPU on a PC with a graphics card, work down this list.

1. Is the card one Ollama can use?

• NVIDIA: a GTX 750 Ti, GTX 900 series or anything newer (compute capability 5.0 and up), with driver 550 or newer — or 570 or newer for the GTX 750 Ti, 900 and 10 series. Older GeForce cards are not supported.
• AMD: on Windows, Ollama uses AMD’s ROCm for the Radeon RX 7900 XTX, 7900 XT, 7900 GRE, 7800 XT, 7700 XT, 7600 XT and 7600, and Vulkan for most other Radeon cards. Vulkan is on by default.
• Intel Arc: through Vulkan, the same way.

2. Update the driver, then Ollama

Install the newest driver from the NVIDIA app, AMD Software or Intel’s site, then update Ollama — its icon by the clock offers updates, or download it again from ollama.com — and restart the PC. An old driver is the most common reason Ollama cannot find a card: its own troubleshooting notes say an outdated AMD driver makes it fall back to the processor.

3. Look for a setting that hides the card

Some guides tell you to set an environment variable to choose a graphics card, and a wrong value hides it altogether. Press Start, type environment variables and open Edit environment variables for your account; check Edit the system environment variables → Environment Variables too. Look for these:

• CUDA_VISIBLE_DEVICES, ROCR_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES or GGML_VK_VISIBLE_DEVICES set to -1, or to a number that is not your card. -1 is how Ollama’s own documentation says to force the processor.
• OLLAMA_VULKAN set to 0, which switches off Vulkan — and with it most AMD and Intel cards.

Delete any you did not set on purpose, then quit Ollama from its icon by the clock and open it again: it reads them only when it starts.

4. Read Ollama’s log

If it still uses the processor, type explorer %LOCALAPPDATA%\Ollama in the Run box (Windows+R) and open server.log. As it starts, Ollama notes each graphics card it found, in lines containing inference compute. If yours is not among them, look for lines about GPU discovery: failure during GPU discovery means it could not talk to the card, which again usually means the driver.

The model doesn’t fit on the card

Two figures under PROCESSOR mean Ollama put as much on the card as it could and left the rest to the processor. What has to fit is more than the download: the model’s weights, the conversation it is holding (the CONTEXT column), and some working memory — all in what is left after Windows, the screen and your other programs have taken their share. Try these in order; each frees room.

1. Shorten the conversation it remembers

The CONTEXT column is how many tokens of conversation the model is set to hold, and the card keeps room for all of them from the start. Ollama picks it by how much memory the card has: 4K tokens under 24 GB, 32K up to 48 GB, and 256K above that — but chat apps and coding tools often ask for more, and a long setting can take gigabytes. 8K tokens is about 6,000 words, plenty for most chats. Lower it with the context length slider in the Ollama app’s settings, or by setting OLLAMA_CONTEXT_LENGTH to 8192 in your environment variables (as above) and restarting Ollama.

2. Free the card’s memory

Other programs keep part of the card’s memory for themselves: browsers, which draw their pages on the card, games and launchers, Discord, animated wallpapers. To see how much, open Task Manager (Ctrl+Shift+Esc), choose Details, right-click a column heading, choose Select columns and tick Dedicated GPU memory. Close what you can, or switch off Use graphics acceleration when available in your browser’s settings.

Then unload the model and ask it something again, because Ollama decides how much goes on the card when it loads a model:

ollama stop gpt-oss:20b

3. Use a smaller download of the same model

Most Ollama models come at 4-bit compression already, but if you downloaded one at 8 bits (a tag ending in q8_0) or 16 (fp16), the tag ending in q4_K_M takes about 40% less memory than q8_0 and loses little. The model’s page on ollama.com lists its tags, and which GGUF file to download explains the names.

4. Or a smaller model

If even that will not fit, a model a size down that runs entirely on the card is usually both faster and more useful than a bigger one crawling along half on the processor. Can my PC run AI? shows, for your graphics card and memory, which models fit and how fast each writes.

5. Check OLLAMA_NUM_PARALLEL

If OLLAMA_NUM_PARALLEL is set above 1, Ollama keeps room for that many conversations at once, each as long as the context length. On one PC for one person, set it to 1 or delete it.

100% GPU, but still slow? It may be overflowing

NVIDIA’s drivers do not refuse a program more memory than the card has: they lend it system memory instead, and pass it to the card over and over as the model writes. Ollama can then report 100% GPU while running far more slowly than it should — often more slowly than if the processor had taken that part itself. In Task Manager’s Details tab, a large figure in the Shared GPU memory column for ollama.exe is the sign.

1. Open NVIDIA Control Panel: right-click the desktop, Show more options, NVIDIA Control Panel.
2. Choose Manage 3D settings, then the Program Settings tab, and add ollama.exe (it is in %LOCALAPPDATA%\Programs\Ollama).
3. Set CUDA - Sysmem Fallback Policy to Prefer No Sysmem Fallback and press Apply.
4. Quit Ollama and open it again. What does not fit then usually goes to the processor instead, which shows up as two figures in ollama ps — and the section above makes it fit.

If all of it is on the card with no overflow and it is still slower than it should be, check that a laptop is plugged in and set to Best performance (Settings → System → Power & battery), and that the card does not slow down to keep cool: a stress test shows whether it does.

The same in LM Studio and the others

LM Studio, llama.cpp, KoboldCpp and Jan are built on the same engine, and the same rules apply. In LM Studio, the model’s load settings have a GPU Offload slider: all the way to the right puts every layer on the card, and Context Length sets the conversation it holds — LM Studio not using the GPU goes through each. In llama.cpp the equivalent is --n-gpu-layers 99, and in KoboldCpp and Jan the GPU layers setting.

How Bottleneck checks this

Bottleneck’s free app does all of this in one click. On its AI models screen, press Check my local AI: it finds the model you are running with Ollama, LM Studio, llama.cpp, KoboldCpp or Jan, sees where it really runs — all on the card, partly on the processor, or overflowing into system memory — times how fast it writes, and sets that beside what your PC should manage with the same model. Nothing leaves your PC.

Where it runs and how fast are free. Pro, $9 once, names what is holding it back on your PC — the conversation length, the programs holding the card’s memory by name, a setting hiding the card, an old driver, memory running in one channel — with the steps to fix each, and compares each check with the last, so you see the change work.