LM Studio not using the GPU on Windows: how to fix it

First: see where the model is running

Load the model and ask it something long. While it writes, open Task Manager (Ctrl+Shift+Esc) and look at Performance:

• On the card: the GPU’s Dedicated GPU memory rose by about the size of the model when it loaded, the GPU is busy while it writes, and the processor stays mostly idle.
• Partly on the processor: the processor is busy on several cores while it writes, and the card’s memory holds less than the model.
• Overflowing: Shared GPU memory climbed by gigabytes. The card ran out of its own memory and is borrowing the PC’s, which is very slow — shared GPU memory explains why.

1. Move GPU Offload all the way up

When you load a model, LM Studio’s load settings include GPU Offload: how many of the model’s layers go on the graphics card, out of how many it has. Slide it to the maximum and load the model again. To keep it, open My Models, click the gear beside the model and save the setting as its default, so it loads that way every time.

If the model then fails to load, or Shared GPU memory climbs, it does not fit in the card’s memory at that setting — steps 3 and 4 make it fit.

2. Use the engine for your graphics card

LM Studio runs models with an engine (a runtime) it downloads for your hardware. In its settings, under Runtime, choose:

• CUDA llama.cpp for an NVIDIA card;
• Vulkan llama.cpp for an AMD or Intel card;
• not the CPU engine, which leaves the card unused.

Update the engine there too, then quit LM Studio, open it again and reload the model. If the card is not listed at all, install the newest graphics driver from the NVIDIA app, AMD Software or Intel and restart the PC.

3. Shorten the context length

Context Length, in the same load settings, is how many tokens of conversation the model can hold, and the card keeps room for all of them from the moment it loads. A long setting can take gigabytes and push layers off the card. 8,192 tokens, about 6,000 words, is enough for most chats; raise it only for long documents, and only as far as still fits.

4. Choose a smaller download

The same model comes in several sizes. If you downloaded a Q8_0 or Q6_K file, the Q4_K_M takes about 40% less memory than Q8_0 and loses little — which GGUF file to download explains the names. If even that does not fit, a smaller model that runs entirely on the card is usually faster and more useful than a bigger one half on the processor. Can my PC run AI? shows which models fit your card.

5. Free the card’s memory

Browsers, games, launchers and Discord keep some of the graphics card’s memory for themselves. In Task Manager’s Details tab, right-click a column heading, choose Select columns and tick Dedicated GPU memory to see how much each holds. Close what you can before loading a model.

6. On NVIDIA: stop it overflowing

NVIDIA’s driver lends system memory to a program that asks for more than the card has, which keeps a model running but slowly. In NVIDIA Control Panel, under Manage 3D settings, on the Program Settings tab, add LM Studio.exe and set CUDA - Sysmem Fallback Policy to Prefer No Sysmem Fallback. Then lower GPU Offload until the model fits, or use the steps above.

7. On a laptop

• Plug it in, and set Power mode to Best performance in Settings, System, Power & battery: on battery, laptops hold the graphics back.
• With two graphics chips, make sure LM Studio uses the stronger one: in Settings, System, Display, Graphics, add LM Studio and choose High performance.

What LM Studio needs

LM Studio’s own requirements for Windows are a processor with AVX2, at least 16 GB of memory and at least 4 GB of graphics memory recommended. A model runs fast only when all of it fits in the card’s memory with about a gigabyte to spare.

How Bottleneck checks this

Bottleneck’s free app has a Check my local AI button on its AI models screen. Load a model in LM Studio, press it and ask LM Studio to write something: Bottleneck watches how much of the card LM Studio uses, whether it overflows into shared memory and how hard the processor works, and says where the model is running. Where it runs is free; Pro names the cause on your PC and the steps to fix it.