The best local LLM for most people with 32GB of memory is Qwen3.8 27B. It’s current, multimodal, strong across chat and code, and its 18GB Ollama build leaves enough room for a useful context on a 24GB GPU or 32GB unified-memory machine. Smaller computers need a smaller answer, not a brutally compressed version of the same model.
Local models are now practical on ordinary consumer hardware, but “best” changes with the job. Coding, document search, creative writing, tool use, and image understanding reward different capabilities. Text LLMs don’t become image generators because they can inspect an image.
The Short Version: Run Qwen3.8 27B as the all-rounder if you have 24GB of fast memory or 32GB of system memory. Pick Qwen3-Coder 30B for repository work. On an 8GB machine, Gemma 3 4B is the honest starting point. It’s better to run a small model cleanly than make a larger one swap itself into uselessness.
Table of Contents
- What You Need
- Seven Models Compared
- Best All-Around
- Best for Reasoning
- Best for Coding
- Best for Writing
- Best for 8GB Systems
- Best for Document Q&A
- Best Lightweight Agent Model
- How to Run Them
- Models to Skip
- How I Chose
- Bottom Line
What You Need to Run a Local LLM
Memory capacity decides whether a model fits. GPU acceleration and memory bandwidth decide whether it answers quickly. A CPU can run a quantized model from system RAM, but large models may generate slowly. A discrete GPU is fast until the model exceeds VRAM and spills work back to system memory.
The file size is only the starting number. The runtime, context cache, operating system, and other applications need memory too. Leave several gigabytes free, and leave more if you intend to use a long context. The RAM for local AI guide maps 8B, 70B, and 120B classes to practical hardware.
Apple Silicon and compact AI workstations use unified memory, which the GPU can access directly. The Mac mini vs Mac Studio comparison, AMD Ryzen AI Halo guide, and Ryzen AI Halo vs RTX Spark comparison explain three very different ways to buy that capacity.
Best Local LLM Models Compared
| Model | Best use | Ollama size | Context listed | Practical memory floor | Main drawback |
|---|---|---|---|---|---|
| Qwen3.8 27B | All-around chat and work | 18GB | 256K | 24GB fast memory or 32GB system RAM | Too large for ordinary 16GB laptops |
| gpt-oss 20B | Reasoning and tool use | 14GB | 128K | 24GB fast memory or 32GB system RAM | Reasoning can be slow and verbose |
| Qwen3-Coder 30B | Coding and agents | 19GB | 256K | 24GB fast memory or 32GB system RAM | Overkill for autocomplete |
| Mistral Small 3.2 24B | Writing and instruction following | 15GB | 128K | 24GB fast memory or 32GB system RAM | Older than Qwen3.8 |
| Gemma 3 4B | Low-RAM chat and images | 3.3GB | 128K | 8GB RAM | Limited reasoning depth |
| Ministral 3 8B | Document Q&A and RAG | 6GB | 256K | 12GB RAM; 16GB preferred | Long context still adds memory use |
| Granite 4 3B | Lightweight agents and tools | 2.1GB | 128K | 8GB RAM | Not a substitute for a larger reasoning model |
Sizes and advertised context windows were checked against the Ollama model library on September 13, 2026. A 256K label doesn’t mean you should allocate 256K tokens on a small machine. Start at 8K or 16K and increase only when the task needs it.
Best All-Around Local LLM: Qwen3.8 27B
Qwen3.8 is the current default because it covers professional writing, research, coding, tools, and image input without forcing a specialist choice. The Ollama build is 18GB and lists a 256K window. It was updated four weeks before this guide, while many still-recommended models are a generation or two behind.
The size is the cost. It won’t fit comfortably beside ordinary applications on a 24GB system-memory computer, and a full 256K context adds an impractical cache. The Qwen3.8 27B RTX 5090 guide explains why one 32GB card can run a four-bit build but still can’t treat the maximum context as free.
Best for Reasoning: gpt-oss 20B
OpenAI’s gpt-oss 20B remains a strong local reasoning and tool-use option. Its 14GB Ollama package is easier to fit than the Qwen models above, and the Apache 2.0 licence is straightforward for many personal and commercial projects.
Reasoning traces cost time and tokens. For short factual chat or rewriting a paragraph, a smaller direct model may feel better. Use gpt-oss when the job benefits from decomposition, tool calls, or a visible reasoning budget, not because “reasoning model” automatically means better prose.
Best Local LLM for Coding: Qwen3-Coder 30B
Qwen3-Coder 30B is the coding pick for a 24GB GPU or 32GB unified-memory machine. It’s a mixture-of-experts model with a 19GB Ollama package and a 256K listed context, so it can work across more of a repository than a small autocomplete model.
It’s too large and deliberate for inline completion. A 7B-class coder may feel much faster when it only needs to finish the current line. The local coding LLM guide separates autocomplete, chat, and agent work by memory tier.
Best for Writing: Mistral Small 3.2 24B
Mistral Small 3.2 is the writing pick because instruction following and controlled repetition matter more here than benchmark theatrics. The 15GB Ollama build also handles image input and tool calls, which makes it useful for turning notes, screenshots, and source material into a structured first draft.
It’s no longer the newest all-rounder, so I wouldn’t buy hardware specifically for it. Qwen3.8 is the better general recommendation when both fit. Mistral earns a place for people who already like its voice and want a mature 24B option with predictable size.
Best for 8GB Systems: Gemma 3 4B
Gemma 3 4B is the honest low-memory choice. Its 3.3GB Ollama build leaves room for the operating system on an 8GB machine and still accepts text and images. That makes it useful for summarizing short documents, extracting structured notes, and ordinary private chat.
A 4B model will miss nuance, make shallow plans, and lose more often on complex code or reasoning. Don’t stretch the context to 128K on an 8GB computer just because the model page lists it. The context cache can consume the headroom that made the model practical.
Best for RAG and Document Q&A: Ministral 3 8B
Ministral 3 8B is a sensible retrieval-augmented generation model because its 6GB package fits a 16GB computer comfortably and its listed 256K window leaves room to test larger document sets. It supports text, images, and tools, so one local service can retrieve passages and format a useful answer.
The model doesn’t fix poor retrieval. Chunking, embeddings, metadata, reranking, and citations determine whether the right passage reaches the prompt. Begin with a small context containing high-quality matches. Dumping a filing cabinet into 256K tokens is expensive avoidance.
Best Lightweight Agent Model: Granite 4 3B
IBM’s Granite 4 3B is only 2.1GB in Ollama and is built with tools and long context in mind. It’s a useful controller for narrow, well-defined local workflows where latency and memory matter more than broad knowledge. A home AI agent that sorts a known folder needs reliability and guardrails more than an eloquent essay.
Three billion parameters don’t provide deep judgment. Keep actions constrained, validate outputs, and require approval for destructive steps. If the agent must interpret ambiguous requests or repair unfamiliar code, move up to Qwen3-Coder or gpt-oss.
How to Actually Run These Models
Use Ollama when you want a repeatable command, local API, or background service. Use LM Studio when you want to browse files, compare quantizations, adjust settings visually, and chat. The Ollama vs LM Studio guide covers their current desktop, CLI, API, and headless options.
The step-by-step local LLM guide handles installation and quantization choices. If you’re still deciding whether the privacy and offline tradeoffs matter, read the plain-English local LLM explanation first.
Small hardware, like the Minisforum M2 Pro, can still be useful. The Lenovo Yoga AI Mini PC analysis is a useful warning that a claimed parameter limit doesn’t describe speed, context headroom, software support, or regional availability.
Local Models I Would Skip in 2026
Skip Llama 2 unless an old application specifically depends on it. It was important, but newer small models provide better instruction following and longer contexts. Familiarity isn’t a performance feature.
Skip the full DeepSeek-R1 671B on normal consumer hardware. Its Ollama build is about 404GB before context and runtime overhead. A heavily compressed version spread across slow memory may technically load while producing answers too slowly to be useful.
Skip extreme 1-bit and 2-bit quantizations when a smaller model fits at four bits. Parameter count looks impressive in a screenshot, but severe compression can erase the quality you bought those parameters to obtain.
How I Chose the Best Local LLM Picks
For this best local LLM shortlist, every recommended model had to have a current, directly downloadable Ollama entry with a published package size and a licence or official model record. I compared use-case claims with public evaluations such as Artificial Analysis and LMArena, then discounted any result that depended on a hosted build or undisclosed inference setup.
Leaderboards don’t reproduce your local result. They may test a larger precision, different prompt template, or server-grade hardware. I used them as evidence of capability, not as a ranking that overrides memory fit, latency, licence, or the actual task.
Bottom Line
Qwen3.8 27B is the best local LLM for a capable 2026 desktop because it handles the widest range of work without feeling like a specialist. Use Qwen3-Coder for code, gpt-oss for deliberate reasoning, Mistral Small for writing, Ministral for document Q&A, and Granite for narrow lightweight agents.
On an 8GB machine, choose Gemma 3 4B and keep the context modest. The smartest local setup isn’t the largest model you can force into memory. It’s the smallest model that does the job quickly enough that you will still use it next week.
