Best Local LLM Models

Choose a model that fits your device and handles the work you need. Start small, use an instruction-tuned version for chat, and compare a few real answers before moving to a larger download.

Llama, Qwen, Gemma, Mistral, Phi and DeepSeek offer different sizes and licenses. The table below names exact variants, so you can compare the files you would run.

Last updated:

Six models to compare

Named releases, not a measured ranking.

Concrete open-weight variants and example local downloads
ModelParameter sizeLicenseExample downloadLocal runtime and fit
Llama 3.2 3B InstructMeta3.21B1B-class option also availableLlama 3.2 Community License2.0 GBQ4_K_MOllama; MLX LM also documents a 4-bit Llama 3.2 3B package. A compact text-only starting point.
Qwen3.5-4BQwen4B classImage and text inputApache 2.03.3 GBllama.cpp packageOllama offers llama.cpp and MLX packages. Choose the package that matches your runtime.
Gemma 4 E2BGoogle2.3B effective5.1B with embeddingsApache 2.04.34 GBQAT Q4_0 + projectorGoogle documents llama.cpp support. Effective parameter count is smaller than the full weight count.
Ministral 3 3B InstructMistral3.4B language+ 0.4B vision encoderApache 2.03.0 GBOllama 3B packageOllama has 3B, 8B and 14B packages. The small variant includes vision support.
Phi-4-mini-instructMicrosoft3.8B classText inputMIT2.5 GBQ4_K_MOllama supports the 3.8B variant. Microsoft designed it for memory-constrained uses and reasoning tasks.
DeepSeek-R1-Distill-Qwen-7BDeepSeek7B class7.62B in this packageMIT; Qwen base Apache 2.04.7 GBQ4_K_MOllama supports the named 7B distill. Keep the full name: its default R1 tag selects a different model.

Download sizes are the sources’ rounded figures on the date above. They describe those packages, not total RAM use. Model support depends on the installed app and runtime version.

Budget for the whole session

A downloaded model occupies storage. Running it also needs memory for the weights, conversation context and temporary work. Leave space for your operating system and other apps.

Google estimates 2.9 GB of GPU or TPU memory to load Gemma 4 E2B at Q4_0 and 4.5 GB for E4B. These figures exclude context and supporting software and vary by tool and environment.

LM Studio recommends 16 GB or more of RAM and suggests smaller models with modest context on 8 GB Macs. MLX LM documents how context-cache size and prompt processing affect memory. Begin with short conversations, then add documents and longer context after checking the result.

Use the hardware-by-hardware guide to narrow your first choice for a phone, laptop or GPU.

Match the file to the app

GGUF is a model container used by llama.cpp and compatible apps. MLX packages target Apple Silicon through MLX-based runtimes. A matching file extension is only the first check: your app must also support the model architecture.

For everyday chat, our pick is Loci. Start with its recommendation or choose another model from the list your device supports. Once you download one, typed chat works offline. To choose a specific file, use LM Studio. For scripts that call a local model over an API, use Ollama.

The offline model download guide explains quantization, companion files and source checks before you download.

Compare on work you recognize

Give each candidate the same short task: rewrite one of your notes, explain a passage you understand, or summarize a document you have read. Check whether it follows your instructions and preserves the facts.

Try the language and document length you expect to use. Keep the model name and quantization with your notes. Change one setting at a time so you can tell what affected the answer. Local models can make errors, including plausible details that the source never contained.

How we chose

We selected concrete variants from six model families, with publisher model cards and documented local distributions. We compared sizes, licenses and runtime options. We did not run a cross-model benchmark or certify these downloads on every device. The links in the table lead to the model cards and exact distribution examples.

Questions and answers

Which local model should I start with?

Start with your app’s hardware recommendation. If you choose files yourself, a supported, quantized instruction model in the 1B to 4B class is a useful first trial. Test it on your own writing or questions before downloading a larger model.

Does a 4 GB download need only 4 GB of RAM?

No. The runtime also needs memory for context and temporary work. Your operating system and other apps need memory too. Download size helps you plan storage; it does not establish the RAM needed for a complete session.

Are all open-weight models open source?

No. Open weights describe access to the model’s learned parameters. Usage and redistribution depend on the license for the exact release. Llama 3.2 has a custom community license; the Qwen3.5, Gemma 4 and Ministral 3 variants listed here use Apache 2.0.

What does the E in Gemma 4 E2B mean?

E means effective parameters. Google lists E2B at 2.3 billion effective parameters and 5.1 billion including embeddings. The smaller number is not the total weight count, so it cannot predict the download size on its own.

Is a DeepSeek distill the full DeepSeek-R1 model?

No. A distill is a smaller model trained using reasoning examples from a larger model. The named 7B Qwen distill differs from the full R1 model and from other distills. Keep the complete model name when comparing files or licenses.

Can I use these models without internet?

You can run a supported local model after downloading its required files and the app that runs it. Cloud models, web search and new downloads still need a connection. Check that you selected a local model before going offline.

Does a larger model always give a better answer?

No. The task, model training, prompt, quantization and context all affect the result. A larger file can also leave less memory for your documents. Compare models on a few real tasks and check the answers against sources you trust.