Ollama from scratch, part 2: model names, pulling, and what will actually run

This is part 2 of a five-part guide to running AI models locally with Ollama. Part 1 got Ollama installed and a model answering questions. This part is about choosing models: reading their names, pulling them, and knowing in advance whether one will run.

In part 1 you ran qwen2.5:0.5b and maybe llama3.2. Those names were handed to you. This part is about the rest of the library, so you can pick a model on purpose instead of copying one from a tutorial.

The library

Ollama has a catalogue of ready-to-use models at ollama.com/library. Each entry has a short description, a pull count, and a list of tags.

The Ollama model library listing llama3.1 and deepseek-r1 with their tags — Every model in the library has a family name and a set of tags. The tags are the sizes and variants you can pull.

To download one:

ollama pull llama3.2

ollama pull only downloads. ollama run downloads if needed and then starts a chat. Once a model is on disk, both are instant.

Reading a model name

A full model name looks like this:

llama3.1:8b-instruct-q4_K_M

It carries four separate pieces of information, split by the : and the -.

llama3.1 : 8b - instruct - q4_K_M
   │        │       │         │
 family   size   variant   quantization

Family and version. llama3.1 is the model family and release. llama3.2, gemma3, qwen2.5, phi4, mistral are other families. Newer is usually better at the same size.

Size. 8b means 8 billion parameters. A parameter is one number inside the model. More parameters means more capable, more memory, and slower. Common sizes are 1b, 3b, 7b, 8b, 14b, 32b, 70b. On a laptop you are usually choosing between 3b and 8b.

Variant. instruct (sometimes chat) means the model was tuned to follow instructions and hold a conversation. The other kind, sometimes tagged text or base, only continues text and will not answer questions well. You almost always want instruct, and for library models it is the default.

Quantization. q4_K_M is how tightly the numbers are packed. Model weights are normally 16-bit numbers. Quantization stores them in fewer bits, 4 here, which shrinks the file roughly fourfold with a small quality cost. It is the single setting that decides whether a model fits on your machine. q4_K_M is the sensible default and the one you get if you do not ask for another. I wrote about what Q4_K_M means in detail in a separate post; for now, "Q4 is the normal choice" is enough.

A bare tag like llama3.1:8b, or just llama3.1, is an alias that points at the default: the 8b size, instruct variant, Q4 quantization. :latest is the same idea and moves when the family is updated.

The tags table for llama3.1 showing 8b at 4.9 GB, 70b at 43 GB and 405b at 243 GB — The tags page for a model lists every size and quantization with its download size. 8b is 4.9 GB, 70b is 43 GB, 405b is 243 GB.

Model size: disk versus memory

The download size, the size on disk, and the amount of memory the model needs are all roughly the same number. For llama3.1:8b that number is about 4.7 GB.

To actually run, the model has to fit in memory, plus a bit of headroom for the conversation. A rough beginner's rule:

memory needed ≈ download size + 1 to 2 GB

So llama3.1:8b at 4.7 GB wants around 6 to 7 GB free. That has to fit in your graphics card's memory (VRAM) to be fast, or in system RAM to run at all, just slower. The detailed post has the real formula, including how a long conversation adds to it. The rule above is fine for choosing a first model.

Checking your machine

You need two numbers: total system memory, and graphics-card memory if you have a dedicated card.

macOS

Apple menu → About This Mac shows your total memory. In Terminal:

sysctl -n hw.memsize

That prints bytes; divide by 1073741824 for gigabytes. On Apple Silicon (M1 and later) the processor and graphics share this memory, so there is no separate number to look for, which is part of why these machines are good at this.

Windows

Open Task Manager with Ctrl+Shift+Esc, go to the Performance tab.

Memory shows total system RAM.
GPU shows Dedicated GPU memory (your VRAM) and Shared GPU memory (borrowed from RAM, slow). The dedicated number is the one that matters.

Linux

free -h                       # "total" on the Mem line is your RAM
nvidia-smi                    # NVIDIA: total VRAM shown top-right
rocm-smi --showmeminfo vram   # AMD

Did it actually fit? Ask `ollama ps`

After you run a model, in another terminal:

$ ollama ps
NAME            ID              SIZE      PROCESSOR    CONTEXT    UNTIL
llama3.1:8b     365c0bd3c000    6.7 GB    100% GPU     4096       4 minutes from now

The PROCESSOR column is the answer.

100% GPU — the whole model is on the graphics card. This is what you want.
100% CPU — running on the processor. Either you have no usable GPU, or the model did not fit in VRAM. It works, just slowly.
51%/49% CPU/GPU — part on each. The commonest "why is this so slow" cause: the model was a little too big for VRAM, so some layers spilled to the CPU.

If you are not on 100% GPU and it is too slow, the fixes in order are: a smaller model, a smaller quantization (q4 instead of q5, or q3 if you are desperate), or a smaller context:

>>> /set parameter num_ctx 2048

A safe first ladder

Start low and climb only when ollama ps says 100% GPU, or CPU speed is tolerable.

Model	Download	Fits comfortably in
`qwen2.5:0.5b`	~0.4 GB	anything
`llama3.2` (3b)	~2 GB	8 GB RAM
`llama3.1:8b`	~4.7 GB	16 GB RAM, or 8 GB VRAM
`qwen2.5:14b`	~9 GB	32 GB RAM, or 12 GB VRAM
`llama3.1:70b`	~40 GB	64 GB RAM, or 48 GB VRAM

Pull one, run it, check ollama ps, and only move up a rung if there is room.

Housekeeping commands

ollama list                 # what you have downloaded
ollama show llama3.1:8b      # architecture, parameters, context length, quant
ollama rm llama3.1:8b        # delete it and any weights nothing else uses
ollama pull llama3.1:8b      # run again later to update to the newest build

ollama show is worth knowing. It prints the facts the name only hints at:

  Model
    architecture        llama
    parameters          8.0B
    context length      131072
    quantization        Q4_K_M

  Capabilities
    completion
    tools

If you want a build that never changes under you, pin the full tag, the way you would pin a version of any dependency:

ollama pull llama3.1:8b-instruct-q4_K_M

What people get wrong

Trusting :latest. It is a moving pointer. Fine for playing, bad for anything you need to behave the same next month. Pin the full tag.
Choosing the biggest model that fits on disk. Disk is not the limit, memory is. A 40 GB model downloads fine and then will not load.
Assuming more parameters is always better. On a machine that cannot hold a 70b model, a 70b model is slower than reading the docs yourself. The largest model that fits entirely in VRAM beats a bigger one that does not, and it is not close.
Ignoring ollama ps. It answers "why is this slow" in one line, every time.

Next in the series

Part 3 covers models that are not in the library: pulling straight from Hugging Face, importing a GGUF file with a Modelfile, what "compatible" means, and how to read the error when a model will not load.