In Part 1 we established that “the AI” is a file of numbers plus an engine to execute it. This time we open the file. By the end you will be able to look at a model name and predict, with arithmetic rather than hope, whether it will run on your machine and roughly how fast. A Model Is a Very Boring File # Strip away the mystique and a language model is an array of floating-point numbers — the weights — organised into named tensors, plus enough metadata to describe how those tensors should be wired together.
Most local AI tutorials hand you a docker compose up and a shrug. This one starts with the architecture, because once you can name the layers, every tutorial after this becomes readable. The Naming Problem # Spend twenty minutes reading about local AI and you will collect a pile of nouns:
Ollama. Qwen. Llama. Gemma. GGUF. llama.cpp. Open WebUI. LM Studio. vLLM. Hugging Face.
Introduction # So about two months back, I moved to another city for work. My home PC — a fairly bulky Fedora machine that I’ve been calling titan — wasn’t something I was going to drag along in a bag. I left it behind at my hometown, plugged in and running on my home network. Before I left, I already had Tailscale configured on it, so at least I knew I could poke at it remotely if I needed to.
