Part 1 established that the model is stateless. Part 2 established that its memory is finite and expensive. Put those together and an awkward question falls out: if the model remembers nothing and can only be handed so much at once, who decides what it gets to see? The answer is the layer you thought was just a chat window. The Layer That Holds Everything the Model Can’t # Open WebUI is a web application. Svelte in the browser, a Python backend, a database on disk. There is no model inside it and no inference happening in it. If Ollama is switched off, Open WebUI still loads perfectly — it simply has nothing to talk to.
In Part 1 we established that “the AI” is a file of numbers plus an engine to execute it. This time we open the file. By the end you will be able to look at a model name and predict, with arithmetic rather than hope, whether it will run on your machine and roughly how fast. A Model Is a Very Boring File # Strip away the mystique and a language model is an array of floating-point numbers — the weights — organised into named tensors, plus enough metadata to describe how those tensors should be wired together.
Most local AI tutorials hand you a docker compose up and a shrug. This one starts with the architecture, because once you can name the layers, every tutorial after this becomes readable. The Naming Problem # Spend twenty minutes reading about local AI and you will collect a pile of nouns:
Ollama. Qwen. Llama. Gemma. GGUF. llama.cpp. Open WebUI. LM Studio. vLLM. Hugging Face.
Introduction # So about two months back, I moved to another city for work. My home PC — a fairly bulky Fedora machine that I’ve been calling titan — wasn’t something I was going to drag along in a bag. I left it behind at my hometown, plugged in and running on my home network. Before I left, I already had Tailscale configured on it, so at least I knew I could poke at it remotely if I needed to.
Introduction # It’s been quite a while since I started homelabbing explicitly. I have already posted about Monitoring My HomeLab With Prometheus and Grafana, but I haven’t posted any video on my channel yet.
So today I want to take some time to talk about what I’m already up to. And for this, I’m going to divided this post into the following sections:
