Understanding Your First Local AI Stack, Part 2: Models, GGUF and Quantisation
·11 mins
In Part 1 we established that “the AI” is a file of numbers plus an engine to execute it. This time we open the file. By the end you will be able to look at a model name and predict, with arithmetic rather than hope, whether it will run on your machine and roughly how fast. A Model Is a Very Boring File # Strip away the mystique and a language model is an array of floating-point numbers — the weights — organised into named tensors, plus enough metadata to describe how those tensors should be wired together.