https://santoshk.dev/tags/gguf/Recent content in GGUF on Fullstack with SantoshHugo -- gohugo.ioen© 2026 Santosh KumarSat, 01 Aug 2026 22:50:57 +0530https://santoshk.dev/posts/2026/understanding-your-first-local-ai-stack-part-2-models-gguf-and-quantisation/Fri, 31 Jul 2026 00:00:00 +0000https://santoshk.dev/posts/2026/understanding-your-first-local-ai-stack-part-2-models-gguf-and-quantisation/<div class="lead text-neutral-500 dark:text-neutral-400 !mb-9 text-xl"> In <a href="https://santoshk.dev/posts/2026/understanding-your-first-local-ai-stack-part-1-the-big-picture/" >Part 1</a> we established that &ldquo;the AI&rdquo; is a file of numbers plus an engine to execute it. This time we open the file. By the end you will be able to look at a model name and predict, with arithmetic rather than hope, whether it will run on your machine and roughly how fast. </div> <h2 class="relative group">A Model Is a Very Boring File <div id="a-model-is-a-very-boring-file" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#a-model-is-a-very-boring-file" aria-label="Anchor">#</a> </span> </h2> <p>Strip away the mystique and a language model is an array of floating-point numbers — the weights — organised into named tensors, plus enough metadata to describe how those tensors should be wired together.</p>