How to Run Underdog Saluki 27B: 2-Bit Qwen3.8-27B Tool-Calling Guide

Underdog Saluki 27B is a roughly two-bits-per-weight build of Qwen3.8-27B that fits in 7.9 GB, against about 54 GB for the full model. Its makers say it keeps tool calling intact and even edges out the full model on their own 120-task benchmark. A new hands-on guide runs it in stock llama.cpp on a single GPU, a Colab T4 or an Apple-silicon Mac, next to the ISTA-DASLab quantization it was built from.

If the claim holds, a capable agent model drops onto hardware a lot of people already own. That's good for privacy, cost and offline use. It is also exactly the kind of claim that deserves a second look.

We like this guide because it treats a vendor number as a hypothesis. It pins every download by size and hash, diffs the two model files tensor by tensor, and scores both on the same seeded 120 tasks from the Berkeley Function Calling Leaderboard using BFCL's own checker. The vendor's benchmark tasks aren't published, so the guide draws its own sample from the same five categories. That's the right instinct.

The file forensics are the interesting finding. Both files share the same 851 tensors. Saluki changes the type of 146 of them, and every change cuts bits. But the bytes tell a different story than the header: only one of 352 quantized matrices is byte-identical to ISTA's, and the unquantized norm and gate vectors moved slightly. Saluki isn't just a re-encoding. Its weights were changed, which fits the makers calling it a preview checkpoint. So any tool-calling win over its parent is a test of that extra training, not of compression alone.

The eyes-open part: a small model that calls tools confidently is still a model that calls tools. The guide's irrelevance tasks, where the right answer is no call at all, are the ones we'd watch hardest, because an agent that fires a function when nothing fits is how local automation does damage. And the guide is honest that 120 tasks, greedy decoding and a single run leave wide error bars. A few tasks of difference sits inside the noise.

The trajectory is clear and mostly welcome. Hybrid architectures like this one, where most layers use linear attention and the KV cache costs a quarter of what full attention would, plus aggressive quantization, are pushing agent-grade models onto laptops. The cost is that provenance gets murky. When a file's header still names one lab as the quantizer while its weights have been altered by another, users need exactly this kind of verification to know what they're running.

Qwen3.8-27B is a hybrid model: 48 of its 64 blocks are Gated DeltaNet linear-attention layers and every fourth block keeps full attention, so the KV cache runs about 64 KiB per token instead of 256 KiB. Saluki builds on ISTA-DASLab's GSQ-RCO quantization of the same model.