Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA

Language models are famously shaky at exact arithmetic. The usual fix is to hand the math to a tool outside the model. qwenGPU, released under MIT by the developer known as @TheVixhal, takes the opposite route: it compiles a four-lane SIMT GPU into 45 untrained layers placed in front of Qwen3.5-2B's own 24, and the model runs real CUDA inside its forward passes.

This is a research demo, not a product. But it's another data point in a small, strange line of work showing that exact computation can live inside a pretrained model without wrecking its language ability.

We find this genuinely clever. The trick uses the Gated DeltaNet recurrent state, the model's own linear attention, as memory. Hold the gate open, make every key one-hot, and each row of state becomes a register or memory cell. A compiler sets every weight. Nothing is trained. The host compiles CUDA to PTX and feeds it in as tokens, and the model writes each launch step itself while the host reads results back without computing them.

The developer's numbers are encouraging but self-reported. Every launch was exact under replay, the reference interpreter matched a real Tesla T4 on 49 of 49 kernels, and the text distribution stayed within a KL of about 7e-4 of the original model. That last one matters most to us. Bolting a machine into a model is only interesting if the model still talks like itself.

Now the limits, and they're real. It runs at roughly 10 to 15 tokens per second on Apple silicon. In advanced mode, where Qwen writes the CUDA itself, it got 4 of 5 right on the first try, and the README itself warns that a 2B model's CUDA is unreliable. So the computation is exact, but the program it's asked to run can still be wrong. That's the trap I'd flag for anyone excited about this direction: exactness at the execution layer can make a wrong answer look more trustworthy, not less.

Where does this lead if it matures? The optimistic read is models that verify their own math internally, with fewer external tool calls and fewer places for things to break. The likelier near-term read is that this stays a research curiosity that sharpens our understanding of what these architectures can represent. Either way, it's the kind of open, inspectable experiment we want more of.

qwenGPU builds on the transformer compiler from Qwenputer by Divyansh Agrawal, which compiled a 32-bit RISC-V CPU into Qwen3.5-4B. Percepta built a WebAssembly interpreter into a transformer earlier this year. Running qwenGPU needs about 16 GB of free memory; no prebuilt checkpoint is published, so users build it locally.