Hugging Face ❤️ TorchTPU: TPUs, Natively

Hugging Face says TorchTPU, a native PyTorch backend for Google's TPUs, now lets Transformers, Diffusers and Kernels run on TPU without porting. Loading a model on a TPU becomes a matter of passing the device name. The older route, PyTorch/XLA, worked but came with lazy tensors, surprise recompilations and hard debugging, and Hugging Face says its own Optimum TPU layer is no longer required for these libraries.

The early numbers are encouraging but honest about gaps. Of the Transformers tests that run on TPU, 96.5% pass. Hugging Face has filed 34 issues against TorchTPU and 19 are fixed. A ragged-dot kernel from the Hub made batch-64 decoding 21.5x faster on Qwen3-30B-A3B and 12.8x faster on Gemma 4 26B-A4B, compared with the default implementation on a TPU v6e-8.

This is a quiet story with real weight. Hardware competition in AI has been held back less by chips than by software. Most models and code are written in PyTorch, and PyTorch has felt most at home on NVIDIA GPUs. When a different accelerator requires rewriting code or learning JAX, teams stay where they are. Making the TPU just another device attacks that friction directly.

We think more real choice in accelerators is good for everyone building with AI. It puts pressure on prices and supply, and it gives researchers and startups more places to get compute. Hugging Face is a sensible partner for this because its libraries sit between most developers and the models they use.

The caveats: TPUs are available only on Google Cloud, so this widens choice among chips while deepening reliance on one cloud. A 96.5% pass rate is good for early work and also means some things will break; Hugging Face says so plainly and is still working on a nightly TPU test run. And the eye-catching speedups compare a tuned kernel to the default path on TPU, not TPU to GPU, so they say nothing yet about which hardware is cheaper for a given job.

I'm watching whether pass rates climb toward parity, whether independent cost comparisons appear, and whether other non-NVIDIA accelerators get the same native treatment.

Google has designed TPUs for about a decade and uses them to train and serve its own models, including Gemini. The TPU software stack grew up around TensorFlow and JAX, leaving PyTorch users on a harder path until now.