Google open-sources TorchTPU so PyTorch code can run on its TPU chips, a day after Amazon did the same for Trainium

Google said on Tuesday, Oct. 20, 2026, at PyTorch Conference 2026, that TorchTPU, its native PyTorch backend for Google's Tensor Processing Units, is now open source on GitHub, along with new vLLM and SGLang serving engines for TPUs and a preview of a TPU agent suite that helps developers port and tune models.

Two days, two clouds, one target. Nvidia's real moat has never been just chips; it's that the world's AI code assumes a GPU. Amazon tried to pry that loose on Monday with TorchNeuron, and Google answered on Tuesday with TorchTPU plus a co-signer Amazon didn't have: Meta's own PyTorch team, and the Hugging Face, vLLM and SGLang folks on top. A one-word device change is a great demo, and Google admits it isn't the same as peak speed. But every friction point removed makes "we're on Nvidia because that's what our code runs on" a weaker excuse, and that's the excuse Nvidia's pricing has been leaning on.

TorchTPU is Google's PyTorch software for its Tensor Processing Units, and Google has published the code on GitHub under the Apache 2.0 license. Apache 2.0 lets anyone use, change, and share the code, including in a product they sell. PyTorch is the open-source toolkit most AI developers use to build, train, and run models. A TPU is Google's own AI chip. Companies use TPUs by renting them on Google Cloud. Google started TorchTPU in mid-2025 and built it with the PyTorch team at Meta. The code is Google's own project. It is not a feature of the main PyTorch release. Those lines are Google's and the PyTorch blog's.

The pitch is that existing PyTorch code runs on TPUs with a small change. The PyTorch blog's example is changing model.to("cuda") to model.to("tpu"). CUDA, in that line, is the usual way PyTorch code names an Nvidia GPU. The blog itself says this gets a model running. It does not get peak TPU speed, because the shapes and operations in a model may have been tuned for GPUs. Those lines are the PyTorch blog's.

What works today, as the PyTorch blog lists it: an eager mode, which runs each step as written, and a compiled mode through torch.compile, which prepares the model before it runs. About 450 core PyTorch operations, a large slice of the everyday commands, not every obscure one. Developers can add their own fast math routines, which Google says can be written with JAX, Pallas, or Helion. Google says TorchTPU ran without extra tuning on the 5,000 models people download most from Hugging Face, the site where ready-made models are shared, including Qwen, DeepSeek, Gemma, Llama, Mistral, and Whisper. It also connects to tools teams already use to serve and train models, including vLLM, SGLang, TorchTitan, Ray, and TorchRec. Google says some of its largest cloud customers already use it in production for training and for inference, the work of answering a request with a trained model. Unlike Google's older PyTorch/XLA path, this runtime does not need JAX, a separate Google math library, installed underneath. Those lines are the PyTorch blog's and Google's.

The partner list is specific. Hugging Face says version 5 of its Transformers library can send a model to a TPU with device_map="tpu". Inferact, the company behind vLLM, and RadixArk, behind SGLang, have new engines for serving models on TPUs. llm-d 1.0, a project with Red Hat and IBM, routes work across fleets that mix TPUs and GPUs. Google also put a TPU agent suite in an early preview on GitHub, to automate setup, diagnostics, and inference tuning, and it updated LiteRT, its software for running models on phones and other devices, so it can run PyTorch models. Those lines are Google's.

Google's own speed figure: TorchTitan training on TorchTPU hit up to 52% model FLOPs utilization on Qwen 3 8B, an 8-billion-parameter model, with near-linear scaling to 8,192 Ironwood TPUs. FLOPs utilization is the share of a chip's math capacity the training run actually used. Near-linear scaling means adding chips kept that efficiency almost intact, out to 8,192 of them. Ironwood is the TPU generation named in that test. The figure is Google's. The posts do not include a speed or cost comparison with Nvidia GPUs.

Jana van Greunen, Meta's director of PyTorch engineering, said, "We cannot be constrained by a single hardware architecture." That quotation is hers.

The day before, Amazon released TorchNeuron, its PyTorch backend for Trainium chips, TechCrunch notes. Most AI apps are written to run on Nvidia GPUs by default. That timing is TechCrunch's.

In plain terms, Google on Tuesday posted TorchTPU, free code so PyTorch programs can run on Google's TPU chips, and said changing one device name is enough to get a model running. Meta's PyTorch team helped build it. Google says peak speed still takes more tuning, and it did not publish a head-to-head against Nvidia. A day earlier, Amazon made the same kind of release for its Trainium chips.