Google has released TorchTPU, its own version of the PyTorch software toolkit, as an open source project. The company announced the move on Tuesday, October 20.
The launch comes one day after Amazon released a similar tool. Both companies are working to reduce Nvidia’s control over how AI apps are built.
Today, most AI apps are written by default to run on Nvidia’s chips, which are known as GPUs.
What PyTorch Is and Why It Matters
PyTorch is a popular open source toolkit. AI app developers and startups use it to build, train, and run AI models.
By some accounts, PyTorch is used by 85% of AI apps. Meta, then called Facebook, first developed it in 2016.
At the time, Nvidia’s GPUs were already the standard hardware for AI research. Because of this, many PyTorch features were built to work specifically with Nvidia chips.
In 2022, Meta handed PyTorch over to the Linux Foundation. The nonprofit manages many important open source projects.
Amazon and Google Release Rival Tools
On Monday, Amazon released its own version of PyTorch, called TorchNeuron. It lets developers move AI apps to Amazon’s homegrown AI chip, Trainium, and to Amazon Web Services without rewriting them.
Amazon is also submitting code to the main PyTorch project. This code separates various features from Nvidia’s chips, making PyTorch more open to any hardware by default.
Google’s TorchTPU works in a similar way. It lets AI apps run on Google’s AI chips, called Tensor Processing Units, or TPUs.
The tool makes it easy to move existing PyTorch apps to TPUs and to Google Cloud. Developers can do this with a simple configuration change and no rewriting of their apps.
Google says it worked directly with several partners on the project. These include Meta, Hugging Face, Inferact, which makes vLLM, and RadixArk, which makes SGLang.
The goal was to make sure TorchTPU’s features and code libraries work well with these popular AI development tools. The original PyTorch already works well with them.
Google also launched a dedicated tool called the TPU agent suite. It helps developers move their apps to Google’s chips and cloud and fix errors along the way.
The company also updated LiteRT, its popular tool for running AI on phones, laptops, and embedded devices. The update helps LiteRT work better with PyTorch AI apps.
