ABAB Pedia

Google Tensor Processing Unit

Google’s Tensor Processing Unit (TPU) is a family of machine-learning accelerators designed for recurring matrix operations in neural networks. The first generation served inference workloads in Google’s data centers. Later generations added training, cluster interconnects, public cloud access and designs specialized for different workloads.

Contents8 sections
Key facts

Deployment in Google's data centers

In a paper for the 2017 International Symposium on Computer Architecture, Norman Jouppi and colleagues documented deployment of the first TPU in Google data centers beginning in 2015. Its matrix unit contained 65,536 eight-bit multiply-accumulators for repeated computations in neural-network inference. The authors evaluated it on networks used in Google’s data centers and considered power consumption and latency alongside throughput.⁠[1]

Training and multi-chip systems

At Google I/O, the company introduced a second-generation Cloud TPU. The first generation had concentrated on inference using an already-trained model. The new hardware also supported training, which adjusts the model’s parameters from examples. Google presented TPU Pods, interconnected accelerators that operate as a system, and outlined plans to make this infrastructure available to cloud customers.⁠[2][1]

Public cloud beta

Google Cloud opened its Cloud TPU beta, giving developers cloud access to accelerators for training and running models. Google managed the systems and drivers, allowing teams to work with its processor designs without building their own installations. TPU identifies the processor family, while Cloud TPU is the service that provides access to corresponding compute resources.⁠[3][4]

Seventh-generation Ironwood

Google introduced Ironwood, its seventh-generation TPU, with an emphasis on the latency, memory bandwidth and chip-to-chip communication required for large-scale inference. On November 6, Google Cloud announced general availability in the coming weeks and identified large-scale training, reinforcement learning and high-volume inference among the supported workloads.⁠[5][6]

Separate architectures for training and inference

At Cloud Next, Google introduced two eighth-generation architectures. TPU 8t was designed for large-scale model training and TPU 8i for low-latency inference. The two designs in a single generation reflected different priorities: throughput across a training cluster and the time taken to serve an inference request.⁠[7]

TPU 8i Max and TPU 8tx

Google Cloud announced two more systems in the eighth generation. TPU 8i Max is designed for long contexts and high compute density per rack, while TPU 8tx addresses the confidentiality requirements of inference. Together with the two designs announced in April, the company identified four eighth-generation systems in one year for different training, inference and deployment needs.⁠[8][7]

Related Organizations, Websites, and Public Accounts

Google’s TPU product and cloud service site. https://cloud.google.com/tpu (opens in a new window)⁠[4]

Sources8 sources

  1. In-Datacenter Performance Analysis of a Tensor Processing Unit (opens in a new window)research.google↑
  2. Build and train machine learning models on our new Google Cloud TPUs (opens in a new window)blog.google↑
  3. Cloud TPU machine learning accelerators now available in beta (opens in a new window)cloud.google.com↑
  4. Cloud Tensor Processing Units (TPUs) (opens in a new window)cloud.google.com↑
  5. Ironwood: The first Google TPU for the age of inference (opens in a new window)blog.google↑
  6. Announcing Ironwood TPUs General Availability and new Axion VMs to power the age of inference (opens in a new window)cloud.google.com↑
  7. Our eighth generation TPUs: two chips for the agentic era (opens in a new window)blog.google↑
  8. Introducing TPU 8tx and TPU 8i Max (opens in a new window)cloud.google.com↑