Google refines its TPUs for inference and confidential computing
Google expands its eighth-generation TPU lineup with two systems, TPU 8tx and TPU 8i Max. One targets sensitive uses with confidential computing, the other large-context inference workloads.
Google expands its eighth-generation TPU lineup with two new configurations, TPU 8tx and TPU 8i Max, as the company says it offers four distinct systems within a single year. The first targets regulated enterprises and government agencies handling sensitive data; the second is aimed at heavy inference workloads, notably agents and very long contexts. Behind these announcements, the challenge is simple: better separate uses to gain in latency, memory, or security, depending on the need.
Two TPUs, two very different problems
TPU 8tx, which uses the same silicon as the TPU 8t, adds a layer of confidential computing thanks to AMD EPYC Turin 5th generation hosts. The principle of confidential computing is to protect data even when processed in memory, in an isolated environment. Google positions it for proprietary or frontier AI models used with regulated data.
The configuration relies on a Boardfly topology, designed to reduce network distance between chips. According to the source, this can lower latency by up to twice compared to the TPU 8t, with a performance impact considered almost negligible. For AI systems chaining inference calls, this detail is not cosmetic: a few hops saved means responses arrive a bit faster, and no one likes a model that takes tea breaks between tokens.
Why AMD comes out ahead in this story
The choice of AMD is no accident. Google selected EPYC Turin to align the PCIe Gen5 x16 interface of the TPU 8t, and a dual-socket platform provides 128 Gen5 lanes, enough to avoid I/O overallocation around four TPUs and high-speed network cards. The source also specifies that a newer host, like AMD EPYC Venice, would practically fall back to Gen5 speeds without providing additional bandwidth to the accelerators.
In other words, the gain is not only technical; it is also political in the good sense of the term, that of cloud providers balancing security, compatibility, and hardware availability. For AMD, this is not a statistical breakthrough across the entire server market, but a strong signal: EPYC enters an acceleration configuration at a major hyperscaler, in an area where confidentiality matters as much as raw power.
TPU 8i Max: more memory for large contexts
TPU 8i Max addresses another bottleneck, memory. Google combines two TPU 8i chips in the same package to double compute, onboard SRAM, and HBM bandwidth. The system offers 384 GB of HBM, about one third more than a single TPU 8i, as well as 17.2 TB/s of HBM bandwidth and 768 MB of on-chip SRAM.
This type of configuration targets uses where memory saturates before compute: processing entire codebases, simultaneous analysis of long videos, or agents that must retain a lot of context. Google thus claims to address the “memory wall,” the limit that blocks workloads when data moves slower than it is computed. TPU 8i Max is therefore not just a doubling of power; it is a targeted response to a very concrete problem of large-scale inference computing.
An increasingly segmented lineup
These announcements show a clear strategy: Google is not seeking a universal TPU, but several variants adapted to distinct profiles. TPU 8tx prioritizes confidentiality and regulated environments; TPU 8i Max favors latency and memory capacity for inference. This specialization by use resembles less a race for the biggest number than a watchmaker’s work on the right compromise. And in AI infrastructures, compromise often makes all the difference.
Key points
- Google adds TPU 8tx and TPU 8i Max to its eighth-generation lineup.
- TPU 8tx targets regulated enterprises and governments.
- Confidential computing relies on AMD EPYC Turin hosts.
- TPU 8i Max doubles the memory and bandwidth of the TPU 8i.
- Google mentions four system configurations within a year.
By the numbers
- 2x — latency potentially reduced with TPU 8tx compared to TPU 8t, according to Google.
- 128 Gen5 lanes — provided by a dual-socket Turin platform.
- 384 GB — HBM memory of the TPU 8i Max.
- 17.2 TB/s — HBM bandwidth announced for TPU 8i Max.
- 768 MB — on-chip SRAM of the TPU 8i Max.