Google TPU 8tx and TPU 8i Max systems
Gear / The story

Google’s TPU 8tx and 8i Max solve different problems: privacy and the memory wait

The new configurations pair existing silicon with different system choices. Confidential computing and memory bandwidth answer different workload constraints.

GeekRadarPublished 3 min read

Google’s expanded TPU range tackles two different obstacles to running large AI workloads: protecting sensitive information while it is being processed, and feeding enough data to the processors. TPU 8tx targets the first; TPU 8i Max targets the second. The distinction is more useful than treating both as another step up a single accelerator ranking.

In its November 9 announcement, Google describes new system configurations built around existing eighth-generation silicon. That matters because a system’s behavior depends on its memory, host processors and connections as well as the arithmetic inside a chip.

TPU 8tx addresses data while it is in use

TPU 8tx combines TPU 8t silicon with fifth-generation AMD EPYC hosts and the low-latency Boardfly network. Google positions it for confidential inference and smaller training workloads, including adapting models with private information. It claims up to twice lower latency than TPU 8t with a near-zero performance penalty for the security approach; those are vendor claims, not results from an independent GeekRadar test.

The underlying security problem is specific. Encrypting a stored file or a network connection does not, by itself, protect the data during computation. Google’s confidential-computing overview describes hardware-backed isolation and protection of data in use, with attestation available to verify trusted environments.

That does not make application permissions irrelevant. A protected computation can still return a result to an authorized caller, and an application can still give that caller more information than intended. Hardware isolation and sensible access rules address different parts of the problem. An organization choosing this approach needs to define what it wants to keep hidden, from whom, and at which point in the workflow.

Google’s four eighth-generation TPU system configurations.
Google’s four eighth-generation TPU system configurations.

TPU 8i Max gives the compute more to work with

TPU 8i Max places two TPU 8i chips in one package. Google says compute, SRAM and high-bandwidth memory bandwidth double relative to TPU 8i. Those are system-resource comparisons, not a guarantee that every application runs twice as fast.

Consider a workload in which the processor repeatedly waits for information. Adding arithmetic capacity alone can leave that waiting time intact. Increasing memory bandwidth alongside compute is intended to preserve the balance between doing work and supplying the data needed for it. Long contexts make that balance particularly relevant, because more information must remain accessible as the model continues producing output.

The announcement does not provide a universal customer price or a result that can be applied unchanged to every model. A useful comparison would keep model, precision, context length, concurrency and latency target consistent. Comparing only peak throughput can reward a configuration that misses the response time an interactive application needs.

Software determines whether the hardware is reachable

Google also points to its PyTorch work, following the native TPU support we examined previously. Easier access matters because a theoretically suitable accelerator has little practical value if adopting it requires an uneconomic rewrite or unsupported operations in a team’s model.

The new systems therefore offer two different reasons to investigate the platform. A privacy-sensitive workload may value the execution boundary; a long-context service may value the memory arrangement. Neither choice should begin with the largest number on a slide. It should begin with the constraint that is currently stopping the application from doing useful work.