TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings

In response to innovations in machine learning (ML) models, production workloads changed radically and rapidly. TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x-7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus ~10x faster overall, which along with OCS flexibility helps large language models. For similar sized systems, it is ~4.3x-4.5x faster than the Graphcore IPU Bow and is 1.2x-1.7x faster and uses 1.3x-1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~3x less energy and produce ~20x less CO2e than contemporary DSAs in a typical on-premise data center.Comment: 15 pages; 16 figures; to be published at ISCA 2023 (the International Symposium on Computer Architecture

In-Datacenter Performance Analysis of a Tensor Processing Unit

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs (caches, out-of-order execution, multithreading, multiprocessing, prefetching, ...) that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.Comment: 17 pages, 11 figures, 8 tables. To appear at the 44th International Symposium on Computer Architecture (ISCA), Toronto, Canada, June 24-28, 201

Gain of Toxicity from ALS/FTD-Linked Repeat Expansions in C9ORF72 Is Alleviated by Antisense Oligonucleotides Targeting GGGGCC-Containing RNAs.

Hexanucleotide expansions in C9ORF72 are the most frequent genetic cause of amyotrophic lateral sclerosis and frontotemporal dementia. Disease mechanisms were evaluated in mice expressing C9ORF72 RNAs with up to 450 GGGGCC repeats or with one or both C9orf72 alleles inactivated. Chronic 50% reduction of C9ORF72 did not provoke disease, while&nbsp;its absence produced splenomegaly, enlarged lymph nodes, and mild social interaction deficits, but not&nbsp;motor dysfunction. Hexanucleotide expansions caused age-, repeat-length-, and expression-level-dependent accumulation of RNA foci and dipeptide-repeat proteins synthesized by AUG-independent translation, accompanied by loss of hippocampal neurons, increased anxiety, and impaired cognitive function. Single-dose injection of antisense oligonucleotides (ASOs) that target repeat-containing RNAs but preserve levels of mRNAs encoding C9ORF72 produced sustained reductions in RNA foci and dipeptide-repeat proteins, and ameliorated behavioral deficits. These efforts identify gain of toxicity as a central disease mechanism caused by repeat-expanded C9ORF72 and establish the feasibility of ASO-mediated therapy

Gain of Toxicity from ALS/FTD-Linked Repeat Expansions in C9ORF72 Is Alleviated by Antisense Oligonucleotides Targeting GGGGCC-Containing RNAs

Hexanucleotide expansions in C9ORF72 are the most frequent genetic cause of amyotrophic lateral sclerosis and frontotemporal dementia. Disease mechanisms were evaluated in mice expressing C9ORF72 RNAs with up to 450 GGGGCC repeats or with one or both C9orf72 alleles inactivated. Chronic 50% reduction of C9ORF72 did not provoke disease, while its absence produced splenomegaly, enlarged lymph nodes, and mild social interaction deficits, but not motor dysfunction. Hexanucleotide expansions caused age-, repeat-length-, and expression-level-dependent accumulation of RNA foci and dipeptide-repeat proteins synthesized by AUG-independent translation, accompanied by loss of hippocampal neurons, increased anxiety, and impaired cognitive function. Single-dose injection of antisense oligonucleotides (ASOs) that target repeat-containing RNAs but preserve levels of mRNAs encoding C9ORF72 produced sustained reductions in RNA foci and dipeptide-repeat proteins, and ameliorated behavioral deficits. These efforts identify gain of toxicity as a central disease mechanism caused by repeat-expanded C9ORF72 and establish the feasibility of ASO-mediated therapy.publisher: Elsevier articletitle: Gain of Toxicity from ALS/FTD-Linked Repeat Expansions in C9ORF72 Is Alleviated by Antisense Oligonucleotides Targeting GGGGCC-Containing RNAs journaltitle: Neuron articlelink: http://dx.doi.org/10.1016/j.neuron.2016.04.006 content_type: article copyright: © 2016 Elsevier Inc.status: publishe

Gain of Toxicity from ALS/FTD-Linked Repeat Expansions in C9ORF72 Is Alleviated by Antisense Oligonucleotides Targeting GGGGCC-Containing RNAs.

Hexanucleotide expansions in C9ORF72 are the most frequent genetic cause of amyotrophic lateral sclerosis and frontotemporal dementia. Disease mechanisms were evaluated in mice expressing C9ORF72 RNAs with up to 450 GGGGCC repeats or with one or both C9orf72 alleles inactivated. Chronic 50% reduction of C9ORF72 did not provoke disease, while&nbsp;its absence produced splenomegaly, enlarged lymph nodes, and mild social interaction deficits, but not&nbsp;motor dysfunction. Hexanucleotide expansions caused age-, repeat-length-, and expression-level-dependent accumulation of RNA foci and dipeptide-repeat proteins synthesized by AUG-independent translation, accompanied by loss of hippocampal neurons, increased anxiety, and impaired cognitive function. Single-dose injection of antisense oligonucleotides (ASOs) that target repeat-containing RNAs but preserve levels of mRNAs encoding C9ORF72 produced sustained reductions in RNA foci and dipeptide-repeat proteins, and ameliorated behavioral deficits. These efforts identify gain of toxicity as a central disease mechanism caused by repeat-expanded C9ORF72 and establish the feasibility of ASO-mediated therapy

Gain of Toxicity from ALS/FTD-Linked Repeat Expansions in C9ORF72 Is Alleviated by Antisense Oligonucleotides Targeting GGGGCC-Containing RNAs

Hexanucleotide expansions in C9ORF72 are the most frequent genetic cause of amyotrophic lateral sclerosis and frontotemporal dementia. Disease mechanisms were evaluated in mice expressing C9ORF72 RNAs with up to 450 GGGGCC repeats or with one or both C9orf72 alleles inactivated. Chronic 50% reduction of C9ORF72 did not provoke disease, while its absence produced splenomegaly, enlarged lymph nodes, and mild social interaction deficits, but no motor dysfunction. Hexanucleotide expansions caused age-, repeat length- and expression level-dependent accumulation of RNA foci and dipeptide-repeat proteins synthesized by AUG-independent translation, accompanied by loss of hippocampal neurons, increased anxiety, and impaired cognitive function. Single dose injection of antisense oligonucleotides (ASOs) that target repeat-containing RNAs but preserve levels of mRNAs encoding C9ORF72 produced sustained reductions in RNA foci and dipeptide-repeat proteins, and ameliorated behavioral deficits. These efforts identify gain-of-toxicity as a central disease mechanism caused by repeat-expanded C9ORF72 and establish the feasibility of ASO-mediated therapy