Breadth-First Pipeline Parallelism

We introduce Breadth-First Pipeline Parallelism, a novel training schedule which optimizes the combination of pipeline and data parallelism. Breadth-First Pipeline Parallelism lowers training time, cost and memory usage by combining a high GPU utilization with a small batch size per GPU, and by making use of fully sharded data parallelism. Experimentally, we observed an increase of up to 43% in training throughput for a 52 billion-parameter model using a small batch size per GPU compared to Megatron-LM, which would reduce the training time and cost by the same amount on a large GPU cluster

Exact Results in Supersymmetric Gauge Theory

Exact results are a key component for understanding any physical theory. Unfortunately in the context of quantum field theory (QFT) they are in general impossible to obtain, and we need some sort of approximations. However there exists certain non-realistic theories rich in exactly computable quantities, and from those exact quantities we can infer various theoretical implications for realistic quantum field theories. Supersymmetric gauge theories stand out among these non-realistic theories as the best compromise between the contradicting requirements of realism and exact computability. This thesis consists of three projects, in which we explore some exact results in supersymmetric quantum field theory. In the first project we define and describe irregular vertex operators in the H_3^+ Wess-Zumino-Witten model. Irregular vertex operators are a QFT-equivalent of irregular singular points in the theory of differential equations, and their study is motivated by a relation to the partition functions of some asymptotically free four-dimensional N=2 supersymmetric gauge theories. The definition is shown to be compatible with previously defined irregular vertex operators in Liouville theory through a known duality between the H_3^+ and Liouville theories. In the second project we use supersymmetric localization to compute the partition function of N=2 supersymmetric gauge theories on a four-sphere in the presence of a surface defect on a two-sphere subspace, taking the form of a two-dimensional gauged linear sigma model. The result generalizes the known results for separate supersymmetric gauge theories on the separate spaces. We obtain a partition function in the form of a standard partition function on S^4, with a modified instanton partition function and an additional insertion corresponding to a shifted version of the S^2 partition function. In the third project we develop a new method for finding the ground states of fermions in the presence of BPS monopoles. We use it to find the ground states in the case of Abelian BPS monopoles in R^3, which were previously unknown

Path representation of su(2)_k states II: Operator construction of the fermionic character and spin-1/2--RSOS factorization

This is the second of two articles (independent of each other) devoted to the analysis of the path description of the states in su(2)_k WZW models. Here we present a constructive derivation of the fermionic character at level k based on these paths. The starting point is the expression of a path in terms of a sequence of nonlocal (formal) operators acting on the vacuum ground-state path. Within this framework, the key step is the construction of the level-k operator sequences out of those at level-1 by the action of a new type of operators. These actions of operators on operators turn out to have a path interpretation: these paths are precisely the finitized RSOS paths related to the unitary minimal models M(k+1,k+2). We thus unravel -- at the level of the path representation of the states --, a direct factorization into a k=1 spinon part times a RSOS factor. It is also pointed out that since there are two fermionic forms describing these finite RSOS paths, the resulting fermionic su(2)_k characters arise in two versions. Finally, the relation between the present construction and the Nagoya spectral decomposition of the path space is sketched.Comment: 28 page

Investigation of Association between PFO Complicated by Cryptogenic Stroke and a Common Variant of the Cardiac Transcription Factor GATA4

Patent foramen ovale (PFO) is associated with clinical conditions including cryptogenic stroke, migraine and varicose veins. Data from studies in humans and mouse suggest that PFO and the secundum form of atrial septal defect (ASDII) exist in an anatomical continuum of septal dysmorphogenesis with a common genetic basis. Mutations in multiple members of the evolutionarily conserved cardiac transcription factor network, including GATA4, cause or predispose to ASDII and PFO. Here, we assessed whether the most prevalent variant of the GATA4 gene, S377G, was significantly associated with PFO or ASD. Our analysis of world indigenous populations showed that GATA4 S377G was largely Caucasian-specific, and so subjects were restricted to those of Caucasian descent. To select for patients with larger PFO, we limited our analysis to those with cryptogenic stroke in which PFO was a subsequent finding. In an initial study of Australian subjects, we observed a weak association between GATA4 S377G and PFO/Stroke relative to Caucasian controls in whom ASD and PFO had been excluded (OR = 2.16; p = 0.02). However, in a follow up study of German Caucasians no association was found with either PFO or ASD. Analysis of combined Australian and German data confirmed the lack of a significant association. Thus, the common GATA4 variant S377G is likely to be relatively benign in terms of its participation in CHD and PFO/Stroke