16,233 research outputs found
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Many deep learning applications benefit from using large models with billions
of parameters. Training these models is notoriously expensive due to the need
for specialized HPC clusters. In this work, we consider alternative setups for
training large models: using cheap "preemptible" instances or pooling existing
resources from multiple regions. We analyze the performance of existing
model-parallel algorithms in these conditions and find configurations where
training larger models becomes less communication-intensive. Based on these
findings, we propose SWARM parallelism, a model-parallel training algorithm
designed for poorly connected, heterogeneous and unreliable devices. SWARM
creates temporary randomized pipelines between nodes that are rebalanced in
case of failure. We empirically validate our findings and compare SWARM
parallelism with existing large-scale training approaches. Finally, we combine
our insights with compression strategies to train a large Transformer language
model with 1B shared parameters (approximately 13B before sharing) on
preemptible T4 GPUs with less than 200Mb/s network.Comment: Accepted to International Conference on Machine Learning (ICML) 2023.
25 pages, 8 figure
Exploring Fully Offloaded GPU Stream-Aware Message Passing
Modern heterogeneous supercomputing systems are comprised of CPUs, GPUs, and
high-speed network interconnects. Communication libraries supporting efficient
data transfers involving memory buffers from the GPU memory typically require
the CPU to orchestrate the data transfer operations. A new offload-friendly
communication strategy, stream-triggered (ST) communication, was explored to
allow offloading the synchronization and data movement operations from the CPU
to the GPU. A Message Passing Interface (MPI) one-sided active target
synchronization based implementation was used as an exemplar to illustrate the
proposed strategy. A latency-sensitive nearest neighbor microbenchmark was used
to explore the various performance aspects of the implementation. The offloaded
implementation shows significant on-node performance advantages over standard
MPI active RMA (36%) and point-to-point (61%) communication. The current
multi-node improvement is less (23% faster than standard active RMA but 11%
slower than point-to-point), but plans are in progress to purse further
improvements.Comment: 12 pages, 17 figure
Life on the land: New lives for agrarian questions
The politics of food, climate, energy, and the yet unfinished work ofending colonialism run square through questions of land. Theclassical agrarian question has taken on new forms, and a newintensity. We look at four dimensions of the agrarian questiontoday: urbanization and labor; care and social reproduction;financialization and global food systems; and social movements.On this 50th anniversary of JPS, we as the journal’s editors invitemore research, vigorous debate, and scholar-activism on theseissues in agrarian politics and beyond. We move into the journal’snext era hoping we might continue to better interpret the worldin order to change it.
Japanese Expert Teachers' Understanding of the Application of Rhythm in Judo: a New Pedagogy
Aim
The aim of this research is to understand the application of rhythm in judo through the experience of expert Japanese coaches.
Background
Scientists and experienced coaches agree rhythm is an important skill in people’s everyday life. There is currently no research that investigates the importance of rhythm in judo. People with a highly developed sense of rhythm, move properly, breathe properly, or begin and finish work at the right time. Where sport is concerned, motion and dance can play an important role not only in the improvement of performance, but also in the reduction, or even prevention of, injuries. Those who are naturally musically inclined (have a musical ear) may find they can improve their technique faster than others, and this is something that, by investigating the way expert coaches understand the application of rhythm in judo, this research seeks to understand.
As Lange, (1970) stated, factors of movement are ‘weight, space, time, and flow on the background of the general flux of movement in proportional arrangements’ (Bradley, 2008; Selioni, 2013; Youngerman, 1976), therefore, this research will investigate the interaction of body and mind. Dance training as well as judo are somatic experiences that have as their ultimate goal the attainment of a skilled body. With quality training an athlete gains an increased awareness of their body which leads to better control of movement and is very important for judo athletes. This training is found in Japanese kabuki dance (Hahn, 2007), the Greek syrtaki dance (Zografou & Pateraki, 2007), and in walking techniques used in the traditional and Olympic sports of Japanese judo and Greek wrestling.
Methods
Interpretative phenomenological analysis (IPA) was the most suitable data analysis approach for this study for a number of reasons, mainly because it was considered to most closely reflect the author's realist epistemological view. The idiographic approach and framework, particularly on IPA, was regarded as a useful framework in which the current topic could meaningfully be explored.
As this study is one of the first to explore this new thematic area, IPA was the preferred approach to address the goal of providing a detailed account of the expert’s experience. Therefore, semi-structured interviews were used as a data source. This is the most conventional form of data collection using IPA and most closely reflects the researcher-participant relationship. Semi-structured interviews provide considerable flexibility by allowing the researcher to be guided by the phenomena of interest to the participant.
In this study, purposive sampling was achieved using inclusion criteria pertaining to the research question.
Using the ranking system criteria based on the belt in combination with age employed by the International Judo Federation (IJF) and Kodokan Judo Institute, six expert coaches of forty years old and over with a minimum belt rank of 6th dan were selected as a sample.
Results
Both interviews and the codification process contributed to new findings regarding the application of rhythm to judo, and judo itself as a pedagogical tool.
The diagrammatic model can be considered a 'guideline' to the phenomena deemed most significant. The personal significance of rhythm in judo was evidenced by the frequency with which the interviewees naturally referred to it during the interviews. A number of interviewees said that it was important for rhythm to be second nature. Rhythm was also described as an integrated and representative
element in the context of training. This framework was seen as essential in providing the reader with a contextualised understanding of the phenomena considered most important for the current research. Interviewees reported various motives for employing training in rhythm such as faster technical development, better attack/defence, fitness, speed, skills acquisition, personal and spiritual growth, competition results.
Conclusions
This study offers first-hand accounts from professional coaches of a previously unknown phenomena, namely the use of rhythm in judo, and sheds insight on how judo experts understand rhythm in terms of training, competition, and personal growth. These findings suggest that outside of training, coaches play an important role in teaching, mentoring, and leading students. In conclusion, the research revealed four important points which form the basis of a new method of teaching judo: pedagogy, skills, rhythm and movement
Approximate Computing Survey, Part I: Terminology and Software & Hardware Approximation Techniques
The rapid growth of demanding applications in domains applying multimedia
processing and machine learning has marked a new era for edge and cloud
computing. These applications involve massive data and compute-intensive tasks,
and thus, typical computing paradigms in embedded systems and data centers are
stressed to meet the worldwide demand for high performance. Concurrently, the
landscape of the semiconductor field in the last 15 years has constituted power
as a first-class design concern. As a result, the community of computing
systems is forced to find alternative design approaches to facilitate
high-performance and/or power-efficient computing. Among the examined
solutions, Approximate Computing has attracted an ever-increasing interest,
with research works applying approximations across the entire traditional
computing stack, i.e., at software, hardware, and architectural levels. Over
the last decade, there is a plethora of approximation techniques in software
(programs, frameworks, compilers, runtimes, languages), hardware (circuits,
accelerators), and architectures (processors, memories). The current article is
Part I of our comprehensive survey on Approximate Computing, and it reviews its
motivation, terminology and principles, as well it classifies and presents the
technical details of the state-of-the-art software and hardware approximation
techniques.Comment: Under Review at ACM Computing Survey
Optimality and Complexity in Measured Quantum-State Stochastic Processes
If an experimentalist observes a sequence of emitted quantum states via
either projective or positive-operator-valued measurements, the outcomes form a
time series. Individual time series are realizations of a stochastic process
over the measurements' classical outcomes. We recently showed that, in general,
the resulting stochastic process is highly complex in two specific senses: (i)
it is inherently unpredictable to varying degrees that depend on measurement
choice and (ii) optimal prediction requires using an infinite number of
temporal features. Here, we identify the mechanism underlying this
complicatedness as generator nonunifilarity -- the degeneracy between sequences
of generator states and sequences of measurement outcomes. This makes it
possible to quantitatively explore the influence that measurement choice has on
a quantum process' degrees of randomness and structural complexity using
recently introduced methods from ergodic theory. Progress in this, though,
requires quantitative measures of structure and memory in observed time series.
And, success requires accurate and efficient estimation algorithms that
overcome the requirement to explicitly represent an infinite set of predictive
features. We provide these metrics and associated algorithms, using them to
design informationally-optimal measurements of open quantum dynamical systems.Comment: 31 pages, 6 appendices, 22 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/qdic.ht
Straggler Mitigation and Latency Optimization in Blockchain-based Hierarchical Federated Learning
Cloud-edge-device hierarchical federated learning (HFL) has been recently
proposed to achieve communication-efficient and privacy-preserving distributed
learning. However, there exist several critical challenges, such as the single
point of failure and potential stragglers in both edge servers and local
devices. To resolve these issues, we propose a decentralized and
straggler-tolerant blockchain-based HFL (BHFL) framework. Specifically, a
Raft-based consortium blockchain is deployed on edge servers to provide a
distributed and trusted computing environment for global model aggregation in
BHFL. To mitigate the influence of stragglers on learning, we propose a novel
aggregation method, HieAvg, which utilizes the historical weights of stragglers
to estimate the missing submissions. Furthermore, we optimize the overall
latency of BHFL by jointly considering the constraints of global model
convergence and blockchain consensus delay. Theoretical analysis and
experimental evaluation show that our proposed BHFL based on HieAvg can
converge in the presence of stragglers, which performs better than the
traditional methods even when the loss function is non-convex and the data on
local devices are non-independent and identically distributed (non-IID)
Aragonese stick dances: a CLIL proposal for PE
Premio extraordinario de Trabajo Fin de Máster curso 2021/2022. Máster en Enseñanza Bilingüe y Aprendizaje Integrado de Contenidos y Lenguas ExtranjerasIn recent years, PE is going through a Copernican change as far as its contents are referred inasmuch as step by step traditional sports and physical activities are giving way to expressive activities much more akin to CLIL such as theatrical performance and dance which, due to their very playful and inclusive nature can improve considerably motivation among those students not so physically gifted that are more often than not thwarted by the competitiveness of usual sports taught in PE. The following bilingual proposal for CLIL PE will focus more concretely on a specific kind of dance seldom used in Physical Education: stick dance. The studies about its pedagogical benefits are few and far between either in Spanish or in English, probably because of its close connection with folklore, which until recently has been disregarded and dismissed in PE as something outdated. The CLIL didactic proposal will be centred upon Naval & Boltaña’s Aragonese stick dances for 2nd year ESO.Recientemente la asignatura de Educación Física está experimentando un cambio copernicano en sus contenidos, por cuanto poco a poco los deportes y actividades físicas tradicionales están dando paso a actividades expresivas más afines al enfoque AICLE, tales como el teatro o la danza, que, dada su misma naturaleza lúdica e inclusiva, pueden mejorar significativamente la motivación entre aquellos estudiantes no muy dotados físicamente y que con mucha frecuencia se ven frustrados por la competitividad de los deportes habitualmente enseñados en Educación Física. La siguiente propuesta bilingüe para Educación Física y AICLE se centrará en un tipo específico de baile rara vez abordado en Educación Física: el baile de bastones. Los estudios al respecto son pocos y aislados, ya sea en español o en ingles, probablemente debido a su estrecha relación con el folclore, lo cual hasta hace poco ha hecho que este tipo de baile haya sido descartado en Educación Física como algo anticuado. La propuesta didáctica aborda los palotiaus (bailes de bastón) aragoneses de Naval y Boltaña para 2º de la ESO, llevada a cabo desde la perspectiva AICLE
- …