7,591 research outputs found

    SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient

    Full text link
    Many deep learning applications benefit from using large models with billions of parameters. Training these models is notoriously expensive due to the need for specialized HPC clusters. In this work, we consider alternative setups for training large models: using cheap "preemptible" instances or pooling existing resources from multiple regions. We analyze the performance of existing model-parallel algorithms in these conditions and find configurations where training larger models becomes less communication-intensive. Based on these findings, we propose SWARM parallelism, a model-parallel training algorithm designed for poorly connected, heterogeneous and unreliable devices. SWARM creates temporary randomized pipelines between nodes that are rebalanced in case of failure. We empirically validate our findings and compare SWARM parallelism with existing large-scale training approaches. Finally, we combine our insights with compression strategies to train a large Transformer language model with 1B shared parameters (approximately 13B before sharing) on preemptible T4 GPUs with less than 200Mb/s network.Comment: Accepted to International Conference on Machine Learning (ICML) 2023. 25 pages, 8 figure

    Practical View-Change-Less Protocol through Rapid View Synchronization

    Full text link
    The emergence of blockchain technology has renewed the interest in consensus-based data management systems that are resilient to failures. To maximize throughput of these systems, we have recently seen several prototype consensus solutions that optimize for throughput at the expense of overall implementation complexity, high costs, and reliability. Due to this, it remains unclear how these prototypes will perform in real-world environments. In this paper, we present the Practical View-Change-Less Protocol PVP, a high-throughput, simple, and reliable consensus protocol. Central to PVP is the combination of (1) a chained consensus design for replicating requests with a reduced message cost; (2) the novel Rapid View Synchronization protocol that enables robust and low-cost failure recovery; and (3) a high-performance concurrent consensus architecture in which independent instances of the chained consensus operate concurrently to process requests with high throughput and without single-replica bottlenecks. Due to the concurrent consensus architecture, PVP greatly outperforms traditional primary-backup consensus protocols such as PBFT (by up to 430%), Narwhal (by up to 296%), and HotStuff (by up to 3803%). Due to its reduced message cost, PVP is even able to outperform RCC, a state-of-the-art high-throughput concurrent consensus protocol, by up to 23%. Furthermore, PVP is able to maintain a stable and low latency and consistently high throughput even during failures.Comment: 16 pages, 14 figure

    Improving the Model Consistency of Decentralized Federated Learning

    Full text link
    To mitigate the privacy leakages and communication burdens of Federated Learning (FL), decentralized FL (DFL) discards the central server and each client only communicates with its neighbors in a decentralized communication network. However, existing DFL suffers from high inconsistency among local clients, which results in severe distribution shift and inferior performance compared with centralized FL (CFL), especially on heterogeneous data or sparse communication topology. To alleviate this issue, we propose two DFL algorithms named DFedSAM and DFedSAM-MGS to improve the performance of DFL. Specifically, DFedSAM leverages gradient perturbation to generate local flat models via Sharpness Aware Minimization (SAM), which searches for models with uniformly low loss values. DFedSAM-MGS further boosts DFedSAM by adopting Multiple Gossip Steps (MGS) for better model consistency, which accelerates the aggregation of local flat models and better balances communication complexity and generalization. Theoretically, we present improved convergence rates O(1KT+1T+1K1/2T3/2(1λ)2)\small \mathcal{O}\big(\frac{1}{\sqrt{KT}}+\frac{1}{T}+\frac{1}{K^{1/2}T^{3/2}(1-\lambda)^2}\big) and O(1KT+1T+λQ+1K1/2T3/2(1λQ)2)\small \mathcal{O}\big(\frac{1}{\sqrt{KT}}+\frac{1}{T}+\frac{\lambda^Q+1}{K^{1/2}T^{3/2}(1-\lambda^Q)^2}\big) in non-convex setting for DFedSAM and DFedSAM-MGS, respectively, where 1λ1-\lambda is the spectral gap of gossip matrix and QQ is the number of MGS. Empirically, our methods can achieve competitive performance compared with CFL methods and outperform existing DFL methods.Comment: ICML202

    optimización da planificación de adquisición de datos LIDAR cara ó modelado 3D de interiores

    Get PDF
    The main objective of this doctoral thesis is the design, validation and implementation of methodologies that allow the geometric and topological modelling of navigable spaces, whether inside buildings or urban environments, to be integrated into three-dimensional geographic information systems (GIS-3D). The input data of this work will consist mainly of point clouds (which can be classified) acquired by LiDAR systems both indoors and outdoors. In addition, the use of BIM infrastructure models and cadastral maps is proposed depending on their availability. Point clouds provide a large amount of environmental information with high accuracy compared to data offered by other acquisition technologies. However, the lack of data structure and volume requires a great deal of processing effort. For this reason, the first step is to structure the data by dividing the input cloud into simpler entities that facilitate subsequent processes. For this first division, the physical elements present in the cloud will be considered, since they can be walls in the case of interior environments or kerbs in the case of exteriors. In order to generate navigation routes adapted to different mobile agents, the next objective will try to establish a semantic subdivision of space according to the functionalities of space. In the case of internal environments, it is possible to use BIM models to evaluate the results and the use of cadastral maps that support the division of the urban environment. Once the navigable space is divided, the design of topologically coherent navigation networks will be parameterized both geometrically and topologically. For this purpose, several spatial discretization techniques, such as 3D tessellations, will be studied to facilitate the establishment of topological relationships, adjacency, connectivity and inclusion between subspaces. Based on the geometric characterization and the topological relations established in the previous phase, the creation of three-dimensional navigation networks with multimodal support will be addressed and different levels of detail will be considered according to the mobility specifications of each agent and its purpose. Finally, the possibility of integrating the networks generated in a GIS-3D visualization system will be considered. For the correct visualization, the level of detail can be adjusted according to geometry and semantics. Aspects such as the type of user or transport, mobility, rights of access to spaces, etc. They must be considered at all times.El objetivo principal de esta tesis doctoral es el diseño, la validación y la implementación de metodologías que permitan el modelado geométrico y topológico de espacios navegables, ya sea de interiores de edificios o entornos urbanos, para integrarse en sistemas de información geográfica tridimensional (SIG). -3D). Los datos de partida de este trabajo consistirán principalmente en nubes de puntos (que pueden estar clasificados) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Además, se propone el uso de modelos BIM de infraestructuras y mapas catastrales en función de su disponibilidad. Las nubes de puntos proporcionan una gran cantidad de información del entorno con gran precisión con respecto a los datos ofrecidos por otras tecnologías de adquisición. Sin embargo, la falta de estructura de datos y su volumen requiere un gran esfuerzo de procesamiento. Por este motivo, el primer paso que se debe realizar consiste en estructurar los datos dividiendo la nube de entrada en entidades más simples que facilitan los procesos posteriores. Para esta primera división se considerarán los elementos físicos presentes en la nube, ya que pueden ser paredes en el caso de entornos interiores o bordillos en el caso de los exteriores. Con el propósito de generar rutas de navegación adaptadas a diferentes agentes móviles, el próximo objetivo intentará establecer una subdivisión semántica del espacio de acuerdo con las funcionalidades del espacio. En el caso de entornos internos, es posible utilizar modelos BIM para evaluar los resultados y el uso de mapas catastrales que sirven de apoyo en la división del entorno urbano. Una vez que se divide el espacio navegable, se parametrizará tanto geométrica como topológicamente al diseño de redes de navegación topológicamente coherentes. Para este propósito, se estudiarán varias técnicas de discretización espacial, como las teselaciones 3D, para facilitar el establecimiento de relaciones topológicas, la adyacencia, la conectividad y la inclusión entre subespacios. A partir de la caracterización geométrica y las relaciones topológicas establecidas en la fase anterior, se abordará la creación de redes de navegación tridimensionales con soporte multimodal y se considerarán diversos niveles de detalle según las especificaciones de movilidad de cada agente y su propósito. Finalmente, se contemplará la posibilidad de integrar las redes generadas en un sistema de visualización tridimensional 3D SIG 3D. Para la correcta visualización, el nivel de detalle se puede ajustar en función de la geometría y la semántica. Aspectos como el tipo de usuario o transporte, movilidad, derechos de acceso a espacios, etc. Deben ser considerados en todo momento.O obxectivo principal desta tese doutoral é o deseño, validación e implementación de metodoloxías que permitan o modelado xeométrico e topolóxico de espazos navegables, ben sexa de interiores de edificios ou de entornos urbanos, ca fin de seren integrados en Sistemas de Información Xeográfica tridimensionais (SIX-3D). Os datos de partida deste traballo constarán principalmente de nubes de puntos (que poden estar clasificadas) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Ademáis plantease o uso de modelos BIM de infraestruturas e mapas catastrais dependendo da súa dispoñibilidade. As nubes de puntos proporcionan unha gran cantidade de información do entorno cunha gran precisión respecto os datos que ofrecen outras tecnoloxías de adquisición. Sen embargo, a falta de estrutura dos datos e a seu volume esixe un amplo esforzo de procesado. Por este motivo o primeiro paso a levar a cabo consiste nunha estruturación dos datos mediante a división da nube de entrada en entidades máis sinxelas que faciliten os procesos posteriores. Para esta primeira división consideraranse elementos físicos presentes na nube como poden ser paredes no caso de entornos interiores ou bordillos no caso de exteriores. Coa finalidade de xerar rutas de navegación adaptadas a distintos axentes móbiles, o seguinte obxectivo tratará de establecer unha subdivisión semántica do espazo de acordo as funcionalidades do espazo. No caso de entornos interiores plantease a posibilidade de empregar modelos BIM para avaliar os resultados e o uso de mapas catastrais que sirvan de apoio na división do entorno urbano. Unha vez divido o espazo navigable parametrizarase tanto xeométricamente como topolóxicamene de cara ao deseño de redes de navegación topolóxicamente coherentes. Para este fin estudaranse varias técnicas de discretización de espazos como como son as teselacións 3D co obxectivo de facilitar establecer relacións topolóxicas, de adxacencia, conectividade e inclusión entre subespazos. A partir da caracterización xeométrica e das relación topolóxicas establecidas na fase previa abordarase a creación de redes de navegación tridimensionais con soporte multi-modal e considerando varios niveis de detalle de acordo as especificacións de mobilidade de cada axente e a súa finalidade. Finalmente comtemplarase a posibilidade de integrar as redes xeradas nun sistema SIX 3D visualización tridimensional. Para a correcta visualización o nivel de detalle poderá axustarse en base a xeometría e a semántica. Aspectos como o tipo de usuario ou transporte, mobilidade, dereitos de acceso a espazos, etc. deberán ser considerados en todo momento

    Architecture and Advanced Electronics Pathways Toward Highly Adaptive Energy- Efficient Computing

    Get PDF
    With the explosion of the number of compute nodes, the bottleneck of future computing systems lies in the network architecture connecting the nodes. Addressing the bottleneck requires replacing current backplane-based network topologies. We propose to revolutionize computing electronics by realizing embedded optical waveguides for onboard networking and wireless chip-to-chip links at 200-GHz carrier frequency connecting neighboring boards in a rack. The control of novel rate-adaptive optical and mm-wave transceivers needs tight interlinking with the system software for runtime resource management

    Anime Studies: media-specific approaches to neon genesis evangelion

    Get PDF
    Anime Studies: Media-Specific Approaches to Neon Genesis Evangelion aims at advancing the study of anime, understood as largely TV-based genre fiction rendered in cel, or cel-look, animation with a strong affinity to participatory cultures and media convergence. Making Neon Genesis Evangelion (Shin Seiki Evangerion, 1995-96) its central case and nodal point, this volumen forground anime as a media with clearly recognizable aesthetic properties, (sub)cultural affordances and situated discourses

    2023-2024 Undergraduate Catalog

    Get PDF
    2023-2024 undergraduate catalog for Morehead State University

    Offene-Welt-Strukturen: Architektur, Stadt- und Naturlandschaft im Computerspiel

    Get PDF
    Welche Rolle spielen Algorithmen für den Bildbau und die Darstellung von Welt und Wetter in Computerspielen? Wie beeinflusst die Gestaltung der Räume, Level und Topografien die Entscheidungen und das Verhalten der Spieler_innen? Ist der Brutalismus der erste genuine Architekturstil der Computerspiele? Welche Bedeutung haben Landschaftsgärten und Nationalparks im Strukturieren von Spielwelten? Wie wird Natur in Zeiten des Klimawandels dargestellt? Insbesondere in den letzten 20 Jahren adaptieren digitale Spielwelten akribischer denn je Merkmale der physisch-realen Welt. Durch aufwändige Produktionsverfahren und komplexe Visualisierungsstrategien wird die Angleichung an unsere übrige Alltagswelt stets in Abhängigkeit von Spielmechanik und Weltlichkeit erzeugt. Wie sich spätestens am Beispiel der Open-World-Spiele zeigt, führt die Übernahme bestimmter Weltbilder und Bildtraditionen zu ideologischen Implikationen, die weit über die bisher im Fokus der Forschung stehenden, aus anderen Medienformaten transferierten Erzählkonventionen hinausgehen. Mit seiner Theorie der Architektur als medialem Scharnier legt der Autor offen, dass digitale Spielwelten medienspezifische Eigenschaften aufweisen, die bisher nicht zu greifen waren und der Erforschung harrten. Durch Verschränken von Konzepten aus u.a. Medienwissenschaft, Game Studies, Philosophie, Architekturtheorie, Humangeografie, Landschaftstheorie und Kunstgeschichte erarbeitet Bonner ein transdisziplinäres Theoriemodell und ermöglicht anhand der daraus entwickelten analytischen Methoden erstmals, die komplexe Struktur heutiger Computerspiele - vom Indie Game bis zur AAA Open World - zu verstehen und zu benennen. Mit "Offene-Welt-Strukturen" wird die Architektonik digitaler Spielwelten umfassend zugänglich
    corecore