Understand Data Preprocessing for Effective End-to-End Training of Deep Neural Networks

In this paper, we primarily focus on understanding the data preprocessing pipeline for DNN Training in the public cloud. First, we run experiments to test the performance implications of the two major data preprocessing methods using either raw data or record files. The preliminary results show that data preprocessing is a clear bottleneck, even with the most efficient software and hardware configuration enabled by NVIDIA DALI, a high-optimized data preprocessing library. Second, we identify the potential causes, exercise a variety of optimization methods, and present their pros and cons. We hope this work will shed light on the new co-design of ``data storage, loading pipeline'' and ``training framework'' and flexible resource configurations between them so that the resources can be fully exploited and performance can be maximized

Commitments to Quantum States

What does it mean to commit to a quantum state? In this work, we propose a simple answer: a commitment to quantum messages is binding if, after the commit phase, the committed state is hidden from the sender's view. We accompany this new definition with several instantiations. We build the first non-interactive succinct quantum state commitments, which can be seen as an analogue of collision-resistant hashing for quantum messages. We also show that hiding quantum state commitments (QSCs) are implied by any commitment scheme for classical messages. All of our constructions can be based on quantum-cryptographic assumptions that are implied by but are potentially weaker than one-way functions. Commitments to quantum states open the door to many new cryptographic possibilities. Our flagship application of a succinct QSC is a quantum-communication version of Kilian's succinct arguments for any language that has quantum PCPs with constant error and polylogarithmic locality. Plugging in the PCP theorem, this yields succinct arguments for NP under significantly weaker assumptions than required classically; moreover, if the quantum PCP conjecture holds, this extends to QMA. At the heart of our security proof is a new rewinding technique for extracting quantum information

Fluid–structure interaction of free convection in a square cavity divided by a flexible membrane and subjected to sinusoidal temperature heating

Purpose: The purpose of the present paper is to model a cavity, which is equally divided vertically by a thin, flexible membrane. The membranes are inevitable components of many engineering devices such as distillation systems and fuel cells. In the present study, a cavity which is equally divided vertically by a thin, flexible membrane is model using the fluid–structure interaction (FSI) associated with a moving grid approach. Design/methodology/approach: The cavity is differentially heated by a sinusoidal time-varying temperature on the left vertical wall, while the right vertical wall is cooled isothermally. There is no thermal diffusion from the upper and lower boundaries. The finite-element Galerkin technique with the aid of an arbitrary Lagrangian–Eulerian procedure is followed in the numerical procedure. The governing equations are transformed into non-dimensional forms to generalize the solution. Findings: The effects of four pertinent parameters are investigated, i.e., Rayleigh number (104 = Ra = 107), elasticity modulus (5 × 1012 = ET = 1016), Prandtl number (0.7 = Pr = 200) and temperature oscillation frequency (2p = f = 240p). The outcomes show that the temperature frequency does not induce a notable effect on the mean values of the Nusselt number and the deformation of the flexible membrane. The convective heat transfer and the stretching of the thin, flexible membrane become higher with a fluid of a higher Prandtl number or with a partition of a lower elasticity modulus. Originality/value: The authors believe that the modeling of natural convection and heat transfer in a cavity with the deformable membrane and oscillating wall heating is a new subject and the results have not been published elsewhere

Rastrelliger systematics inferred from mitochondrial cytochrome b sequences

The fish genus Rastrelliger is composed of three morphologically recognized species; Rastrelliger kanagurta, Rastrelliger brachysoma and Rastrelliger faughni. In this study, cytochrome b gene sequencing was applied to address the systematics and phylogenetic relationships of these species. In agreement with previous morphological data, the results corroborate monophyletic discrimination between all the species. However, inconsistent bootstrap support (< 50 to 88%) between R. kanagurta and R. brachysoma was observed indicating limited divergence between these two species. R. faughni is recognized as the most basal species for this genus with high statistical support (99 and 100%). Diversification of Rastrelliger might have happen in two epochs, Miocene and early Pleistocene

Two-step hyperparameter optimization method: Accelerating hyperparameter search by using a fraction of a training dataset

Hyperparameter optimization (HPO) is an important step in machine learning (ML) model development, but common practices are archaic -- primarily relying on manual or grid searches. This is partly because adopting advanced HPO algorithms introduces added complexity to the workflow, leading to longer computation times. This poses a notable challenge to ML applications, as suboptimal hyperparameter selections curtail the potential of ML model performance, ultimately obstructing the full exploitation of ML techniques. In this article, we present a two-step HPO method as a strategic solution to curbing computational demands and wait times, gleaned from practical experiences in applied ML parameterization work. The initial phase involves a preliminary evaluation of hyperparameters on a small subset of the training dataset, followed by a re-evaluation of the top-performing candidate models post-retraining with the entire training dataset. This two-step HPO method is universally applicable across HPO search algorithms, and we argue it has attractive efficiency gains. As a case study, we present our recent application of the two-step HPO method to the development of neural network emulators for aerosol activation. Although our primary use case is a data-rich limit with many millions of samples, we also find that using up to 0.0025% of the data (a few thousand samples) in the initial step is sufficient to find optimal hyperparameter configurations from much more extensive sampling, achieving up to 135-times speedup. The benefits of this method materialize through an assessment of hyperparameters and model performance, revealing the minimal model complexity required to achieve the best performance. The assortment of top-performing models harvested from the HPO process allows us to choose a high-performing model with a low inference cost for efficient use in global climate models (GCMs)