Sequential Subset Matching for Dataset Distillation

Dataset distillation is a newly emerging task that synthesizes a small-size dataset used in training deep neural networks (DNNs) for reducing data storage and model training costs. The synthetic datasets are expected to capture the essence of the knowledge contained in real-world datasets such that the former yields a similar performance as the latter. Recent advancements in distillation methods have produced notable improvements in generating synthetic datasets. However, current state-of-the-art methods treat the entire synthetic dataset as a unified entity and optimize each synthetic instance equally. This static optimization approach may lead to performance degradation in dataset distillation. Specifically, we argue that static optimization can give rise to a coupling issue within the synthetic data, particularly when a larger amount of synthetic data is being optimized. This coupling issue, in turn, leads to the failure of the distilled dataset to extract the high-level features learned by the deep neural network (DNN) in the latter epochs. In this study, we propose a new dataset distillation strategy called Sequential Subset Matching (SeqMatch), which tackles this problem by adaptively optimizing the synthetic data to encourage sequential acquisition of knowledge during dataset distillation. Our analysis indicates that SeqMatch effectively addresses the coupling issue by sequentially generating the synthetic instances, thereby enhancing its performance significantly. Our proposed SeqMatch outperforms state-of-the-art methods in various datasets, including SVNH, CIFAR-10, CIFAR-100, and Tiny ImageNet. Our code is available at https://github.com/shqii1j/seqmatch

EXPERIMENTAL RESEARCH ON THE VIBRATION CHARACTERISTICS OF BRIDGE'S HORIZONTAL ROTATION SYSTEM

As a new construction method, the bridge horizontal rotation construction method can reduce the impact of traffic under the bridge. During the horizontal rotation of the bridge, the overall structure will inevitably lead to a vibration response due to the construction error of the contact surface of the spherical hinge. Due to the large weight of the structure and the longer cantilever of the superstructure, the vibration at the spherical hinge will be amplified at the girder end, which will adversely affect the stability of the structure. Taking a 10,000-ton rotating bridge as a reference, a scaled model was made to test the vibration of the girder during the rotating process of the horizontal rotating system.And by analyzing the frequency domain curve of girder vibration and the results of simulation calculation, it is found that the vertical vibration displacement response is related to the first three modes of longitudinal bending of the girder structure, but has nothing to do with the higher modes or other modes. Applying the harmonic response analysis module in ANSYS software method, it is proposed that the structural vibration effect will reach the smallest by controlling the rotating speed in order to control the excitation frequency within the first-order mode frequency of girder. Also in this research, the expression of the relationship between the vertical vibration velocity and acceleration of the girder end of the horizontal rotation system and the vibration frequency of the girder is established. Based on that, it is proposed that the stability of the horizontal rotation can be predicted by monitoring the vertical velocity and acceleration of the cantilever girder end during the horizontal rotation

MECHANICAL BEHAVIOR OF HORIZONTAL SWIVEL SYSTEM WITH UHPC SPHERICAL HINGE UNDER SEISMIC ACTION

In the process of rotation, the total weight of the bridge structure is jointly supported by the spherical hinge and the supporting structure, and its lateral stability is poor. It is easy to lose stability under the action of dynamic loads such as seismic action effect. The present paper takes a 10,000-ton continuous rigid frame swivel bridge as the re-search object, analyzes the dynamic response of the seismic action to the horizontal swivel system, and establishes several structure simulation models. Eighteen seismic waves in three directions that meet the calculation requirements are screened for time history analysis and compared with the response spectrum method. Finally, an optimization algorithm for the seismic response of the bridge under horizontal swivel system is proposed based on the mode superposition method. The UHPC spherical hinge bears all the vertical forces and 20% of the bending moment caused by the seismic action, the support structure bearing the remaining 80% of the bending moment. The optimization algorithm proposed in this paper features high accuracy

Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation

Model-based deep learning has achieved astounding successes due in part to the availability of large-scale realworld data. However, processing such massive amounts of data comes at a considerable cost in terms of computations, storage, training and the search for good neural architectures. Dataset distillation has thus recently come to the fore. This paradigm involves distilling information from large real-world datasets into tiny and compact synthetic datasets such that processing the latter yields similar performances as the former. State-of-the-art methods primarily rely on learning the synthetic dataset by matching the gradients obtained during training between the real and synthetic data. However, these gradient-matching methods suffer from the accumulated trajectory error caused by the discrepancy between the distillation and subsequent evaluation. To alleviate the adverse impact of this accumulated trajectory error, we propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7% on a subset of images of the ImageNet dataset with higher resolution images. We also validate the effectiveness and generalizability of our method with datasets of different resolutions and demonstrate its applicability to neural architecture search

Efficient Sharpness-aware Minimization for Improved Training of Neural Networks

Overparametrized Deep Neural Networks (DNNs) often achieve astounding performances, but may potentially result in severe generalization error. Recently, the relation between the sharpness of the loss landscape and the generalization error has been established by Foret et al. (2020), in which the Sharpness Aware Minimizer (SAM) was proposed to mitigate the degradation of the generalization. Unfortunately, SAM s computational cost is roughly double that of base optimizers, such as Stochastic Gradient Descent (SGD). This paper thus proposes Efficient Sharpness Aware Minimizer (ESAM), which boosts SAM s efficiency at no cost to its generalization performance. ESAM includes two novel and efficient training strategies-StochasticWeight Perturbation and Sharpness-Sensitive Data Selection. In the former, the sharpness measure is approximated by perturbing a stochastically chosen set of weights in each iteration; in the latter, the SAM loss is optimized using only a judiciously selected subset of data that is sensitive to the sharpness. We provide theoretical explanations as to why these strategies perform well. We also show, via extensive experiments on the CIFAR and ImageNet datasets, that ESAM enhances the efficiency over SAM from requiring 100% extra computations to 40% vis-a-vis base optimizers, while test accuracies are preserved or even improved

Early Cenozoic diorite and diabase from Doumer Island, Antarctic Peninsula: zircon U-Pb geochronology, petrogenesis and tectonic implications

In order to understand the petrogenesis and tectonic setting of diorite and diabase units on Doumer Island, Antarctic Peninsula, this paper reports new laser ablation—inductively coupled plasma—mass spectrometry (LA-ICP-MS) zircon U-Pb, geochemical, and Lu-Hf isotopic data for the magmatism. The diorite and diabase samples yielded zircon U-Pb ages of 55.4 ± 0.3 and 52.8 ± 0.4 Ma, respectively. These samples are enriched in the large ion lithophile elements and the light rare earth elements, and are depleted in the high field strength elements. The zircons in these samples yield Hf (t) values from 9.03 to 11.87 and model ages (TDM2) of 342–524 Ma. The major, trace, rare earth element (REE), and Hf isotopic data for the diorites indicate that these units were formed by the mixing of magmas generated by (a) the partial melting of mantle wedge material that experienced fluid-metasomatism in a subduction zone setting, and (b) the melting of juvenile crustal material induced by the upwelling of mantle-derived magmas in a subduction–collision setting. The diabase units contain higher total REE concentrations than the diorite, indicating they were derived from a different source region. These samples also have higher Mg# values and contain lower concentrations of Cr and Ni than the diorites, and have weakly negative Nb and Ta anomalies with Nb/Ta values of <3. The zircons in these samples yield Hf (t) values from 9.08 to 11.11 and model ages (TDM2) of 389– 503 Ma. The major, trace, REE, and Hf isotopic compositions of the diabase units indicate that that they were derived from the mixing of depleted mantle-derived magmas with magmas generated by the melting of juvenile crustal material which was induced by the upwelling of the mantle into the crust. Overall the Cenozoic diorite and diabase on Doumer Island is related to subduction environment

3D phase field modeling of multi-dendrites evolution in solidification and validation by synchrotron x-ray tomography

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. In this paper, the dynamics of multi-dendrite concurrent growth and coarsening of an Al-15 wt.% Cu alloy was studied using a highly computationally efficient 3D phase field model and real-time synchrotron X-ray micro-tomography. High fidelity multi-dendrite simulations were achieved and the results were compared directly with the time-evolved tomography datasets to quantify the relative importance of multi-dendritic growth and coarsening. Coarsening mechanisms under different solidification conditions were further elucidated. The dominant coarsening mechanisms change from small arm melting and interdendritic groove advancement to coalescence when the solid volume fraction approaches ~0.70. Both tomography experiments and phase field simulations indicated that multi-dendrite coarsening obeys the classical Lifshitz–Slyozov–Wagner theory Rn − Rn0=kc(t − t0), but with a higher constant of n = 4.3