Statistical Models for Co-occurrence Data

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. In this contribution we develop a statistical framework for analyzing co-occurrence data in a general setting where elementary observations are joint occurrences of pairs of abstract objects from two finite sets. The main challenge for statistical models in this context is to overcome the inherent data sparseness and to estimate the probabilities for pairs which were rarely observed or even unobserved in a given sample set. Moreover, it is often of considerable interest to extract grouping structure or to find a hierarchical data organization. A novel family of mixture models is proposed which explain the observed data by a finite number of shared aspects or clusters. This provides a common framework for statistical inference and structure discovery and also includes several recently proposed models as special cases. Adopting the maximum likelihood principle, EM algorithms are derived to fit the model parameters. We develop improved versions of EM which largely avoid overfitting problems and overcome the inherent locality of EM--based optimization. Among the broad variety of possible applications, e.g., in information retrieval, natural language processing, data mining, and computer vision, we have chosen document retrieval, the statistical analysis of noun/adjective co-occurrence and the unsupervised segmentation of textured images to test and evaluate the proposed algorithms

Fabrication, characterization of high-entropy alloys and deep learning-based inspection in metal additive manufacturing

Alloying has been used to confer desirable properties to materials. It typically involves the addition of small amounts of secondary elements to a primary element. In the past decade, however, a new alloying strategy that involves the combination of multiple principal elements in high concentrations to create new materials called high- entropy alloys (HEAs) has been in vogue. In the first part, the investigation focused on the fabrication process and property assessment of the additive manufactured HEA to broaden its engineering applications. Additive manufacturing (AM) is based on manufacturing philosophy through the layer-by-layer method and accomplish the near net-shaped components fabrication. Attempt was made to coat AlCoCrFeNi HEA on an AISI 304 stainless steel substrate to integrate their properties, however, it failed due to the cracks at the interface. The implementation of an intermediate layer improved the bond and eliminated the cracks. Next, an AlCoCrFeNiTi0.5 HEA coating was fabricated on the Ti6Al4V substrate, and its isothermal oxidation behavior was studied. The HEA coating effectively improved the Ti6Al4V substrate\u27s oxidation resistance due to the formation of continuous protective oxides. In the second part, research efforts were made on the deep learning-based quality inspection of additive manufactured products. The traditional inspection process has relied on manual recognition, which could suffer from low efficiency and potential bias. A neural-network approach was developed toward robust real-world AM anomaly detection. The results indicate the promising application of the neural network in the AM industry --Abstract, page iv

Modeling the Phase-Change Memory Material, Ge2Sb2Te5, with a Machine-Learned Interatomic Potential

The phase-change material, Ge2Sb2Te5, is the canonical material ingredient for next-generation storage-class memory devices used in novel computing architectures, but fundamental questions remain regarding its atomic structure and physico-chemical properties. Here, we introduce a machine-learning (ML)-based interatomic potential that enables large-scale atomistic simulations of liquid, amorphous, and crystalline Ge2Sb2Te5 with an unprecedented combination of speed and density-functional theory (DFT) level of accuracy. Two applications exemplify the usefulness of such an ML-driven approach: we generate a 7,200-atom structural model, hitherto inaccessible with DFT simulations, that affords new insight into the medium-range structural order; and we create an ensemble of uncorrelated, smaller structures, for studies of their chemical bonding with statistical significance. Our work opens the way for new atomistic insights into the fascinating and chemically complex class of phase-change materials that are used in real non-volatile memory devices

KernelWarehouse: Towards Parameter-Efficient Dynamic Convolution

Dynamic convolution learns a linear mixture of $n$ static kernels weighted with their sample-dependent attentions, demonstrating superior performance compared to normal convolution. However, existing designs are parameter-inefficient: they increase the number of convolutional parameters by $n$ times. This and the optimization difficulty lead to no research progress in dynamic convolution that can allow us to use a significant large value of $n$ (e.g., $n>100$ instead of typical setting $n<10$) to push forward the performance boundary. In this paper, we propose $KernelWarehouse$, a more general form of dynamic convolution, which can strike a favorable trade-off between parameter efficiency and representation power. Its key idea is to redefine the basic concepts of "$kernels$" and "$assembling$ $kernels$" in dynamic convolution from the perspective of reducing kernel dimension and increasing kernel number significantly. In principle, KernelWarehouse enhances convolutional parameter dependencies within the same layer and across successive layers via tactful kernel partition and warehouse sharing, yielding a high degree of freedom to fit a desired parameter budget. We validate our method on ImageNet and MS-COCO datasets with different ConvNet architectures, and show that it attains state-of-the-art results. For instance, the ResNet18|ResNet50|MobileNetV2|ConvNeXt-Tiny model trained with KernelWarehouse on ImageNet reaches 76.05%|81.05%|75.52%|82.51% top-1 accuracy. Thanks to its flexible design, KernelWarehouse can even reduce the model size of a ConvNet while improving the accuracy, e.g., our ResNet18 model with 36.45%|65.10% parameter reduction to the baseline shows 2.89%|2.29% absolute improvement to top-1 accuracy.Comment: This research work was completed and submitted in early May 2023. Code and pre-trained models are available at https://github.com/OSVAI/KernelWarehous