In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defer

Enabling machine learning classifiers to defer their decision to a downstream expert when the expert is more accurate will ensure improved safety and performance. This objective can be achieved with the learning-to-defer framework which aims to jointly learn how to classify and how to defer to the expert. In recent studies, it has been theoretically shown that popular estimators for learning to defer parameterized with softmax provide unbounded estimates for the likelihood of deferring which makes them uncalibrated. However, it remains unknown whether this is due to the widely used softmax parameterization and if we can find a softmax-based estimator that is both statistically consistent and possesses a valid probability estimator. In this work, we first show that the cause of the miscalibrated and unbounded estimator in prior literature is due to the symmetric nature of the surrogate losses used and not due to softmax. We then propose a novel statistically consistent asymmetric softmax-based surrogate loss that can produce valid estimates without the issue of unboundedness. We further analyze the non-asymptotic properties of our method and empirically validate its performance and calibration on benchmark datasets.Comment: NeurIPS 202

On the Importance of Feature Separability in Predicting Out-Of-Distribution Error

Estimating the generalization performance is practically challenging on out-of-distribution (OOD) data without ground truth labels. While previous methods emphasize the connection between distribution difference and OOD accuracy, we show that a large domain gap not necessarily leads to a low test accuracy. In this paper, we investigate this problem from the perspective of feature separability, and propose a dataset-level score based upon feature dispersion to estimate the test accuracy under distribution shift. Our method is inspired by desirable properties of features in representation learning: high inter-class dispersion and high intra-class compactness. Our analysis shows that inter-class dispersion is strongly correlated with the model accuracy, while intra-class compactness does not reflect the generalization performance on OOD data. Extensive experiments demonstrate the superiority of our method in both prediction performance and computational efficiency

Weakly Supervised Regression with Interval Targets

This paper investigates an interesting weakly supervised regression setting called regression with interval targets (RIT). Although some of the previous methods on relevant regression settings can be adapted to RIT, they are not statistically consistent, and thus their empirical performance is not guaranteed. In this paper, we provide a thorough study on RIT. First, we proposed a novel statistical model to describe the data generation process for RIT and demonstrate its validity. Second, we analyze a simple selection method for RIT, which selects a particular value in the interval as the target value to train the model. Third, we propose a statistically consistent limiting method for RIT to train the model by limiting the predictions to the interval. We further derive an estimation error bound for our limiting method. Finally, extensive experiments on various datasets demonstrate the effectiveness of our proposed method.Comment: Accepted by ICML 202

A Method for Out-of-Distribution Detection in Encrypted Mobile Traffic Classification

The widespread use of encrypted communication in mobile networks poses significant challenges in accurately classifying traffic. Detecting out-of-distribution (OOD) samples, which significantly deviate from known classes, adds complexity to the task. This paper proposes a feature analysis-based OOD detection scheme for traffic classification in Long-Term Evolution (LTE) systems. Our method utilizes Long Short-Term Memory (LSTM) networks for feature extraction, capturing the feature vectors of the traffic series. Principal Component Analysis (PCA) is then applied to obtain principal and residual principal components. Leveraging the residual feature vector, we construct an OOD score to quantify deviation from the ID dataset. Extensive experiments on a large-scale encrypted mobile traffic dataset demonstrate the superiority of our approach, achieving high accuracy in OOD detection compared to existing techniques. Our method contributes to enhanced security and reliable traffic classification in LTE systems, addressing challenges posed by OOD samples

Raman Study of Layered Breathing Kagome Lattice Semiconductor Nb3Cl8

Niobium chloride (Nb3Cl8) is a layered 2D semiconducting material with many exotic properties including a breathing kagome lattice, a topological flat band in its band structure, and a crystal structure that undergoes a structural and magnetic phase transition at temperatures below 90 K. Despite being a remarkable material with fascinating new physics, the understanding of its phonon properties is at its infancy. In this study, we investigate the phonon dynamics of Nb3Cl8 in bulk and few layer flakes using polarized Raman spectroscopy and density functional theory (DFT) analysis to determine the material's vibrational modes, as well as their symmetrical representations and atomic displacements. We experimentally resolved 12 phonon modes, 5 of which are A1g modes while the remaining 7 are Eg modes, which is in strong agreement with our DFT calculation. Layer-dependent results suggest that the Raman peak positions are mostly insensitive to changes in layer thickness, while peak intensity and FWHM are affected. Raman measurements as a function of excitation wavelength (473-785 nm) show a significant increase of the peak intensities when using a 473 nm excitation source, suggesting a near resonant condition. Temperature-dependent Raman experiments carried out above and below the transition temperature did not show any change in the symmetries of the phonon modes, suggesting that the structural phase transition is likely from the high temperature P3m1 phase to the low-temperature R3m phase. Magneto-Raman measurements carried out at 140 and 2 K between -2 to 2 T show that the Raman modes are not magnetically coupled. Overall, our study presented here significantly advances the fundamental understanding of layered Nb3Cl8 material which can be further exploited for future applications.Comment: 18 pages, 8 figures, 1 tabl

Enhanced Osseointegration of Hierarchically Structured Ti Implant with Electrically Bioactive SnO₂-TiO₂ Bilayered Surface

The poor osseointegration of Ti implant significantly compromise its application in load-bearing bone repair and replacement. Electrically bioactive coating inspirited from heterojunction on Ti implant can benefit osseointegration but cannot avoid the stress shielding effect between bone and implant. To resolve this conflict, hierarchically structured Ti implant with electrically bioactive SnO2–TiO2 bilayered surface has been developed to enhance osseointegration. Benefiting from the electric cue offered by the built-in electrical field of SnO2–TiO2 heterojunction and the topographic cue provided by the hierarchical surface structure to bone regeneration, the osteoblastic function of basic multicellular units around the implant is significantly improved. Because the individual TiO2 or SnO2 coating with uniform surface exhibits no electrical bioactivity, the effects of electric and topographic cues to osseointegration have been decoupled via the analysis of in vivo performance for the placed Ti implant with different surfaces. The developed Ti implant shows significantly improved osseointegration with excellent bone–implant contact, improved mineralization of extracellular matrix, and increased push-out force. These results suggest that the synergistic strategy of combing electrical bioactivity with hierarchical surface structure provides a new platform for developing advanced endosseous implants

Observation of flat and weakly dispersing bands in a van der Waals semiconductor Nb3Br8 with breathing kagome lattice

Niobium halides, Nb3X8 (X = Cl,Br,I), which are predicted two-dimensional magnets, have recently gotten attention due to their breathing kagome geometry. Here, we have studied the electronic structure of Nb3Br8 by using angle-resolved photoemission spectroscopy (ARPES) and first-principles calculations. ARPES results depict the presence of multiple flat and weakly dispersing bands. These bands are well explained by the theoretical calculations, which show they have Nb d character indicating their origination from the Nb atoms forming the breathing kagome plane. This van der Waals material can be easily thinned down via mechanical exfoliation to the ultrathin limit and such ultrathin samples are stable as depicted from the time-dependent Raman spectroscopy measurements at room temperature. These results demonstrate that Nb3Br8 is an excellent material not only for studying breathing kagome induced flat band physics and its connection with magnetism, but also for heterostructure fabrication for application purposes.Comment: 24 pages, 12 figures, Supplemental Material include