Knowing I Don\u27t Know

https://scholarworks.umt.edu/grad_portfolios/1322/thumbnail.jp

Incremental Deep Neural Network Learning using Classification Confidence Thresholding

Most modern neural networks for classification fail to take into account the concept of the unknown. Trained neural networks are usually tested in an unrealistic scenario with only examples from a closed set of known classes. In an attempt to develop a more realistic model, the concept of working in an open set environment has been introduced. This in turn leads to the concept of incremental learning where a model with its own architecture and initial trained set of data can identify unknown classes during the testing phase and autonomously update itself if evidence of a new class is detected. Some problems that arise in incremental learning are inefficient use of resources to retrain the classifier repeatedly and the decrease of classification accuracy as multiple classes are added over time. This process of instantiating new classes is repeated as many times as necessary, accruing errors. To address these problems, this paper proposes the Classification Confidence Threshold approach to prime neural networks for incremental learning to keep accuracies high by limiting forgetting. A lean method is also used to reduce resources used in the retraining of the neural network. The proposed method is based on the idea that a network is able to incrementally learn a new class even when exposed to a limited number samples associated with the new class. This method can be applied to most existing neural networks with minimal changes to network architecture.Comment: Accepted to IEEE TNNL

Moving Towards Open Set Incremental Learning: Readily Discovering New Authors

The classification of textual data often yields important information. Most classifiers work in a closed world setting where the classifier is trained on a known corpus, and then it is tested on unseen examples that belong to one of the classes seen during training. Despite the usefulness of this design, often there is a need to classify unseen examples that do not belong to any of the classes on which the classifier was trained. This paper describes the open set scenario where unseen examples from previously unseen classes are handled while testing. This further examines a process of enhanced open set classification with a deep neural network that discovers new classes by clustering the examples identified as belonging to unknown classes, followed by a process of retraining the classifier with newly recognized classes. Through this process the model moves to an incremental learning model where it continuously finds and learns from novel classes of data that have been identified automatically. This paper also develops a new metric that measures multiple attributes of clustering open set data. Multiple experiments across two author attribution data sets demonstrate the creation an incremental model that produces excellent results.Comment: Accepted to Future of Information and Communication Conference (FICC) 202

Superfluid transitions in bosonic atom-molecule mixtures near Feshbach resonance

We study bosonic atoms near a Feshbach resonance, and predict that in addition to a standard normal and atomic superfluid phases, this system generically exhibits a distinct phase of matter: a molecular superfluid, where molecules are superfluid while atoms are not. We explore zero- and finite-temperature properties of the molecular superfluid (a bosonic, strong-coupling analog of a BCS superconductor), and study quantum and classical phase transitions between the normal, molecular superfluid and atomic superfluid states.Comment: 4 revtex pages, 3 eps figures; submitted to PR

5-Year survival of pediatric anterior cruciate ligament reconstruction with living donor hamstring tendon grafts

Background: It is well accepted that there is a higher incidence of repeat anterior cruciate ligament (ACL) injuries in the pediatric population after ACL reconstruction (ACLR) with autograft tissue compared with adults. Hamstring autograft harvest may contribute to the risk for repeat ACL injuries in this high functional demand group. A novel method is the use of a living donor hamstring tendon (LDHT) graft from a parent; however, there is currently limited research on the outcomes of this technique, particularly beyond the short term. Purpose/Hypothesis: The purpose was to determine the medium-term survival of the ACL graft and the contralateral ACL (CACL) after primary ACLR with the use of an LDHT graft from a parent in those aged less than 18 years and to identify factors associated with subsequent ACL injuries. It was hypothesized that ACLR with the use of an LDHT provides acceptable midterm outcomes in pediatric patients. Study Design: Case series; Level of evidence, 4. Methods: Between 2005 and 2014, 247 (of 265 eligible) consecutive patients in a prospective database, having undergone primary ACLR with the use of an LDHT graft and aged less than 18 years, were included. Outcomes were assessed at a minimum of 2 years after surgery including data on ACL reinjuries, International Knee Documentation Committee (IKDC) scores, and current symptoms, as well as factors associated with the ACL reinjury risk were investigated. Results: Patients were reviewed at a mean of 4.5 years (range, 24-127 months [10.6 years]) after ACLR with an LDHT graft. Fifty-one patients (20.6%) sustained an ACL graft rupture, 28 patients (11.3%) sustained a CACL rupture, and 2 patients sustained both an ACL graft rupture and a CACL rupture (0.8%). Survival of the ACL graft was 89%, 82%, and 76% at 1, 2, and 5 years, respectively. Survival of the CACL was 99%, 94%, and 86% at 1, 2, and 5 years, respectively. Survival of the ACL graft was favorable in patients with Tanner stage 1-2 at the time of surgery versus those with Tanner stage 3-5 at 5 years (87% vs 69%, respectively; hazard ratio, 3.7; P = .01). The mean IKDC score was 91.7. A return to preinjury levels of activity was reported by 59.1%. Conclusion: After ACLR with an LDHT graft from a parent in those aged less than 18 years, a second ACL injury (ACL graft or CACL injury) occurred in 1 in 3 patients. The 5-year survival rate of the ACL graft was 76%, and the 5-year survival rate of the CACL was 86%. High IKDC scores and continued participation in sports were maintained over the medium term. Importantly, there was favorable survival of the ACL graft in patients with Tanner stage 1-2 compared with patients with Tanner stage 3-5 over 5 years. Patients with Tanner stage 1-2 also had a significantly lower incidence of second ACL injuries over 5 years compared with those with Tanner stage 3-5, occurring in 1 in 5 patients. Thus, an LDHT graft from a parent is an appropriate graft for physically immature children

Utilizing Priming to Identify Optimal Class Ordering to Alleviate Catastrophic Forgetting

In order for artificial neural networks to begin accurately mimicking biological ones, they must be able to adapt to new exigencies without forgetting what they have learned from previous training. Lifelong learning approaches to artificial neural networks attempt to strive towards this goal, yet have not progressed far enough to be realistically deployed for natural language processing tasks. The proverbial roadblock of catastrophic forgetting still gate-keeps researchers from an adequate lifelong learning model. While efforts are being made to quell catastrophic forgetting, there is a lack of research that looks into the importance of class ordering when training on new classes for incremental learning. This is surprising as the ordering of "classes" that humans learn is heavily monitored and incredibly important. While heuristics to develop an ideal class order have been researched, this paper examines class ordering as it relates to priming as a scheme for incremental class learning. By examining the connections between various methods of priming found in humans and how those are mimicked yet remain unexplained in life-long machine learning, this paper provides a better understanding of the similarities between our biological systems and the synthetic systems while simultaneously improving current practices to combat catastrophic forgetting. Through the merging of psychological priming practices with class ordering, this paper is able to identify a generalizable method for class ordering in NLP incremental learning tasks that consistently outperforms random class ordering.Comment: Accepted to IEEE International Conference on Semantic Computing (ICSC) 202

Revealing the tidal scars of the Small Magellanic Cloud

Due to their close proximity, the Large and Small Magellanic Clouds (SMC/LMC) provide natural laboratories for understanding how galaxies form and evolve. With the goal of determining the structure and dynamical state of the SMC, we present new spectroscopic data for $\sim$ 3000 SMC red giant branch stars observed using the AAOmega spectrograph at the Anglo-Australian Telescope. We complement our data with further spectroscopic measurements from previous studies that used the same instrumental configuration and proper motions from the \textit{Gaia} Data Release 2 catalogue. Analysing the photometric and stellar kinematic data, we find that the SMC centre of mass presents a conspicuous offset from the velocity centre of its associated \mbox{H\,{\sc i}} gas, suggesting that the SMC gas is likely to be far from dynamical equilibrium. Furthermore, we find evidence that the SMC is currently undergoing tidal disruption by the LMC within 2\,kpc of the centre of the SMC, and possibly all the way in to the very core. This is evidenced by a net outward motion of stars from the SMC centre along the direction towards the LMC and apparent tangential anisotropy at all radii. The latter is expected if the SMC is undergoing significiant tidal stripping, as we demonstrate using a suite of $N$-body simulations of the SMC/LMC system disrupting around the Milky Way. These results suggest that dynamical models for the SMC that assume a steady state will need to be revisited.Comment: Revised version submitted to MNRAS after referee report, 18 pages, 18 figure

A randomized controlled trial of PEEK versus titanium interference screws for anterior cruciate ligament reconstruction with 2-year follow-up

Purpose: To compare the clinical performance of ACL reconstruction with PEEK and titanium interference screws at 2 years and to evaluate a novel method of measuring tunnel volume. Study Design: Randomized controlled trial; Level of evidence, 1. Methods: A total of 133 patients underwent arthroscopic ACL reconstruction with 4-strand hamstring autografts and were randomized to have titanium or PEEK interference screws for femoral and tibial tunnel fixation. At 2 years, subjective Lysholm and International Knee Documentation Committee scores were assessed and clinical examination performed. At 12 months, MRI was performed to assess graft incorporation and cyst formation, and a novel technique was employed to measure tunnel volumes. Results: There were no significant differences in graft rerupture rate, contralateral ACL rupture rate, subjective outcomes, or objective outcomes. In the titanium and PEEK groups, MRI demonstrated high overall rates of graft integration (96%-100% and 90%-93%, respectively) and ligamentization (89% and 84%) and low rates of synovitis (22% and 10%) and cyst formation (0%-18% and 13%-15%). There was a higher proportion of patients with incomplete graft integration within the femoral tunnel in the PEEK group as compared with the titanium group (10% vs 0%, P = .03); however, the authors suggest that metal artifact precluded proper assessment of the graft in the titanium group by MRI. Tunnel volumes also appeared to be equivalent in the 2 groups and were measured with a novel technique that was highly reproducible in the PEEK group secondary to the absence of flare. Conclusion: Two-year clinical analysis of PEEK interference screws for femoral and tibial fixation of ACL reconstructions showed equivalent clinical performance to titanium interference screws. Given the excellent mechanical characteristics, biological compatibility, and absence of metal artifact on MRI, PEEK has become our material of choice for interference screw fixation in ACL reconstruction