Effects of classification context on categorization in natural categories

The patterns of classification of borderline instances of eight common taxonomic categories were examined under three different instructional conditions to test two predictions: first, that lack of a specified context contributes to vagueness in categorization, and second, that altering the purpose of classification can lead to greater or lesser dependence on similarity in classification. The instructional conditions contrasted purely pragmatic with more technical/quasi-legal contexts as purposes for classification, and these were compared with a no-context control. The measures of category vagueness were between-subjects disagreement and within-subjects consistency, and the measures of similarity based categorization were category breadth and the correlation of instance categorization probability with mean rated typicality, independently measured in a neutral context. Contrary to predictions, none of the measures of vagueness, reliability, category breadth, or correlation with typicality were generally affected by the instructional setting as a function of pragmatic versus technical purposes. Only one subcondition, in which a situational context was implied in addition to a purposive context, produced a significant change in categorization. Further experiments demonstrated that the effect of context was not increased when participants talked their way through the task, and that a technical context did not elicit more all-or-none categorization than did a pragmatic context. These findings place an important boundary condition on the effects of instructional context on conceptual categorization

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

Predicting neural architecture performance is a challenging task and is crucial to neural architecture design and search. Existing approaches either rely on neural performance predictors which are limited to modeling architectures in a predefined design space involving specific sets of operators and connection rules, and cannot generalize to unseen architectures, or resort to zero-cost proxies which are not always accurate. In this paper, we propose GENNAPE, a Generalized Neural Architecture Performance Estimator, which is pretrained on open neural architecture benchmarks, and aims to generalize to completely unseen architectures through combined innovations in network representation, contrastive pretraining, and fuzzy clustering-based predictor ensemble. Specifically, GENNAPE represents a given neural network as a Computation Graph (CG) of atomic operations which can model an arbitrary architecture. It first learns a graph encoder via Contrastive Learning to encourage network separation by topological features, and then trains multiple predictor heads, which are soft-aggregated according to the fuzzy membership of a neural network. Experiments show that GENNAPE pretrained on NAS-Bench-101 can achieve superior transferability to 5 different public neural network benchmarks, including NAS-Bench-201, NAS-Bench-301, MobileNet and ResNet families under no or minimum fine-tuning. We further introduce 3 challenging newly labelled neural network benchmarks: HiAML, Inception and Two-Path, which can concentrate in narrow accuracy ranges. Extensive experiments show that GENNAPE can correctly discern high-performance architectures in these families. Finally, when paired with a search algorithm, GENNAPE can find architectures that improve accuracy while reducing FLOPs on three families.Comment: AAAI 2023 Oral Presentation; includes supplementary materials with more details on introduced benchmarks; 14 Pages, 6 Figures, 10 Table

Transfer and the Fuzzy-Trace Theory

Transfer occurs when something is learned under particular circumstances and is applied in a new, somehow different, situation. This paper will argue that fuzzy-trace theory can be used to explain the process of transfer. The advantage of fuzzy-trace theory is found in a dual-process theory of memory. Fuzzy-trace theory explains a broad range of phenomena and has the strength to conquer the elusive problem of transfer. Trace-cue compatibility theory is a theory of memory retrieval. By combining the trace-cue compatibility theory with fuzzy-trace theory, we get a method for treating both memory storage and memory retrieval. This combination provides a powerful mechanism for understanding the results of classic experiments on transfer. We can explain transfer in terms of particular forms of memory being cued by an event. In many cases, the cued memory is an analog for the target item. When the target analog has been mapped onto the appropriate memory trace, transfer can occur

IDENAS: Internal Dependency Exploration for Neural Architecture Search

Machine learning is a powerful tool for extracting valuable information and making various predictions from diverse datasets. Traditional algorithms rely on well-defined input and output variables however, there are scenarios where the distinction between the input and output variables and the underlying, associated (input and output) layers of the model, are unknown. Neural Architecture Search (NAS) and Feature Selection have emerged as promising solutions in such scenarios. This research proposes IDENAS, an Internal Dependency-based Exploration for Neural Architecture Search, integrating NAS with feature selection. The methodology explores internal dependencies in the complete parameter space for classification involving 1D sensor and 2D image data as well. IDENAS employs a modified encoder-decoder model and the Sequential Forward Search (SFS) algorithm, combining input-output configuration search with embedded feature selection. Experimental results demonstrate IDENASs superior performance in comparison to other algorithms, showcasing its effectiveness in model development pipelines and automated machine learning. On average, IDENAS achieved significant modelling improvements, underscoring its significant contribution to advancing the state-of-the-art in neural architecture search and feature selection integration.Comment: 57 pages, 19 figures + appendix, the related software code can be found under the link: https://github.com/viharoszsolt/IDENA

i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systems

Input features play a crucial role in DNN-based recommender systems with thousands of categorical and continuous fields from users, items, contexts, and interactions. Noisy features and inappropriate embedding dimension assignments can deteriorate the performance of recommender systems and introduce unnecessary complexity in model training and online serving. Optimizing the input configuration of DNN models, including feature selection and embedding dimension assignment, has become one of the essential topics in feature engineering. However, in existing industrial practices, feature selection and dimension search are optimized sequentially, i.e., feature selection is performed first, followed by dimension search to determine the optimal dimension size for each selected feature. Such a sequential optimization mechanism increases training costs and risks generating suboptimal input configurations. To address this problem, we propose a differentiable neural input razor (i-Razor) that enables joint optimization of feature selection and dimension search. Concretely, we introduce an end-to-end differentiable model to learn the relative importance of different embedding regions of each feature. Furthermore, a flexible pruning algorithm is proposed to achieve feature filtering and dimension derivation simultaneously. Extensive experiments on two large-scale public datasets in the Click-Through-Rate (CTR) prediction task demonstrate the efficacy and superiority of i-Razor in balancing model complexity and performance.Comment: Accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE

Neural Architecture Search: Insights from 1000 Papers

In the past decade, advances in deep learning have resulted in breakthroughs in a variety of areas, including computer vision, natural language understanding, speech recognition, and reinforcement learning. Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas. Neural architecture search (NAS), the process of automating the design of neural architectures for a given task, is an inevitable next step in automating machine learning and has already outpaced the best human-designed architectures on many tasks. In the past few years, research in NAS has been progressing rapidly, with over 1000 papers released since 2020 (Deng and Lindauer, 2021). In this survey, we provide an organized and comprehensive guide to neural architecture search. We give a taxonomy of search spaces, algorithms, and speedup techniques, and we discuss resources such as benchmarks, best practices, other surveys, and open-source libraries

Representations for Cognitive Vision : a Review of Appearance-Based, Spatio-Temporal, and Graph-Based Approaches

The emerging discipline of cognitive vision requires a proper representation of visual information including spatial and temporal relationships, scenes, events, semantics and context. This review article summarizes existing representational schemes in computer vision which might be useful for cognitive vision, a and discusses promising future research directions. The various approaches are categorized according to appearance-based, spatio-temporal, and graph-based representations for cognitive vision. While the representation of objects has been covered extensively in computer vision research, both from a reconstruction as well as from a recognition point of view, cognitive vision will also require new ideas how to represent scenes. We introduce new concepts for scene representations and discuss how these might be efficiently implemented in future cognitive vision systems

Algebraic description of spacetime foam

A mathematical formalism for treating spacetime topology as a quantum observable is provided. We describe spacetime foam entirely in algebraic terms. To implement the correspondence principle we express the classical spacetime manifold of general relativity and the commutative coordinates of its events by means of appropriate limit constructions.Comment: 34 pages, LaTeX2e, the section concerning classical spacetimes in the limit essentially correcte