Hybrid solutions to the feature interaction problem

In this paper we assume a competitive marketplace where the features are developed by different enterprises, which cannot or will not exchange information. We present a classification of feature interaction in this setting and introduce an on-line technique which serves as a basis for the two novel <i>hybrid</i> approaches presented. The approaches are hybrid as they are neither strictly off-line nor on-line, but combine aspects of both. The two approaches address different kinds of feature interactions, and thus are complimentary. Together they provide a complete solution by addressing interaction detection and resolution. We illustrate the techniques within the communication networks domain

Towards Hybrid-grained Feature Interaction Selection for Deep Sparse Network

Deep sparse networks are widely investigated as a neural network architecture for prediction tasks with high-dimensional sparse features, with which feature interaction selection is a critical component. While previous methods primarily focus on how to search feature interaction in a coarse-grained space, less attention has been given to a finer granularity. In this work, we introduce a hybrid-grained feature interaction selection approach that targets both feature field and feature value for deep sparse networks. To explore such expansive space, we propose a decomposed space which is calculated on the fly. We then develop a selection algorithm called OptFeature, which efficiently selects the feature interaction from both the feature field and the feature value simultaneously. Results from experiments on three large real-world benchmark datasets demonstrate that OptFeature performs well in terms of accuracy and efficiency. Additional studies support the feasibility of our method.Comment: NeurIPS 2023 poste

Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection

There is currently a large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks. Previous works attempting to close this gap have failed to fully investigate the exponentially growing number of feature combinations which deep networks consider automatically during training. In this work, we develop a tractable selection algorithm to efficiently identify the necessary feature combinations by leveraging techniques in feature interaction detection. Our proposed Sparse Interaction Additive Networks (SIAN) construct a bridge from these simple and interpretable models to fully connected neural networks. SIAN achieves competitive performance against state-of-the-art methods across multiple large-scale tabular datasets and consistently finds an optimal tradeoff between the modeling capacity of neural networks and the generalizability of simpler methods