Feature engineering, a crucial step of machine learning, aims to extract
useful features from raw data to improve data quality. In recent years, great
efforts have been devoted to Automated Feature Engineering (AutoFE) to replace
expensive human labor. However, existing methods are computationally demanding
due to treating AutoFE as a coarse-grained black-box optimization problem over
a discrete space. In this work, we propose an efficient gradient-based method
called DIFER to perform differentiable automated feature engineering in a
continuous vector space. DIFER selects potential features based on evolutionary
algorithm and leverages an encoder-predictor-decoder controller to optimize
existing features. We map features into the continuous vector space via the
encoder, optimize the embedding along the gradient direction induced by the
predicted score, and recover better features from the optimized embedding by
the decoder. Extensive experiments on classification and regression datasets
demonstrate that DIFER can significantly improve the performance of various
machine learning algorithms and outperform current state-of-the-art AutoFE
methods in terms of both efficiency and performance.Comment: 8 pages, 5 figure