5 research outputs found
Improve Deep Forest with Learnable Layerwise Augmentation Policy Schedule
As a modern ensemble technique, Deep Forest (DF) employs a cascading
structure to construct deep models, providing stronger representational power
compared to traditional decision forests. However, its greedy multi-layer
learning procedure is prone to overfitting, limiting model effectiveness and
generalizability. This paper presents an optimized Deep Forest, featuring
learnable, layerwise data augmentation policy schedules. Specifically, We
introduce the Cut Mix for Tabular data (CMT) augmentation technique to mitigate
overfitting and develop a population-based search algorithm to tailor
augmentation intensity for each layer. Additionally, we propose to incorporate
outputs from intermediate layers into a checkpoint ensemble for more stable
performance. Experimental results show that our method sets new
state-of-the-art (SOTA) benchmarks in various tabular classification tasks,
outperforming shallow tree ensembles, deep forests, deep neural network, and
AutoML competitors. The learned policies also transfer effectively to Deep
Forest variants, underscoring its potential for enhancing non-differentiable
deep learning modules in tabular signal processing
Multi-Level Deep Cascade Trees for Conversion Rate Prediction in Recommendation System
Developing effective and efficient recommendation methods is very challenging for modern e-commerce platforms. Generally speaking, two essential modules named “ClickThrough Rate Prediction” (CTR) and “Conversion Rate Prediction” (CVR) are included, where CVR module is a crucial factor that affects the final purchasing volume directly. However, it is indeed very challenging due to its sparseness nature. In this paper, we tackle this problem by proposing multiLevel Deep Cascade Trees (ldcTree), which is a novel decision tree ensemble approach. It leverages deep cascade structures by stacking Gradient Boosting Decision Trees (GBDT) to effectively learn feature representation. In addition, we propose to utilize the cross-entropy in each tree of the preceding GBDT as the input feature representation for next level GBDT, which has a clear explanation, i.e., a traversal from root to leaf nodes in the next level GBDT corresponds to the combination of certain traversals in the preceding GBDT. The deep cascade structure and the combination rule enable the proposed ldcTree to have a stronger distributed feature representation ability. Moreover, inspired by ensemble learning, we propose an Ensemble ldcTree (E-ldcTree) to encourage the model’s diversity and enhance the representation ability further. Finally, we propose an improved Feature learning method based on EldcTree (F-EldcTree) for taking adequate use of weak and strong correlation features identified by pretrained GBDT models. Experimental results on off-line data set and online deployment demonstrate the effectiveness of the proposed methods