Feature Selection via Binary Simultaneous Perturbation Stochastic Approximation

Feature selection (FS) has become an indispensable task in dealing with today's highly complex pattern recognition problems with massive number of features. In this study, we propose a new wrapper approach for FS based on binary simultaneous perturbation stochastic approximation (BSPSA). This pseudo-gradient descent stochastic algorithm starts with an initial feature vector and moves toward the optimal feature vector via successive iterations. In each iteration, the current feature vector's individual components are perturbed simultaneously by random offsets from a qualified probability distribution. We present computational experiments on datasets with numbers of features ranging from a few dozens to thousands using three widely-used classifiers as wrappers: nearest neighbor, decision tree, and linear support vector machine. We compare our methodology against the full set of features as well as a binary genetic algorithm and sequential FS methods using cross-validated classification error rate and AUC as the performance criteria. Our results indicate that features selected by BSPSA compare favorably to alternative methods in general and BSPSA can yield superior feature sets for datasets with tens of thousands of features by examining an extremely small fraction of the solution space. We are not aware of any other wrapper FS methods that are computationally feasible with good convergence properties for such large datasets.Comment: This is the Istanbul Sehir University Technical Report #SHR-ISE-2016.01. A short version of this report has been accepted for publication at Pattern Recognition Letter

Feature selection using stochastic approximation with Barzilai and Borwein non-monotone gains

With recent emergence of machine learning problems with massive number of features, feature selection (FS) has become an ever-increasingly important tool to mitigate the effects of the so-called curse of dimensionality. FS aims to eliminate redundant and irrelevant features for models that are faster to train, easier to understand, and less prone to overfitting. This study presents a wrapper FS method based on Simultaneous Perturbation Stochastic Approximation (SPSA) with Barzilai and Borwein (BB) non-monotone gains within a pseudo-gradient descent framework wherein performance is measured via cross-validation. We illustrate that SPSA with BB gains (SPSA-BB) provides dramatic improvements in terms of the number of iterations for convergence with minimal degradation in cross-validated error performance over the current state-of-the art approach with monotone gains (SPSA-MON). In addition, SPSA-BB requires only one internal parameter and therefore it eliminates the need for careful fine-tuning of numerous other internal parameters as in SPSA-MON or comparable meta-heuristic FS methods such as genetic algorithms (GA). Our particular implementation includes gradient averaging as well as gain smoothing for better convergence properties. We present computational experiments on various public datasets with Nearest Neighbors and Naive Bayes classifiers as wrappers. We present comparisons of SPSA-BB against full set of features, SPSA-MON, as well as seven popular meta-heuristics based FS algorithms including GA and particle swarm optimization. Our results indicate that SPSA-BB converges to a good feature set in about 50 iterations on the average regardless of the number of features (whether a dozen or more than 1000 features) and its performance is quite competitive. SPSA-BB can be considered extremely fast for a wrapper method and therefore it stands as a high-performing new feature selection method that is also computationally feasible in practice

Human exposure to aerosol from indoor gas stove cooking and the resulting nervous system responses

Our knowledge of the effects of exposure to indoor ultrafine particles (sub-100 nm, #/cm3) on human brain activity is very limited. The effects of cooking ultrafine particles (UFP) on healthy adults were assessed using an electroencephalograph (EEGs) for brain response. Peak ultrafine particle concentrations were approximately 3 × 105 particle/cm3, and the average level was 1.64 × 105 particle/cm3. The average particle number emission rate (S) and the average number decay rate (a+k) for chicken frying in brain experiments were calculated to be 2.82 × 1012 (SD = 1.83 × 1012, R2 = 0.91, p = 0.0013) particles/min, 0.47 (SD = 0.30, R2 = 0.90, p < 0.0001) min−1, respectively. EEGs were recorded before and during cooking (14 min) and 30 min after the cooking sessions. The brain fast-wave band (beta) decreased during exposure, similar to people with neurodegenerative diseases. It subsequently increased to its pre-exposure condition for 70% of the study participants after 30 min. The brain slow-wave band to fast-wave band ratio (theta/beta ratio) increased during and after exposure, similar to observed behavior in early-stage Alzheimer's disease (AD) patients. The brain then tended to return to its normal condition within 30 min following the exposure. This study suggests that chronically exposed people to high concentrations of cooking aerosol might progress toward AD

Human exposure to aerosol from indoor gas stove cooking and the resulting nervous system responses

Our knowledge of the effects of exposure to indoor ultrafine particles (sub-100 nm, #/cm3) on human brain activity is very limited. The effects of cooking ultrafine particles (UFP) on healthy adults were assessed using an electroencephalograph (EEGs) for brain response. Peak ultrafine particle concentrations were approximately 3 × 105 particle/cm3, and the average level was 1.64 × 105 particle/cm3. The average particle number emission rate (S) and the average number decay rate (a+k) for chicken frying in brain experiments were calculated to be 2.82 × 1012 (SD = 1.83 × 1012, R2 = 0.91, p = 0.0013) particles/min, 0.47 (SD = 0.30, R2 = 0.90, p < 0.0001) min−1, respectively. EEGs were recorded before and during cooking (14 min) and 30 min after the cooking sessions. The brain fast-wave band (beta) decreased during exposure, similar to people with neurodegenerative diseases. It subsequently increased to its pre-exposure condition for 70% of the study participants after 30 min. The brain slow-wave band to fast-wave band ratio (theta/beta ratio) increased during and after exposure, similar to observed behavior in early-stage Alzheimer's disease (AD) patients. The brain then tended to return to its normal condition within 30 min following the exposure. This study suggests that chronically exposed people to high concentrations of cooking aerosol might progress toward AD

Human exposure to aerosol from indoor gas stove cooking and the resulting nervous system responses.

Our knowledge of the effects of exposure to indoor ultrafine particles (sub-100 nm, #/cm3 ) on human brain activity is very limited. The effects of cooking ultrafine particles (UFP) on healthy adults were assessed using an electroencephalograph (EEGs) for brain response. Peak ultrafine particle concentrations were approximately 3 × 105 particle/cm3, and the average level was 1.64 × 105 particle/cm3 . The average particle number emission rate (S) and the average number decay rate (a+k) for chicken frying in brain experiments were calculated to be 2.82 × 1012 (SD = 1.83 × 1012 , R2  = 0.91, p = 0.0013) particles/min, 0.47 (SD = 0.30, R2  = 0.90, p < 0.0001) min-1 , respectively. EEGs were recorded before and during cooking (14 min) and 30 min after the cooking sessions. The brain fast-wave band (beta) decreased during exposure, similar to people with neurodegenerative diseases. It subsequently increased to its pre-exposure condition for 70% of the study participants after 30 min. The brain slow-wave band to fast-wave band ratio (theta/beta ratio) increased during and after exposure, similar to observed behavior in early-stage Alzheimer's disease (AD) patients. The brain then tended to return to its normal condition within 30 min following the exposure. This study suggests that chronically exposed people to high concentrations of cooking aerosol might progress toward AD