A hybrid approach

Costa-Mendes, R., Oliveira, T., Castelli, M., & Cruz-Jesus, F. (2021). A machine learning approximation of the 2015 Portuguese high school student grades: A hybrid approach. Education and Information Technologies, 26(2), 1527-1547. https://doi.org/10.1007/s10639-020-10316-yThis article uses an anonymous 2014–15 school year dataset from the Directorate-General for Statistics of Education and Science (DGEEC) of the Portuguese Ministry of Education as a means to carry out a predictive power comparison between the classic multilinear regression model and a chosen set of machine learning algorithms. A multilinear regression model is used in parallel with random forest, support vector machine, artificial neural network and extreme gradient boosting machine stacking ensemble implementations. Designing a hybrid analysis is intended where classical statistical analysis and artificial intelligence algorithms are blended to augment the ability to retain valuable conclusions and well-supported results. The machine learning algorithms attain a higher level of predictive ability. In addition, the stacking appropriateness increases as the base learner output correlation matrix determinant increases and the random forest feature importance empirical distributions are correlated with the structure of p-values and the statistical significance test ascertains of the multiple linear model. An information system that supports the nationwide education system should be designed and further structured to collect meaningful and precise data about the full range of academic achievement antecedents. The article concludes that no evidence is found in favour of smaller classes.publishersversionpublishe

A Quantum Space of Representation

Costa-mendes, R., Cruz-jesus, F., Oliveira, T., & Castelli, M. (2022). Deep Learning in Predicting High School Grades: A Quantum Space of Representation. Emerging Science Journal, 6, 166-187. https://doi.org/10.28991/ESJ-2022-SIED-012. Funding: This study was funded by FCT – Fundação para a Ciência e Tecnologia (DSAIPA/DS/0032/2018)This paper applies deep learning to the prediction of Portuguese high school grades. A deep multilayer perceptron and a multiple linear regression implementation are undertaken. The objective is to demonstrate the adequacy of deep learning as a quantitative explanatory paradigm when compared with the classical econometrics approach. The results encompass point predictions, prediction intervals, variable gradients, and the impact of an increase in the class size on grades. Deep learning’s generalization error is lower in the student grade prediction, and its prediction intervals are more accurate. The deep multilayer perceptron gradient empirical distributions largely align with the regression coefficient estimates, indicating a satisfactory regression fit. Based on gradient discrepancies, a student’s mother being an employer does not seem to be a positive factor. A benign paradigm shift concerning the balance between home and career affairs for both genders should be reinforced. The deep multilayer perceptron broadens the spectrum of possibilities, providing a quantum solution hinged on a universal approximator. In the case of an academic achievement-critical factor such as class size, where the literature is neither unanimous on its importance nor its direction, the multilayer perceptron formed three distinct clusters per the individual gradient signals.publishersversionpublishe

Feature selection in credit risk modeling: an international evidence

This paper aims to discover a suitable combination of contemporary feature selection techniques and robust prediction classifiers. As such, to examine the impact of the feature selection method on classifier performance, we use two Chinese and three other real-world credit scoring datasets. The utilized feature selection methods are the least absolute shrinkage and selection operator (LASSO), multivariate adaptive regression splines (MARS). In contrast, the examined classifiers are the classification and regression trees (CART), logistic regression (LR), artificial neural network (ANN), and support vector machines (SVM). Empirical findings confirm that LASSO’s feature selection method, followed by robust classifier SVM, demonstrates remarkable improvement and outperforms other competitive classifiers. Moreover, ANN also offers improved accuracy with feature selection methods; LR only can improve classification efficiency through performing feature selection via LASSO. Nonetheless, CART does not provide any indication of improvement in any combination. The proposed credit scoring modeling strategy may use to develop policy, progressive ideas, operational guidelines for effective credit risk management of lending, and other financial institutions. The finding of this study has practical value, as to date, there is no consensus about the combination of feature selection method and prediction classifiers

A comparison between Recurrent Neural Networks and classical machine learning approaches In Laser induced breakdown spectroscopy

Recurrent Neural Networks are classes of Artificial Neural Networks that establish connections between different nodes form a directed or undirected graph for temporal dynamical analysis. In this research, the laser induced breakdown spectroscopy (LIBS) technique is used for quantitative analysis of aluminum alloys by different Recurrent Neural Network (RNN) architecture. The fundamental harmonic (1064 nm) of a nanosecond Nd:YAG laser pulse is employed to generate the LIBS plasma for the prediction of constituent concentrations of the aluminum standard samples. Here, Recurrent Neural Networks based on different networks, such as Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Simple Recurrent Neural Network (Simple RNN), and as well as Recurrent Convolutional Networks comprising of Conv-SimpleRNN, Conv-LSTM and Conv-GRU are utilized for concentration prediction. Then a comparison is performed among prediction by classical machine learning methods of support vector regressor (SVR), the Multi Layer Perceptron (MLP), Decision Tree algorithm, Gradient Boosting Regression (GBR), Random Forest Regression (RFR), Linear Regression, and k-Nearest Neighbor (KNN) algorithm. Results showed that the machine learning tools based on Convolutional Recurrent Networks had the best efficiencies in prediction of the most of the elements among other multivariate methods

Predicting Long-Term Mortality in TAVI Patients Using Machine Learning Techniques

Background: Whereas transcatheter aortic valve implantation (TAVI) has become the gold standard for aortic valve stenosis treatment in high-risk patients, it has recently been extended to include intermediate risk patients. However, the mortality rate at 5 years is still elevated. The aim of the present study was to develop a novel machine learning (ML) approach able to identify the best predictors of 5-year mortality after TAVI among several clinical and echocardiographic variables, which may improve the long-term prognosis. Methods: We retrospectively enrolled 471 patients undergoing TAVI. More than 80 pre-TAVI variables were collected and analyzed through different feature selection processes, which allowed for the identification of several variables with the highest predictive value of mortality. Different ML models were compared. Results: Multilayer perceptron resulted in the best performance in predicting mortality at 5 years after TAVI, with an area under the curve, positive predictive value, and sensitivity of 0.79, 0.73, and 0.71, respectively. Conclusions: We presented an ML approach for the assessment of risk factors for long-term mortality after TAVI to improve clinical prognosis. Fourteen potential predictors were identified with the organic mitral regurgitation (myxomatous or calcific degeneration of the leaflets and/or annulus) which showed the highest impact on 5 years mortality

Univariate and Multivariate Regression Models for Short-Term Wind Energy Forecasting

Wind energy resource is a never-ending resource that is categorized under renewable energy. Electricity generated from the wind when the wind blows across the wind turbine system produces high kinetic energy once it goes through the wind blades, rotating and turning it into useful mechanical energy. That motion of the generator produces electricity. However, in Malaysia, the inconsistency in terms of wind speed required for wind turbines to operate efficiently and generate a suitable amount of electrical power is a major problem. Different locations have different weather parameters that affect wind speed and wind energy production. Wind energy forecasting is performed in this paper using linear, nonlinear, and deep learning models for a small-scale wind turbine. The paper focuses on comparing and correlating the performance of univariate and multivariate input parameters with wind speed as its primary feature using short-term forecasting with a time horizon of 1 hour ahead. The set location is at Mersing, Johor, where it is prominently one of the locations in Malaysia with a constant and high amount of wind speed. It is found that Huber Regressor, Gradient Boosting, and Convolutional Neural Network (CNN) are shown to be powerful in prediction. Huber Regressor has the best Mean Absolute Error (MAE) of 0.597 and Root Mean Square Error (RMSE) of 0.797, while Gradient Boosting has the best learning rate (R2) at 0.637. CNN has the best MAPE at 30.861 and is shown to be the most optimum forecasting model for a univariate parameter. The results show that the outcome of the evaluation does not vary significantly depending on the criteria chosen in the data selection

Univariate and Multivariate Regression Models for Short-Term Wind Energy Forecasting

Wind energy resource is a never-ending resource that is categorized under renewable energy. Electricity generated from the wind when the wind blows across the wind turbine system produces high kinetic energy once it goes through the wind blades, rotating and turning it into useful mechanical energy. That motion of the generator produces electricity. However, in Malaysia, the inconsistency in terms of wind speed required for wind turbines to operate efficiently and generate a suitable amount of electrical power is a major problem. Different locations have different weather parameters that affect wind speed and wind energy production. Wind energy forecasting is performed in this paper using linear, nonlinear, and deep learning models for a small-scale wind turbine. The paper focuses on comparing and correlating the performance of univariate and multivariate input parameters with wind speed as its primary feature using short-term forecasting with a time horizon of 1 hour ahead. The set location is at Mersing, Johor, where it is prominently one of the locations in Malaysia with a constant and high amount of wind speed. It is found that Huber Regressor, Gradient Boosting, and Convolutional Neural Network (CNN) are shown to be powerful in prediction. Huber Regressor has the best Mean Absolute Error (MAE) of 0.597 and Root Mean Square Error (RMSE) of 0.797, while Gradient Boosting has the best learning rate (R2) at 0.637. CNN has the best MAPE at 30.861 and is shown to be the most optimum forecasting model for a univariate parameter. The results show that the outcome of the evaluation does not vary significantly depending on the criteria chosen in the data selection