Computational complexity analysis of decision tree algorithms

YesDecision tree is a simple but powerful learning technique that is considered as one of the famous learning algorithms that have been successfully used in practice for various classification tasks. They have the advantage of producing a comprehensible classification model with satisfactory accuracy levels in several application domains. In recent years, the volume of data available for learning is dramatically increasing. As a result, many application domains are faced with a large amount of data thereby posing a major bottleneck on the computability of learning techniques. There are different implementations of the decision tree using different techniques. In this paper, we theoretically and experimentally study and compare the computational power of the most common classical top-down decision tree algorithms (C4.5 and CART). This work can serve as part of review work to analyse the computational complexity of the existing decision tree classifier algorithm to gain understanding of the operational steps with the aim of optimizing the learning algorithm for large datasets

Computational complexity analysis of decision tree algorithms

YesDecision tree is a simple but powerful learning technique that is considered as one of the famous learning algorithms that have been successfully used in practice for various classification tasks. They have the advantage of producing a comprehensible classification model with satisfactory accuracy levels in several application domains. In recent years, the volume of data available for learning is dramatically increasing. As a result, many application domains are faced with a large amount of data thereby posing a major bottleneck on the computability of learning techniques. There are different implementations of the decision tree using different techniques. In this paper, we theoretically and experimentally study and compare the computational power of the most common classical top-down decision tree algorithms (C4.5 and CART). This work can serve as part of review work to analyse the computational complexity of the existing decision tree classifier algorithm to gain understanding of the operational steps with the aim of optimizing the learning algorithm for large datasets

Fast hyperboloid decision tree algorithms

Hyperbolic geometry is gaining traction in machine learning for its effectiveness at capturing hierarchical structures in real-world data. Hyperbolic spaces, where neighborhoods grow exponentially, offer substantial advantages and consistently deliver state-of-the-art results across diverse applications. However, hyperbolic classifiers often grapple with computational challenges. Methods reliant on Riemannian optimization frequently exhibit sluggishness, stemming from the increased computational demands of operations on Riemannian manifolds. In response to these challenges, we present hyperDT, a novel extension of decision tree algorithms into hyperbolic space. Crucially, hyperDT eliminates the need for computationally intensive Riemannian optimization, numerically unstable exponential and logarithmic maps, or pairwise comparisons between points by leveraging inner products to adapt Euclidean decision tree algorithms to hyperbolic space. Our approach is conceptually straightforward and maintains constant-time decision complexity while mitigating the scalability issues inherent in high-dimensional Euclidean spaces. Building upon hyperDT we introduce hyperRF, a hyperbolic random forest model. Extensive benchmarking across diverse datasets underscores the superior performance of these models, providing a swift, precise, accurate, and user-friendly toolkit for hyperbolic data analysis

Computational Complexity for Physicists

These lecture notes are an informal introduction to the theory of computational complexity and its links to quantum computing and statistical mechanics.Comment: references updated, reprint available from http://itp.nat.uni-magdeburg.de/~mertens/papers/complexity.shtm

Tree-based Intelligent Intrusion Detection System in Internet of Vehicles

The use of autonomous vehicles (AVs) is a promising technology in Intelligent Transportation Systems (ITSs) to improve safety and driving efficiency. Vehicle-to-everything (V2X) technology enables communication among vehicles and other infrastructures. However, AVs and Internet of Vehicles (IoV) are vulnerable to different types of cyber-attacks such as denial of service, spoofing, and sniffing attacks. In this paper, an intelligent intrusion detection system (IDS) is proposed based on tree-structure machine learning models. The results from the implementation of the proposed intrusion detection system on standard data sets indicate that the system has the ability to identify various cyber-attacks in the AV networks. Furthermore, the proposed ensemble learning and feature selection approaches enable the proposed system to achieve high detection rate and low computational cost simultaneously.Comment: Accepted in IEEE Global Communications Conference (GLOBECOM) 201

Analisis Perbandingan Decision Tree Algoritma C4.5 dengan algoritma lainnya: Sistematic Literature Review

Decision tree is one of the popular methods in data analysis and machine learning. The C4.5 algorithm is one of the most widely used decision tree algorithms because of its ability to produce decision rules that can be understood easily. However, various variations and developments of other decision tree algorithms have emerged, offering improved performance and new features. This study aims to carry out a comparative analysis between the C4.5 decision tree algorithm and several other decision tree algorithms that have been developed. The method used in this research is a systematic literature review, in which the researcher conducts a structured search and evaluation of relevant scientific articles. Researchers will compare the performance of the C4.5 algorithm with other algorithms based on several criteria, including predictive accuracy, computational complexity, interpretability of decision rules, and ability to handle unbalanced data. The results of the analysis show that the selection of a decision tree algorithm must be based on the specific purpose of the analysis and the characteristics of the data used. If the interpretability of decision rules is a major factor, the C4.5 algorithm remains a good choice. However, if predictive accuracy and handling of unbalanced data is a priority, algorithms such as Random Forest, Naive Bayes, or KNN may be a better choice.Decision tree merupakan salah satu metode populer dalam analisis data dan pembelajaran mesin. Algoritma C4.5 adalah salah satu algoritma decision tree yang banyak digunakan karena kemampuannya dalam menghasilkan aturan keputusan yang dapat dipahami dengan mudah. Namun, berbagai variasi dan pengembangan algoritma decision tree lainnya telah muncul, menawarkan peningkatan kinerja dan fitur-fitur baru. Penelitian ini bertujuan untuk melakukan analisis perbandingan antara algoritma decision tree C4.5 dengan beberapa algoritma decision tree lainnya yang telah dikembangkan. Metode yang digunakan dalam penelitian ini yaitu sistematic literature review, di mana peneliti melakukan pencarian terstruktur dan evaluasi terhadap artikel-artikel ilmiah yang relevan. Peneliti akan membandingkan kinerja algoritma C4.5 dengan algoritma lainnya berdasarkan beberapa kriteria, termasuk akurasi prediksi, kompleksitas komputasional, interpretabilitas aturan keputusan, dan kemampuan menangani data yang tidak seimbang. Hasil analisis memperlihatkan bahwa pemilihan algoritma decision tree harus didasarkan pada tujuan spesifik analisis dan karakteristik data yang digunakan. Jika interpretabilitas aturan keputusan menjadi faktor utama, algoritma C4.5 tetap menjadi pilihan yang baik. Namun, jika akurasi prediksi dan penanganan data yang tidak seimbang menjadi prioritas, algoritma-algoritma seperti Random Forest, Naive Bayes, atau KNN dapat menjadi pilihan yang lebih baik

Methods for evaluating Decision Problems with Limited Information

LImited Memory Influence Diagrams (LIMIDs) are general models of decision problems for representing limited memory policies (Lauritzen and Nilsson (2001)). The evaluation of LIMIDs can be done by Single Policy Updating that produces a local maximum strategy in which no single policy modification can increase the expected utility. This paper examines the quality of the obtained local maximum strategy and proposes three different methods for evaluating LIMIDs. The first algorithm, Temporal Policy Updating, resembles Single Policy Updating. The second algorithm, Greedy Search, successively updates the policy that gives the highest expected utility improvement. The final algorithm, Simulating Annealing, differs from the two preceeding by allowing the search to take some downhill steps to escape a local maximum. A careful comparison of the algorithms is provided both in terms of the quality of the obtained strategies, and in terms of implementation of the algorithms including some considerations of the computational complexity

The Folklore of Sorting Algorithms

The objective of this paper is to review the folklore knowledge seen in research work devoted on synthesis, optimization, and effectiveness of various sorting algorithms. We will examine sorting algorithms in the folklore lines and try to discover the tradeoffs between folklore and theorems. Finally, the folklore knowledge on complexity values of the sorting algorithms will be considered, verified and subsequently converged in to theorems