Support Vector Machine Histogram: New Analysis and Architecture Design Method of Deep Convolutional Neural Network

Deep convolutional neural network (DCNN) is a kind of hierarchical neural network models and attracts attention in recent years since it has shown high classification performance. DCNN can acquire the feature representation which is a parameter indicating the feature of the input by learning. However, its internal analysis and the design of the network architecture have many unclear points and it cannot be said that it has been sufficiently elucidated. We propose the novel DCNN analysis method “Support vector machine (SVM) histogram” as a prescription to deal with these problems. This is a method that examines the spatial distribution of DCNN extracted feature representation by using the decision boundary of linear SVM. We show that we can interpret DCNN hierarchical processing using this method. In addition, by using the result of SVM histogram, DCNN architecture design becomes possible. In this study, we designed the architecture of the application to large scale natural image dataset. In the result, we succeeded in showing higher accuracy than the original DCNN

Survey on the Family of the Recursive-Rule Extraction Algorithm

In this paper, we first review the theoretical and historical backgrounds on rule extraction from neural network ensembles. Because the structures of previous neural network ensembles were quite complicated, research on an efficient rule extraction algorithm from neural network ensembles has been sparse, even though a practical need exists for rule extraction in Big Data datasets. We describe the Recursive-Rule extraction (Re-RX) algorithm, which is an important step toward handling large datasets. Then we survey the family of the Recursive-Rule extraction algorithm, i.e. the Multiple-MLP Ensemble Re-RX algorithm, and present concrete applications in financial and medical domains that require extremely high accuracy for classification rules. Finally, we mention two promising ideas to considerably enhance the accuracy of the Multiple-MLP Ensemble Re-RX algorithm. We also discuss developments in the near future that will make the Multiple-MLP Ensemble Re-RX algorithm much more accurate, concise, and comprehensible rule extraction from mixed datasets

Soft computing-based methods for semantic service retrieval

University of Technology Sydney. Faculty of Engineering and Information Technology.Nowadays, a large number of business services have been advertised to customers via online channels. To access the published services, the customers typically search for the services by using search engines. Consequently, in order to meet the customers' desires, many researchers have focused on improving performance of the retrieval process. In the recent past, semantic technologies have played an important role in service retrieval and service querying. A service retrieval system consists of two main processes; service annotation and service querying. Annotating services semantically enables machines to understand the purpose of services, while semantic service querying helps machines to expand user queries by considering meanings of query terms, and retrieve services which are relevant to the queries. Because of dealing with semantics of services and queries, both processes can further assist in intelligent and precise service retrieval, selection and composition. In terms of semantic service annotation, a key issue is the manual nature of service annotation. Manual service annotation requires not just large amount of time, but updating the annotation is infrequent and, hence, annotation of the service description changes may be out-of-date. Although some researchers have studied semantic service annotation, they have focused only on Web services, not business service information. Moreover, their approaches are semi-automated, so service providers are still required to select appropriate service annotations. Similar to semantic service annotation, existing literature in semantic service querying has focused on processing Web pages or Web services, not business service information. In addition, because of issues of ubiquity, heterogeneity, and ambiguity of services, the use of soft computing methods offers an interesting solution for handling complex tasks in service retrieval. Unfortunately, based on the literature review, no soft-computing based methods have been used for semantic service annotation or semantic service querying. In this research, intelligent soft-computing driven methods are developed to improve the performance of a semantic retrieval system for business services. The research includes three main parts, namely, intelligent methods for semantically annotating services, querying service concepts, and retrieving services based on relevant concepts. Furthermore, a prototype of a service retrieval system is built to validate the developed intelligent methods. The research proposes three semantic-based methods; ECBR, Vector-based and Classification-based, for accomplishing each research part. The experimental results present that the Classification-based method, which is based on soft-computing techniques, performs well in the service annotation and outperforms both the ECBR and the Vector-based methods in the service querying and service retrieval

Similarity hash based scoring of portable executable files for efficient malware detection in IoT

YesThe current rise in malicious attacks shows that existing security systems are bypassed by malicious files. Similarity hashing has been adopted for sample triaging in malware analysis and detection. File similarity is used to cluster malware into families such that their common signature can be designed. This paper explores four hash types currently used in malware analysis for portable executable (PE) files. Although each hashing technique produces interesting results, when applied independently, they have high false detection rates. This paper investigates into a central issue of how different hashing techniques can be combined to provide a quantitative malware score and to achieve better detection rates. We design and develop a novel approach for malware scoring based on the hashes results. The proposed approach is evaluated through a number of experiments. Evaluation clearly demonstrates a significant improvement (> 90%) in true detection rates of malware

A General Framework for Robust G-Invariance in G-Equivariant Networks

We introduce a general method for achieving robust group-invariance in group-equivariant convolutional neural networks ($G$-CNNs), which we call the $G$-triple-correlation ($G$-TC) layer. The approach leverages the theory of the triple-correlation on groups, which is the unique, lowest-degree polynomial invariant map that is also complete. Many commonly used invariant maps - such as the max - are incomplete: they remove both group and signal structure. A complete invariant, by contrast, removes only the variation due to the actions of the group, while preserving all information about the structure of the signal. The completeness of the triple correlation endows the $G$-TC layer with strong robustness, which can be observed in its resistance to invariance-based adversarial attacks. In addition, we observe that it yields measurable improvements in classification accuracy over standard Max $G$-Pooling in $G$-CNN architectures. We provide a general and efficient implementation of the method for any discretized group, which requires only a table defining the group's product structure. We demonstrate the benefits of this method for $G$-CNNs defined on both commutative and non-commutative groups - $SO(2)$, $O(2)$, $SO(3)$, and $O(3)$ (discretized as the cyclic $C8$, dihedral $D16$, chiral octahedral $O$ and full octahedral $O_h$ groups) - acting on $\mathbb{R}^2$ and $\mathbb{R}^3$ on both $G$-MNIST and $G$-ModelNet10 datasets

A survey on feature drift adaptation: Definition, benchmark, challenges and future directions

Data stream mining is a fast growing research topic due to the ubiquity of data in several real-world problems. Given their ephemeral nature, data stream sources are expected to undergo changes in data distribution, a phenomenon called concept drift. This paper focuses on one specific type of drift that has not yet been thoroughly studied, namely feature drift. Feature drift occurs whenever a subset of features becomes, or ceases to be, relevant to the learning task; thus, learners must detect and adapt to these changes accordingly. We survey existing work on feature drift adaptation with both explicit and implicit approaches. Additionally, we benchmark several algorithms and a naive feature drift detection approach using synthetic and real-world datasets. The results from our experiments indicate the need for future research in this area as even naive approaches produced gains in accuracy while reducing resources usage. Finally, we state current research topics, challenges and future directions for feature drift adaptation