Inspecting class hierarchies in classification-based metric learning models

Most classification models treat all misclassifications equally. However, different classes may be related, and these hierarchical relationships must be considered in some classification problems. These problems can be addressed by using hierarchical information during training. Unfortunately, this information is not available for all datasets. Many classification-based metric learning methods use class representatives in embedding space to represent different classes. The relationships among the learned class representatives can then be used to estimate class hierarchical structures. If we have a predefined class hierarchy, the learned class representatives can be assessed to determine whether the metric learning model learned semantic distances that match our prior knowledge. In this work, we train a softmax classifier and three metric learning models with several training options on benchmark and real-world datasets. In addition to the standard classification accuracy, we evaluate the hierarchical inference performance by inspecting learned class representatives and the hierarchy-informed performance, i.e., the classification performance, and the metric learning performance by considering predefined hierarchical structures. Furthermore, we investigate how the considered measures are affected by various models and training options. When our proposed ProxyDR model is trained without using predefined hierarchical structures, the hierarchical inference performance is significantly better than that of the popular NormFace model. Additionally, our model enhances some hierarchy-informed performance measures under the same training options. We also found that convolutional neural networks (CNNs) with random weights correspond to the predefined hierarchies better than random chance.Comment: The main manuscript is 22 pages. The whole paper is 49 pages. The codes for our experiments will be available in https://github.com/hjk92g/Inspecting_Hierarchies_ML . The plankton datasets are available from the Norwegian Marine Data Center (MicroS: https://doi.org/10.21335/NMDC-2102309336 , MicroL: https://doi.org/10.21335/NMDC-573815973 , MesoZ: https://doi.org/10.21335/NMDC-1805578916

Zipf's law, Hierarchical Structure, and Shuffling-Cards Model for Urban Development

A new angle of view is proposed to find the simple rules dominating complex systems and regular patterns behind random phenomena such as cities. Hierarchy of cities reflects the ubiquitous structure frequently observed in the natural world and social institutions. Where there is a hierarchy with cascade structure, there is a rank-size distribution following Zipf's law, and vice versa. The hierarchical structure can be described with a set of exponential functions that are identical in form to Horton-Strahler's laws on rivers and Gutenberg-Richter's laws on earthquake energy. From the exponential models, we can derive four power laws such as Zipf's law indicative of fractals and scaling symmetry. Research on the hierarchy is revealing for us to understand how complex systems are self-organized. A card-shuffling model is built to interpret the relation between Zipf's law and hierarchy of cities. This model can be expanded to explain the general empirical power-law distributions across the individual physical and social sciences, which are hard to be comprehended within the specific scientific domains.Comment: 28 pages, 8 figure

A study of hierarchical and flat classification of proteins

Automatic classification of proteins using machine learning is an important problem that has received significant attention in the literature. One feature of this problem is that expert-defined hierarchies of protein classes exist and can potentially be exploited to improve classification performance. In this article we investigate empirically whether this is the case for two such hierarchies. We compare multi-class classification techniques that exploit the information in those class hierarchies and those that do not, using logistic regression, decision trees, bagged decision trees, and support vector machines as the underlying base learners. In particular, we compare hierarchical and flat variants of ensembles of nested dichotomies. The latter have been shown to deliver strong classification performance in multi-class settings. We present experimental results for synthetic, fold recognition, enzyme classification, and remote homology detection data. Our results show that exploiting the class hierarchy improves performance on the synthetic data, but not in the case of the protein classification problems. Based on this we recommend that strong flat multi-class methods be used as a baseline to establish the benefit of exploiting class hierarchies in this area

GRASS: Generative Recursive Autoencoders for Shape Structures

We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures. Our key insight is that 3D shapes are effectively characterized by their hierarchical organization of parts, which reflects fundamental intra-shape relationships such as adjacency and symmetry. We develop a recursive neural net (RvNN) based autoencoder to map a flat, unlabeled, arbitrary part layout to a compact code. The code effectively captures hierarchical structures of man-made 3D objects of varying structural complexities despite being fixed-dimensional: an associated decoder maps a code back to a full hierarchy. The learned bidirectional mapping is further tuned using an adversarial setup to yield a generative model of plausible structures, from which novel structures can be sampled. Finally, our structure synthesis framework is augmented by a second trained module that produces fine-grained part geometry, conditioned on global and local structural context, leading to a full generative pipeline for 3D shapes. We demonstrate that without supervision, our network learns meaningful structural hierarchies adhering to perceptual grouping principles, produces compact codes which enable applications such as shape classification and partial matching, and supports shape synthesis and interpolation with significant variations in topology and geometry.Comment: Corresponding author: Kai Xu ([email protected]

A Hierarchical Allometric Scaling Analysis of Chinese Cities: 1991-2014

The law of allometric scaling based on Zipf distributions can be employed to research hierarchies of cities in a geographical region. However, the allometric patterns are easily influenced by random disturbance from the noises in observational data. In theory, both the allometric growth law and Zipf's law are related to the hierarchical scaling laws associated with fractal structure. In this paper, the scaling laws of hierarchies with cascade structure are used to study Chinese cities, and the method of R/S analysis is applied to analyzing the change trend of the allometric scaling exponents. The results show that the hierarchical scaling relations of Chinese cities became clearer and clearer from 1991 to 2014 year; the global allometric scaling exponent values fluctuated around 0.85, and the local scaling exponent approached to 0.85. The Hurst exponent of the allometric parameter change is greater than 0.5, indicating persistence and a long-term memory of urban evolution. The main conclusions can be reached as follows: the allometric scaling law of cities represents an evolutionary order rather than an invariable rule, which emerges from self-organized process of urbanization, and the ideas from allometry and fractals can be combined to optimize spatial and hierarchical structure of urban systems in future city planning.Comment: 28 pages, 10 figures, 5 table