Utilizing Class Information for Deep Network Representation Shaping

Statistical characteristics of deep network representations, such as sparsity and correlation, are known to be relevant to the performance and interpretability of deep learning. When a statistical characteristic is desired, often an adequate regularizer can be designed and applied during the training phase. Typically, such a regularizer aims to manipulate a statistical characteristic over all classes together. For classification tasks, however, it might be advantageous to enforce the desired characteristic per class such that different classes can be better distinguished. Motivated by the idea, we design two class-wise regularizers that explicitly utilize class information: class-wise Covariance Regularizer (cw-CR) and class-wise Variance Regularizer (cw-VR). cw-CR targets to reduce the covariance of representations calculated from the same class samples for encouraging feature independence. cw-VR is similar, but variance instead of covariance is targeted to improve feature compactness. For the sake of completeness, their counterparts without using class information, Covariance Regularizer (CR) and Variance Regularizer (VR), are considered together. The four regularizers are conceptually simple and computationally very efficient, and the visualization shows that the regularizers indeed perform distinct representation shaping. In terms of classification performance, significant improvements over the baseline and L1/L2 weight regularization methods were found for 21 out of 22 tasks over popular benchmark datasets. In particular, cw-VR achieved the best performance for 13 tasks including ResNet-32/110.Comment: Published in AAAI 201

Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks

Compared to the traditional machine learning models, deep neural networks (DNN) are known to be highly sensitive to the choice of hyperparameters. While the required time and effort for manual tuning has been rapidly decreasing for the well developed and commonly used DNN architectures, undoubtedly DNN hyperparameter optimization will continue to be a major burden whenever a new DNN architecture needs to be designed, a new task needs to be solved, a new dataset needs to be addressed, or an existing DNN needs to be improved further. For hyperparameter optimization of general machine learning problems, numerous automated solutions have been developed where some of the most popular solutions are based on Bayesian Optimization (BO). In this work, we analyze four fundamental strategies for enhancing BO when it is used for DNN hyperparameter optimization. Specifically, diversification, early termination, parallelization, and cost function transformation are investigated. Based on the analysis, we provide a simple yet robust algorithm for DNN hyperparameter optimization - DEEP-BO (Diversified, Early-termination-Enabled, and Parallel Bayesian Optimization). When evaluated over six DNN benchmarks, DEEP-BO mostly outperformed well-known solutions including GP-Hedge, BOHB, and the speed-up variants that use Median Stopping Rule or Learning Curve Extrapolation. In fact, DEEP-BO consistently provided the top, or at least close to the top, performance over all the benchmark types that we have tested. This indicates that DEEP-BO is a robust solution compared to the existing solutions. The DEEP-BO code is publicly available at <uri>https://github.com/snu-adsl/DEEP-BO</uri>

Evaluation on Recovery of Glass and Plastics from Compact Fluorescent Lamps (CFLs) by Air Separation Unit

Compact Fluorescent Lamps (CFLs) are composed of glass, plastic, non-ferrous metal, ferrous metal, paper, plastic, rubber, and so on. In order to separate glass and plastic among CFLs components, air separation unit is applied using the difference in specific gravity. Since specific gravities of glass, plastic, non-ferrous metals, rubber, etc. were widely spread, it can be separated by the different specific gravity between 0.40 and 4.36. In air separation unit, particle size and air speed are controlled to recover glass and plastics among the components of CFLs. In other words, it can be removed paper and vinyl to recover glass and plastics. The specific gravities of paper and vinyl in CFLs are 0.45 and 0.88, respectively. And the specific gravities of glass and plastics are almost similar to be 2.2 - 2.6. In air separation unit, the used particle size of the components from CFLs is less than 6 mm. Since phosphor powder and ferrous metals are recovered prior to the air separation unit, the components are not involved those materials. By utilizing a vertical and zigzag type of air separation unit, thereafter, recovery of glass and plastics is estimated with changing air speed. As the air speed increased from 3.08 m/s to 6.75 m/s, separation efficiency of glass and plastics increased from 42.0% to 99.3%. Due to the experimental results of air separation unit, it can find that paper and vinyl from the components of CFLs be efficiently removed by the air separation unit

The Effect Of Chief Executive Officers Turnover On International Financial Reporting Standards Reconciliation

We investigate the impact of Chief Executive Officer (CEO) changes on International Financial Reporting Standards (IFRS) reconciliation. Since January 1st, 2011 all Korean listed companies are required to adopt IFRS in their separate and consolidated accounts. To aid investors in evaluating corporate performance over time, the companies must restate the K-GAAP financial statements for 2010 under IFRS. We find that negative IFRS reconciliation is more frequent for firms with CEO turnover in 2011. The result suggests that new CEOs have an incentive to report lower earnings through IFRS reconciliation for the purpose of big bath. Additionally, in order to examine whether new CEOs’ incentive of the negative IFRS reconciliation is existed in different corporate governance levels, we classify the companies into strong and weak corporate governance. From the test, we find that their incentive of negative IFRS reconciliation is disappeared (existed) in the companies with strong (weak) corporate governance.   This study will contribute to academics and disclosure-related practitioners by providing valuable information of the CEO incentive regarding IFRS reconciliation. We believe that our empirical evidence will be helpful to market participants when they make a business decisions in case of CEO turnover

The boundary of Rauzy fractal and discrete tilings

The Rauzy fractal is a domain in the two-dimensional plane constructed by the Rauzy substitution, a substitution rule on three letters. The Rauzy fractal has a fractal-like boundary, and the currently known its constructions is not only for its boundary but also for the entire domain. In this paper, we show that all points in the Rauzy fractal have a layered structure. We propose two methods of constructing the Rauzy fractal using layered structures. We show how such layered structures can be used to construct the boundary of the Rauzy fractal with less computation than conventional methods. There is a self-replicating pattern in one of the layered structure in the Rauzy fractal. We introduce a notion of self-replicating word and visualize how some self-replicating words on three letters creates discrete tiling of the two dimensional plane

The Influence Of Firm’s Fair Value System On Earnings Quality Under IFRS

This paper analyzes the influence of firms’ fair value system on earnings quality under IFRS. Korean firms are required to adopt IFRS in 2011. IFRS adoption was expected to increase value relevance of book value of equity and benefit information users’ decision making. However, prior Korean studies report that value relevance of book value of equity is indifferent between under K-GAAP and IFRS. We consider that the indifference in value relevance of book value of equity after IFRS adoption is due to different level of fair value system among firms. We investigate whether the different level of fair value system among firms lead to the difference in earnings quality. Furthermore, we examine how each firm’s fair value system affect earnings quality under IFRS.  This study finds following results. First, firms with weak fair value system smooth income more frequently. Second, firms with weak fair value system experience small amount of positive profit and slight increase in net income compared to prior period more frequently. Third, firms with weak fair value system make less timely loss recognition. Lastly, book value of equity and goodwill has low relative value relevance for weak fair value systemic firms, while both book value of equity and goodwill have incremental value relevance for firms with strong fair value evaluation system