Collaborative Layer-wise Discriminative Learning in Deep Neural Networks

Intermediate features at different layers of a deep neural network are known to be discriminative for visual patterns of different complexities. However, most existing works ignore such cross-layer heterogeneities when classifying samples of different complexities. For example, if a training sample has already been correctly classified at a specific layer with high confidence, we argue that it is unnecessary to enforce rest layers to classify this sample correctly and a better strategy is to encourage those layers to focus on other samples. In this paper, we propose a layer-wise discriminative learning method to enhance the discriminative capability of a deep network by allowing its layers to work collaboratively for classification. Towards this target, we introduce multiple classifiers on top of multiple layers. Each classifier not only tries to correctly classify the features from its input layer, but also coordinates with other classifiers to jointly maximize the final classification performance. Guided by the other companion classifiers, each classifier learns to concentrate on certain training examples and boosts the overall performance. Allowing for end-to-end training, our method can be conveniently embedded into state-of-the-art deep networks. Experiments with multiple popular deep networks, including Network in Network, GoogLeNet and VGGNet, on scale-various object classification benchmarks, including CIFAR100, MNIST and ImageNet, and scene classification benchmarks, including MIT67, SUN397 and Places205, demonstrate the effectiveness of our method. In addition, we also analyze the relationship between the proposed method and classical conditional random fields models.Comment: To appear in ECCV 2016. Maybe subject to minor changes before camera-ready versio

Transmitter-side antennas correlation in SVD-assisted MIMO systems

MIMO techniques allow increasing wireless channel performance by decreasing the BER and increasing the channel throughput and in consequence are included in current mobile communication standards. MIMO techniques are based on benefiting the existence of multipath in wireless communications and the application of appropriate signal processing techniques. The singular value decomposition (SVD) is a popular signal processing technique which, based on the perfect channel state information (PCSI) knowledge at both the transmitter and receiver sides, removes inter-antenna interferences and improves channel performance. Nevertheless, the proximity of the multiple antennas at each front-end produces the so called antennas correlation effect due to the similarity of the various physical paths. In consequence, antennas correlation drops the MIMO channel performance. This investigation focuses on the analysis of a MIMO channel under transmitter-side antennas correlation conditions. First, antennas correlation is analyzed and characterized by the correlation coefficients. The analysis describes the relation between antennas correlation and the appearance of predominant layers which significantly affect the channel performance. Then, based on the SVD, pre- and post-processing is applied to remove inter-antenna interferences. Finally, bit- and power allocation strategies are applied to reach the best performance. The resulting BER reveals that antennas correlation effect diminishes the channel performance and that not necessarily all MIMO layers must be activated to obtain the best performance

Predictive modeling of die filling of the pharmaceutical granules using the flexible neural tree

In this work, a computational intelligence (CI) technique named flexible neural tree (FNT) was developed to predict die filling performance of pharmaceutical granules and to identify significant die filling process variables. FNT resembles feedforward neural network, which creates a tree-like structure by using genetic programming. To improve accuracy, FNT parameters were optimized by using differential evolution algorithm. The performance of the FNT-based CI model was evaluated and compared with other CI techniques: multilayer perceptron, Gaussian process regression, and reduced error pruning tree. The accuracy of the CI model was evaluated experimentally using die filling as a case study. The die filling experiments were performed using a model shoe system and three different grades of microcrystalline cellulose (MCC) powders (MCC PH 101, MCC PH 102, and MCC DG). The feed powders were roll-compacted and milled into granules. The granules were then sieved into samples of various size classes. The mass of granules deposited into the die at different shoe speeds was measured. From these experiments, a dataset consisting true density, mean diameter (d50), granule size, and shoe speed as the inputs and the deposited mass as the output was generated. Cross-validation (CV) methods such as 10FCV and 5x2FCV were applied to develop and to validate the predictive models. It was found that the FNT-based CI model (for both CV methods) performed much better than other CI models. Additionally, it was observed that process variables such as the granule size and the shoe speed had a higher impact on the predictability than that of the powder property such as d50. Furthermore, validation of model prediction with experimental data showed that the die filling behavior of coarse granules could be better predicted than that of fine granules

Virtual Sensor Based on a Deep Learning Approach for Estimating Efficiency in Chillers

P. 307-319Intensive use of heating, ventilation and air conditioning (HVAC) systems in buildings entails an analysis and monitoring of their e ciency. Cooling systems are key facilities in large buildings, and par- ticularly critical in hospitals, where chilled water production is needed as an auxiliary resource for a large number of devices. A chiller plant is often composed of several HVAC units running at the same time, be- ing impossible to assess the individual cooling production and e ciency, since a sensor is seldom installed due to the high cost. We propose a virtual sensor that provides an estimation of the cooling production, based on a deep learning architecture that features a 2D CNN (Convolu- tional Neural Network) to capture relevant features on two-way matrix arrangements of chiller data involving thermodynamic variables and the refrigeration circuits of the chiller unit. Our approach has been tested on an air-cooled chiller in the chiller plant at a hospital, and compared to other state-of-the-art methods using 10-fold cross-validation. Our re- sults report the lowest errors among the tested methods and include a comparison of the true and estimated cooling production and e ciency for a period of several daysS

MEG Can Map Short and Long-Term Changes in Brain Activity following Deep Brain Stimulation for Chronic Pain

Deep brain stimulation (DBS) has been shown to be clinically effective for some forms of treatment-resistant chronic pain, but the precise mechanisms of action are not well understood. Here, we present an analysis of magnetoencephalography (MEG) data from a patient with whole-body chronic pain, in order to investigate changes in neural activity induced by DBS for pain relief over both short- and long-term. This patient is one of the few cases treated using DBS of the anterior cingulate cortex (ACC). We demonstrate that a novel method, null-beamforming, can be used to localise accurately brain activity despite the artefacts caused by the presence of DBS electrodes and stimulus pulses. The accuracy of our source localisation was verified by correlating the predicted DBS electrode positions with their actual positions. Using this beamforming method, we examined changes in whole-brain activity comparing pain relief achieved with deep brain stimulation (DBS ON) and compared with pain experienced with no stimulation (DBS OFF). We found significant changes in activity in pain-related regions including the pre-supplementary motor area, brainstem (periaqueductal gray) and dissociable parts of caudal and rostral ACC. In particular, when the patient reported experiencing pain, there was increased activity in different regions of ACC compared to when he experienced pain relief. We were also able to demonstrate long-term functional brain changes as a result of continuous DBS over one year, leading to specific changes in the activity in dissociable regions of caudal and rostral ACC. These results broaden our understanding of the underlying mechanisms of DBS in the human brain

Integration and fusion of standard automated perimetry and optical coherence tomography data for improved automated glaucoma diagnostics

<p>Abstract</p> <p>Background</p> <p>The performance of glaucoma diagnostic systems could be conceivably improved by the integration of functional and structural test measurements that provide relevant and complementary information for reaching a diagnosis. The purpose of this study was to investigate the performance of data fusion methods and techniques for simple combination of Standard Automated Perimetry (SAP) and Optical Coherence Tomography (OCT) data for the diagnosis of glaucoma using Artificial Neural Networks (ANNs).</p> <p>Methods</p> <p>Humphrey 24-2 SITA standard SAP and StratusOCT tests were prospectively collected from a randomly selected population of 125 healthy persons and 135 patients with glaucomatous optic nerve heads and used as input for the ANNs. We tested commercially available standard parameters as well as novel ones (fused OCT and SAP data) that exploit the spatial relationship between visual field areas and sectors of the OCT peripapillary scan circle. We evaluated the performance of these SAP and OCT derived parameters both separately and in combination.</p> <p>Results</p> <p>The diagnostic accuracy from a combination of fused SAP and OCT data (95.39%) was higher than that of the best conventional parameters of either instrument, i.e. SAP Glaucoma Hemifield Test (p < 0.001) and OCT Retinal Nerve Fiber Layer Thickness ≥ 1 quadrant (p = 0.031). Fused OCT and combined fused OCT and SAP data provided similar Area under the Receiver Operating Characteristic Curve (AROC) values of 0.978 that were significantly larger (p = 0.047) compared to ANNs using SAP parameters alone (AROC = 0.945). On the other hand, ANNs based on the OCT parameters (AROC = 0.970) did not perform significantly worse than the ANNs based on the fused or combined forms of input data. The use of fused input increased the number of tests that were correctly classified by both SAP and OCT based ANNs.</p> <p>Conclusions</p> <p>Compared to the use of SAP parameters, input from the combination of fused OCT and SAP parameters, and from fused OCT data, significantly increased the performance of ANNs. Integrating parameters by including a priori relevant information through data fusion may improve ANN classification accuracy compared to currently available methods.</p

Functional kinds: a skeptical look

The functionalist approach to kinds has suffered recently due to its association with law-based approaches to induction and explanation. Philosophers of science increasingly view nomological approaches as inappropriate for the special sciences like psychology and biology, which has led to a surge of interest in approaches to natural kinds that are more obviously compatible with mechanistic and model-based methods, especially homeostatic property cluster theory. But can the functionalist approach to kinds be weaned off its dependency on laws? Dan Weiskopf has recently offered a reboot of the functionalist program by replacing its nomological commitments with a model-based approach more closely derived from practice in psychology. Roughly, Weiskopf holds that the natural kinds of psychology will be the functional properties that feature in many empirically successful cognitive models, and that those properties need not be localized to parts of an underlying mechanism. I here skeptically examine the three modeling practices that Weiskopf thinks introduce such non-localizable properties: fictionalization, reification, and functional abstraction. In each case, I argue that recognizing functional properties introduced by these practices as autonomous kinds comes at clear cost to those explanations’ counterfactual explanatory power. At each step, a tempting functionalist response is parochialism: to hold that the false or omitted counterfactuals fall outside the modeler’s explanatory aims, and so should not be counted against functional kinds. I conclude by noting the dangers this attitude poses to scientific disagreement, inviting functionalists to better articulate how the individuation conditions for functional kinds might outstrip the perspective of a single modeler

Increase of universality in human brain during mental imagery from visual perception

BACKGROUND: Different complex systems behave in a similar way near their critical points of phase transitions which leads to an emergence of a universal scaling behaviour. Universality indirectly implies a long-range correlation between constituent subsystems. As the distributed correlated processing is a hallmark of higher complex cognition, I investigated a measure of universality in human brain during perception and mental imagery of complex real-life visual object like visual art. METHODOLOGY/PRINCIPAL FINDINGS: A new method was presented to estimate the strength of hidden universal structure in a multivariate data set. In this study, I investigated this method in the electrical activities (electroencephalogram signals) of human brain during complex cognition. Two broad groups--artists and non-artists--were studied during the encoding (perception) and retrieval (mental imagery) phases of actual paintings. Universal structure was found to be stronger in visual imagery than in visual perception, and this difference was stronger in artists than in non-artists. Further, this effect was found to be largest in the theta band oscillations and over the prefrontal regions bilaterally. CONCLUSIONS/SIGNIFICANCE: Phase transition like dynamics was observed in the electrical activities of human brain during complex cognitive processing, and closeness to phase transition was higher in mental imagery than in real perception. Further, the effect of long-term training on the universal scaling was also demonstrated