Statistically Motivated Second Order Pooling

Second-order pooling, a.k.a.~bilinear pooling, has proven effective for deep learning based visual recognition. However, the resulting second-order networks yield a final representation that is orders of magnitude larger than that of standard, first-order ones, making them memory-intensive and cumbersome to deploy. Here, we introduce a general, parametric compression strategy that can produce more compact representations than existing compression techniques, yet outperform both compressed and uncompressed second-order models. Our approach is motivated by a statistical analysis of the network's activations, relying on operations that lead to a Gaussian-distributed final representation, as inherently used by first-order deep networks. As evidenced by our experiments, this lets us outperform the state-of-the-art first-order and second-order models on several benchmark recognition datasets.Comment: Accepted to ECCV 2018. Camera ready version. 14 page, 5 figures, 3 table

Towards explaining deep neural networks through graph analysis

Due to its potential to solve complex tasks, deep learning is being used across many different areas. The complexity of neural networks however makes it difficult to explain the whole decision process used by the model, which makes understanding deep learning models an active research topic. In this work we address this issue by extracting the knowledge acquired by trained Deep Neural Networks (DNNs) and representing this knowledge in a graph. The proposed graph encodes statistical correlations between neurons’ activation values in order to expose the relationship between neurons in the hidden layers with both the input layer and output classes. Two initial experiments in image classification were conducted to evaluate whether the proposed graph can help understanding and explaining DNNs. We first show how it is possible to explore the proposed graph to find what neurons are the most important for predicting each class. Then, we use graph analysis to detect groups of classes that are more similar to each other and how these similarities affect the DNN. Finally, we use heatmaps to visualize what parts of the input layer are responsible for activating each neuron in hidden layers. The results show that by building and analysing the proposed graph it is possible to gain relevant insights of the DNN’s inner workings

Cross validation of bi-modal health-related stress assessment

This study explores the feasibility of objective and ubiquitous stress assessment. 25 post-traumatic stress disorder patients participated in a controlled storytelling (ST) study and an ecologically valid reliving (RL) study. The two studies were meant to represent an early and a late therapy session, and each consisted of a "happy" and a "stress triggering" part. Two instruments were chosen to assess the stress level of the patients at various point in time during therapy: (i) speech, used as an objective and ubiquitous stress indicator and (ii) the subjective unit of distress (SUD), a clinically validated Likert scale. In total, 13 statistical parameters were derived from each of five speech features: amplitude, zero-crossings, power, high-frequency power, and pitch. To model the emotional state of the patients, 28 parameters were selected from this set by means of a linear regression model and, subsequently, compressed into 11 principal components. The SUD and speech model were cross-validated, using 3 machine learning algorithms. Between 90% (2 SUD levels) and 39% (10 SUD levels) correct classification was achieved. The two sessions could be discriminated in 89% (for ST) and 77% (for RL) of the cases. This report fills a gap between laboratory and clinical studies, and its results emphasize the usefulness of Computer Aided Diagnostics (CAD) for mental health care

National survey and analysis of barriers to the utilisation of the 2005 Mental Capacity Act by people with bipolar disorder in England and Wales

Background: The Mental Capacity Act (2005) (MCA) provides a legal framework for advance planning for both health and welfare in England and Wales for people if they lose mental capacity e.g. through mania or severe depression. Aims: To determine the proportion of people with bipolar disorder (BD) who utilise advance planning, their experience of using it and barriers to its implementation. Methods: National survey of people with clinical diagnosis of BD of their knowledge, use and experience of the MCA. Thematically analysed qualitative interviews with maximum variance sample of people with BD. Results: 544 respondents with BD participated in the survey; 18 in the qualitative study. 403 (74.1%) believed making plans about their personal welfare if they lost capacity to be very important. 199 (36.6%) participants knew about the MCA. 54 (10%), 62 (11%) and 21 (4%) participants made advanced decisions to refuse treatment, advance statements and lasting power of attorney respectively. Barriers included not understanding its different forms, unrealistic expectations and advance plans ignored by services. Conclusion: In BD the demand for advance plans about welfare with loss of capacity was high but utilisation of the MCA was low with barriers at service user, clinician and organisation levels

Structure Discovery in Mixed Order Hyper Networks

Background  Mixed Order Hyper Networks (MOHNs) are a type of neural network in which the interactions between inputs are modelled explicitly by weights that can connect any number of neurons. Such networks have a human readability that networks with hidden units lack. They can be used for regression, classification or as content addressable memories and have been shown to be useful as fitness function models in constraint satisfaction tasks. They are fast to train and, when their structure is fixed, do not suffer from local minima in the cost function during training. However, their main drawback is that the correct structure (which neurons to connect with weights) must be discovered from data and an exhaustive search is not possible for networks of over around 30 inputs.  Results  This paper presents an algorithm designed to discover a set of weights that satisfy the joint constraints of low training error and a parsimonious model. The combined structure discovery and weight learning process was found to be faster, more accurate and have less variance than training an MLP.  Conclusions  There are a number of advantages to using higher order weights rather than hidden units in a neural network but discovering the correct structure for those weights can be challenging. With the method proposed in this paper, the use of high order networks becomes tractable

Symmetry, Reference Frames, and Relational Quantities in Quantum Mechanics

We propose that observables in quantum theory are properly understood as representatives of symmetry-invariant quantities relating one system to another, the latter to be called a reference system. We provide a rigorous mathematical language to introduce and study quantum reference systems, showing that the orthodox "absolute" quantities are good representatives of observable relative quantities if the reference state is suitably localised. We use this relational formalism to critique the literature on the relationship between reference frames and superselection rules, settling a long-standing debate on the subject

Movement Behavior of High-Heeled Walking: How Does the Nervous System Control the Ankle Joint during an Unstable Walking Condition?

The human locomotor system is flexible and enables humans to move without falling even under less than optimal conditions. Walking with high-heeled shoes constitutes an unstable condition and here we ask how the nervous system controls the ankle joint in this situation? We investigated the movement behavior of high-heeled and barefooted walking in eleven female subjects. The movement variability was quantified by calculation of approximate entropy (ApEn) in the ankle joint angle and the standard deviation (SD) of the stride time intervals. Electromyography (EMG) of the soleus (SO) and tibialis anterior (TA) muscles and the soleus Hoffmann (H-) reflex were measured at 4.0 km/h on a motor driven treadmill to reveal the underlying motor strategies in each walking condition. The ApEn of the ankle joint angle was significantly higher (p<0.01) during high-heeled (0.38±0.08) than during barefooted walking (0.28±0.07). During high-heeled walking, coactivation between the SO and TA muscles increased towards heel strike and the H-reflex was significantly increased in terminal swing by 40% (p<0.01). These observations show that high-heeled walking is characterized by a more complex and less predictable pattern than barefooted walking. Increased coactivation about the ankle joint together with increased excitability of the SO H-reflex in terminal swing phase indicates that the motor strategy was changed during high-heeled walking. Although, the participants were young, healthy and accustomed to high-heeled walking the results demonstrate that that walking on high-heels needs to be controlled differently from barefooted walking. We suggest that the higher variability reflects an adjusted neural strategy of the nervous system to control the ankle joint during high-heeled walking

Can computational efficiency alone drive the evolution of modularity in neural networks?

Some biologists have abandoned the idea that computational efficiency in processing multipart tasks or input sets alone drives the evolution of modularity in biological networks. A recent study confirmed that small modular (neural) networks are relatively computationally-inefficient but large modular networks are slightly more efficient than non-modular ones. The present study determines whether these efficiency advantages with network size can drive the evolution of modularity in networks whose connective architecture can evolve. The answer is no, but the reason why is interesting. All simulations (run in a wide variety of parameter states) involving gradualistic connective evolution end in non-modular local attractors. Thus while a high performance modular attractor exists, such regions cannot be reached by gradualistic evolution. Non-gradualistic evolutionary simulations in which multi-modularity is obtained through duplication of existing architecture appear viable. Fundamentally, this study indicates that computational efficiency alone does not drive the evolution of modularity, even in large biological networks, but it may still be a viable mechanism when networks evolve by non-gradualistic means

Statistical Methods in Recent HIV Noninferiority Trials: Reanalysis of 11 Trials

Background: In recent years the ‘‘noninferiority’ ’ trial has emerged as the new standard design for HIV drug development among antiretroviral patients often with a primary endpoint based on the difference in success rates between the two treatment groups. Different statistical methods have been introduced to provide confidence intervals for that difference. The main objective is to investigate whether the choice of the statistical method changes the conclusion of the trials. Methods: We presented 11 trials published in 2010 using a difference in proportions as the primary endpoint. In these trials, 5 different statistical methods have been used to estimate such confidence intervals. The five methods are described and applied to data from the 11 trials. The noninferiority of the new treatment is not demonstrated if the prespecified noninferiority margin it includes in the confidence interval of the treatment difference. Results: Results indicated that confidence intervals can be quite different according to the method used. In many situations, however, conclusions of the trials are not altered because point estimates of the treatment difference were too far from the prespecified noninferiority margins. Nevertheless, in few trials the use of different statistical methods led to different conclusions. In particular the use of ‘‘exact’ ’ methods can be very confusing. Conclusion: Statistical methods used to estimate confidence intervals in noninferiority trials have a strong impact on th

Validation and Use of 22Na Turnover to Measure Food Intake in Free-Ranging Lizards

As the food intake of free-ranging animals has proved to be difficult to measure by traditional means, the feasibility of using radioactive Na to measure food consumption in a small scincid lizard (Lampropholis guichenoti) was assessed. This technique has previously been used only for several species of mammal. A significant relationship between food intake and Na turnover was found in the laboratory, with Na turnover underestimating intake by 7.6%. The food intake of free-ranging members of a field population was estimated by 22Na turnover to be 9.55, 0.65, 9.39 and 13.75 mg dry weight (day)-1 during autumn, winter, spring and summer respectively. Estimates of assimilated and expended energy from these food intake values agree closely with data reported for other lizards using alternative techniques. This study also describes the technical innovations which were necessary to study lizards weighing less than 1 g; and it suggests that 22Na can provide an easy, reliable and inexpensive means of studying the energetics of many free-living animals