On the Importance of Normalisation Layers in Deep Learning with Piecewise Linear Activation Units

Deep feedforward neural networks with piecewise linear activations are currently producing the state-of-the-art results in several public datasets. The combination of deep learning models and piecewise linear activation functions allows for the estimation of exponentially complex functions with the use of a large number of subnetworks specialized in the classification of similar input examples. During the training process, these subnetworks avoid overfitting with an implicit regularization scheme based on the fact that they must share their parameters with other subnetworks. Using this framework, we have made an empirical observation that can improve even more the performance of such models. We notice that these models assume a balanced initial distribution of data points with respect to the domain of the piecewise linear activation function. If that assumption is violated, then the piecewise linear activation units can degenerate into purely linear activation units, which can result in a significant reduction of their capacity to learn complex functions. Furthermore, as the number of model layers increases, this unbalanced initial distribution makes the model ill-conditioned. Therefore, we propose the introduction of batch normalisation units into deep feedforward neural networks with piecewise linear activations, which drives a more balanced use of these activation units, where each region of the activation function is trained with a relatively large proportion of training samples. Also, this batch normalisation promotes the pre-conditioning of very deep learning models. We show that by introducing maxout and batch normalisation units to the network in network model results in a model that produces classification results that are better than or comparable to the current state of the art in CIFAR-10, CIFAR-100, MNIST, and SVHN datasets

Orbital period analysis of eclipsing Z Cam-type Dwarf Nova EM Cygni: Evidence of magnetic braking and a third body

Combining with our newest CCD times of light minimum of EM Cygni, all 45 available times of light minimum including 7 data with large scatters are compiled and the updated O-C analysis is made. The bestfit for the O-C diagram of EM Cygni is a quadratic-plus-sinusoidal fit. The secular orbital period decrease rate -2.5(\pm 0.3)x10^{-11} s s^{-1} means that magnetic braking effect with a rate of mass loss via stellar wind, 2.3x10^{-10}Msunyr^{-1}, is needed for explaining the observed orbital period decrease. Moreover, for explaining the significant cyclical period change with a period of \sim 17.74(\pm 0.01)yr shown in the O-C diagram, magnetic activity cycles and light travel-time effect are discussed in detail. The O-C diagram of EM Cygni cannot totally rule the possibility of multi-periodic modulation out due to the gaps presented after 25000 cycles. Based on the hypothesis of a K-type third star in literature, light trave-time effect may be a more plausible explanation. However, the low orbital inclination of the third body (\sim 7.4 degree) suggests that the hypothetic K-type third star may be captured by EM Cygni. But assuming the spectral contamination from a block of circumbinary material instead of a K-type third star, the third star may be a brown dwarf in case of the coplanar orbit with parent binary.Comment: 11 pages, 2 figures, accepted for publication in PAS

Organic rankine cycle with positive displacement expander and variable working fluid composition

Organic Rankine Cycles are often used in the exploitation of low-temperature heat sources. The relatively small temperature differential available to these projects makes them particularly vulnerable to changing ambient conditions, especially if an air-cooled condenser is used. The authors have recently demonstrated that a dynamic ORC with a variable working fluid composition, tuned to match the condensing temperature with the heat sink, can be used to achieve a considerable increase in year-round power generation under such conditions [1]. However, this assumed the expander was a turbine capable of operating at multiple pressure ratios for large scale applications. This paper will investigate if small scale ORC systems that use positive-displacement expanders with fixed expansion ratios could also benefit from this new concept. In this paper, a numerical model was firstly developed. A comprehensive analysis was then conducted for a case study. The results showed that the dynamic Organic Rankine Cycle concept can be applied to lower-power applications that use that use positive-displacement expanders with fixed expansion ratios and still result in improvements in year-round energy generation

Using a side-branched volume to tune the acoustic field in a looped-tube travelling-wave thermoacoustic engine with a RC load

Travelling-wave thermoacoustic engine utilises a compact acoustic network to obtain a right time-phasing between the acoustic velocity and pressure oscillations within the regenerator to force gas parcels to experience a Stirling-like thermodynamic cycle. As such, thermal energy can be converted to mechanical work (i.e., high-intensity pressure waves). It is therefore crucial to control the time-phasing carefully to improve the performance of thermoacoustic engines. Various ways have been proposed and demonstrated for adjusting time-phasing, including both passive and active methods. The aim of this study is to introduce a new passive phase tuning method (i.e., a side-branched acoustic volume) to tune the time-phasing within a looped-tube travelling wave thermoacoustic engine. The proposed concept has been investigated both numerically and experimentally in this research. An experimental rig was simulated and designed using DeltaEC software (Design Environment for Low-amplitude ThermoAcoustic Energy Conversion). It was then constructed according to the obtained theoretical model. The result of this study showed a qualitative agreement between experimental measurement and numerical simulations, demonstrating that the proposed technique can effectively adjust the phase angle between the acoustic velocity and pressure oscillations within the loop-tube thermoacoustic engines, and improve its performance