Shakeout: A New Approach to Regularized Deep Neural Network Training

Recent years have witnessed the success of deep neural networks in dealing with a plenty of practical problems. Dropout has played an essential role in many successful deep neural networks, by inducing regularization in the model training. In this paper, we present a new regularized training approach: Shakeout. Instead of randomly discarding units as Dropout does at the training stage, Shakeout randomly chooses to enhance or reverse each unit's contribution to the next layer. This minor modification of Dropout has the statistical trait: the regularizer induced by Shakeout adaptively combines $L_0$, $L_1$ and $L_2$ regularization terms. Our classification experiments with representative deep architectures on image datasets MNIST, CIFAR-10 and ImageNet show that Shakeout deals with over-fitting effectively and outperforms Dropout. We empirically demonstrate that Shakeout leads to sparser weights under both unsupervised and supervised settings. Shakeout also leads to the grouping effect of the input units in a layer. Considering the weights in reflecting the importance of connections, Shakeout is superior to Dropout, which is valuable for the deep model compression. Moreover, we demonstrate that Shakeout can effectively reduce the instability of the training process of the deep architecture.Comment: Appears at T-PAMI 201

Asymptotic preserving and uniformly unconditionally stable finite difference schemes for kinetic transport equations

In this paper, uniformly unconditionally stable first and second order finite difference schemes are developed for kinetic transport equations in the diffusive scaling. We first derive an approximate evolution equation for the macroscopic density, from the formal solution of the distribution function, which is then discretized by following characteristics for the transport part with a backward finite difference semi-Lagrangian approach, while the diffusive part is discretized implicitly. After the macroscopic density is available, the distribution function can be efficiently solved even with a fully implicit time discretization, since all discrete velocities are decoupled, resulting in a low-dimensional linear system from spatial discretizations at each discrete velocity. Both first and second order discretizations in space and in time are considered. The resulting schemes can be shown to be asymptotic preserving (AP) in the diffusive limit. Uniformly unconditional stabilities are verified from a Fourier analysis based on eigenvalues of corresponding amplification matrices. Numerical experiments, including high dimensional problems, have demonstrated the corresponding orders of accuracy both in space and in time, uniform stability, AP property, and good performances of our proposed approach

High miR-34a and miR-26b expressions inhibit prostate cancer cell OPCN-1 proliferation and enhances apoptosis

Purpose: To investigate the effects of miR-34a and miR-26b on the targeted genes, LEF1 and EphA2, and proliferation and apoptosis of OPCN-1.Methods: Sixty specimens of cancer tissue (CT) and equivalent tissue adjacent to tumors (TAT) were collected from prostate cancer patients. Quantitative reverse transcription polymerase chain reaction (qRT-PCR) was used to determine the mRNA expression levels of miR-34a, miR-26b, LEF1, and EphA2 in the above tissues, while protein expression levels of LEF1 and EphA2 were evaluated by Western blot.Results: Compared with TAT, the expression levels of miR-26b and miR-34a in CT decreased significantly (p < 0.05), whereas the mRNA and protein expression levels of EphA2 and LEF1 in CT significantly increased (p < 0.05). TargetScanHuman7.2 assay data revealed that miR-26b targeted EphA2, while miR-34a targeted LEF1. MiR-26b MG showed decreased EphA2 mRNA and protein levels when compared with miR-26b-NC group after overexpression. The miR-34a MG exhibited decreased expression levels of LEF1 mRNA and protein compared with the miR-34a-NC group. Between 48 and 72 h, miR-26b MG grew more slowly than miR-26b-NC group; miR-34a MG also showed significantly slower growth than miR-34a-NC group. The miR-26b MG and miR-34a MG groups displayed higher apoptosis rate than miR-26b-NC and miR-34a-NC groups, respectively.Conclusion: High expressions of miR-34a and miR-26b targeted the inhibition of LEF1 and EphA2, respectively, indicating that they inhibit the proliferation, and also control the increased apoptosis rate of OPCN-1 cells. Hence, miR-34a and miR-26b are probable molecular targets for the development of new prostate cancer drugs

Post-Newtonian parameters of ghost-free parity-violating gravities

We investigate the slow-motion and weak-field approximation of the general ghost-free parity-violating (PV) theory of gravity in the parametrized post-Newtonian (PPN) framework and derive the perturbative field equations, which are modified by the PV terms of this theory. The complete PPN parameters are obtained by solving the perturbative field equations. We find that all the PPN parameters are exactly the same as those in general relativity, except for an extra parameter $\kappa$, which is caused by the new curl-type term in the gravitomagnetic sector of the metric in this theory. We calculate the precession effects of gyroscopes in this theory and constrain the model parameters by the observations of the Gravity Probe B experiment.Comment: 20 pages, 1 figur

Single phase pressure drop in round cylindrical headers of parallel flow MCHXs

This paper presents the investigation of the pressure drop in headers and development of correlation for pressure loss coefficient for single phase flow through round cylindrical headers of parallel MCHXs. The working fluid was compressed air flowing through header with 1 - 20 m/s based on smallest cross section while the velocity through micro-channels was in the range 6 - 30 m/s. The experimental results indicate that the pressure loss coefficient of inlet header is a linear function of the ratio of velocities through micro-channel tube and header, except for the first two micro-channel tubes; the pressure loss coefficient of outlet header is a quadratic function of the ratio of velocities through micro-channel tube and header, and decreases as the velocities through upstream micro-channel tubes increase. Correlations for predicting pressure drop of inlet header and outlet header are developed and agreement for 98% of experimental data is within a ±15 Pa

Unsupervised String Transformation Learning for Entity Consolidation

Data integration has been a long-standing challenge in data management with many applications. A key step in data integration is entity consolidation. It takes a collection of clusters of duplicate records as input and produces a single "golden record" for each cluster, which contains the canonical value for each attribute. Truth discovery and data fusion methods, as well as Master Data Management (MDM) systems, can be used for entity consolidation. However, to achieve better results, the variant values (i.e., values that are logically the same with different formats) in the clusters need to be consolidated before applying these methods. For this purpose, we propose a data-driven method to standardize the variant values based on two observations: (1) the variant values usually can be transformed to the same representation (e.g., "Mary Lee" and "Lee, Mary") and (2) the same transformation often appears repeatedly across different clusters (e.g., transpose the first and last name). Our approach first uses an unsupervised method to generate groups of value pairs that can be transformed in the same way (i.e., they share a transformation). Then the groups are presented to a human for verification and the approved ones are used to standardize the data. In a real-world dataset with 17,497 records, our method achieved 75% recall and 99.5% precision in standardizing variant values by asking a human 100 yes/no questions, which completely outperformed a state of the art data wrangling tool

Testing parity symmetry of gravity with gravitational waves

The examination of parity symmetry in gravitational interactions has drawn increasing attention. Although Einstein's General Relativity is parity-conserved, numerous theories of parity-violating (PV) gravity in different frameworks have recently been proposed for different motivations. In this review, we briefly summarize the recent progress of these theories, and focus on the observable effects of PV terms in the gravitational waves (GWs), which are mainly reflected in the difference between the left-hand and right-hand polarization modes. We are primarily concerned with the implications of these theories for GWs generated by the compact binary coalescences and the primordial GWs generated in the early Universe. The deviation of GW waveforms and/or primordial power spectrum can always be quantified by the energy scale of parity violation of the theory. Applying the current and future GW observation from laser interferometers and cosmic microwave background radiation, the current and potential constraints on the PV energy scales are presented, which indicates that the parity symmetry of gravity can be tested in high energy scale in this new era of gravitational waves.Comment: 22 pages, no figure

Investigation of Application of Suction Line Heat Exchanger in R290 Air Conditioner with Small Diameter Copper Tube

R290 is a potential refrigerant replacing R22 because of its zero Ozone Depletion Potential (ODP) and virtually zero Global Warming Potential (GWP). However, R290 is flammable and requires excellent containment to avoid leakage and reduce the risk of fire. The use of small diameter copper tube (5 mm or even smaller) is an effective way to reduce refrigerant charge and thus reduce the risk of fire in the event of a refrigerant leak. However, employing small diameter copper tube will increase pressure drop and consequently reduce system performance. A suction line heat exchanger which employs the low temperature refrigerant in suction line to cool down the refrigerant before expansion value is a potential solution to improve system performance because R290 has low discharge temperature compared with HFC refrigerants (e.g. R22, R410A). This paper presents an investigation of application of a suction line heat exchanger in an R290 air conditioner with small diameter copper tube. A theoretical analysis is proposed at first to investigate the effect of the suction line heat exchanger on capacity and system energy efficiency under variable evaporating and cooling temperatures. A prototype R290 air conditioner with and without a suction line heat exchanger is tested in order to explore the effect of a suction line heat exchanger on system performance and refrigerant charge in real working conditions. Finally, a refrigerant circuit solution for heat pump air conditioners is proposed. The results of theoretical analysis indicate that the capacity and system energy efficiency increase linearly with the heat exchange of the suctionline heat exchanger, and the suction line heat exchanger can improves capacity by up to 12% and system energy efficiency by up to 4% under both cooling and heating modes. This is because the suction line heat exchanger increases the sub-cooling but has less impact on compressor power due to good thermal properties of R290. Further, the evaporating and condensation temperature have insignificant impact on the performance of suction line heat exchanger. The experimental results show that the suction line heat exchanger improves the cooling capacity and system efficiency by 5.3% and 4.5%, respectively. These results agree well with that of the system analysis. The sub-cooling temperature increases 10.2o C and the discharge temperature increases 25.4o C. Further, the suction line heat exchanger reduces the refrigerant charge by as much as 6%. This is because suction line heat exchanger increases the discharge temperature, and thus the superheat region of the condenser increases resulting in less refrigerant in the condenser. Overall, the use of a suction line heat exchanger in a system with small diameter copper tube improves the performance of R290 and more importantly reduces the refrigerant charge

Positive loop-closed automata: a decidable class of hybrid systems

AbstractThe model-checking problem for real-time and hybrid systems is very difficult, even for a well-formed class of hybrid systems—the class of linear hybrid automata—the problem is still undecidable in general. So an important question for the analysis and design of real-time and hybrid systems is the identification of subclasses of such systems and corresponding restricted classes of analysis problems that can be settled algorithmically. In this paper, we show that for a class of linear hybrid automata called positive loop-closed automata, the satisfaction problem for linear duration properties can be solved by linear programming. We extend the traditional regular expressions with duration constraints and use them as a language to describe the behaviour of this class of linear hybrid automata. The extended notation is called duration-constrained regular expressions. Based on this formalism, we show that the model-checking problem can be reduced formally to linear programs