On Breiman's Dilemma in Neural Networks: Phase Transitions of Margin Dynamics

Margin enlargement over training data has been an important strategy since perceptrons in machine learning for the purpose of boosting the robustness of classifiers toward a good generalization ability. Yet Breiman shows a dilemma (Breiman, 1999) that a uniform improvement on margin distribution \emph{does not} necessarily reduces generalization errors. In this paper, we revisit Breiman's dilemma in deep neural networks with recently proposed spectrally normalized margins. A novel perspective is provided to explain Breiman's dilemma based on phase transitions in dynamics of normalized margin distributions, that reflects the trade-off between expressive power of models and complexity of data. When data complexity is comparable to the model expressiveness in the sense that both training and test data share similar phase transitions in normalized margin dynamics, two efficient ways are derived to predict the trend of generalization or test error via classic margin-based generalization bounds with restricted Rademacher complexities. On the other hand, over-expressive models that exhibit uniform improvements on training margins, as a distinct phase transition to test margin dynamics, may lose such a prediction power and fail to prevent the overfitting. Experiments are conducted to show the validity of the proposed method with some basic convolutional networks, AlexNet, VGG-16, and ResNet-18, on several datasets including Cifar10/100 and mini-ImageNet.Comment: 34 page

Sparse Matrix-based Random Projection for Classification

As a typical dimensionality reduction technique, random projection can be simply implemented with linear projection, while maintaining the pairwise distances of high-dimensional data with high probability. Considering this technique is mainly exploited for the task of classification, this paper is developed to study the construction of random matrix from the viewpoint of feature selection, rather than of traditional distance preservation. This yields a somewhat surprising theoretical result, that is, the sparse random matrix with exactly one nonzero element per column, can present better feature selection performance than other more dense matrices, if the projection dimension is sufficiently large (namely, not much smaller than the number of feature elements); otherwise, it will perform comparably to others. For random projection, this theoretical result implies considerable improvement on both complexity and performance, which is widely confirmed with the classification experiments on both synthetic data and real data

Investování do vybraných technologických společností

Import 02/11/2016Financial market is a place that people trade different financial securities to earn the profits in general. Capital market plays a significant role in the financial market. It provides people funds to invest or debt financing for long term, which greater than one year. Because of the uncertainty of the capital market in the future, investing in capital market is not easy. Therefore, my aim is to evaluate investments in the capital market. The main objective of this thesis is to evaluate performance of the selected technological companies in the capital market. Due to Apple company and Google company are both leading companies in IT industry, they are chosen to compare to each other. According to the investing triangle, we evaluate these two companies by using return, risk and liquidity. Comparing Apple company and Google company, it can let us understand more about the condition and situation of the IT industry in the USA, and the results will help us to figure out which choice is better for the investors according to the different criteria. The criteria which includes stock prices, market values, dividends, annual returns, monthly returns and risks are counted to help us evaluate the advantages and disadvantages of both Apple company and Google company. In this thesis, the first part is the principles of investing in the financial market in general. The classification, history, roles and functions, main instruments and basic principles of investing in capital market are all included in the first part. The second part introduces mainly the development history of Apple company and Google company, and basic financial characteristics such as ROA and ROE shows us how profitable these two companies are. Then market indicators, for example, EPS, P/E ratio and dividend payout ratio will be introduced and analyzed in the second part as well. The third part is the core of my study. In this part, the main objective is comparison of both Apple’s and Google’s stock performance. It composes of monthly and annual returns and cumulative returns. Risk is calculated by using standard deviation method. Liquidity is also mentioned to show the volume of the stocks. By comparison of Apple company and Google company, we draw a conclusion that based on the used criteria we could suggest to invest in Apple’s stock rather than Google’s, with high dividend and lower risk.Financial market is a place that people trade different financial securities to earn the profits in general. Capital market plays a significant role in the financial market. It provides people funds to invest or debt financing for long term, which greater than one year. Because of the uncertainty of the capital market in the future, investing in capital market is not easy. Therefore, my aim is to evaluate investments in the capital market. The main objective of this thesis is to evaluate performance of the selected technological companies in the capital market. Due to Apple company and Google company are both leading companies in IT industry, they are chosen to compare to each other. According to the investing triangle, we evaluate these two companies by using return, risk and liquidity. Comparing Apple company and Google company, it can let us understand more about the condition and situation of the IT industry in the USA, and the results will help us to figure out which choice is better for the investors according to the different criteria. The criteria which includes stock prices, market values, dividends, annual returns, monthly returns and risks are counted to help us evaluate the advantages and disadvantages of both Apple company and Google company. In this thesis, the first part is the principles of investing in the financial market in general. The classification, history, roles and functions, main instruments and basic principles of investing in capital market are all included in the first part. The second part introduces mainly the development history of Apple company and Google company, and basic financial characteristics such as ROA and ROE shows us how profitable these two companies are. Then market indicators, for example, EPS, P/E ratio and dividend payout ratio will be introduced and analyzed in the second part as well. The third part is the core of my study. In this part, the main objective is comparison of both Apple’s and Google’s stock performance. It composes of monthly and annual returns and cumulative returns. Risk is calculated by using standard deviation method. Liquidity is also mentioned to show the volume of the stocks. By comparison of Apple company and Google company, we draw a conclusion that based on the used criteria we could suggest to invest in Apple’s stock rather than Google’s, with high dividend and lower risk.154 - Katedra financívelmi dobř

Two-axis-twisting spin squeezing by multi-pass quantum erasure

Many-body entangled states are key elements in quantum information science and quantum metrology. One important problem in establishing a high degree of many-body entanglement using optical techniques is the leakage of the system information via the light that creates such entanglement. We propose an all-optical interference-based approach to erase this information. Unwanted atom-light entanglement can be removed by destructive interference of three or more successive atom-light interactions, with only the desired effective atom-atom interaction left. This quantum erasure protocol allows implementation of Heisenberg-limited spin squeezing using coherent light and a cold or warm atomic ensemble. Calculations show that significant improvement in the squeezing exceeding 10 dB is obtained compared to previous methods, and substantial spin squeezing is attainable even under moderate experimental conditions. Our method enables the efficient creation of many-body entangled states with simple setups, and thus is promising for advancing technologies in quantum metrology and quantum information processing.Comment: 10 pages, 4 figures. We have improved the presentation and added a new section, in which we have generalized the scheme from a three-pass scheme to multi-pass schem

Sparse Binary Matrices of LDPC codes for Compressed Sensing

Document correspondant à la page publiée dans les actes de la conférenceInternational audienceCompressed sensing shows that one undetermined measurement matrix can losslessly compress sparse signals if this matrix satisfies Restricted Isometry Property (RIP). However, in practice there are still no explicit approaches to construct such matrices. Gaussian matrices and Fourier matrices are first proved satisfying RIP with high probabilities. Recently, sparse random binary matrices with lower computation load also expose comparable performance with Gaussian matrices. But they are all constructed randomly, and unstable in orthogonality. In this paper, inspired by these observations, we propose to construct structured sparse binary matrices which are stable in orthogonality. The solution lies in the algorithms that construct parity-check matrices of low-density parity-check (LDPC) codes. Experiments verify that proposed matrices significantly outperform aforementioned three types of matrices. And significantly, for this type of matrices with a given size, the optimal matrix for compressed sensing can be approximated and constructed according to some rules

Investigating the influence of special on-off attacks on challenge-based collaborative intrusion detection networks

Intrusions are becoming more complicated with the recent development of adversarial techniques. To boost the detection accuracy of a separate intrusion detector, the collaborative intrusion detection network (CIDN) has thus been developed by allowing intrusion detection system (IDS) nodes to exchange data with each other. Insider attacks are a great threat for such types of collaborative networks, where an attacker has the authorized access within the network. In literature, a challenge-based trust mechanism is effective at identifying malicious nodes by sending challenges. However, such mechanisms are heavily dependent on two assumptions, which would cause CIDNs to be vulnerable to advanced insider attacks in practice. In this work, we investigate the influence of advanced on–off attacks on challenge-based CIDNs, which can respond truthfully to one IDS node but behave maliciously to another IDS node. To evaluate the attack performance, we have conducted two experiments under a simulated and a real CIDN environment. The obtained results demonstrate that our designed attack is able to compromise the robustness of challenge-based CIDNs in practice; that is, some malicious nodes can behave untruthfully without a timely detection