Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approach

Modern neural networks are highly overparameterized, with capacity to substantially overfit to training data. Nevertheless, these networks often generalize well in practice. It has also been observed that trained networks can often be "compressed" to much smaller representations. The purpose of this paper is to connect these two empirical observations. Our main technical result is a generalization bound for compressed networks based on the compressed size. Combined with off-the-shelf compression algorithms, the bound leads to state of the art generalization guarantees; in particular, we provide the first non-vacuous generalization guarantees for realistic architectures applied to the ImageNet classification problem. As additional evidence connecting compression and generalization, we show that compressibility of models that tend to overfit is limited: We establish an absolute limit on expected compressibility as a function of expected generalization error, where the expectations are over the random choice of training examples. The bounds are complemented by empirical results that show an increase in overfitting implies an increase in the number of bits required to describe a trained network.Comment: 16 pages, 1 figure. Accepted at ICLR 201

The efficient interleaving of digital-video-broadcasting-satellite 2nd generations system

The DVB-S2 system is designed as a toolbox to permit the execution of the satellite programs. Interleaver is an essential part of the DVB-S2 system. The current general block interleaver in DVB-S2 is not best, which leads to high BER and maybe not satisfy the requirements of the system. The purpose of this paper is to study the several interleaver types and comparative analyses are done between them to find which of these give better performance. Simulations results obtained prove that the 2D interleavers minimize BER more than other interleavers of DVB-S2. Further, the performance of 2D interleaver is better on a system that required a low SNR

DIGITAL VIDEO BROADCASTING VIA SATELLITE (DVB-S)

This paper provides a brief introduction to the DVB-S system based on [EN-300-421]. The DVB-S system provides directto-home (DTH) services for consumer integrated receiver decoders (IRD), as well as collective antenna systems (satellitemaster antenna television SMATV) and cable television head-end stations. The overview covers the physical layer thatcomprises adaptation, framing, coding, interleaving and modulation, and discusses error performance requirements toachieve quality of service (QoS) targets.Keywords: system provides direct-to-homey, satellite master antenna television and achieves quality of service

Deepr: A Convolutional Net for Medical Records

Feature engineering remains a major bottleneck when creating predictive systems from electronic medical records. At present, an important missing element is detecting predictive regular clinical motifs from irregular episodic records. We present Deepr (short for Deep record), a new end-to-end deep learning system that learns to extract features from medical records and predicts future risk automatically. Deepr transforms a record into a sequence of discrete elements separated by coded time gaps and hospital transfers. On top of the sequence is a convolutional neural net that detects and combines predictive local clinical motifs to stratify the risk. Deepr permits transparent inspection and visualization of its inner working. We validate Deepr on hospital data to predict unplanned readmission after discharge. Deepr achieves superior accuracy compared to traditional techniques, detects meaningful clinical motifs, and uncovers the underlying structure of the disease and intervention space

WIMAX LINK PERFORMANCE ANALYSIS FOR WIRELESS AUTOMATION APPLICATIONS

Wireless broadband access technologies are rapidly growing and a corresponding growth in the demand of its applicability transcends faster internet access, high speed file download and different multimedia applications such as voice calls, video streaming, teleconferencing etc, to industrial operations and automation. Industrial and automation systems perform operations that requires the transmission of real time information from one end to another through high-performance wireless broadband communication links. WiMAX, based on IEEE 802.16 standard is one of the wireless broadband access technologies that has overcome location, speed, and access limitations of the traditional Digital Subscriber Line and Wireless Fidelity, and offers high efficient data rates. This thesis presents detailed analysis of operational WiMAX link performance parameters such as throughput, latency, jitter, and packet loss for suitable applicability in wireless automation applications. The theoretical background of components and functionalities of WiMAX physical and MAC layers as well as the network performance features are presented. The equipment deployed for this field experiment are Alvarion BreeZeMAX 3000 fixed WiMAX equipment operating in the 3.5 GHz licensed band with channel bandwidth of 3.5 MHz. The deployed equipment consisting of MBSE and CPE are installed and commissioned prior to field tests. Several measurements are made in three link quality scenarios (sufficient, good and excellent) in the University of Vaasa campus. Observations and results obtained are discussed and analyzed.fi=Opinnäytetyö kokotekstinä PDF-muodossa.|en=Thesis fulltext in PDF format.|sv=Lärdomsprov tillgängligt som fulltext i PDF-format

Information embedding and retrieval in 3D printed objects

Deep learning and convolutional neural networks have become the main tools of computer vision. These techniques are good at using supervised learning to learn complex representations from data. In particular, under limited settings, the image recognition model now performs better than the human baseline. However, computer vision science aims to build machines that can see. It requires the model to be able to extract more valuable information from images and videos than recognition. Generally, it is much more challenging to apply these deep learning models from recognition to other problems in computer vision. This thesis presents end-to-end deep learning architectures for a new computer vision field: watermark retrieval from 3D printed objects. As it is a new area, there is no state-of-the-art on many challenging benchmarks. Hence, we first define the problems and introduce the traditional approach, Local Binary Pattern method, to set our baseline for further study. Our neural networks seem useful but straightfor- ward, which outperform traditional approaches. What is more, these networks have good generalization. However, because our research field is new, the problems we face are not only various unpredictable parameters but also limited and low-quality training data. To address this, we make two observations: (i) we do not need to learn everything from scratch, we know a lot about the image segmentation area, and (ii) we cannot know everything from data, our models should be aware what key features they should learn. This thesis explores these ideas and even explore more. We show how to use end-to-end deep learning models to learn to retrieve watermark bumps and tackle covariates from a few training images data. Secondly, we introduce ideas from synthetic image data and domain randomization to augment training data and understand various covariates that may affect retrieve real-world 3D watermark bumps. We also show how the illumination in synthetic images data to effect and even improve retrieval accuracy for real-world recognization applications