321 research outputs found
Coding Strategies for Genetic Algorithms and Neural Nets
The interaction between coding and learning rules in neural nets (NNs), and between coding and genetic operators in genetic algorithms (GAs) is discussed. The underlying principle advocated is that similar things in "the world" should have similar codes. Similarity metrics are suggested for the coding of images and numerical quantities in neural nets, and for the coding of neural network structures in genetic algorithms.
A principal component analysis of natural images yields receptive fields resembling horizontal and vertical edge and bar detectors. The orientation sensitivity of the "bar detector" components is found to match a psychophysical model, suggesting that the brain may make some use of principal components in its visual processing.
Experiments are reported on the effects of different input and output codings on the accuracy of neural nets handling numeric data. It is found that simple analogue and interpolation codes are most successful. Experiments on the coding of image data demonstrate the sensitivity of final performance to the internal structure of the net.
The interaction between the coding of the target problem and reproduction operators of mutation and recombination in GAs are discussed and illustrated. The possibilities for using GAs to adapt aspects of NNs are considered. The permutation problem, which affects attempts to use GAs both to train net weights and adapt net structures, is illustrated and methods to reduce it suggested. Empirical tests using a simulated net design problem to reduce evaluation times indicate that the permutation problem may not be as severe as has been thought, but suggest the utility of a sorting recombination operator, that matches hidden units according to the number of connections they have in common.
A number of experiments using GAs to design network structures are reported, both to specify a net to be trained from random weights, and to prune a pre-trained net. Three different coding methods are tried, and various sorting recombination operators evaluated. The results indicate that appropriate sorting can be beneficial, but the effects are problem-dependent. It is shown that the GA tends to overfit the net to the particular set of test criteria, to the possible detriment of wider generalisation ability. A method of testing the ability of a GA to make progress in the presence of noise, by adding a penalty flag, is described
Burrows–Wheeler compression: Principles and reflections
AbstractAfter a general description of the Burrows–Wheeler transform and a brief survey of recent work on processing its output, the paper examines the coding of the zero-runs from the MTF recoding stage, an aspect with little prior treatment. It is concluded that the original scheme proposed by Wheeler is extremely efficient and unlikely to be much improved.The paper then proposes some new interpretations and uses of the Burrows–Wheeler transform, with new insights and approaches to lossless compression, perhaps including techniques from error correction
Dynamical properties of the Pascal adic transformation
We study the dynamics of a transformation that acts on infinite paths in the
graph associated with Pascal's triangle. For each ergodic invariant measure the
asymptotic law of the return time to cylinders is given by a step function. We
construct a representation of the system by a subshift on a two-symbol alphabet
and then prove that the complexity function of this subshift is asymptotic to a
cubic, the frequencies of occurrence of blocks behave in a regular manner, and
the subshift is topologically weak mixing
Turbo codes and turbo algorithms
In the first part of this paper, several basic ideas that prompted the coming of turbo codes are commented on. We then present some personal points of view on the main advances obtained in past years on turbo coding and decoding such as the circular trellis termination of recursive systematic convolutional codes and double-binary turbo codes associated with Max-Log-MAP decoding. A novel evaluation method, called genieinitialised iterative processing (GIIP), is introduced to assess the error performance of iterative processing. We show that using GIIP produces a result that can be viewed as a lower bound of the maximum likelihood iterative decoding and detection performance. Finally, two wireless communication systems are presented to illustrate recent applications of the turbo principle, the first one being multiple-input/multiple-output channel iterative detection and the second one multi-carrier modulation with linear precoding
Bytes Are All You Need: Transformers Operating Directly On File Bytes
Modern deep learning approaches usually transform inputs into a
modality-specific form. For example, the most common deep learning approach to
image classification involves decoding image file bytes into an RGB tensor
which is passed into a neural network. Instead, we investigate performing
classification directly on file bytes, without the need for decoding files at
inference time. Using file bytes as model inputs enables the development of
models which can operate on multiple input modalities. Our model,
\emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of
when training and testing directly on TIFF file bytes using a
transformer backbone with configuration similar to DeiT-Ti ( accuracy
when operating on RGB images). Without modifications or hyperparameter tuning,
ByteFormer achieves classification accuracy when operating on WAV
files from the Speech Commands v2 dataset (compared to state-of-the-art
accuracy of ). Additionally, we demonstrate that ByteFormer has
applications in privacy-preserving inference. ByteFormer is capable of
performing inference on particular obfuscated input representations with no
loss of accuracy. We also demonstrate ByteFormer's ability to perform inference
with a hypothetical privacy-preserving camera which avoids forming full images
by consistently masking of pixel channels, while still achieving
accuracy on ImageNet. Our code will be made available at
https://github.com/apple/ml-cvnets/tree/main/examples/byteformer
An efficient projector-based passivity test for descriptor systems
An efficient passivity test based on canonical projector techniques is proposed for descriptor systems (DSs) widely encountered in circuit and system modeling. The test features a natural flow that first evaluates the index of a DS, followed by possible decoupling into its proper and improper subsystems. Explicit state-space formulations for respective subsystems are derived to facilitate further processing such as model order reduction and/or passivity enforcement. Efficient projector construction and a fast generalized Hamiltonian test for the proper-part passivity are also elaborated. Numerical examples then confirm the superiority of the proposed method over existing passivity tests for DSs based on linear matrix inequalities or skew-Hamiltonian/Hamiltonian matrix pencils. © 2010 IEEE.published_or_final_versio
- …