321 research outputs found

    Coding Strategies for Genetic Algorithms and Neural Nets

    Get PDF
    The interaction between coding and learning rules in neural nets (NNs), and between coding and genetic operators in genetic algorithms (GAs) is discussed. The underlying principle advocated is that similar things in "the world" should have similar codes. Similarity metrics are suggested for the coding of images and numerical quantities in neural nets, and for the coding of neural network structures in genetic algorithms. A principal component analysis of natural images yields receptive fields resembling horizontal and vertical edge and bar detectors. The orientation sensitivity of the "bar detector" components is found to match a psychophysical model, suggesting that the brain may make some use of principal components in its visual processing. Experiments are reported on the effects of different input and output codings on the accuracy of neural nets handling numeric data. It is found that simple analogue and interpolation codes are most successful. Experiments on the coding of image data demonstrate the sensitivity of final performance to the internal structure of the net. The interaction between the coding of the target problem and reproduction operators of mutation and recombination in GAs are discussed and illustrated. The possibilities for using GAs to adapt aspects of NNs are considered. The permutation problem, which affects attempts to use GAs both to train net weights and adapt net structures, is illustrated and methods to reduce it suggested. Empirical tests using a simulated net design problem to reduce evaluation times indicate that the permutation problem may not be as severe as has been thought, but suggest the utility of a sorting recombination operator, that matches hidden units according to the number of connections they have in common. A number of experiments using GAs to design network structures are reported, both to specify a net to be trained from random weights, and to prune a pre-trained net. Three different coding methods are tried, and various sorting recombination operators evaluated. The results indicate that appropriate sorting can be beneficial, but the effects are problem-dependent. It is shown that the GA tends to overfit the net to the particular set of test criteria, to the possible detriment of wider generalisation ability. A method of testing the ability of a GA to make progress in the presence of noise, by adding a penalty flag, is described

    Burrows–Wheeler compression: Principles and reflections

    Get PDF
    AbstractAfter a general description of the Burrows–Wheeler transform and a brief survey of recent work on processing its output, the paper examines the coding of the zero-runs from the MTF recoding stage, an aspect with little prior treatment. It is concluded that the original scheme proposed by Wheeler is extremely efficient and unlikely to be much improved.The paper then proposes some new interpretations and uses of the Burrows–Wheeler transform, with new insights and approaches to lossless compression, perhaps including techniques from error correction

    Dynamical properties of the Pascal adic transformation

    Full text link
    We study the dynamics of a transformation that acts on infinite paths in the graph associated with Pascal's triangle. For each ergodic invariant measure the asymptotic law of the return time to cylinders is given by a step function. We construct a representation of the system by a subshift on a two-symbol alphabet and then prove that the complexity function of this subshift is asymptotic to a cubic, the frequencies of occurrence of blocks behave in a regular manner, and the subshift is topologically weak mixing

    Turbo codes and turbo algorithms

    Get PDF
    In the first part of this paper, several basic ideas that prompted the coming of turbo codes are commented on. We then present some personal points of view on the main advances obtained in past years on turbo coding and decoding such as the circular trellis termination of recursive systematic convolutional codes and double-binary turbo codes associated with Max-Log-MAP decoding. A novel evaluation method, called genieinitialised iterative processing (GIIP), is introduced to assess the error performance of iterative processing. We show that using GIIP produces a result that can be viewed as a lower bound of the maximum likelihood iterative decoding and detection performance. Finally, two wireless communication systems are presented to illustrate recent applications of the turbo principle, the first one being multiple-input/multiple-output channel iterative detection and the second one multi-carrier modulation with linear precoding

    Bytes Are All You Need: Transformers Operating Directly On File Bytes

    Full text link
    Modern deep learning approaches usually transform inputs into a modality-specific form. For example, the most common deep learning approach to image classification involves decoding image file bytes into an RGB tensor which is passed into a neural network. Instead, we investigate performing classification directly on file bytes, without the need for decoding files at inference time. Using file bytes as model inputs enables the development of models which can operate on multiple input modalities. Our model, \emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of 77.33%77.33\% when training and testing directly on TIFF file bytes using a transformer backbone with configuration similar to DeiT-Ti (72.2%72.2\% accuracy when operating on RGB images). Without modifications or hyperparameter tuning, ByteFormer achieves 95.42%95.42\% classification accuracy when operating on WAV files from the Speech Commands v2 dataset (compared to state-of-the-art accuracy of 98.7%98.7\%). Additionally, we demonstrate that ByteFormer has applications in privacy-preserving inference. ByteFormer is capable of performing inference on particular obfuscated input representations with no loss of accuracy. We also demonstrate ByteFormer's ability to perform inference with a hypothetical privacy-preserving camera which avoids forming full images by consistently masking 90%90\% of pixel channels, while still achieving 71.35%71.35\% accuracy on ImageNet. Our code will be made available at https://github.com/apple/ml-cvnets/tree/main/examples/byteformer

    An efficient projector-based passivity test for descriptor systems

    Get PDF
    An efficient passivity test based on canonical projector techniques is proposed for descriptor systems (DSs) widely encountered in circuit and system modeling. The test features a natural flow that first evaluates the index of a DS, followed by possible decoupling into its proper and improper subsystems. Explicit state-space formulations for respective subsystems are derived to facilitate further processing such as model order reduction and/or passivity enforcement. Efficient projector construction and a fast generalized Hamiltonian test for the proper-part passivity are also elaborated. Numerical examples then confirm the superiority of the proposed method over existing passivity tests for DSs based on linear matrix inequalities or skew-Hamiltonian/Hamiltonian matrix pencils. © 2010 IEEE.published_or_final_versio
    • …
    corecore