1,189 research outputs found
Growing Efficient Deep Networks by Structured Continuous Sparsification
We develop an approach to training deep networks while dynamically adjusting
their architecture, driven by a principled combination of accuracy and sparsity
objectives. Unlike conventional pruning approaches, our method adopts a gradual
continuous relaxation of discrete network structure optimization and then
samples sparse subnetworks, enabling efficient deep networks to be trained in a
growing and pruning manner. Extensive experiments across CIFAR-10, ImageNet,
PASCAL VOC, and Penn Treebank, with convolutional models for image
classification and semantic segmentation, and recurrent models for language
modeling, show that our training scheme yields efficient networks that are
smaller and more accurate than those produced by competing pruning methods
Phrase table pruning for Statistical Machine Translation
Phrase-Based Statistical Machine Translation systems model the translation process using pairs of corresponding sequences of words extracted from parallel corpora. These biphrases are stored in phrase tables that typically contain several millions such entries, making it di cult to assess their quality without going to the end of the translation process. Our work is based on the examplifying study of phrase tables generated from the Europarl data, from French to English. We give some statistical information about the biphrases contained in the phrase table, evaluate the coverage of previously unseen sentences and analyse the e ects of pruning on the translation
SOS: Selective Objective Switch for Rapid Immunofluorescence Whole Slide Image Classification
The difficulty of processing gigapixel whole slide images (WSIs) in clinical
microscopy has been a long-standing barrier to implementing computer aided
diagnostic systems. Since modern computing resources are unable to perform
computations at this extremely large scale, current state of the art methods
utilize patch-based processing to preserve the resolution of WSIs. However,
these methods are often resource intensive and make significant compromises on
processing time. In this paper, we demonstrate that conventional patch-based
processing is redundant for certain WSI classification tasks where high
resolution is only required in a minority of cases. This reflects what is
observed in clinical practice; where a pathologist may screen slides using a
low power objective and only switch to a high power in cases where they are
uncertain about their findings. To eliminate these redundancies, we propose a
method for the selective use of high resolution processing based on the
confidence of predictions on downscaled WSIs --- we call this the Selective
Objective Switch (SOS). Our method is validated on a novel dataset of 684
Liver-Kidney-Stomach immunofluorescence WSIs routinely used in the
investigation of autoimmune liver disease. By limiting high resolution
processing to cases which cannot be classified confidently at low resolution,
we maintain the accuracy of patch-level analysis whilst reducing the inference
time by a factor of 7.74.Comment: Accepted for publication at CVPR202
Efficient Transformers with Dynamic Token Pooling
Transformers achieve unrivalled performance in modelling language, but remain
inefficient in terms of memory and time complexity. A possible remedy is to
reduce the sequence length in the intermediate layers by pooling fixed-length
segments of tokens. Nevertheless, natural units of meaning, such as words or
phrases, display varying sizes. To address this mismatch, we equip language
models with a dynamic-pooling mechanism, which predicts segment boundaries in
an autoregressive fashion. We compare several methods to infer boundaries,
including end-to-end learning through stochastic re-parameterisation,
supervised learning (based on segmentations from subword tokenizers or spikes
in conditional entropy), as well as linguistically motivated boundaries. We
perform character-level evaluation on texts from multiple datasets and
morphologically diverse languages. The results demonstrate that dynamic
pooling, which jointly segments and models language, is both faster and more
accurate than vanilla Transformers and fixed-length pooling within the same
computational budget
Deep Learning for Free-Hand Sketch: A Survey
Free-hand sketches are highly illustrative, and have been widely used by
humans to depict objects or stories from ancient times to the present. The
recent prevalence of touchscreen devices has made sketch creation a much easier
task than ever and consequently made sketch-oriented applications increasingly
popular. The progress of deep learning has immensely benefited free-hand sketch
research and applications. This paper presents a comprehensive survey of the
deep learning techniques oriented at free-hand sketch data, and the
applications that they enable. The main contents of this survey include: (i) A
discussion of the intrinsic traits and unique challenges of free-hand sketch,
to highlight the essential differences between sketch data and other data
modalities, e.g., natural photos. (ii) A review of the developments of
free-hand sketch research in the deep learning era, by surveying existing
datasets, research topics, and the state-of-the-art methods through a detailed
taxonomy and experimental evaluation. (iii) Promotion of future work via a
discussion of bottlenecks, open problems, and potential research directions for
the community.Comment: This paper is accepted by IEEE TPAM
Building task-oriented machine translation systems
La principal meta de esta tesis es desarrollar sistemas de traduccion interactiva que presenten mayor
sinergia con sus usuarios potenciales. Por ello, el objetivo es hacer los sistemas estado del arte mas
ergonomicos, intuitivos y eficientes, con el fin de que el experto humano se sienta mas comodo al utilizarlos.
Con este fin se presentan diferentes t�ecnicas enfocadas a mejorar la adaptabilidad y el tiempo
de respuesta de los sistemas de traduccion automatica subyacentes, as�ÿ como tambien se presenta una
estrategia cuya finalidad es mejorar la interaccion hombre-m�aquina. Todo ello con el proposito ultimo
de rellenar el hueco existente entre el estado del arte en traduccion automatica y las herramientas que los
traductores humanos tienen a su disposici�on.
En lo que respecta al tiempo de respuesta de los sistemas de traducci�on autom�atica, en esta tesis se
presenta una t�ecnica de poda de los par�ametros de los modelos de traducci�on actuales, cuya intuici�on est�a
basada en el concepto de segmentaci�on biling¤ue, pero que termina por evolucionar hacia una estrategia de
re-estimaci�on de dichos par�ametros. Utilizando esta estrategia se obtienen resultados experimentales que
demuestran que es posible podar la tabla de segmentos hasta en un 97%, sin mermar por ello la calidad
de las traducciones obtenidas. Adem�as, estos resultados son coherentes en diferentes pares de lenguas,
lo cual evidencia que la t�ecnica que se presenta aqu�ÿ es efectiva en un entorno de traducci�on autom�atica
tradicional, y por lo tanto podr�ÿa ser utilizada directamente en un escenario de post-edici�on. Sin embargo,
los experimentos llevados a cabo en traducci�on interactiva son ligeramente menos convincentes, pues
implican la necesidad de llegar a un compromiso entre el tiempo de respuesta y la calidad de los sufijos
producidos.
Por otra parte, se presentan dos t�ecnicas de adaptaci�on, con el prop�osito de mejorar la adaptabilidad
de los sistemas de traducci�on autom�atica. La primeraSanchis Trilles, G. (2012). Building task-oriented machine translation systems [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/17174Palanci
A Formal Model of Ambiguity and its Applications in Machine Translation
Systems that process natural language must cope with and resolve ambiguity. In this dissertation, a model of language processing is advocated in which multiple inputs and multiple analyses of inputs are considered concurrently and a single analysis is only a last resort. Compared to conventional models, this approach can be understood as replacing single-element inputs and outputs with weighted sets of inputs and outputs. Although processing components must deal with sets (rather than individual elements), constraints are imposed on the elements of these sets, and the representations from existing models may be reused. However, to deal efficiently with large (or infinite) sets, compact representations of sets that share structure between elements, such as weighted finite-state transducers and synchronous context-free grammars, are necessary. These representations and algorithms for manipulating them are discussed in depth in depth.
To establish the effectiveness and tractability of the proposed processing model, it is applied to several problems in machine translation. Starting with spoken language translation, it is shown that translating a set of transcription hypotheses yields better translations compared to a baseline in which a single (1-best) transcription hypothesis is selected and then translated, independent of the translation model formalism used. More subtle forms of ambiguity that arise even in text-only translation (such as decisions conventionally made during system development about how to preprocess text) are then discussed, and it is shown that the ambiguity-preserving paradigm can be employed in these cases as well, again leading to improved translation quality. A model for supervised learning that learns from training data where sets (rather than single elements) of correct labels are provided for each training instance and use it to learn a model of compound word segmentation is also introduced, which is used as a preprocessing step in machine translation
- …