54 research outputs found

    M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval

    Full text link
    This work investigates the use of large-scale, pre-trained models (CLIP and HuBERT) for multilingual speech-image retrieval. For non-English speech-image retrieval, we outperform the current state-of-the-art performance by a wide margin when training separate models for each language, and show that a single model which processes speech in all three languages still achieves retrieval scores comparable with the prior state-of-the-art. We identify key differences in model behavior and performance between English and non-English settings, presumably attributable to the English-only pre-training of CLIP and HuBERT. Finally, we show that our models can be used for mono- and cross-lingual speech-text retrieval and cross-lingual speech-speech retrieval, despite never having seen any parallel speech-text or speech-speech data during training.Comment: Submitted to ICASSP 202

    Algorithm XXX: SHEPPACK: Modiļ¬ed Shepard Algorithm for Interpolation of Scattered Multivariate Data

    Get PDF
    Scattered data interpolation problems arise in many applications. Shepardā€™s method for constructing a global interpolant by blending local interpolants using local-support weight functions usually creates reasonable approximations. SHEPPACK is a Fortran 95 package containing ļ¬ve versions of the modified Shepard algorithm: quadratic (Fortran 95 translations of Algorithms 660, 661, and 798), cubic (Fortran 95 translation of Algorithm 791), and linear variations of the original Shepard algorithm. An option to the linear Shepard code is a statistically robust ļ¬t, intended to be used when the data is known to contain outliers. SHEPPACK also includes a hybrid robust piecewise linear estimation algorithm RIPPLE (residual initiated polynomial-time piecewise linear estimation) intended for data from piecewise linear functions in arbitrary dimension m. The main goal of SHEPPACK is to provide users with a single consistent package containing most existing polynomial variations of Shepardā€™s algorithm. The algorithms target data of different dimensions. The linear Shepard algorithm, robust linear Shepard algorithm, and RIPPLE are the only algorithms in the package that are applicable to arbitrary dimensional data

    AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models

    Full text link
    Audio-visual representation learning aims to develop systems with human-like perception by utilizing correlation between auditory and visual information. However, current models often focus on a limited set of tasks, and generalization abilities of learned representations are unclear. To this end, we propose the AV-SUPERB benchmark that enables general-purpose evaluation of unimodal audio/visual and bimodal fusion representations on 7 datasets covering 5 audio-visual tasks in speech and audio processing. We evaluate 5 recent self-supervised models and show that none of these models generalize to all tasks, emphasizing the need for future study on improving universal model performance. In addition, we show that representations may be improved with intermediate-task fine-tuning and audio event classification with AudioSet serves as a strong intermediate task. We release our benchmark with evaluation code and a model submission platform to encourage further research in audio-visual learning.Comment: Submitted to ICASSP 2024; Evaluation Code: https://github.com/roger-tseng/av-superb Submission Platform: https://av.superbbenchmark.or

    Comparative functional analysis of aquaporins/glyceroporins in mammals and anurans

    Get PDF
    Maintenance of fluid homeostasis is critical to establishing and maintaining normal physiology. The landmark discovery of membrane water channels (aquaporins; AQPs) ushered in a new area in osmoregulatory biology that has drawn from and contributed to diverse branches of biology, from molecular biology and genomics to systems biology and evolution, and from microbial and plant biology to animal and translational physiology. As a result, the study of AQPs provides a unique and integrated backdrop for exploring the relationships between genes and genome systems, the regulation of gene expression, and the physiologic consequences of genetic variation. The wide species distribution of AQP family members and the evolutionary conservation of the family indicate that the control of membrane water flux is a critical biological process. AQP function and regulation is proving to be central to many of the pathways involved in individual physiologic systems in both mammals and anurans. In mammals, AQPs are essential to normal secretory and absorptive functions of the eye, lung, salivary gland, sweat glands, gastrointestinal tract, and kidney. In urinary, respiratory, and gastrointestinal systems, AQPs are required for proper urine concentration, fluid reabsorption, and glandular secretions. In anurans, AQPs are important in mediating physiologic responses to changes in the external environment, including those that occur during metamorphosis and adaptation from an aquatic to terrestrial environment and thermal acclimation in anticipation of freezing. Therefore, an understanding of AQP function and regulation is an important aspect of an integrated approach to basic biological research

    Culture of selected organisms in recirculating and flow-through systems using thermal effluent

    No full text
    Due to the character of the original source materials and the nature of batch digitization, quality control issues may be present in this document. Please report any quality issues you encounter to [email protected], referencing the URI of the item.Bibliography: leaves 75-79.Not availabl

    The Diffusion of e-Services in Danish Municipalities

    No full text
    • ā€¦
    corecore