49 research outputs found
One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verification
The application of speech self-supervised learning (SSL) models has achieved
remarkable performance in speaker verification (SV). However, there is a
computational cost hurdle in employing them, which makes development and
deployment difficult. Several studies have simply compressed SSL models through
knowledge distillation (KD) without considering the target task. Consequently,
these methods could not extract SV-tailored features. This paper suggests
One-Step Knowledge Distillation and Fine-Tuning (OS-KDFT), which incorporates
KD and fine-tuning (FT). We optimize a student model for SV during KD training
to avert the distillation of inappropriate information for the SV. OS-KDFT
could downsize Wav2Vec 2.0 based ECAPA-TDNN size by approximately 76.2%, and
reduce the SSL model's inference time by 79% while presenting an EER of 0.98%.
The proposed OS-KDFT is validated across VoxCeleb1 and VoxCeleb2 datasets and
W2V2 and HuBERT SSL models. Experiments are available on our GitHub
In situ electrochemical surface modification for high-voltage LiCoO2 in lithium ion batteries
High-voltage LiCoO2 has been revisited to improve the energy density of lithium ion batteries. LiCoO2 can deliver the reversible capacity of about 200 mA h g(-1) when the upper cut-off voltage increases to 4.55 V (vs. Li/Li+). However, the high upper cut-off voltage causes the severe failures of LiCoO2 such as structural degradation, electrolyte decomposition, and Co dissolution. Various surface-modified LiCoO2 materials have been introduced to suppress electrolyte decomposition and Co dissolution, thereby leading to the improved electrochemical performance. Most of the coated LiCoO2 materials are obtained through a conventional coating process such as sol-gel synthesis, which is complex and high-cost. In this paper, the in situ electrochemical coating method is introduced as a simple and low-cost coating process, where the electrolyte additive of Mg salts is electrochemically decomposed to form a MgF2-based coating layer on the LiCoO2 surface. LiCoO2 electrochemically coated with MgF2 suppresses Co dissolution in electrolytes, resulting in excellent electrochemical performance such as high reversible capacity of 198 mA h g(-1) and stable cycle performance over 100 cycles in the voltage range between 3 and 4.55 V (vs. Li/Li+) at 45 degrees C. The formation mechanism of MgF2 is also demonstrated through ex situ XPS and XANES analyses.
Integrated Parameter-Efficient Tuning for General-Purpose Audio Models
The advent of hyper-scale and general-purpose pre-trained models is shifting
the paradigm of building task-specific models for target tasks. In the field of
audio research, task-agnostic pre-trained models with high transferability and
adaptability have achieved state-of-the-art performances through fine-tuning
for downstream tasks. Nevertheless, re-training all the parameters of these
massive models entails an enormous amount of time and cost, along with a huge
carbon footprint. To overcome these limitations, the present study explores and
applies efficient transfer learning methods in the audio domain. We also
propose an integrated parameter-efficient tuning (IPET) framework by
aggregating the embedding prompt (a prompt-based learning approach), and the
adapter (an effective transfer learning method). We demonstrate the efficacy of
the proposed framework using two backbone pre-trained audio models with
different characteristics: the audio spectrogram transformer and wav2vec 2.0.
The proposed IPET framework exhibits remarkable performance compared to
fine-tuning method with fewer trainable parameters in four downstream tasks:
sound event classification, music genre classification, keyword spotting, and
speaker verification. Furthermore, the authors identify and analyze the
shortcomings of the IPET framework, providing lessons and research directions
for parameter efficient tuning in the audio domain.Comment: 5 pages, 3 figures, submit to ICASSP202
Convolution channel separation and frequency sub-bands aggregation for music genre classification
In music, short-term features such as pitch and tempo constitute long-term
semantic features such as melody and narrative. A music genre classification
(MGC) system should be able to analyze these features. In this research, we
propose a novel framework that can extract and aggregate both short- and
long-term features hierarchically. Our framework is based on ECAPA-TDNN, where
all the layers that extract short-term features are affected by the layers that
extract long-term features because of the back-propagation training. To prevent
the distortion of short-term features, we devised the convolution channel
separation technique that separates short-term features from long-term feature
extraction paths. To extract more diverse features from our framework, we
incorporated the frequency sub-bands aggregation method, which divides the
input spectrogram along frequency bandwidths and processes each segment. We
evaluated our framework using the Melon Playlist dataset which is a large-scale
dataset containing 600 times more data than GTZAN which is a widely used
dataset in MGC studies. As the result, our framework achieved 70.4% accuracy,
which was improved by 16.9% compared to a conventional framework
Localised degradation within sulfide-based all-solid-state electrodes visualised by Raman mapping
The distribution of degradation products, before and after cycling, within common sulfide-based solid electrolytes (β-LiPS, LiPSCl and LiGePS) was mapped using Raman microscopy. All composite electrodes displayed the appearance of side reaction products after the initial charge-discharge cycle, located at the site of a LiNiMnCoO particle
PAS: Partial Additive Speech Data Augmentation Method for Noise Robust Speaker Verification
Background noise reduces speech intelligibility and quality, making speaker
verification (SV) in noisy environments a challenging task. To improve the
noise robustness of SV systems, additive noise data augmentation method has
been commonly used. In this paper, we propose a new additive noise method,
partial additive speech (PAS), which aims to train SV systems to be less
affected by noisy environments. The experimental results demonstrate that PAS
outperforms traditional additive noise in terms of equal error rates (EER),
with relative improvements of 4.64% and 5.01% observed in SE-ResNet34 and
ECAPA-TDNN. We also show the effectiveness of proposed method by analyzing
attention modules and visualizing speaker embeddings.Comment: 5 pages, 2 figures, 1 table, accepted to CKAIA2023 as a conference
pape
Write-rationing garbage collection for hybrid memories
Emerging Non-Volatile Memory (NVM) technologies offer high capacity and energy efficiency compared to DRAM, but suffer from limited write endurance and longer latencies. Prior work seeks the best of both technologies by combining DRAM and NVM in hybrid memories to attain low latency, high capacity, energy efficiency, and durability. Coarse-grained hardware and OS optimizations then spread writes out (wear-leveling) and place highly mutated pages in DRAM to extend NVM lifetimes. Unfortunately even with these coarse-grained methods, popular Java applications exact impractical NVM lifetimes of 4 years or less.
This paper shows how to make hybrid memories practical, without changing the programming model, by enhancing garbage collection in managed language runtimes. We find object write behaviors offer two opportunities: (1) 70% of writes occur to newly allocated objects, and (2) 2% of objects capture 81% of writes to mature objects. We introduce writerationing garbage collectors that exploit these fine-grained behaviors. They extend NVM lifetimes by placing highly mutated objects in DRAM and read-mostly objects in NVM. We implement two such systems. (1) Kingsguard-nursery places new allocation in DRAM and survivors in NVM, reducing NVM writes by 5x versus NVM only with wear-leveling. (2) Kingsguard-writers (KG-W) places nursery objects in DRAM and survivors in a DRAM observer space. It monitors all mature object writes and moves unwritten mature objects from DRAM to NVM. Because most mature objects are unwritten, KG-W exploits NVM capacity while increasing NVM lifetimes by 11x. It reduces the energy-delay product by 32% over DRAM-only and 29% over NVM-only. This work opens up new avenues for making hybrid memories practical
Superionic lithium transport via multiple coordination environments defined by two-anion packing
Fast cation transport in solids underpins energy storage. Materials design has focused on structures that can define transport pathways with minimal cation coordination change, restricting attention to a small part of chemical space. Motivated by the greater structural diversity of binary intermetallics than that of the metallic elements, we used two anions to build a pathway for three-dimensional superionic lithium ion conductivity that exploits multiple cation coordination environments. Li
7
Si
2
S
7
I is a pure lithium ion conductor created by an ordering of sulphide and iodide that combines elements of hexagonal and cubic close-packing analogously to the structure of NiZr. The resulting diverse network of lithium positions with distinct geometries and anion coordination chemistries affords low barriers to transport, opening a large structural space for high cation conductivity.
</jats:p