Neural2Speech: A Transfer Learning Framework for Neural-Driven Speech Reconstruction

Reconstructing natural speech from neural activity is vital for enabling direct communication via brain-computer interfaces. Previous efforts have explored the conversion of neural recordings into speech using complex deep neural network (DNN) models trained on extensive neural recording data, which is resource-intensive under regular clinical constraints. However, achieving satisfactory performance in reconstructing speech from limited-scale neural recordings has been challenging, mainly due to the complexity of speech representations and the neural data constraints. To overcome these challenges, we propose a novel transfer learning framework for neural-driven speech reconstruction, called Neural2Speech, which consists of two distinct training phases. First, a speech autoencoder is pre-trained on readily available speech corpora to decode speech waveforms from the encoded speech representations. Second, a lightweight adaptor is trained on the small-scale neural recordings to align the neural activity and the speech representation for decoding. Remarkably, our proposed Neural2Speech demonstrates the feasibility of neural-driven speech reconstruction even with only 20 minutes of intracranial data, which significantly outperforms existing baseline methods in terms of speech fidelity and intelligibility.Comment: To appear in 2024 IEEE International Conference on Acoustics, Speech and Signal Processin

Sprachenlernen, Sprachpolitik, Sprache in den Medien und vieles mehr : Der Forschungsverbund „Educational Linguistics“ stellt sich vor

No thumbnai

The symbiotic bacteria Alcaligenes faecalis of the entomopathogenic nematodes Oscheius spp. exhibit potential biocontrol of plant- and entomopathogenic fungi

Soil-dwelling entomopathogenic nematodes (EPNs) kill arthropod hosts by injecting their symbiotic bacteria into the host hemolymph and feed on the bacteria and the tissue of the dying host for several generations cycles until the arthropod cadaver is completely depleted. The EPN-bacteria-arthropod cadaver complex represents a rich energy source for the surrounding opportunistic soil fungal biota and other competitors. We hypothesized that EPNs need to protect their food source until depletion and that the EPN symbiotic bacteria produce volatile and non-volatile exudations that deter different soil fungal groups in the soil. We isolated the symbiotic bacteria species (Alcaligenes faecalis) from the EPN Oscheius spp. and ran infectivity bioassays against entomopathogenic fungi (EPF) as well as against plant pathogenic fungi (PPF). We found that both volatile and non-volatile symbiotic bacterial exudations had negative effects on both EPF and PPF. Such deterrent function on functionally different fungal strains suggests a common mode of action of A.faecalis bacterial exudates, which has the potential to influence the structure of soil microbial communities, and could be integrated into pest management programs for increasing crop protection against fungal pathogens

Distinct and predictive chromatin signatures of transcriptional promoters and enhancers in the human genome.

Eukaryotic gene transcription is accompanied by acetylation and methylation of nucleosomes near promoters, but the locations and roles of histone modifications elsewhere in the genome remain unclear. We determined the chromatin modification states in high resolution along 30 Mb of the human genome and found that active promoters are marked by trimethylation of Lys4 of histone H3 (H3K4), whereas enhancers are marked by monomethylation, but not trimethylation, of H3K4. We developed computational algorithms using these distinct chromatin signatures to identify new regulatory elements, predicting over 200 promoters and 400 enhancers within the 30-Mb region. This approach accurately predicted the location and function of independently identified regulatory elements with high sensitivity and specificity and uncovered a novel functional enhancer for the carnitine transporter SLC22A5 (OCTN2). Our results give insight into the connections between chromatin modifications and transcriptional regulatory activity and provide a new tool for the functional annotation of the human genome. Activation of eukaryotic gene transcription involves the coordination of a multitude of transcription factors and cofactors on regulatory DNA sequences such as promoters and enhancers and on the chromatin structure containing these elements 1-3 . Promoters are located at the 5¢ ends of genes immediately surrounding the transcriptional start site (TSS) and serve as the point of assembly of the transcriptional machinery and initiation of transcription 4 . Enhancers contribute to the activation of their target genes from positions upstream, downstream or within a target or neighboring gene Recent investigations using chromatin immunoprecipitation (ChIP) and microarray (ChIP-chip) experiments have described the chromatin architecture of transcriptional promoters in yeast, fly and mammalian systems 9 . In a manner largely conserved across species, active promoters are marked by acetylation of various residues of histones H3 and H4 and methylation of H3K4, particularly trimethylation of this residue. Nucleosome depletion is also a general characteristic of active promoters in yeast and flies, although this feature remains to be thoroughly examined in mammalian systems. Although some studies suggest that distal regulatory elements like enhancers may be marked by similar histone modification patterns 10-13 , the distinguishing chromatin features of promoters and enhancers have yet to be determined, hindering our understanding of a predictive histone code for different classes of regulatory elements. Here, we present high-resolution maps of multiple histone modifications and transcriptional regulators in 30 Mb of the human genome, demonstrating that active promoters and enhancers are associated with distinct chromatin signatures that can be used to predict these regulatory elements in the human genome. RESULTS Chromatin architecture and transcription factor localization We performed ChIP-chip analysis 14 to determine the chromatin architecture along 44 human loci selected by the ENCODE consortium as common targets for genomic analysis 15 , totaling 30 Mb