Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference

A popular approach to streaming speech translation is to employ a single offline model with a \textit{wait-$k$} policy to support different latency requirements, which is simpler than training multiple online models with different latency constraints. However, there is a mismatch problem in using a model trained with complete utterances for streaming inference with partial input. We demonstrate that speech representations extracted at the end of a streaming input are significantly different from those extracted from a complete utterance. To address this issue, we propose a new approach called Future-Aware Streaming Translation (FAST) that adapts an offline ST model for streaming input. FAST includes a Future-Aware Inference (FAI) strategy that incorporates future context through a trainable masked embedding, and a Future-Aware Distillation (FAD) framework that transfers future context from an approximation of full speech to streaming input. Our experiments on the MuST-C EnDe, EnEs, and EnFr benchmarks show that FAST achieves better trade-offs between translation quality and latency than strong baselines. Extensive analyses suggest that our methods effectively alleviate the aforementioned mismatch problem between offline training and online inference.Comment: work in progres

BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing

The emergence of large language models (LLMs) has sparked significant interest in extending their remarkable language capabilities to speech. However, modality alignment between speech and text still remains an open problem. Current solutions can be categorized into two strategies. One is a cascaded approach where outputs (tokens or states) of a separately trained speech recognition system are used as inputs for LLMs, which limits their potential in modeling alignment between speech and text. The other is an end-to-end approach that relies on speech instruction data, which is very difficult to collect in large quantities. In this paper, we address these issues and propose the BLSP approach that Bootstraps Language-Speech Pre-training via behavior alignment of continuation writing. We achieve this by learning a lightweight modality adapter between a frozen speech encoder and an LLM, ensuring that the LLM exhibits the same generation behavior regardless of the modality of input: a speech segment or its transcript. The training process can be divided into two steps. The first step prompts an LLM to generate texts with speech transcripts as prefixes, obtaining text continuations. In the second step, these continuations are used as supervised signals to train the modality adapter in an end-to-end manner. We demonstrate that this straightforward process can extend the capabilities of LLMs to speech, enabling speech recognition, speech translation, spoken language understanding, and speech conversation, even in zero-shot cross-lingual scenarios

Repair of critical sized cranial defects with BMP9-transduced calvarial cells delivered in a thermoresponsive scaffold

<div><p>Large skeletal defects caused by trauma, congenital malformations, and post-oncologic resections of the calvarium present major challenges to the reconstructive surgeon. We previously identified BMP-9 as the most osteogenic BMP in vitro and in vivo. Here we sought to investigate the bone regenerative capacity of murine-derived calvarial mesenchymal progenitor cells (iCALs) transduced by BMP-9 in the context of healing critical-sized calvarial defects. To accomplish this, the transduced cells were delivered to the defect site within a thermoresponsive biodegradable scaffold consisting of poly(polyethylene glycol citrate-co-N-isopropylacrylamide mixed with gelatin (PPCN-g). A total of three treatment arms were evaluated: PPCN-g alone, PPCN-g seeded with iCALs expressing GFP, and PPCN-g seeded with iCALs expressing BMP-9. Defects treated only with PPCN-g scaffold did not statistically change in size when evaluated at eight weeks postoperatively (p = 0.72). Conversely, both animal groups treated with iCALs showed significant reductions in defect size after 12 weeks of follow-up (BMP9-treated: p = 0.0025; GFP-treated: p = 0.0042). However, H&E and trichrome staining revealed more complete osseointegration and mature bone formation only in the BMP9-treated group. These results suggest that BMP9-transduced iCALs seeded in a PPCN-g thermoresponsive scaffold is capable of inducing bone formation in vivo and is an effective means of creating tissue engineered bone for critical sized defects.</p></div

Assessment of the Trichrome histology.

<p>Histologic analysis of tissue microsections harvested 12 weeks post-treatment. (A, B) Chondroid matrix and a smaller proportion of mature bone is shown in the defect site of a PPCN + iCAL + AdGFP-treated mouse. (C, D) A visibly higher proportion of mature bone can be appreciated in samples taken from the defect sites treated with PPCN + iCAL + AdBMP9.</p

Schematic representation of the overall experimental design.

<p>(A) Murine calvarial cells (CALs) were immortalized via retrovirally introduced SV40 large T antigen to produce iCALs. iCALs were then transduced with BMP9 via adenoviral vector and mixed with PPCN scaffolding material. (B) qPCR analysis demonstrating relative expression of BMP9 in iCALs infected with ad-BMP9 (blue bars) compared to control (ad-GFP)-infected cells (red bars). Gene transcript expression was normalized against GAPDH expression. (C) The mixture was subsequently tested in our murine craniofacial defect model. Four millimeter diameter full-thickness calvarial defects were created in the left parietal bone of 8-week-old male athymic (nu/nu) mice. The newly created empty defect reveals the underlying dura mater. PPCN alone or PPCN and adenovirally transduced immortalized calvarial cells were used to fill the defect site.</p

Time-course microCT imaging of the calvarial defects.

<p>At 24–48 hours postoperatively, baseline microCT imaging was performed and analyzed to determine defect volume. Follow-up imaging and analysis was performed at 2, 4, 6, 8, and 12 weeks postoperatively to quantify residual defect volume and new bone ossification. Representative images are shown.</p

Histologic evaluation of the BMP9-induced calvarial defect repair.

<p>Histologic analysis of tissue microsections harvested 12 weeks post-treatment. Yellow arrows indicate the original defect borders. (A) The defect site of this AdGFP-treated mouse shows incomplete healing; there is some ingrowth of bone, but fibrous tissue fills part of the defect. (B) Conversely, the defect site of an AdBMP9-treated mouse has been completely bridged with new bone. All sections show no trace of PPCN material, indicating complete resorption.</p

Quantitative analysis of the BMP9-induced calvarial defect repair.

<p>(A, B, C) Average defect volumes (mm³) were calculated at 24–48 hours, 2, 4, 6, 8, and 12 weeks postoperatively using volumetric reconstructions generated in Amira®. Asterisks indicate a significant (p < 0.05) difference in average defect volume between test groups at the specified time point, as determined by one-way ANOVA. (D, E, F) The change in defect volume over time was used to deduce the percentage of baseline defect volume filled with new bone.</p

NEL-Like Molecule-1 (Nell1) Is Regulated by Bone Morphogenetic Protein 9 (BMP9) and Potentiates BMP9-Induced Osteogenic Differentiation at the Expense of Adipogenesis in Mesenchymal Stem Cells

Background: BMP9 induces both osteogenic and adipogenic differentiation of mesenchymal stem cells (MSCs). Nell1 is a secretory glycoprotein with osteoinductive and anti-adipogenic activities. We investigated the role of Nell1 in BMP9-induced osteogenesis and adipogenesis in MSCs. Methods: Previously characterized MSCs iMEFs were used. Overexpression of BMP9 and NELL1 or silencing of mouse Nell1 was mediated by adenoviral vectors. Early and late osteogenic and adipogenic markers were assessed by staining techniques and qPCR analysis. In vivo activity was assessed in an ectopic bone formation model of athymic mice. Results: We demonstrate that Nell1 expression was up-regulated by BMP9. Exogenous Nell1 potentiated BMP9-induced late stage osteogenic differentiation while inhibiting the early osteogenic marker. Forced Nell1 expression enhanced BMP9-induced osteogenic regulators/markers and inhibited BMP9-upregulated expression of adipogenic regulators/markers in MSCs. In vivo ectopic bone formation assay showed that exogenous Nell1 expression enhanced mineralization and maturity of BMP9-induced bone formation, while inhibiting BMP9-induced adipogenesis. Conversely, silencing Nell1 expression in BMP9-stimulated MSCs led to forming immature chondroid-like matrix. Conclusion: Our findings indicate that Nell1 can be up-regulated by BMP9, which in turn accelerates and augments BMP9-induced osteogenesis. Exogenous Nell1 may be exploited to enhance BMP9-induced bone formation while overcoming BMP9-induced adipogenesis in regenerative medicine

NEL-Like Molecule-1 (Nell1) Is Regulated by Bone Morphogenetic Protein 9 (BMP9) and Potentiates BMP9-Induced Osteogenic Differentiation at the Expense of Adipogenesis in Mesenchymal Stem Cells

Background: BMP9 induces both osteogenic and adipogenic differentiation of mesenchymal stem cells (MSCs). Nell1 is a secretory glycoprotein with osteoinductive and anti-adipogenic activities. We investigated the role of Nell1 in BMP9-induced osteogenesis and adipogenesis in MSCs. Methods: Previously characterized MSCs iMEFs were used. Overexpression of BMP9 and NELL1 or silencing of mouse Nell1 was mediated by adenoviral vectors. Early and late osteogenic and adipogenic markers were assessed by staining techniques and qPCR analysis. In vivo activity was assessed in an ectopic bone formation model of athymic mice. Results: We demonstrate that Nell1 expression was up-regulated by BMP9. Exogenous Nell1 potentiated BMP9-induced late stage osteogenic differentiation while inhibiting the early osteogenic marker. Forced Nell1 expression enhanced BMP9-induced osteogenic regulators/markers and inhibited BMP9-upregulated expression of adipogenic regulators/markers in MSCs. In vivo ectopic bone formation assay showed that exogenous Nell1 expression enhanced mineralization and maturity of BMP9-induced bone formation, while inhibiting BMP9-induced adipogenesis. Conversely, silencing Nell1 expression in BMP9-stimulated MSCs led to forming immature chondroid-like matrix. Conclusion: Our findings indicate that Nell1 can be up-regulated by BMP9, which in turn accelerates and augments BMP9-induced osteogenesis. Exogenous Nell1 may be exploited to enhance BMP9-induced bone formation while overcoming BMP9-induced adipogenesis in regenerative medicine