Off-target at-scale Scale Down Model verification of a marketed biopharmaceutical

According to authority guidelines, process understanding is considered the key factor for a successful control strategy in submissions for biopharmaceutical production processes. This knowledge is usually gained in scale down model (SDM) experiments and therefore is subject to the ability of the SDM to reliably predict behavior of cell culture processes in manufacturing scale. In order to prove predictability, SDM runs at target conditions are typically compared to manufacturing scale target runs only. At the same time, predictability is also assumed within the whole design space described or even beyond as evaluated in process design studies. The proof at off-target conditions, however, is rarely provided. In this presentation, the author will exhibit an example where a well-established scale down model was used to determine off-target conditions to manufacture clinical material of a marketed product in 10,000 L scale with a deliberately deviant from target but nevertheless well-defined product quality profile. The desired quality attribute profile was achieved in two manufacturing runs with remarkable accuracy providing additional evidence of the SDM predictability. Hence, confidence is increased in the capability of the resultant control system which has been submitted to authorities for a post licensure manufacturing change

Don't Forget to Fight! : Singapore's History Education and War Commemoration, 1945-2005

Ph.DDOCTOR OF PHILOSOPH

ESPnet-ONNX: Bridging a Gap Between Research and Production

In the field of deep learning, researchers often focus on inventing novel neural network models and improving benchmarks. In contrast, application developers are interested in making models suitable for actual products, which involves optimizing a model for faster inference and adapting a model to various platforms (e.g., C++ and Python). In this work, to fill the gap between the two, we establish an effective procedure for optimizing a PyTorch-based research-oriented model for deployment, taking ESPnet, a widely used toolkit for speech processing, as an instance. We introduce different techniques to ESPnet, including converting a model into an ONNX format, fusing nodes in a graph, and quantizing parameters, which lead to approximately 1.3-2$\times$ speedup in various tasks (i.e., ASR, TTS, speech translation, and spoken language understanding) while keeping its performance without any additional training. Our ESPnet-ONNX will be publicly available at https://github.com/espnet/espnet_onnxComment: Accepted to APSIPA ASC 202

InterMPL: Momentum Pseudo-Labeling with Intermediate CTC Loss

This paper presents InterMPL, a semi-supervised learning method of end-to-end automatic speech recognition (ASR) that performs pseudo-labeling (PL) with intermediate supervision. Momentum PL (MPL) trains a connectionist temporal classification (CTC)-based model on unlabeled data by continuously generating pseudo-labels on the fly and improving their quality. In contrast to autoregressive formulations, such as the attention-based encoder-decoder and transducer, CTC is well suited for MPL, or PL-based semi-supervised ASR in general, owing to its simple/fast inference algorithm and robustness against generating collapsed labels. However, CTC generally yields inferior performance than the autoregressive models due to the conditional independence assumption, thereby limiting the performance of MPL. We propose to enhance MPL by introducing intermediate loss, inspired by the recent advances in CTC-based modeling. Specifically, we focus on self-conditional and hierarchical conditional CTC, that apply auxiliary CTC losses to intermediate layers such that the conditional independence assumption is explicitly relaxed. We also explore how pseudo-labels should be generated and used as supervision for intermediate losses. Experimental results in different semi-supervised settings demonstrate that the proposed approach outperforms MPL and improves an ASR model by up to a 12.1% absolute performance gain. In addition, our detailed analysis validates the importance of the intermediate loss.Comment: Submitted to ICASSP202

BECTRA: Transducer-based End-to-End ASR with BERT-Enhanced Encoder

We present BERT-CTC-Transducer (BECTRA), a novel end-to-end automatic speech recognition (E2E-ASR) model formulated by the transducer with a BERT-enhanced encoder. Integrating a large-scale pre-trained language model (LM) into E2E-ASR has been actively studied, aiming to utilize versatile linguistic knowledge for generating accurate text. One crucial factor that makes this integration challenging lies in the vocabulary mismatch; the vocabulary constructed for a pre-trained LM is generally too large for E2E-ASR training and is likely to have a mismatch against a target ASR domain. To overcome such an issue, we propose BECTRA, an extended version of our previous BERT-CTC, that realizes BERT-based E2E-ASR using a vocabulary of interest. BECTRA is a transducer-based model, which adopts BERT-CTC for its encoder and trains an ASR-specific decoder using a vocabulary suitable for a target task. With the combination of the transducer and BERT-CTC, we also propose a novel inference algorithm for taking advantage of both autoregressive and non-autoregressive decoding. Experimental results on several ASR tasks, varying in amounts of data, speaking styles, and languages, demonstrate that BECTRA outperforms BERT-CTC by effectively dealing with the vocabulary mismatch while exploiting BERT knowledge.Comment: Submitted to ICASSP202

Influence of Active Fingertip Contact with a Stable Surface on Postural Sway and Electromyographic Activities of the Lower Extremity Muscles Immediately after Descending a Step

Recent studies have shown that active contact cues from a fingertip provide information that leads to reduced postural sway during static standing. Although the falling risk increases immediately after descending a step, little is known about the influence of active fingertip contact with a stable surface. The purpose of this study was to investigate the influence of fingertip contact on the sway and the EMG (Electromyographic) activities of lower extremity muscles immediately after descending a step. Nine healthy male volunteers participated in this study. Sway was measured by the center of pressure (COP) and compared under three conditions: (1) standing without touching, (2) standing with the right index fingertip lightly touching (<1N), and (3) standing forcefully touching (5~10N). EMG activities were measured associated with sway and compared. More areas of COP were observed while standing without touching than while standing with light or forceful touching, but there was no significant difference between the two touching conditions. No significant differences in the length of COP and the EMG activities were observed among the three conditions. The results suggested that the fingertip touch contact with a stable surface decreased sway immediately after descending a step by finely controlling the lower extremity muscles