Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model

Numerous studies in the field of music generation have demonstrated impressive performance, yet virtually no models are able to directly generate music to match accompanying videos. In this work, we develop a generative music AI framework, Video2Music, that can match a provided video. We first curated a unique collection of music videos. Then, we analysed the music videos to obtain semantic, scene offset, motion, and emotion features. These distinct features are then employed as guiding input to our music generation model. We transcribe the audio files into MIDI and chords, and extract features such as note density and loudness. This results in a rich multimodal dataset, called MuVi-Sync, on which we train a novel Affective Multimodal Transformer (AMT) model to generate music given a video. This model includes a novel mechanism to enforce affective similarity between video and music. Finally, post-processing is performed based on a biGRU-based regression model to estimate note density and loudness based on the video features. This ensures a dynamic rendering of the generated chords with varying rhythm and volume. In a thorough experiment, we show that our proposed framework can generate music that matches the video content in terms of emotion. The musical quality, along with the quality of music-video matching is confirmed in a user study. The proposed AMT model, along with the new MuVi-Sync dataset, presents a promising step for the new task of music generation for videos

OF@TEIN: An OpenFlow-enabled SDN Testbed over International SmartX Rack Sites

In this paper, we will discuss our on-going effort for OF@TEIN SDN(Software-Defined Networking) testbed, which currently spans over Korea and fiveSouth-East Asian (SEA) collaborators with internationally deployed OpenFlowenabledSmartX Racks

Predictive biomarkers for 5-fluorouracil and oxaliplatin-based chemotherapy in gastric cancers via profiling of patient-derived xenografts.

Gastric cancer (GC) is commonly treated by chemotherapy using 5-fluorouracil (5-FU) derivatives and platinum combination, but predictive biomarker remains lacking. We develop patient-derived xenografts (PDXs) from 31 GC patients and treat with a combination of 5-FU and oxaliplatin, to determine biomarkers associated with responsiveness. When the PDXs are defined as either responders or non-responders according to tumor volume change after treatment, the responsiveness of PDXs is significantly consistent with the respective clinical outcomes of the patients. An integrative genomic and transcriptomic analysis of PDXs reveals that pathways associated with cell-to-cell and cell-to-extracellular matrix interactions enriched among the non-responders in both cancer cells and the tumor microenvironment (TME). We develop a 30-gene prediction model to determine the responsiveness to 5-FU and oxaliplatin-based chemotherapy and confirm the significant poor survival outcomes among cases classified as non-responder-like in three independent GC cohorts. Our study may inform clinical decision-making when designing treatment strategies

Anomaly Detection of the Brake Operating Unit on Metro Vehicles Using a One-Class LSTM Autoencoder

Detecting anomalies in the Brake Operating Unit (BOU) braking system of metro trains is very important for trains’ reliability and safety. However, current periodic maintenance and inspection cannot detect anomalies at an early stage. In addition, constructing a stable and accurate anomaly detection system is a very challenging task. Hence, in this work, we propose a method for detecting anomalies of BOU on metro vehicles using a one-class long short-term memory (LSTM) autoencoder. First, we extracted brake cylinder (BC) pressure data from the BOU data since one of the anomaly cases of metro trains is that BC pressure relief time is delayed by 4 s. After that, extracted BC pressure data is split into subsequences which are fed into our proposed one-class LSTM autoencoder which consists of two LSTM blocks (encoder and decoder). The one-class LSTM autoencoder is trained using training data which only consists of normal subsequences. To detect anomalies from test data that contain abnormal subsequences, the mean absolute error (MAE) for each subsequence is calculated. When the error is larger than a predefined threshold which was set to the maximum value of MAE in the training (normal) dataset, we can declare that example an anomaly. We conducted the experiments with the BOU data of metro trains in Korea. Experimental results show that our proposed method can detect anomalies of the BOU data well