The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusion

In recent years, video generation has become a prominent generative tool and has drawn significant attention. However, there is little consideration in audio-to-video generation, though audio contains unique qualities like temporal semantics and magnitude. Hence, we propose The Power of Sound (TPoS) model to incorporate audio input that includes both changeable temporal semantics and magnitude. To generate video frames, TPoS utilizes a latent stable diffusion model with textual semantic information, which is then guided by the sequential audio embedding from our pretrained Audio Encoder. As a result, this method produces audio reactive video contents. We demonstrate the effectiveness of TPoS across various tasks and compare its results with current state-of-the-art techniques in the field of audio-to-video generation. More examples are available at https://ku-vai.github.io/TPoS/Comment: ICCV202

Probability-Based Multi-Label Classification Considering Correlation Between Labels— Focusing on DSM-5 Depressive Disorder Diagnostic Criteria

The incidence of depressive disorder in Korea is the highest among OECD countries. The proportion of patients in their 20s is the highest. However, social gaze and false perception are causing problems such as not visiting the hospital or delaying the visit. Accordingly, we suggest a Korean model for predicting depressive disorders using data from online communities widely used by people in their 20s. In many countries, including South Korea, depressive disorders are diagnosed using DSM-5 (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition) published by the American Psychiatric Association. We propose a model that predicts the probability of a user’s speech corresponding to nine criteria for diagnosing DSM-5 depressive disorder, following advice obtained through periodic meetings with a psychiatrist. The prediction performance was improved by using the correlation between each criterion in the model implementation stage

Ordered assemblies of Fe3O4 and a donor-acceptor-type pi-conjugated polymer in nanoparticles for enhanced photoacoustic and magnetic effects

We report that the ordered structure in the assemblies of iron oxide nanoparticles in conjugated polymer nanoparticles is the key to achieve better properties to realize multimodal theranostic agents for magnetic resonance and photoacoustic imaging. Hybrid nanoparticles of a conjugated polymer (PCPDTBT), a phospholipid (D8PE) with a primary amine polar head, and iron oxide (Fe3O4) nanoparticles were prepared by a phase-separated film shattering process by varying the iron oxide concentration while maintaining a fixed mixing ratio of PCPDTBT and D8PE. Notably, the hybrid nanoparticles assembled at a molar mixing ratio of 1:1:0.8 (PCPDTBT/D8PE/Fe3O4) exhibited the shortest transversal relaxation time, T-2, and a photoacoustic signal 22 times higher than that obtained at the 1:1:0 mixing ratio. Structural analysis by X-ray diffraction together with the measurements of energy transfer by transient absorption spectroscopy confirmed that the structural ordering of these hybrid nanoparticles was responsible for their enhanced photoacoustic and magnetic properties

Conjugated polymer nano-ellipsoids assembled with octanoic acid and their polyurethane nanocomposites with simultaneous thermal storage and antibacterial activity

We prepared conjugated polymer nano-ellipsoids (CPNs) via emulsification of chloroform phase using the octanoic acid (OA) in dimethyl formamide (DMF), followed by the removal of chloroform by heating. The resulting CPN DMF solutions were thoroughly mixed with polyurethane (PU) DMF solutions to form composite films upon solvent removal, with uniformly distributed CPNs due to hydrogen bonds between PU matrix and the CPNs. Superior photothermal and antibacterial properties of the PU:CPN composite films were observed, presenting the usefulness of CPNs as an efficient light harvester and thermal storage material, and the OA as an antibacterial material for multifunctional fiber applications. (C) 2018 The Korean Society of Industrial and Engineering Chemistry