Zero-Shot Video Moment Retrieval from Frozen Vision-Language Models

Accurate video moment retrieval (VMR) requires universal visual-textual correlations that can handle unknown vocabulary and unseen scenes. However, the learned correlations are likely either biased when derived from a limited amount of moment-text data which is hard to scale up because of the prohibitive annotation cost (fully-supervised), or unreliable when only the video-text pairwise relationships are available without fine-grained temporal annotations (weakly-supervised). Recently, the vision-language models (VLM) demonstrate a new transfer learning paradigm to benefit different vision tasks through the universal visual-textual correlations derived from large-scale vision-language pairwise web data, which has also shown benefits to VMR by fine-tuning in the target domains. In this work, we propose a zero-shot method for adapting generalisable visual-textual priors from arbitrary VLM to facilitate moment-text alignment, without the need for accessing the VMR data. To this end, we devise a conditional feature refinement module to generate boundary-aware visual features conditioned on text queries to enable better moment boundary understanding. Additionally, we design a bottom-up proposal generation strategy that mitigates the impact of domain discrepancies and breaks down complex-query retrieval tasks into individual action retrievals, thereby maximizing the benefits of VLM. Extensive experiments conducted on three VMR benchmark datasets demonstrate the notable performance advantages of our zero-shot algorithm, especially in the novel-word and novel-location out-of-distribution setups.Comment: Accepted by WACV 202

Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training

The correlation between the vision and text is essential for video moment retrieval (VMR), however, existing methods heavily rely on separate pre-training feature extractors for visual and textual understanding. Without sufficient temporal boundary annotations, it is non-trivial to learn universal video-text alignments. In this work, we explore multi-modal correlations derived from large-scale image-text data to facilitate generalisable VMR. To address the limitations of image-text pre-training models on capturing the video changes, we propose a generic method, referred to as Visual-Dynamic Injection (VDI), to empower the model's understanding of video moments. Whilst existing VMR methods are focusing on building temporal-aware video features, being aware of the text descriptions about the temporal changes is also critical but originally overlooked in pre-training by matching static images with sentences. Therefore, we extract visual context and spatial dynamic information from video frames and explicitly enforce their alignments with the phrases describing video changes (e.g. verb). By doing so, the potentially relevant visual and motion patterns in videos are encoded in the corresponding text embeddings (injected) so to enable more accurate video-text alignments. We conduct extensive experiments on two VMR benchmark datasets (Charades-STA and ActivityNet-Captions) and achieve state-of-the-art performances. Especially, VDI yields notable advantages when being tested on the out-of-distribution splits where the testing samples involve novel scenes and vocabulary.Comment: CVPR202

Inhibition of autophagy by 3-MA enhances IL-24-induced apoptosis in human oral squamous cell carcinoma cells

Abstract Background Interleukin-24(IL-24), also referred to as melanoma differentiation-associated gene-7(mda-7), is a unique member of the IL-10 gene family, which displays nearly ubiquitous cancer-specific toxicity. The most notable feature of IL-24 is selectively induced growth suppression and apoptosis in various cancer cells, with no harmful effects toward normal cells. Autophagy is a self-protective mechanism in many kinds of tumor cells that respond to anticancer treatment. It is reported that autophagy inhibition could enhance the effects of many kinds of anticancer treatments, including gene therapy. However, whether IL-24 is effective to treat oral squamous cell carcinomas (OSCC) and if autophagy inhibition could improve the anticancer effect of IL-24 towards OSCC is has not been detected. Methods MTT assays were carried out to determine the cell proliferation; Transfection was used to gene transfer; Western Blot was performed to detect the protein level of LC3II, P62, Beclin 1, Cleaved caspase-3, β-Tubulin and β-actin; Apoptosis rates and cell cycle alteration were analyzed using flow cytometry; Autophagy induction was confirmed by MDC staining, GFP-LC3 staining and transmission electron microscopy. Amount of IL-24 in the culture medium was quantified by ELISA. Apoptosis in vivo was analyzed by TUNEL assay. HE staining was used to observe the morphology of the samples. Results In the present study, we proved that IL-24 have a novel anticancer effect towards KB cells and that autophagy inhibition could improve the anticancer effect of IL-24. IL-24 treated cells showed autophagy characteristics and autophagy inhibition by 3-methyladenine (3-MA) significantly enhanced IL-24-induced apoptosis. Similar results were obtained in the KB cells xenograft tumor model. Conclusions These results suggest that the combination of autophagy inhibitors and IL-24 based on the AdLTR2EF1α-mediated gene transfer could be a promising way to cure OSCC.http://deepblue.lib.umich.edu/bitstream/2027.42/113230/1/13046_2015_Article_211.pd

Bio-inspired magnetic-driven folded diaphragm for biomimetic robot

Soft robotics holds promise for realizing easy control over complex locomotion and deformation. Lin et al. report an earthworm-inspired untethered magnetic robot that consists one-piece-mold folded diaphragm to achieve large three-dimensional deformation with inside-volume change capability

Cu-doped CdS and its application in CdTe thin film solar cell

Cu is widely used in the back contact formation of CdTe thin film solar cells. However, Cu is easily to diffuse from the back contact into the CdTe absorber layer and even to the cell junction interface CdS/CdTe. This phenomenon is generally believed to be the main factor affecting the CdTe solar cell stability. In this study Cu was intentionally doped in CdS thin film to study its effect on the microstructural, optical and electrical properties of the CdS material. Upon Cu doping, the VCd− and the surface-state-related photoluminescence emissions were dramatically decreased/quenched. The presence of Cu atom hindered the recrystallization/coalescence of the nano-sized grains in the as-deposited CdS film during the air and the CdCl2 annealing. CdTe thin film solar cell fabricated with Cu-doped CdS window layers demonstrated much decreased fill factor, which was induced by the increased space-charge recombination near the p-n junction and the worsened junction crystalline quality. Temperature dependent current-voltage curve measurement indicated that the doped Cu in the CdS window layer was not stable at both room and higher temperatures

HMM-Based Method for Aircraft Environmental Control System Turbofan Rolling Bearing Fault Diagnosis

In response to the high-noise, nonlinear, and nonstationary characteristics of vibration signals from aircraft environmental control system (ECS) turbofan rolling bearings, this paper proposes a diagnostic method for the degree of ECS turbofan bearing faults based on the Hidden Markov Model (HMM). Experimental results demonstrate that HMM can accurately diagnose and predict faults in ECS turbofan rolling bearings. The HMM method enhances diagnostic accuracy, and its effectiveness and feasibility in fault diagnosis based on different rolling bearing fault instances are elaborated. By employing the HMM model to establish precise models from decomposed dynamic data, it successfully identifies faults such as the fracture of the bearing cage under biased load conditions, although its performance in recognizing overheating faults is suboptimal

Atmospheric-Window-Matching Hierarchical Broadband Infrared Absorber Realized by Lithography-Free Fabrication

An ultra-broadband selective absorber has been realized with a hierarchical structure through integrating vacuum impedance-matched structure, quarter wavelength structure, and gradient refractive index structure. Through optimizing the design parameters of the proposed hierarchical structure, an ultra-broadband infrared absorber covering the three major atmospheric windows (0.7–2.5, 3–5, and 8–14 μm) has been numerically and experimentally demonstrated. An overall absorption up to 80% covering all the three major atmospheric infrared windows and a ratio of the total absorptions within and beyond the windows as high as 5.88 has been achieved with the developed absorber. The high absorption and spectral selectivity of the absorber make it promising for sensitive broadband infrared spectroscopy detection. The proposed hierarchical structure also provides great design freedom with many tunable factors, making it convenient to extend the design for other applications. In addition, we developed a cost-effective lithography-free method for the fabrication of this structure. The design flexibility and fabrication convenience of this hierarchical structure render it suitable for the development of tailored selective broadband absorbers for targeted applications

Combination of multiple active sites in N, O co‐doped defective carbon materials for high performance aqueous supercapacitors

Abstract Supercapacitors have been used in a broad range of fields including electronics, transportation, and energies. Electrode materials with high capacitance and good rate performance are crucial for the future development and application of supercapacitors. Herein, we prepared N, O co‐doped defective carbon blocks (NO‐DCBs) with abundant active sites through carbonization and ball‐milling of polyimide. The as‐obtained NO‐DCBs exhibit high atomic content of N and O inherited from polyamic acid (PAA) precursor, as well as large amounts of intrinsic defects introduced by ball‐milling. Benefiting from the synergy of pseudocapacitance and electrical double‐layer capacitance provided by heteroatoms and intrinsic defects respectively, the NO‐DCBs assembled symmetric aqueous capacitor shows high capacitance of 329 F g−1 at 0.1 A g−1, good rate performance of 48% capacitance retention at 50 A g−1, and superb cycling stability. This work promotes the deep understanding of the synergy effect of functional groups and intrinsic defects for capacitive energy storage, and broadens the avenue for structural design of active sites in carbon materials