15 research outputs found
Recommended from our members
Move2Hear : active audio-visual source separation
We introduce the active audio-visual source separation problem, where an
agent must move intelligently in order to better isolate the sounds coming from an object of interest in its environment. The agent hears multiple audio sources simultaneously (e.g., a person speaking down the hall in a noisy household) and must use its eyes and ears to automatically separate out the sounds originating
from the target object within a limited time budget. Towards this goal, we introduce a reinforcement learning approach that trains movement policies controlling the agent’s camera and microphone placement over time, guided by the improvement in predicted audio separation quality. We demonstrate our approach in scenarios
motivated by both augmented reality (system is already co-located with the target object) and mobile robotics (agent begins arbitrarily far from the target object). Using state-of-the-art realistic audio-visual simulations in 3D environments, we demonstrate our model’s ability to find minimal movement sequences with maximal payoff for audio source separation.Computer Science
Few-Shot Audio-Visual Learning of Environment Acoustics
Room impulse response (RIR) functions capture how the surrounding physical
environment transforms the sounds heard by a listener, with implications for
various applications in AR, VR, and robotics. Whereas traditional methods to
estimate RIRs assume dense geometry and/or sound measurements throughout the
environment, we explore how to infer RIRs based on a sparse set of images and
echoes observed in the space. Towards that goal, we introduce a
transformer-based method that uses self-attention to build a rich acoustic
context, then predicts RIRs of arbitrary query source-receiver locations
through cross-attention. Additionally, we design a novel training objective
that improves the match in the acoustic signature between the RIR predictions
and the targets. In experiments using a state-of-the-art audio-visual simulator
for 3D environments, we demonstrate that our method successfully generates
arbitrary RIRs, outperforming state-of-the-art methods and -- in a major
departure from traditional methods -- generalizing to novel environments in a
few-shot manner. Project: http://vision.cs.utexas.edu/projects/fs_rir.Comment: Accepted to NeurIPS 202
EVALUATION OF ANTI CANCER POTENTIAL OF METHANOL EXTRACT OF CURCUMA ZEDOARIA.
Objective: Evaluation of anti cancer activity of methanol extracts of Curcuma zedoaria against Ehrlich's ascities carcinoma (EAC) cell line in Swiss albino mice. Method: In vitro cytotoxicity assay has been evaluated by using the trypan blue and MTT assay method. In vivo anti cancer activity was performed by using EAC cells induced mice groups (n=12), at the doses of 100 and 200 mg/kg b.w. respectively, half of the mice were sacrificed and the restwere kept alive for life span parameter. The anti cancer potential of MECZ was assessed by evaluating tumor volume, viable and nonviable tumorcell count, tumor weight, hematological parameters and biochemical estimations. Furthermore, antioxidant parameters were assayed by estimatingliver tissue enzymes. Result: Dose dependent cytotoxicity was observed in (* p < 0.05) Trypan blue and MTT assay method. In vivo anti cancer parameters like tumorvolume, tumor weight, and viable cell count were decreased compared to the EAC control group. MECZ treated EAC cell–bearing mice had anincreased life span to that of EAC control group. Hematological and serum biochemical profiles were restored to normal levels in MECZ-treated micecompared to the EAC control group. Among the tissue parameters lipid peroxidation, reduced glutathione,superoxide dismutase, and catalasetoward normal levels compared to the EAC control group. In short, Conclusion: MECZ exhibited remarkable antitumor activity in Swiss albino mice, which is attributed to its augmentation of endogenous antioxidantactivities and cytotoxic nature. Keywords: Curcuma zedoaria, Zingiberaceae, EAC cell line, antitumor activity, 5-Flurouraci
Chat2Map: Efficient Scene Mapping from Multi-Ego Conversations
Can conversational videos captured from multiple egocentric viewpoints reveal
the map of a scene in a cost-efficient way? We seek to answer this question by
proposing a new problem: efficiently building the map of a previously unseen 3D
environment by exploiting shared information in the egocentric audio-visual
observations of participants in a natural conversation. Our hypothesis is that
as multiple people ("egos") move in a scene and talk among themselves, they
receive rich audio-visual cues that can help uncover the unseen areas of the
scene. Given the high cost of continuously processing egocentric visual
streams, we further explore how to actively coordinate the sampling of visual
information, so as to minimize redundancy and reduce power use. To that end, we
present an audio-visual deep reinforcement learning approach that works with
our shared scene mapper to selectively turn on the camera to efficiently chart
out the space. We evaluate the approach using a state-of-the-art audio-visual
simulator for 3D scenes as well as real-world video. Our model outperforms
previous state-of-the-art mapping methods, and achieves an excellent
cost-accuracy tradeoff. Project: http://vision.cs.utexas.edu/projects/chat2map.Comment: Accepted to CVPR 202