LCCM-VC: Learned Conditional Coding Modes for Video Compression

End-to-end learning-based video compression has made steady progress over the last several years. However, unlike learning-based image coding, which has already surpassed its handcrafted counterparts, learning-based video coding still has some ways to go. In this paper we present learned conditional coding modes for video coding (LCCM-VC), a video coding model that achieves state-of-the-art results among learning-based video coding methods. Our model utilizes conditional coding engines from the recent conditional augmented normalizing flows (CANF) pipeline, and introduces additional coding modes to improve compression performance. The compression efficiency is especially good in the high-quality/high-bitrate range, which is important for broadcast and video-on-demand streaming applications. The implementation of LCCM-VC is available at https://github.com/hadihdz/lccm_vcComment: 5 pages, 3 figures, IEEE ICASSP 202

Learned Scalable Video Coding For Humans and Machines

Video coding has traditionally been developed to support services such as video streaming, videoconferencing, digital TV, and so on. The main intent was to enable human viewing of the encoded content. However, with the advances in deep neural networks (DNNs), encoded video is increasingly being used for automatic video analytics performed by machines. In applications such as automatic traffic monitoring, analytics such as vehicle detection, tracking and counting, would run continuously, while human viewing could be required occasionally to review potential incidents. To support such applications, a new paradigm for video coding is needed that will facilitate efficient representation and compression of video for both machine and human use in a scalable manner. In this manuscript, we introduce the first end-to-end learnable video codec that supports a machine vision task in its base layer, while its enhancement layer supports input reconstruction for human viewing. The proposed system is constructed based on the concept of conditional coding to achieve better compression gains. Comprehensive experimental evaluations conducted on four standard video datasets demonstrate that our framework outperforms both state-of-the-art learned and conventional video codecs in its base layer, while maintaining comparable performance on the human vision task in its enhancement layer. We will provide the implementation of the proposed system at www.github.com upon completion of the review process.Comment: 14 pages, 16 figure

Unsupervised Video Summarization via Reinforcement Learning and a Trained Evaluator

This paper presents a novel approach for unsupervised video summarization using reinforcement learning. It aims to address the existing limitations of current unsupervised methods, including unstable training of adversarial generator-discriminator architectures and reliance on hand-crafted reward functions for quality evaluation. The proposed method is based on the concept that a concise and informative summary should result in a reconstructed video that closely resembles the original. The summarizer model assigns an importance score to each frame and generates a video summary. In the proposed scheme, reinforcement learning, coupled with a unique reward generation pipeline, is employed to train the summarizer model. The reward generation pipeline trains the summarizer to create summaries that lead to improved reconstructions. It comprises a generator model capable of reconstructing masked frames from a partially masked video, along with a reward mechanism that compares the reconstructed video from the summary against the original. The video generator is trained in a self-supervised manner to reconstruct randomly masked frames, enhancing its ability to generate accurate summaries. This training pipeline results in a summarizer model that better mimics human-generated video summaries compared to methods relying on hand-crafted rewards. The training process consists of two stable and isolated training steps, unlike adversarial architectures. Experimental results demonstrate promising performance, with F-scores of 62.3 and 54.5 on TVSum and SumMe datasets, respectively. Additionally, the inference stage is 300 times faster than our previously reported state-of-the-art method

Synthesis and Effects of 4,5-Diaryl-2-(2-alkylthio-5-imidazolyl) Imidazoles as Selective Cyclooxygenase Inhibitors

Objective(s)In recent years highly selective COX-2inhibitors were withdrawn from the market because of an increased risk of cardiovascular complications. In this study we were looking for potent compounds with moderate selectivity for cox-2. So, four analogues of 4, 5-diaryl-2-(2-alkylthio-5-imidazolyl) imidazole derivatives were synthesized and their anti-inflammatory and anti-nociceptive activities were evaluated on male BALB/c mice (25-30 g). Molecular modeling and in vitro COX-1 and COX-2 isozyme inhibition studies were also performed. Materials and Methods2-(2-Alkylthio-5-imidazolyl)-4,5-diphenylimidazole compounds were obtained by the reaction of benzyl with 2-alkylthio-1-benzylimidazole-5-carbaldehyde, in the presence of ammonium acetate. Spectroscopic data and elemental analysis of compounds were obtained and their structures elucidated. Anti-nociception effects were examined using writhing test in mice. The effect of the analogues (7.5, 30, 52.5 and 75 mg/kg) against acute inflammation were studied using xylene-induced ear edema test in mice. Celecoxib (75 mg/kg) was used as positive control.ResultsAll four analogues exhibited anti-nociceptive activity against acetic acid induced writhing, but did not show significant analgesic effect (P< 0.05) compared with celecoxib. It was shown that analogues injected 30 min before xylene application reduced the weight of edematic ears. All analogues were found to have less selectivity for COX-2 in comparison to celecoxib. ConclusionInjected doses of synthesised analogues possesses favorite anti-nociceptive effect and also has anti-inflammatory effects, but comparing with celecoxib this effect is not significantly different. On the other hand selectivity index for analogues is less than celecoxib and so we expect less cardiovascular side effects for these compounds

Learned Multimodal Compression for Autonomous Driving

Autonomous driving sensors generate an enormous amount of data. In this paper, we explore learned multimodal compression for autonomous driving, specifically targeted at 3D object detection. We focus on camera and LiDAR modalities and explore several coding approaches. One approach involves joint coding of fused modalities, while others involve coding one modality first, followed by conditional coding of the other modality. We evaluate the performance of these coding schemes on the nuScenes dataset. Our experimental results indicate that joint coding of fused modalities yields better results compared to the alternatives.6 pages, 5 figures, IEEE MMSP 202

Mutual Information Analysis in Multimodal Learning Systems

In recent years, there has been a significant increase in applications of multimodal signal processing and analysis, largely driven by the increased availability of multimodal datasets and the rapid progress in multimodal learning systems. Well-known examples include autonomous vehicles, audiovisual generative systems, vision-language systems, and so on. Such systems integrate multiple signal modalities: text, speech, images, video, LiDAR, etc., to perform various tasks. A key issue for understanding such systems is the relationship between various modalities and how it impacts task performance. In this paper, we employ the concept of mutual information (MI) to gain insight into this issue. Taking advantage of the recent progress in entropy modeling and estimation, we develop a system called InfoMeter to estimate MI between modalities in a multimodal learning system. We then apply InfoMeter to analyze a multimodal 3D object detection system over a large-scale dataset for autonomous driving. Our experiments on this system suggest that a lower MI between modalities is beneficial for detection accuracy. This new insight may facilitate improvements in the development of future multimodal learning systems.Comment: 6 pages, 7 figures, IEEE MIPR 202

Research Paper: Effectiveness of Corticosteroid Therapy for Caustic Esophageal Injury

Background: Delayed caustic injury complications are common, especially in developing countries, and several treatments have been proposed to prevent the resulting esophageal strictures so far. Although inflammatory nature of caustic injury makes the anti-inflammatory agents a viable option, few studies have investigated these agents. High-dose corticosteroids therapy for reduction of stricture formation in the esophagus after the ingestion of caustic material is still a controversial topic. In this regard, this study aimed to determine the impact of high doses of methylprednisolone in preventing esophageal stricture.Methods: A total of 112 patients with grade II esophageal caustic injury, diagnosed by esophagogastroscopy within 24 hours of injury, were enrolled in our study. The treatment group (n=44) received methylprednisolone (1 g/d for 3 days), pantoprazole, ceftriaxone, and metronidazole and the control group (n=58) received the same regimen excluding methylprednisolone. Endoscopic and radiologic findings were used to compare the severity of the damage to the esophagus and stomach between the two groups.Results: After 8 months of follow-up, stricture development was observed in 3 (5.6%) patients in the treatment group and in 11 (19%) patients in the control group. The difference was statistically significant (P=0.038). The gastric outlet obstruction was observed in 4 (7.4%) patients in the treatment group and in 19 (32.7%) patients in the control group. Again, the difference was statistically significant (P&lt;0.05). There were not any side effects due to the high doses of methylprednisolone in the study group.Conclusion: High doses of methylprednisolone can prevent the development of esophageal stricture in grade II of caustic injury