Putting the Object Back into Video Object Segmentation

We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result. Recent works on VOS employ bottom-up pixel-level memory reading which struggles due to matching noise, especially in the presence of distractors, resulting in lower performance in more challenging data. In contrast, Cutie performs top-down object-level memory reading by adapting a small set of object queries for restructuring and interacting with the bottom-up pixel features iteratively with a query-based object transformer (qt, hence Cutie). The object queries act as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation. Together with foreground-background masked attention, Cutie cleanly separates the semantics of the foreground object from the background. On the challenging MOSE dataset, Cutie improves by 8.7 J&F over XMem with a similar running time and improves by 4.2 J&F over DeAOT while running three times as fast. Code is available at: https://hkchengrex.github.io/CutieComment: Project page: https://hkchengrex.github.io/Cuti

Per-Clip Video Object Segmentation

Recently, memory-based approaches show promising results on semi-supervised video object segmentation. These methods predict object masks frame-by-frame with the help of frequently updated memory of the previous mask. Different from this per-frame inference, we investigate an alternative perspective by treating video object segmentation as clip-wise mask propagation. In this per-clip inference scheme, we update the memory with an interval and simultaneously process a set of consecutive frames (i.e. clip) between the memory updates. The scheme provides two potential benefits: accuracy gain by clip-level optimization and efficiency gain by parallel computation of multiple frames. To this end, we propose a new method tailored for the per-clip inference. Specifically, we first introduce a clip-wise operation to refine the features based on intra-clip correlation. In addition, we employ a progressive matching mechanism for efficient information-passing within a clip. With the synergy of two modules and a newly proposed per-clip based training, our network achieves state-of-the-art performance on Youtube-VOS 2018/2019 val (84.6% and 84.6%) and DAVIS 2016/2017 val (91.9% and 86.1%). Furthermore, our model shows a great speed-accuracy trade-off with varying memory update intervals, which leads to huge flexibility.Comment: CVPR 2022; Code is available at https://github.com/pkyong95/PCVO

A Generalized Framework for Video Instance Segmentation

The handling of long videos with complex and occluded sequences has recently emerged as a new challenge in the video instance segmentation (VIS) community. However, existing methods have limitations in addressing this challenge. We argue that the biggest bottleneck in current approaches is the discrepancy between training and inference. To effectively bridge this gap, we propose a Generalized framework for VIS, namely GenVIS, that achieves state-of-the-art performance on challenging benchmarks without designing complicated architectures or requiring extra post-processing. The key contribution of GenVIS is the learning strategy, which includes a query-based training pipeline for sequential learning with a novel target label assignment. Additionally, we introduce a memory that effectively acquires information from previous states. Thanks to the new perspective, which focuses on building relationships between separate frames or clips, GenVIS can be flexibly executed in both online and semi-online manner. We evaluate our approach on popular VIS benchmarks, achieving state-of-the-art results on YouTube-VIS 2019/2021/2022 and Occluded VIS (OVIS). Notably, we greatly outperform the state-of-the-art on the long VIS benchmark (OVIS), improving 5.6 AP with ResNet-50 backbone. Code is available at https://github.com/miranheo/GenVIS.Comment: CVPR 202

Male-Specific W4P/R Mutation in the Pre-S1 Region of Hepatitis B Virus, Increasing the Risk of Progression of Liver Diseases in Chronic Patients

The issue of hepatitis B virus (HBV) mutations possibly leading to a gender disparity in the progression of liver diseases has not been explored. We aimed to elucidate the relationships of the novel pre-S1 mutations, W4P/R, with the progression of liver diseases and male predominance in a South Korean chronic cohort by use of a molecular epidemiologic study. We developed a fluorescence resonance energy transfer (FRET)-based real-time PCR (RT-PCR) assay for the detection of the W4P/R mutations and applied it to 292 chronic HBV patients. The pre-S1 mutations from 247 (84.6%) of a total of 292 patients were detected by this assay. W4P/R mutants were found to be significantly related to severe liver diseases (hepatocellular carcinoma [HCC] and liver cirrhosis, 12.4% [19/153] of patients, versus chronic hepatitis and asymptomatic carriage, 1.1% [1/94] of patients) (P<0.001). All of the W4P/R mutants were found in males only. The novel HBV pre-S1 mutations, W4P/R, may be associated with disease severity in male patients chronically infected with HBV genotype C. The W4P/R mutations may provide in part an explanation for the relatively high ratio of male to female incidence in HCC generation in South Korean chronic HBV patients.OAIID:oai:osos.snu.ac.kr:snu2013-01/102/0000006653/6SEQ:6PERF_CD:SNU2013-01EVAL_ITEM_CD:102USER_ID:0000006653ADJUST_YN:YEMP_ID:A077651DEPT_CD:806CITE_RATE:4.068FILENAME:male-specific w4p_r mutation in the pre-s1.pdfDEPT_NM:의과학과EMAIL:[email protected]_YN:YCONFIRM:

Development of Anthropometry-Based Equations for the Estimation of the Total Body Water in Koreans

For developing race-specific anthropometry-based total body water (TBW) equations, we measured TBW using bioelectrical impedance analysis (TBWBIA) in 2,943 healthy Korean adults. Among them, 2,223 were used as a reference group. Two equations (TBWK1 and TBWK2) were developed based on age, sex, height, and body weight. The adjusted R2 was 0.908 for TBWK1 and 0.910 for TBWK2. The remaining 720 subjects were used for the validation of our results. Watson (TBWW) and Hume-Weyers (TBWH) formulas were also used. In men, TBWBIA showed the highest correlation with TBWH, followed by TBWK1, TBWK2 and TBWW. TBWK1 and TBWK2 showed the lower root mean square errors (RMSE) and mean prediction errors (ME) than TBWW and TBWH. On the Bland-Altman plot, the correlations between the differences and means were smaller for TBWK2 than for TBWK1. On the contrary, TBWBIA showed the highest correlation with TBWW, followed by TBWK2, TBWK1, and TBWH in females. RMSE was smallest in TBWW, followed by TBWK2, TBWK1 and TBWH. ME was closest to zero for TBWK2, followed by TBWK1, TBWW and TBWH. The correlation coefficients between the means and differences were highest in TBWW, and lowest in TBWK2. In conclusion, TBWK2 provides better accuracy with a smaller bias than the TBWW or TBWH in males. TBWK2 shows a similar accuracy, but with a smaller bias than TBWW in females