1,255 research outputs found

    Temporal Attention-Gated Model for Robust Sequence Classification

    Full text link
    Typical techniques for sequence classification are designed for well-segmented sequences which have been edited to remove noisy or irrelevant parts. Therefore, such methods cannot be easily applied on noisy sequences expected in real-world applications. In this paper, we present the Temporal Attention-Gated Model (TAGM) which integrates ideas from attention models and gated recurrent networks to better deal with noisy or unsegmented sequences. Specifically, we extend the concept of attention model to measure the relevance of each observation (time step) of a sequence. We then use a novel gated recurrent network to learn the hidden representation for the final prediction. An important advantage of our approach is interpretability since the temporal attention weights provide a meaningful value for the salience of each time step in the sequence. We demonstrate the merits of our TAGM approach, both for prediction accuracy and interpretability, on three different tasks: spoken digit recognition, text-based sentiment analysis and visual event recognition.Comment: Accepted by CVPR 201

    κ°•μΈν•œ λŒ€ν™”ν˜• μ˜μƒ λΆ„ν•  μ•Œκ³ λ¦¬μ¦˜μ„ μœ„ν•œ μ‹œλ“œ 정보 ν™•μž₯ 기법에 λŒ€ν•œ 연ꡬ

    Get PDF
    ν•™μœ„λ…Όλ¬Έ (박사) -- μ„œμšΈλŒ€ν•™κ΅ λŒ€ν•™μ› : κ³΅κ³ΌλŒ€ν•™ 전기·컴퓨터곡학뢀, 2021. 2. 이경무.Segmentation of an area corresponding to a desired object in an image is essential to computer vision problems. This is because most algorithms are performed in semantic units when interpreting or analyzing images. However, segmenting the desired object from a given image is an ambiguous issue. The target object varies depending on user and purpose. To solve this problem, an interactive segmentation technique has been proposed. In this approach, segmentation was performed in the desired direction according to interaction with the user. In this case, seed information provided by the user plays an important role. If the seed provided by a user contain abundant information, the accuracy of segmentation increases. However, providing rich seed information places much burden on the users. Therefore, the main goal of the present study was to obtain satisfactory segmentation results using simple seed information. We primarily focused on converting the provided sparse seed information to a rich state so that accurate segmentation results can be derived. To this end, a minimum user input was taken and enriched it through various seed enrichment techniques. A total of three interactive segmentation techniques was proposed based on: (1) Seed Expansion, (2) Seed Generation, (3) Seed Attention. Our seed enriching type comprised expansion of area around a seed, generation of new seed in a new position, and attention to semantic information. First, in seed expansion, we expanded the scope of the seed. We integrated reliable pixels around the initial seed into the seed set through an expansion step composed of two stages. Through the extended seed covering a wider area than the initial seed, the seed's scarcity and imbalance problems was resolved. Next, in seed generation, we created a seed at a new point, but not around the seed. We trained the system by imitating the user behavior through providing a new seed point in the erroneous region. By learning the user's intention, our model could e ciently create a new seed point. The generated seed helped segmentation and could be used as additional information for weakly supervised learning. Finally, through seed attention, we put semantic information in the seed. Unlike the previous models, we integrated both the segmentation process and seed enrichment process. We reinforced the seed information by adding semantic information to the seed instead of spatial expansion. The seed information was enriched through mutual attention with feature maps generated during the segmentation process. The proposed models show superiority compared to the existing techniques through various experiments. To note, even with sparse seed information, our proposed seed enrichment technique gave by far more accurate segmentation results than the other existing methods.μ˜μƒμ—μ„œ μ›ν•˜λŠ” 물체 μ˜μ—­μ„ μž˜λΌλ‚΄λŠ” 것은 컴퓨터 λΉ„μ „ λ¬Έμ œμ—μ„œ ν•„μˆ˜μ μΈ μš”μ†Œμ΄λ‹€. μ˜μƒμ„ ν•΄μ„ν•˜κ±°λ‚˜ 뢄석할 λ•Œ, λŒ€λΆ€λΆ„μ˜ μ•Œκ³ λ¦¬μ¦˜λ“€μ΄ 의미둠적인 λ‹¨μœ„ 기반으둜 λ™μž‘ν•˜κΈ° λ•Œλ¬Έμ΄λ‹€. κ·ΈλŸ¬λ‚˜ μ˜μƒμ—μ„œ 물체 μ˜μ—­μ„ λΆ„ν• ν•˜λŠ” 것은 λͺ¨ν˜Έν•œ λ¬Έμ œμ΄λ‹€. μ‚¬μš©μžμ™€ λͺ©μ μ— 따라 μ›ν•˜λŠ” 물체 μ˜μ—­μ΄ 달라지기 λ•Œλ¬Έμ΄λ‹€. 이λ₯Ό ν•΄κ²°ν•˜κΈ° μœ„ν•΄ μ‚¬μš©μžμ™€μ˜ ꡐλ₯˜λ₯Ό 톡해 μ›ν•˜λŠ” λ°©ν–₯으둜 μ˜μƒ 뢄할을 μ§„ν–‰ν•˜λŠ” λŒ€ν™”ν˜• μ˜μƒ λΆ„ν•  기법이 μ‚¬μš©λœλ‹€. μ—¬κΈ°μ„œ μ‚¬μš©μžκ°€ μ œκ³΅ν•˜λŠ” μ‹œλ“œ 정보가 μ€‘μš”ν•œ 역할을 ν•œλ‹€. μ‚¬μš©μžμ˜ μ˜λ„λ₯Ό λ‹΄κ³  μžˆλŠ” μ‹œλ“œ 정보가 μ •ν™•ν• μˆ˜λ‘ μ˜μƒ λΆ„ν• μ˜ 정확도도 μ¦κ°€ν•˜κ²Œ λœλ‹€. κ·ΈλŸ¬λ‚˜ ν’λΆ€ν•œ μ‹œλ“œ 정보λ₯Ό μ œκ³΅ν•˜λŠ” 것은 μ‚¬μš©μžμ—κ²Œ λ§Žμ€ 뢀담을 주게 λœλ‹€. κ·ΈλŸ¬λ―€λ‘œ κ°„λ‹¨ν•œ μ‹œλ“œ 정보λ₯Ό μ‚¬μš©ν•˜μ—¬ λ§Œμ‘±ν• λ§Œν•œ λΆ„ν•  κ²°κ³Όλ₯Ό μ–»λŠ” 것이 μ£Όμš” λͺ©μ μ΄ λœλ‹€. μš°λ¦¬λŠ” 제곡된 ν¬μ†Œν•œ μ‹œλ“œ 정보λ₯Ό λ³€ν™˜ν•˜λŠ” μž‘μ—…μ— μ΄ˆμ μ„ λ‘μ—ˆλ‹€. λ§Œμ•½ μ‹œλ“œ 정보가 ν’λΆ€ν•˜κ²Œ λ³€ν™˜λœλ‹€λ©΄ μ •ν™•ν•œ μ˜μƒ λΆ„ν•  κ²°κ³Όλ₯Ό 얻을 수 있기 λ•Œλ¬Έμ΄λ‹€. κ·ΈλŸ¬λ―€λ‘œ λ³Έ ν•™μœ„ λ…Όλ¬Έμ—μ„œλŠ” μ‹œλ“œ 정보λ₯Ό ν’λΆ€ν•˜κ²Œ ν•˜λŠ” 기법듀을 μ œμ•ˆν•œλ‹€. μ΅œμ†Œν•œμ˜ μ‚¬μš©μž μž…λ ₯을 κ°€μ •ν•˜κ³  이λ₯Ό λ‹€μ–‘ν•œ μ‹œλ“œ ν™•μž₯ 기법을 톡해 λ³€ν™˜ν•œλ‹€. μš°λ¦¬λŠ” μ‹œλ“œ ν™•λŒ€, μ‹œλ“œ 생성, μ‹œλ“œ 주의 집쀑에 κΈ°λ°˜ν•œ 총 μ„Έ κ°€μ§€μ˜ λŒ€ν™”ν˜• μ˜μƒ λΆ„ν•  기법을 μ œμ•ˆν•œλ‹€. 각각 μ‹œλ“œ μ£Όλ³€μœΌλ‘œμ˜ μ˜μ—­ ν™•λŒ€, μƒˆλ‘œμš΄ 지점에 μ‹œλ“œ 생성, 의미둠적 정보에 μ£Όλͺ©ν•˜λŠ” ν˜•νƒœμ˜ μ‹œλ“œ ν™•μž₯ 기법을 μ‚¬μš©ν•œλ‹€. λ¨Όμ € μ‹œλ“œ ν™•λŒ€μ— κΈ°λ°˜ν•œ κΈ°λ²•μ—μ„œ μš°λ¦¬λŠ” μ‹œλ“œμ˜ μ˜μ—­ ν™•μž₯을 λͺ©ν‘œλ‘œ ν•œλ‹€. 두 λ‹¨κ³„λ‘œ κ΅¬μ„±λœ ν™•λŒ€ 과정을 톡해 처음 μ‹œλ“œ μ£Όλ³€μ˜ λΉ„μŠ·ν•œ 픽셀듀을 μ‹œλ“œ μ˜μ—­μœΌλ‘œ νŽΈμž…ν•œλ‹€. μ΄λ ‡κ²Œ ν™•μž₯된 μ‹œλ“œλ₯Ό μ‚¬μš©ν•¨μœΌλ‘œμ¨ μ‹œλ“œμ˜ ν¬μ†Œν•¨κ³Ό λΆˆκ· ν˜•μœΌλ‘œ μΈν•œ 문제λ₯Ό ν•΄κ²°ν•  수 μžˆλ‹€. λ‹€μŒμœΌλ‘œ μ‹œλ“œ 생성에 κΈ°λ°˜ν•œ κΈ°λ²•μ—μ„œ μš°λ¦¬λŠ” μ‹œλ“œ 주변이 μ•„λ‹Œ μƒˆλ‘œμš΄ 지점에 μ‹œλ“œλ₯Ό μƒμ„±ν•œλ‹€. μš°λ¦¬λŠ” μ˜€μ°¨κ°€ λ°œμƒν•œ μ˜μ—­μ— μ‚¬μš©μžκ°€ μƒˆλ‘œμš΄ μ‹œλ“œλ₯Ό μ œκ³΅ν•˜λŠ” λ™μž‘μ„ λͺ¨λ°©ν•˜μ—¬ μ‹œμŠ€ν…œμ„ ν•™μŠ΅ν•˜μ˜€λ‹€. μ‚¬μš©μžμ˜ μ˜λ„λ₯Ό ν•™μŠ΅ν•¨μœΌλ‘œμ¨ 효과적으둜 μ‹œλ“œλ₯Ό 생성할 수 μžˆλ‹€. μƒμ„±λœ μ‹œλ“œλŠ” μ˜μƒ λΆ„ν• μ˜ 정확도λ₯Ό 높일 뿐만 μ•„λ‹ˆλΌ μ•½μ§€λ„ν•™μŠ΅μ„ μœ„ν•œ λ°μ΄ν„°λ‘œμ¨ ν™œμš©λ  수 μžˆλ‹€. λ§ˆμ§€λ§‰μœΌλ‘œ μ‹œλ“œ 주의 집쀑을 ν™œμš©ν•œ κΈ°λ²•μ—μ„œ μš°λ¦¬λŠ” 의미둠적 정보λ₯Ό μ‹œλ“œμ— λ‹΄λŠ”λ‹€. 기쑴에 μ œμ•ˆν•œ 기법듀과 달리 μ˜μƒ λΆ„ν•  λ™μž‘κ³Ό μ‹œλ“œ ν™•μž₯ λ™μž‘μ΄ ν†΅ν•©λœ λͺ¨λΈμ„ μ œμ•ˆν•œλ‹€. μ‹œλ“œ μ •λ³΄λŠ” μ˜μƒ λΆ„ν•  λ„€νŠΈμ›Œν¬μ˜ νŠΉμ§•λ§΅κ³Ό μƒν˜Έ ꡐλ₯˜ν•˜λ©° κ·Έ 정보가 풍뢀해진닀. μ œμ•ˆν•œ λͺ¨λΈλ“€μ€ λ‹€μ–‘ν•œ μ‹€ν—˜μ„ 톡해 κΈ°μ‘΄ 기법 λŒ€λΉ„ μš°μˆ˜ν•œ μ„±λŠ₯을 κΈ°λ‘ν•˜μ˜€λ‹€. 특히 μ‹œλ“œκ°€ λΆ€μ‘±ν•œ μƒν™©μ—μ„œ μ‹œλ“œ ν™•μž₯ 기법듀은 ν›Œλ₯­ν•œ λŒ€ν™”ν˜• μ˜μƒ λΆ„ν•  μ„±λŠ₯을 λ³΄μ˜€λ‹€.1 Introduction 1 1.1 Previous Works 2 1.2 Proposed Methods 4 2 Interactive Segmentation with Seed Expansion 9 2.1 Introduction 9 2.2 Proposed Method 12 2.2.1 Background 13 2.2.2 Pyramidal RWR 16 2.2.3 Seed Expansion 19 2.2.4 Re nement with Global Information 24 2.3 Experiments 27 2.3.1 Dataset 27 2.3.2 Implement Details 28 2.3.3 Performance 29 2.3.4 Contribution of Each Part 30 2.3.5 Seed Consistency 31 2.3.6 Running Time 33 2.4 Summary 34 3 Interactive Segmentation with Seed Generation 37 3.1 Introduction 37 3.2 Related Works 40 3.3 Proposed Method 41 3.3.1 System Overview 41 3.3.2 Markov Decision Process 42 3.3.3 Deep Q-Network 46 3.3.4 Model Architecture 47 3.4 Experiments 48 3.4.1 Implement Details 48 3.4.2 Performance 49 3.4.3 Ablation Study 53 3.4.4 Other Datasets 55 3.5 Summary 58 4 Interactive Segmentation with Seed Attention 61 4.1 Introduction 61 4.2 Related Works 64 4.3 Proposed Method 65 4.3.1 Interactive Segmentation Network 65 4.3.2 Bi-directional Seed Attention Module 67 4.4 Experiments 70 4.4.1 Datasets 70 4.4.2 Metrics 70 4.4.3 Implement Details 71 4.4.4 Performance 71 4.4.5 Ablation Study 76 4.4.6 Seed enrichment methods 79 4.5 Summary 82 5 Conclusions 87 5.1 Summary 89 Bibliography 90 ꡭ문초둝 103Docto

    Semi-supervised Salient Object Detection with Effective Confidence Estimation

    Full text link
    The success of existing salient object detection models relies on a large pixel-wise labeled training dataset, which is time-consuming and expensive to obtain. We study semi-supervised salient object detection, with access to a small number of labeled samples and a large number of unlabeled samples. Specifically, we present a pseudo label based learn-ing framework with a Conditional Energy-based Model. We model the stochastic nature of human saliency labels using the stochastic latent variable of the Conditional Energy-based Model. It further enables generation of a high-quality pixel-wise uncertainty map, highlighting the reliability of corresponding pseudo label generated for the unlabeled sample. This minimises the contribution of low-certainty pseudo labels in optimising the model, preventing the error propagation. Experimental results show that the proposed strategy can effectively explore the contribution of unlabeled data. With only 1/16 labeled samples, our model achieves competitive performance compared with state-of-the-art fully-supervised models
    • …
    corecore