SHINE: Deep Learning-Based Accessible Parking Management System

The ongoing expansion of urban areas facilitated by advancements in science and technology has resulted in a considerable increase in the number of privately owned vehicles worldwide, including in South Korea. However, this gradual increment in the number of vehicles has inevitably led to parking-related issues, including the abuse of disabled parking spaces (hereafter referred to as accessible parking spaces) designated for individuals with disabilities. Traditional license plate recognition (LPR) systems have proven inefficient in addressing such a problem in real-time due to the high frame rate of surveillance cameras, the presence of natural and artificial noise, and variations in lighting and weather conditions that impede detection and recognition by these systems. With the growing concept of parking 4.0, many sensors, IoT and deep learning-based approaches have been applied to automatic LPR and parking management systems. Nonetheless, the studies show a need for a robust and efficient model for managing accessible parking spaces in South Korea. To address this, we have proposed a novel system called, Shine, which uses the deep learning-based object detection algorithm for detecting the vehicle, license plate, and disability badges (referred to as cards, badges, or access badges hereafter) and verifies the rights of the driver to use accessible parking spaces by coordinating with the central server. Our model, which achieves a mean average precision of 92.16%, is expected to address the issue of accessible parking space abuse and contributes significantly towards efficient and effective parking management in urban environments

조건부 텍스트 생성 시스템에 대한 사실 관계의 일관성 평가

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2022. 8. 정교민.최근의 사전학습 언어모델의 활용을 통한 조건부 텍스트 생성 시스템들의 발전에도 불구하고, 시스템들의 사실 관계의 일관성은 여전히 충분하지 않은 편이다. 그러나 널리 사용되는 n-그램 기반 유사성 평가 기법은 사실 일관성 평가에 매우 취약하다. 따라서, 사실 일관된 텍스트 생성 시스템을 개발하기 위해서는 먼저 시스템의 사실 관계를 제대로 평가할 수 있는 자동 평가 기법이 필요하다. 본 논문에서는 다양한 조건부 텍스트 생성 시스템에 대해, 이전 평가 기법보다 사실 관계 일관성 평가에서 인간의 판단과 매우 높은 상관관계를 보여주는 4가지 평가 기법을 제안한다. 이 기법들은 (1) 보조 태스크 활용 및 (2) 데이터 증강 기법 등을 활용한다. 첫째로, 우리는 중요한 핵심 단어또는 핵심 구문에 초점을 맞춘 두 가지 다른 보조 태스크를 활용하여 두 가지 사실 관계의 일관성 평가 기법을 제안한다. 우리는 먼저 핵심 구문의 가중치 예측 태스크를 이전 평가 기법에 결합하여 주관식 질의 응답을 위한 평가 기법을 제안한다. 또한, 우리는 질의 생성 및 응답을 활용하여 키워드에 대한 질의를 생성하고, 이미지와 캡션에 대한 질문의 답을 비교하여 사실 일관성을 확인하는 QACE를 제안한다. 둘째로, 우리는 보조 태스크 활용과 달리, 데이터 기반 방식의 학습을 통해 두 가지의 평가 기법을 제안한다. 구체적으로, 우리는 증강된 일관성 없는 텍스트를 일관성 있는 텍스트와 구분하도록 훈련한다. 먼저 규칙 기반 변형을 통한 불일치 캡션 생성으로 이미지 캡션 평가 지표 UMIC을 제안한다. 다음 단계로, 마스킹된 소스와 마스킹된 요약을 사용하여 일관성이 없는 요약을 생성하는 MFMA를 통해 평가 지표를 개발한다. 마지막으로, 데이터 기반 사실 일관성 평가 기법 개발의 확장으로, 시스템의 사실 관계 오류를 수정할 수 있는 빠른 사후 교정 시스템을 제안한다.Despite the recent advances of conditional text generation systems leveraged from pre-trained language models, factual consistency of the systems are still not sufficient. However, widely used n-gram similarity metrics are vulnerable to evaluate the factual consistency. Hence, in order to develop a factual consistent system, an automatic factuality metric is first necessary. In this dissertation, we propose four metrics that show very higher correlation with human judgments than previous metrics in evaluating factual consistency, for diverse conditional text generation systems. To build such metrics, we utilize (1) auxiliary tasks and (2) data augmentation methods. First, we focus on the keywords or keyphrases that are critical for evaluating factual consistency and propose two factual consistency metrics using two different auxiliary tasks. We first integrate the keyphrase weights prediction task to the previous metrics to propose a KPQA (Keyphrase Prediction for Question Answering)-metric for generative QA. Also, we apply question generation and answering to develop a captioning metric QACE (Question Answering for Captioning Evaluation). QACE generates questions on the keywords of the candidate. QACE checks the factual consistency by comparing the answers of these questions for the source image and the caption. Secondly, different from using auxiliary tasks, we directly train a metric with a data-driven approach to propose two metrics. Specifically, we train a metric to distinguish augmented inconsistent texts with the consistent text. We first modify the original reference captions to generate inconsistent captions using several rule-based methods such as substituting keywords to propose UMIC (Unreferenced Metric for Image Captioning). As a next step, we introduce a MFMA (Mask-and-Fill with Masked-Article)-metric by generating inconsistent summary using the masked source and the masked summary. Finally, as an extension of developing data-driven factual consistency metrics, we also propose a faster post-editing system that can fix the factual errors in the system.1 Introduction 1 2 Background 10 2.1 Text Evaluation Metrics 10 2.1.1 N-gram Similarity Metrics 10 2.1.2 Embedding Similarity Metrics 12 2.1.3 Auxiliary Task Based Metrics 12 2.1.4 Entailment Based Metrics 13 2.2 Evaluating Automated Metrics 14 3 Integrating Keyphrase Weights for Factual Consistency Evaluation 15 3.1 Related Work 17 3.2 Proposed Approach: KPQA-Metric 18 3.2.1 KPQA 18 3.2.2 KPQA Metric 19 3.3 Experimental Setup and Dataset 23 3.3.1 Dataset 23 3.3.2 Implementation Details 26 3.4 Empirical Results 27 3.4.1 Comparison with Other Methods 27 3.4.2 Analysis 29 3.5 Conclusion 35 4 Question Generation and Question Answering for Factual Consistency Evaluation 36 4.1 Related Work 37 4.2 Proposed Approach: QACE 38 4.2.1 Question Generation 38 4.2.2 Question Answering 39 4.2.3 Abstractive Visual Question Answering 40 4.2.4 QACE Metric 42 4.3 Experimental Setup and Dataset 43 4.3.1 Dataset 43 4.3.2 Implementation Details 44 4.4 Empirical Results 45 4.4.1 Comparison with Other Methods 45 4.4.2 Analysis 46 4.5 Conclusion 48 5 Rule-Based Inconsistent Data Augmentation for Factual Consistency Evaluation 49 5.1 Related Work 51 5.2 Proposed Approach: UMIC 52 5.2.1 Modeling 52 5.2.2 Negative Samples 53 5.2.3 Contrastive Learning 55 5.3 Experimental Setup and Dataset 56 5.3.1 Dataset 56 5.3.2 Implementation Details 60 5.4 Empirical Results 61 5.4.1 Comparison with Other Methods 61 5.4.2 Analysis 62 5.5 Conclusion 65 6 Inconsistent Data Augmentation with Masked Generation for Factual Consistency Evaluation 66 6.1 Related Work 68 6.2 Proposed Approach: MFMA and MSM 70 6.2.1 Mask-and-Fill with Masked Article 71 6.2.2 Masked Summarization 72 6.2.3 Training Factual Consistency Checking Model 72 6.3 Experimental Setup and Dataset 73 6.3.1 Dataset 73 6.3.2 Implementation Details 74 6.4 Empirical Results 75 6.4.1 Comparison with Other Methods 75 6.4.2 Analysis 78 6.5 Conclusion 84 7 Factual Error Correction for Improving Factual Consistency 85 7.1 Related Work 87 7.2 Proposed Approach: RFEC 88 7.2.1 Problem Formulation 88 7.2.2 Training Dataset Construction 89 7.2.3 Evidence Sentence Retrieval 90 7.2.4 Entity Retrieval Based Factual Error Correction 90 7.3 Experimental Setup and Dataset 92 7.3.1 Dataset 92 7.3.2 Implementation Details 93 7.4 Empirical Results 93 7.4.1 Comparison with Other Methods 93 7.4.2 Analysis 95 7.5 Conclusion 95 8 Conclusion 97 Abstract (In Korean) 118박

Visual Abductive Reasoning Meets Driving Hazard Prediction: Problem Formulation and Dataset

This paper addresses the problem of predicting hazards that drivers may encounter while driving a car. We formulate it as a task of anticipating impending accidents using a single input image captured by car dashcams. Unlike existing approaches to driving hazard prediction that rely on computational simulations or anomaly detection from videos, this study focuses on high-level inference from static images. The problem needs predicting and reasoning about future events based on uncertain observations, which falls under visual abductive reasoning. To enable research in this understudied area, a new dataset named the DHPR (Driving Hazard Prediction and Reasoning) dataset is created. The dataset consists of 15K dashcam images of street scenes, and each image is associated with a tuple containing car speed, a hypothesized hazard description, and visual entities present in the scene. These are annotated by human annotators, who identify risky scenes and provide descriptions of potential accidents that could occur a few seconds later. We present several baseline methods and evaluate their performance on our dataset, identifying remaining issues and discussing future directions. This study contributes to the field by introducing a novel problem formulation and dataset, enabling researchers to explore the potential of multi-modal AI for driving hazard prediction.Comment: Main Paper: 10 pages, Supplementary Materials: 25 page

Augmented reality and its aspects: a case study for heating systems

Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do ParanáThanks to the advances of technology in various domains, and the mixing between real and virtual worlds. Allowed this master’s thesis to explore concepts related to virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR). The development and comparison of Android applications and Microsoft HoloLens aimed to solve a deadlock in the recognition of instructions by the users. We used an interactive manual of assembly and disassembly for taps of residential heaters. Therefore, this work deals with three main parts. Firstly, the exploration of the concepts of VR, AR, MR, and XR. Secondly, 3D modeling and animations techniques. Finally, the development of applications using Vuforia, Wikitude, and MRTK. The users tried our application “HeaterGuideAR” to verify the effectiveness of the instruction passed by the interactive manual. Only a few users had some difficulties at the beginning of the trials. Thus, it was necessary to provide aid tools. However, other users were able to disassemble the faucet without any external help. We suggest continuing this work with more explorations, models, and situations.Graças aos últimos avanços tecnológicos em diversas áreas deram a possibilidade de fazer a mistura do mundo real com o virtual. É com este intuito que esta tese de mestrado veio expor os conceitos relacionados à realidade virtual (RV), realidade aumentada (RA), realidade mista (RM) e realidade estendida (RE). O desenvolvimento e comparação de aplicativos Android e Microsoft HoloLens teve como objetivo resolver um impasse no entendimento de instruções por parte dos usuários. Utilizamos um manual interativo para montagem e desmontagem de torneiras de aquecedores residenciais. Este trabalho, portanto, lida com três partes principais. Na primeira, a exploração dos conceitos de RV, RA, RM e RE. Na segunda, modelagem 3D e técnicas de animações. E por fim, o desenvolvimento de aplicações usando Vuforia, Wikitude e MRTK. A aplicação “HeaterGuideAR” foi testada pelos usuários afim de verificar a eficácia da instrução passada pelo manual interativo. Apenas alguns usuários tiveram algumas dificuldades no início dos testes. Sendo que, foi necessário fornecer algumas ferramentas de auxílio. Mesmo assim, outros usuários conseguiram desmontar a torneira sem ajuda externa. Sugerimos continuar este trabalho com mais explorações, modelos e situações.Mestrado de dupla diplomação com a UTFPR - Universidade Tecnológica Federal do Paran

Outdoor Education and Mobile Learning: an Autobiographical Narrative Using Application-Based Information and Resources

Although mobile learning using smartphones and applications or apps have the potential to inform and educate individuals in an outdoor environment, users may find that connectivity issues and basic knowledge of outdoor environments, including both physical and emotional, could be limited by what this technology provided. This study provided my perspective as both participant and researcher on a journey over 150 miles on the Colorado Trail, using my iPhone as my primary tool for navigation and information for learning how to survive in an outdoor environment. From the beginning, the physical effects were difficult to overcome, but it was the psychological toll that became my greatest obstacle and the one element where mobile learning in the outdoor environment proved to have the greatest value. While this was one perspective, in a single study, by one participant, in which mobile learning in an outdoor environment took place, there were several themes that developed in regards to data connection, the use of fluid apps, the usefulness of static apps, and the restrictions of power in rural mountainous environments. These themes were emphasized to help future researchers further develop this information to help in the continued development of outdoor education using mobile learning

Towards the development of a cost-effective Image-Sensing-Smart-Parking Systems (ISenSmaP)

Finding parking in a busy city has been a major daily problem in today’s busy life. Researchers have proposed various parking spot detection systems to overcome the problem of spending a long time searching for a parking spot. These works include a wide variety of sensors to detect the presence of a vehicle in a parking spot. These approaches are expensive to implement and ineffective in extreme weather conditions in an outdoor parking environment. As a result, a cost-effective, dependable, and time-saving parking solution is much more desirable. In this thesis, we proposed and developed an image processing-based real-time parking-spot detection system using deep-learning algorithms. In this regard, we annotated the images using the Visual Geometry Group (VGG) annotator and preprocessed the dataset using the image contrast enhancement technique that attempts to solve the illumination changes in pictures captured in an open space, followed by training the model using the Mask-R-CNN (Region-Based Convolutional Neural Network) and Faster-RCNN algorithms. ROIs (Regions of interest) are used later to determine the vacancy status of each parking spot. Our experimental results demonstrate the effectiveness of our developed parking systems as we achieved a mean Average Precision (mAP) of 0.999 for the PKLot dataset and a mAP of 0.9758 for custom datasets. Furthermore, as part of the smart parking application, we developed an Android App that can be used by the end users. Our proposed intelligent parking system is scalable, cost-effective, and to the best of our knowledge, it offers higher parking spot detection accuracy than any other solutions in this domain

Digitally augmented sketch-planning

Thesis (M.C.P.)--Massachusetts Institute of Technology, Dept. of Urban Studies and Planning, 2002.Includes bibliographical references (p. 73-74).While many aspects of the planning profession have changed radically in light of recent technological advances, the practice of sketching plans has remained largely unaffected. There may be good reasons for eschewing computers in the design arena such as that their use may detract from the liberty of the design thinking process. This thesis suggests that this reluctance may be overcome by changing the practice from one of emulation with digital tools to one of "augmentation". In addressing a perceived need to bring computation to the design table a solution called the "digitally augmented sketch planning environment" (DASPE) has been developed. Making use of video projection, DASPE augments the design space with digital visualization and analysis tools and allows planners to sketch using either conventional media or a pen stylus on a digitizing table. Plans can be sketched in the conventional manner, then "hardened" into three dimensional computer models without the need to leave the design space.by Kenneth Goulding.M.C.P