Recent Trends in Computational Intelligence

Traditional models struggle to cope with complexity, noise, and the existence of a changing environment, while Computational Intelligence (CI) offers solutions to complicated problems as well as reverse problems. The main feature of CI is adaptability, spanning the fields of machine learning and computational neuroscience. CI also comprises biologically-inspired technologies such as the intellect of swarm as part of evolutionary computation and encompassing wider areas such as image processing, data collection, and natural language processing. This book aims to discuss the usage of CI for optimal solving of various applications proving its wide reach and relevance. Bounding of optimization methods and data mining strategies make a strong and reliable prediction tool for handling real-life applications

Soft Biometric Analysis: MultiPerson and RealTime Pedestrian Attribute Recognition in Crowded Urban Environments

Traditionally, recognition systems were only based on human hard biometrics. However, the ubiquitous CCTV cameras have raised the desire to analyze human biometrics from far distances, without people attendance in the acquisition process. Highresolution face closeshots are rarely available at far distances such that facebased systems cannot provide reliable results in surveillance applications. Human soft biometrics such as body and clothing attributes are believed to be more effective in analyzing human data collected by security cameras. This thesis contributes to the human soft biometric analysis in uncontrolled environments and mainly focuses on two tasks: Pedestrian Attribute Recognition (PAR) and person reidentification (reid). We first review the literature of both tasks and highlight the history of advancements, recent developments, and the existing benchmarks. PAR and person reid difficulties are due to significant distances between intraclass samples, which originate from variations in several factors such as body pose, illumination, background, occlusion, and data resolution. Recent stateoftheart approaches present endtoend models that can extract discriminative and comprehensive feature representations from people. The correlation between different regions of the body and dealing with limited learning data is also the objective of many recent works. Moreover, class imbalance and correlation between human attributes are specific challenges associated with the PAR problem. We collect a large surveillance dataset to train a novel gender recognition model suitable for uncontrolled environments. We propose a deep residual network that extracts several posewise patches from samples and obtains a comprehensive feature representation. In the next step, we develop a model for multiple attribute recognition at once. Considering the correlation between human semantic attributes and class imbalance, we respectively use a multitask model and a weighted loss function. We also propose a multiplication layer on top of the backbone features extraction layers to exclude the background features from the final representation of samples and draw the attention of the model to the foreground area. We address the problem of person reid by implicitly defining the receptive fields of deep learning classification frameworks. The receptive fields of deep learning models determine the most significant regions of the input data for providing correct decisions. Therefore, we synthesize a set of learning data in which the destructive regions (e.g., background) in each pair of instances are interchanged. A segmentation module determines destructive and useful regions in each sample, and the label of synthesized instances are inherited from the sample that shared the useful regions in the synthesized image. The synthesized learning data are then used in the learning phase and help the model rapidly learn that the identity and background regions are not correlated. Meanwhile, the proposed solution could be seen as a data augmentation approach that fully preserves the label information and is compatible with other data augmentation techniques. When reid methods are learned in scenarios where the target person appears with identical garments in the gallery, the visual appearance of clothes is given the most importance in the final feature representation. Clothbased representations are not reliable in the longterm reid settings as people may change their clothes. Therefore, developing solutions that ignore clothing cues and focus on identityrelevant features are in demand. We transform the original data such that the identityrelevant information of people (e.g., face and body shape) are removed, while the identityunrelated cues (i.e., color and texture of clothes) remain unchanged. A learned model on the synthesized dataset predicts the identityunrelated cues (shortterm features). Therefore, we train a second model coupled with the first model and learns the embeddings of the original data such that the similarity between the embeddings of the original and synthesized data is minimized. This way, the second model predicts based on the identityrelated (longterm) representation of people. To evaluate the performance of the proposed models, we use PAR and person reid datasets, namely BIODI, PETA, RAP, Market1501, MSMTV2, PRCC, LTCC, and MIT and compared our experimental results with stateoftheart methods in the field. In conclusion, the data collected from surveillance cameras have low resolution, such that the extraction of hard biometric features is not possible, and facebased approaches produce poor results. In contrast, soft biometrics are robust to variations in data quality. So, we propose approaches both for PAR and person reid to learn discriminative features from each instance and evaluate our proposed solutions on several publicly available benchmarks.This thesis was prepared at the University of Beria Interior, IT Instituto de Telecomunicações, Soft Computing and Image Analysis Laboratory (SOCIA Lab), Covilhã Delegation, and was submitted to the University of Beira Interior for defense in a public examination session

Visual Analysis of Extremely Dense Crowded Scenes

Visual analysis of dense crowds is particularly challenging due to large number of individuals, occlusions, clutter, and fewer pixels per person which rarely occur in ordinary surveillance scenarios. This dissertation aims to address these challenges in images and videos of extremely dense crowds containing hundreds to thousands of humans. The goal is to tackle the fundamental problems of counting, detecting and tracking people in such images and videos using visual and contextual cues that are automatically derived from the crowded scenes. For counting in an image of extremely dense crowd, we propose to leverage multiple sources of information to compute an estimate of the number of individuals present in the image. Our approach relies on sources such as low confidence head detections, repetition of texture elements (using SIFT), and frequency-domain analysis to estimate counts, along with confidence associated with observing individuals, in an image region. Furthermore, we employ a global consistency constraint on counts using Markov Random Field which caters for disparity in counts in local neighborhoods and across scales. We tested this approach on crowd images with the head counts ranging from 94 to 4543 and obtained encouraging results. Through this approach, we are able to count people in images of high-density crowds unlike previous methods which are only applicable to videos of low to medium density crowded scenes. However, the counting procedure just outputs a single number for a large patch or an entire image. With just the counts, it becomes difficult to measure the counting error for a query image with unknown number of people. For this, we propose to localize humans by finding repetitive patterns in the crowd image. Starting with detections from an underlying head detector, we correlate them within the image after their selection through several criteria: in a pre-defined grid, locally, or at multiple scales by automatically finding the patches that are most representative of recurring patterns in the crowd image. Finally, the set of generated hypotheses is selected using binary integer quadratic programming with Special Ordered Set (SOS) Type 1 constraints. Human Detection is another important problem in the analysis of crowded scenes where the goal is to place a bounding box on visible parts of individuals. Primarily applicable to images depicting medium to high density crowds containing several hundred humans, it is a crucial pre-requisite for many other visual tasks, such as tracking, action recognition or detection of anomalous behaviors, exhibited by individuals in a dense crowd. For detecting humans, we explore context in dense crowds in the form of locally-consistent scale prior which captures the similarity in scale in local neighborhoods with smooth variation over the image. Using the scale and confidence of detections obtained from an underlying human detector, we infer scale and confidence priors using Markov Random Field. In an iterative mechanism, the confidences of detections are modified to reflect consistency with the inferred priors, and the priors are updated based on the new detections. The final set of detections obtained are then reasoned for occlusion using Binary Integer Programming where overlaps and relations between parts of individuals are encoded as linear constraints. Both human detection and occlusion reasoning in this approach are solved with local neighbor-dependent constraints, thereby respecting the inter-dependence between individuals characteristic to dense crowd analysis. In addition, we propose a mechanism to detect different combinations of body parts without requiring annotations for individual combinations. Once human detection and localization is performed, we then use it for tracking people in dense crowds. Similar to the use of context as scale prior for human detection, we exploit it in the form of motion concurrence for tracking individuals in dense crowds. The proposed method for tracking provides an alternative and complementary approach to methods that require modeling of crowd flow. Simultaneously, it is less likely to fail in the case of dynamic crowd flows and anomalies by minimally relying on previous frames. The approach begins with the automatic identification of prominent individuals from the crowd that are easy to track. Then, we use Neighborhood Motion Concurrence to model the behavior of individuals in a dense crowd, this predicts the position of an individual based on the motion of its neighbors. When the individual moves with the crowd flow, we use Neighborhood Motion Concurrence to predict motion while leveraging five-frame instantaneous flow in case of dynamically changing flow and anomalies. All these aspects are then embedded in a framework which imposes hierarchy on the order in which positions of individuals are updated. The results are reported on eight sequences of medium to high density crowds and our approach performs on par with existing approaches without learning or modeling patterns of crowd flow. We experimentally demonstrate the efficacy and reliability of our algorithms by quantifying the performance of counting, localization, as well as human detection and tracking on new and challenging datasets containing hundreds to thousands of humans in a given scene

적분 및 매개변수 기법 융합을 이용한 스마트폰 다중 동작에서 보행 항법

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 박찬국.In this dissertation, an IA-PA fusion-based PDR (Pedestrian Dead Reckoning) using low-cost inertial sensors is proposed to improve the indoor position estimation. Specifically, an IA (Integration Approach)-based PDR algorithm combined with measurements from PA (Parametric Approach) is constructed so that the algorithm is operated even in various poses that occur when a pedestrian moves with a smartphone indoors. In addition, I propose an algorithm that estimates the device attitude robustly in a disturbing situation by an ellipsoidal method. In addition, by using the machine learning-based pose recognition, it is possible to improve the position estimation performance by varying the measurement update according to the poses. First, I propose an adaptive attitude estimation based on ellipsoid technique to accurately estimate the direction of movement of a smartphone device. The AHRS (Attitude and Heading Reference System) uses an accelerometer and a magnetometer as measurements to calculate the attitude based on the gyro and to compensate for drift caused by gyro sensor errors. In general, the attitude estimation performance is poor in acceleration and geomagnetic disturbance situations, but in order to effectively improve the estimation performance, this dissertation proposes an ellipsoid-based adaptive attitude estimation technique. When a measurement disturbance comes in, it is possible to update the measurement more accurately than the adaptive estimation technique without considering the direction by adjusting the measurement covariance with the ellipsoid method considering the direction of the disturbance. In particular, when the disturbance only comes in one axis, the proposed algorithm can use the measurement partly by updating the other two axes considering the direction. The proposed algorithm shows its effectiveness in attitude estimation under disturbances through the rate table and motion capture equipment. Next, I propose a PDR algorithm that integrates IA and PA that can be operated in various poses. When moving indoors using a smartphone, there are many degrees of freedom, so various poses such as making a phone call, texting, and putting a pants pocket are possible. In the existing smartphone-based positioning algorithms, the position is estimated based on the PA, which can be used only when the pedestrian's walking direction and the device's direction coincide, and if it does not, the position error due to the mismatch in angle is large. In order to solve this problem, this dissertation proposes an algorithm that constructs state variables based on the IA and uses the position vector from the PA as a measurement. If the walking direction and the device heading do not match based on the pose recognized through machine learning technique, the position is updated in consideration of the direction calculated using PCA (Principal Component Analysis) and the step length obtained through the PA. It can be operated robustly even in various poses that occur. Through experiments considering various operating conditions and paths, it is confirmed that the proposed method stably estimates the position and improves performance even in various indoor environments.본 논문에서는 저가형 관성센서를 이용한 보행항법시스템 (PDR: Pedestrian Dead Reckoning)의 성능 향상 알고리즘을 제안한다. 구체적으로 보행자가 실내에서 스마트폰을 들고 이동할 때 발생하는 다양한 동작 상황에서도 운용될 수 있도록, 매개변수 기반 측정치를 사용하는 적분 기반의 보행자 항법 알고리즘을 구성한다. 또한 타원체 기반 자세 추정 알고리즘을 구성하여 외란 상황에서도 강인하게 자세를 추정하는 알고리즘을 제안한다. 추가적으로 기계학습 기반의 동작 인식 정보를 이용, 동작에 따른 측정치 업데이트를 달리함으로써 위치 추정 성능을 향상시킨다. 먼저 스마트폰 기기의 이동 방향을 정확하게 추정하기 위해 타원체 기법 기반 적응 자세 추정을 제안한다. 자세 추정 기법 (AHRS: Attitude and Heading Reference System)은 자이로를 기반으로 자세를 계산하고 자이로 센서오차에 의해 발생하는 드리프트를 보정하기 위해 측정치로 가속도계와 지자계를 사용한다. 일반적으로 가속 및 지자계 외란 상황에서는 자세 추정 성능이 떨어지는데, 추정 성능을 효과적으로 향상시키기 위해 본 논문에서는 타원체 기반 적응 자세 추정 기법을 제안한다. 측정치 외란이 들어오는 경우, 외란의 방향을 고려하여 타원체 기법으로 측정치 공분산을 조정해줌으로써 방향을 고려하지 않은 적응 추정 기법보다 정확하게 측정치 업데이트를 할 수 있다. 특히 외란이 한 축으로만 들어오는 경우, 제안한 알고리즘은 방향을 고려해 나머지 두 축에 대해서는 업데이트 해줌으로써 측정치를 부분적으로 사용할 수 있다. 레이트 테이블, 모션 캡쳐 장비를 통해 제안한 알고리즘의 자세 성능이 향상됨을 확인하였다. 다음으로 다양한 동작에서도 운용 가능한 적분 및 매개변수 기법을 융합하는 보행항법 알고리즘을 제안한다. 스마트폰을 이용해 실내를 이동할 때에는 자유도가 크기 때문에 전화 걸기, 문자, 바지 주머니 넣기 등 다양한 동작이 발생 가능하다. 기존의 스마트폰 기반 보행 항법에서는 매개변수 기법을 기반으로 위치를 추정하는데, 이는 보행자의 진행 방향과 기기의 방향이 일치하는 경우에만 사용 가능하며 일치하지 않는 경우 자세 오차로 인한 위치 오차가 크게 발생한다. 이러한 문제를 해결하기 위해 본 논문에서는 적분 기반 기법을 기반으로 상태변수를 구성하고 매개변수 기법을 통해 나오는 위치 벡터를 측정치로 사용하는 알고리즘을 제안한다. 만약 기계학습을 통해 인식한 동작을 바탕으로 진행 방향과 기기 방향이 일치하지 않는 경우, 주성분 분석을 통해 계산한 진행방향을 이용해 진행 방향을, 매개변수 기법을 통해 얻은 보폭으로 거리를 업데이트해 줌으로써 보행 중 발생하는 여러 동작에서도 강인하게 운용할 수 있다. 다양한 동작 상황 및 경로를 고려한 실험을 통해 위에서 제안한 방법이 다양한 실내 환경에서도 안정적으로 위치를 추정하고 성능이 향상됨을 확인하였다.Chapter 1 Introduction 1 1.1 Motivation and Background 1 1.2 Objectives and Contribution 5 1.3 Organization of the Dissertation 6 Chapter 2 Pedestrian Dead Reckoning System 8 2.1 Overview of Pedestrian Dead Reckoning 8 2.2 Parametric Approach 9 2.2.1 Step detection algorithm 11 2.2.2 Step length estimation algorithm 13 2.2.3 Heading estimation 14 2.3 Integration Approach 15 2.3.1 Extended Kalman filter 16 2.3.2 INS-EKF-ZUPT 19 2.4 Activity Recognition using Machine Learning 21 2.4.1 Challenges in HAR 21 2.4.2 Activity recognition chain 22 Chapter 3 Attitude Estimation in Smartphone 26 3.1 Adaptive Attitude Estimation in Smartphone 26 3.1.1 Indirect Kalman filter-based attitude estimation 26 3.1.2 Conventional attitude estimation algorithms 29 3.1.3 Adaptive attitude estimation using ellipsoidal methods 30 3.2 Experimental Results 36 3.2.1 Simulation 36 3.2.2 Rate table experiment 44 3.2.3 Handheld rotation experiment 46 3.2.4 Magnetic disturbance experiment 49 3.3 Summary 53 Chapter 4 Pedestrian Dead Reckoning in Multiple Poses of a Smartphone 54 4.1 System Overview 55 4.2 Machine Learning-based Pose Classification 56 4.2.1 Training dataset 57 4.2.2 Feature extraction and selection 58 4.2.3 Pose classification result using supervised learning in PDR 62 4.3 Fusion of the Integration and Parametric Approaches in PDR 65 4.3.1 System model 67 4.3.2 Measurement model 67 4.3.3 Mode selection 74 4.3.4 Observability analysis 76 4.4 Experimental Results 82 4.4.1 AHRS results 82 4.4.2 PCA results 84 4.4.3 IA-PA results 88 4.5 Summary 100 Chapter 5 Conclusions 103 5.1 Summary of the Contributions 103 5.2 Future Works 105 국문초록 125 Acknowledgements 127Docto

The Hanabi Challenge: A New Frontier for AI Research

From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.Comment: 32 pages, 5 figures, In Press (Artificial Intelligence

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Algorithms for Autonomous Personal Navigation Systems

Personal positioning is a challenging topic in the area of navigation mainly because of the cost, size and power consumption constraints imposed on the hardware. Satellite based positioning techniques can meet the requirements for many applications, but cover well only outdoor environment. Problems like weak satellite signals make the positioning impossible indoors. Urban canyons are also difficult areas for GNSS based navigation because of large multipath errors and satellite signal outages. Many applications require seamless positioning in all environments. However, there is no overall solution for navigation in GNSS denied environment, which is reliable, accurate, cost effective and quickly installed. Recently developed systems for indoor positioning often require pre-installed infrastructure. Another approach is to use fully autonomous navigation systems based on self-contained sensors and street or indoor maps. This thesis is concerned with autonomous personal navigation devices, which do not rely on the reception of external information, like satellite or terrestrial signals. The three proposed algorithms can be integrated into personal navigation systems. The first algorithm computes positioning for a map aided navigation system designed for land vehicles traveling on road network. The novelty is in application of particle filtering to vehicle navigation using road network database. The second algorithm is aimed at map aided vehicle navigation indoors. The novelty is in the method for correction of position and heading. The third algorithm computes solution for pedestrian navigation system, which is based on body mounted inertial measurement unit and models of human gait