Contrastive Classification and Representation Learning with Probabilistic Interpretation

Cross entropy loss has served as the main objective function for classification-based tasks. Widely deployed for learning neural network classifiers, it shows both effectiveness and a probabilistic interpretation. Recently, after the success of self supervised contrastive representation learning methods, supervised contrastive methods have been proposed to learn representations and have shown superior and more robust performance, compared to solely training with cross entropy loss. However, cross entropy loss is still needed to train the final classification layer. In this work, we investigate the possibility of learning both the representation and the classifier using one objective function that combines the robustness of contrastive learning and the probabilistic interpretation of cross entropy loss. First, we revisit a previously proposed contrastive-based objective function that approximates cross entropy loss and present a simple extension to learn the classifier jointly. Second, we propose a new version of the supervised contrastive training that learns jointly the parameters of the classifier and the backbone of the network. We empirically show that our proposed objective functions show a significant improvement over the standard cross entropy loss with more training stability and robustness in various challenging settings

Continual Novelty Detection

Novelty Detection methods identify samples that are not representative of a model's training set thereby flagging misleading predictions and bringing a greater flexibility and transparency at deployment time. However, research in this area has only considered Novelty Detection in the offline setting. Recently, there has been a growing realization in the computer vision community that applications demand a more flexible framework - Continual Learning - where new batches of data representing new domains, new classes or new tasks become available at different points in time. In this setting, Novelty Detection becomes more important, interesting and challenging. This work identifies the crucial link between the two problems and investigates the Novelty Detection problem under the Continual Learning setting. We formulate the Continual Novelty Detection problem and present a benchmark, where we compare several Novelty Detection methods under different Continual Learning settings. We show that Continual Learning affects the behaviour of novelty detection algorithms, while novelty detection can pinpoint insights in the behaviour of a continual learner. We further propose baselines and discuss possible research directions. We believe that the coupling of the two problems is a promising direction to bring vision models into practice

Homography-based ground plane detection using a single on-board camera

This study presents a robust method for ground plane detection in vision-based systems with a non-stationary camera. The proposed method is based on the reliable estimation of the homography between ground planes in successive images. This homography is computed using a feature matching approach, which in contrast to classical approaches to on-board motion estimation does not require explicit ego-motion calculation. As opposed to it, a novel homography calculation method based on a linear estimation framework is presented. This framework provides predictions of the ground plane transformation matrix that are dynamically updated with new measurements. The method is specially suited for challenging environments, in particular traffic scenarios, in which the information is scarce and the homography computed from the images is usually inaccurate or erroneous. The proposed estimation framework is able to remove erroneous measurements and to correct those that are inaccurate, hence producing a reliable homography estimate at each instant. It is based on the evaluation of the difference between the predicted and the observed transformations, measured according to the spectral norm of the associated matrix of differences. Moreover, an example is provided on how to use the information extracted from ground plane estimation to achieve object detection and tracking. The method has been successfully demonstrated for the detection of moving vehicles in traffic environments

Language Model Applications to Spelling with Brain-Computer Interfaces

Within the Ambient Assisted Living (AAL) community, Brain-Computer Interfaces (BCIs) have raised great hopes as they provide alternative communication means for persons with disabilities bypassing the need for speech and other motor activities. Although significant advancements have been realized in the last decade, applications of language models (e.g., word prediction, completion) have only recently started to appear in BCI systems. The main goal of this article is to review the language model applications that supplement non-invasive BCI-based communication systems by discussing their potential and limitations, and to discern future trends. First, a brief overview of the most prominent BCI spelling systems is given, followed by an in-depth discussion of the language models appli

TB165: Chemical and Physical Properties of the Danforth, Elliotsville, Peacham, and Penquis Soil Map Units

The soils reported in this bulletin have developed in several different parent materials. The Danforth soil has developed from very deep, well drained, loose, high coarse fragment till derived from slate and fine-grained metasandstone. The Elliottsville soils have developed in moderately deep, well drained till derived from slates, metasandstones, phyllite and schists. The Penquis soils developed in moderately deep, well drained till of similar lithology as Elliottsville, but with a higher component of weathered and crushable rock fragments throughout the soil profile. Peacham soils are developed in very deep, very poorly drained, dense till derived from phyllite, schist, and granite.https://digitalcommons.library.umaine.edu/aes_techbulletin/1041/thumbnail.jp

Solid Energetic Material Based on Aluminum Micropowder Modified by Microwave Radiation

The paper discusses the application of pulsed microwave radiation for the modification of crystalline components of a high-energy material (HEsM). The model aluminized mixture with increased heat of combustion was studied. The mixture contained 15 wt.% aluminum micron powder, which was modified by microwave irradiation. It was found that the HEM thermogram has an exo-effect with the maximum at 364.3 °C. The use of a modified powder in the HEM composition increased the energy release during combustion by 11% from 5.6 kJ/g to 6.2 kJ/g. The reason for this effect is the increase in the reactivity of aluminum powder after microwave irradiation. In this research, we confirmed that the powders do not lose the stored energy, even as part of the HEM produced on their basis. A laser projection imaging system with brightness amplification was used to estimate the speed of combustion front propagation over the material surface. Measurement of the burning rate revealed a slight difference in the burning rates of HEMs based on irradiated and non-irradiated aluminum micropowders. This property can be demanded in practice, allowing a greater release of energy while maintaining the volume of energetic material

Data S1: Data

We present the evaluation of two well-known, low-cost consumer-grade EEG devices: the Emotiv EPOC and the Neurosky MindWave. Problems with using the consumer-grade EEG devices (BCI illiteracy, poor technical characteristics, and adverse EEG artefacts) are discussed. The experimental evaluation of the devices, performed with 10 subjects asked to perform concentration/relaxation and blinking recognition tasks, is given. The results of statistical analysis show that both devices exhibit high variability and non-normality of attention and meditation data, which makes each of them difficult to use as an input to control tasks. BCI illiteracy may be a significant problem, as well as setting up of the proper environment of the experiment. The results of blinking recognition show that using the Neurosky device means recognition accuracy is less than 50%, while the Emotiv device has achieved a recognition accuracy of more than 75%; for tasks that require concentration and relaxation of subjects, the Emotiv EPOC device has performed better (as measured by the recognition accuracy) by ∼9%. Therefore, the Emotiv EPOC device may be more suitable for control tasks using the attention/meditation level or eye blinking than the Neurosky MindWave device

From Multi-Channel Vision Towards Active Exploration

This thesis is a collection of three studies investigating the multi-channel processing of visual information in biologically-inspired computer vision systems. These three studies are interconnected and supported by an auxiliary work on object recognition.The first study (Chapter 2) is focused on a biologically-inspired multichannel vision approach to independent motion detection (IMD). The goal is to detect objects that move independently from the moving observer. For example, a video camera mounted in a car "sees" a constantly moving environment while the car is driving. In this case, the motion (perceived by the camera) is caused by the self-motion of the car and the independent motion of other objects (e.g., vehicles or pedestrians). The task then is to differentiate the independently moving object (IMOs) from the motion induced by the moving observer in the (static with respect to Earth) environment. In this chapter we propose an approach for IMD, which uses several channels extracted from the input visual stream to create a so-called independent motion (IM) map, which is a map where the intensity of each pixel encodes the likelihood of the pixel being a part of an IMO. Several extensions of the proposed IMD model are presented and described in this study. All these extended models involve an additional appearance-based object recognition channel, which is used to upgrade the representation of the detected independent motion from the pixel-based formto the object-based (set of IMO locations and descriptions) one.In the second study (Chapter 4) we move from the passive exploration of the surrounding world, addressed in the previous study, towards an active exploration. By the active exploration here we mean the ability of the system to move (or, more precisely, rotate) both cameras of the considered stereo setup. As a first step towards a complete active exploration scenario, we considered its simplified case of horizontal vergence control (VC). The goal of the latter is to verge both cameras on the target object. By vergence here we mean the horizontal (pan) rotation of both cameras in opposite directions, which brings the fixation point (intersection of the cameras' optical axes) onto the surface of the target object. The considered here vergence requires only horizontal rotation of both cameras, which can be easily modeled on the given (pan-tilt) robotic head by a symmetric pan-rotation of both cameras in opposite directions, while keeping the common tilt angle fixed. In Chapter 4 we propose and evaluate two neural models for vergence control. Both models use input stereo images to estimate the desired vergence angle (the angle between cameras' optical axes). The first model assumes that the gaze direction of the robotic head is orthogonal to the baseline and that the stimulus is a frontoparallel plane orthogonal to the gaze direction. The second model goes beyond these assumptions and operates reliably in the general case where all restrictions on the orientation of the gaze, as well as the target position, type and orientation, are dropped.In the third study (Chapter 5) we go to the next level of active exploration hierarchy by considering vergence and version eye movements. By the version eye movement we consider the rotational movements of both eyes in the same direction. In this chapter, we propose a novel model, called vergence-version control with attention effects (VVCA), where object recognition is used as a channel for controlling version/vergence eye movements in a biologically-plausible way. Besides purely theoretical (simulated) results, the proposed VVCA model has a real-world embodiment in the form of a robotic setup, working under real-time control of VVCA model, which was adapted specifically for this case (real-time performance).We have also extensively worked on object recognition, the results of which have been employed in all of the studies mentioned. For appearance-based object recognition (used in IMD and VC studies) we involve the well-known recognition paradigm - the convolutional neural network (CNN). In Chapter 3 we present and describe an extended version of CNN, called myCNN, which can be regarded as a fusion of a conventional CNN with hierarchical cortex-like mechanisms.List of Figures List of Tables List of Abbreviations List of Notations 1 General Introduction 1.1 Introduction 1.2 Objectives 1.3 Connection to EU Projects 1.3.1 MCCOOP 1.3.2 DRIVSCO 1.3.3 EYESHOTS 1.4 Thesis structure 2 Independent Motion Detection 2.1 Introduction 2.2 Methods 2.2.1 Binocular disparity 2.2.2 Optic flow 2.2.3 Normalized coordinates 2.2.4 Self-motion 2.2.5 Elevation 2.2.6 Labeling of video sequences 2.2.7 Cue fusion and classification 2.3 Results 2.3.1 Self-motion 2.3.2 Ground plane and elevation 2.3.3 Labeling 2.3.4 Cue fusion and classification 2.4 Discussion and Conclusion 3 Convolutional Neural Network for Object Recognition 3.1 Introduction 3.2 Convolutional neural network 3.2.1 Structure 3.3 Training 3.3.1 Stochastic Diagonal Levenberg-Marquardt 3.3.2 Stochastic Meta-Descent 3.4 Extended convolutional network 3.4.1 M-layer 3.4.2 Implementation 3.5 Applications of convolutional networks 3.5.1 Character recognition 3.5.2 Face detection 3.5.3 Generic object recognition 3.5.4 IMO classification 3.5.5 Vergence control 3.6 Discussion and Conclusion 4 Vergence Control Based on Distributed Disparity 4.1 Introduction 4.2 Methods 4.2.1 Vergence control framework 4.2.2 Vergence performance measures 4.3 Experiments 4.4 Results 4.5 Discussion and Conclusions 5 Vergence-Version Control with Attention Effects 5.1 Introduction 5.1.1 Version eye movements (saccades) 5.1.2 Models of saccades 5.1.3 Visual attention 5.2 Methods 5.2.1 Description of the proposed model 5.2.2 Data processing workflow 5.2.3 Environment module 5.2.4 Robotic head model (RHM) 5.2.5 Disparity representation module (V1) 5.2.6 Object Recognition System (ORS) 5.2.7 Eye Movement System (EMS) 5.3 Results 5.3.1 Object recognition system 5.3.2 Vergence 5.3.3 Full VVCA model demo 5.4 Discussion and Conclusions 6 Conclusions and Future Research Perspectives 6.1 General Discussion on the Main Findings 6.1.1 Independent Motion Detection study 6.1.2 Vergence Control study 6.1.3 Vergence-Version Control with Attention Effects study 6.1.4 Object Recognition study 6.2 Limitations 6.3 Future Research Perspectives 6.3.1 Technological developments 6.3.2 Bio-medical developments A Appendix (Materials) A.1 Software development platforms A.2 Developed software tools A.2.1 myLabel A.2.2 myCNN A.2.3 myRaytracer A.3 Hardware setups A.4 Test car A.4.1 In-car stereo video setup A.4.2 In-car LIDAR sensor setup A.5 Robotic setup Summary Samenvatting (Summary in Dutch) Резюме (Summary in Russian) Bibliography Curriculum Vitaenrpages: 150status: publishe

An Approach to On-Road Vehicle Detection, Description and Tracking

We present an approach to detecting independently moving objects (IMOs) in stereo video sequences acquired by onboard cameras on a moving vehicle. The proposed approach is based on the processing of two independent information streams: an independent motion detection stream and an object recognition stream. Fusion of these streams outputs allows our system to segment IMOs, track them, and even estimate some of their properties.status: publishe