How do Cross-View and Cross-Modal Alignment Affect Representations in Contrastive Learning?

Various state-of-the-art self-supervised visual representation learning approaches take advantage of data from multiple sensors by aligning the feature representations across views and/or modalities. In this work, we investigate how aligning representations affects the visual features obtained from cross-view and cross-modal contrastive learning on images and point clouds. On five real-world datasets and on five tasks, we train and evaluate 108 models based on four pretraining variations. We find that cross-modal representation alignment discards complementary visual information, such as color and texture, and instead emphasizes redundant depth cues. The depth cues obtained from pretraining improve downstream depth prediction performance. Also overall, cross-modal alignment leads to more robust encoders than pre-training by cross-view alignment, especially on depth prediction, instance segmentation, and object detection

Hearing What You Cannot See: Acoustic Vehicle Detection Around Corners

This work proposes to use passive acoustic perception as an additional sensing modality for intelligent vehicles. We demonstrate that approaching vehicles behind blind corners can be detected by sound before such vehicles enter in line-of-sight. We have equipped a research vehicle with a roof-mounted microphone array, and show on data collected with this sensor setup that wall reflections provide information on the presence and direction of occluded approaching vehicles. A novel method is presented to classify if and from what direction a vehicle is approaching before it is visible, using as input Direction-of-Arrival features that can be efficiently computed from the streaming microphone array data. Since the local geometry around the ego-vehicle affects the perceived patterns, we systematically study several environment types, and investigate generalization across these environments. With a static ego-vehicle, an accuracy of 0.92 is achieved on the hidden vehicle classification task. Compared to a state-of-the-art visual detector, Faster R-CNN, our pipeline achieves the same accuracy more than one second ahead, providing crucial reaction time for the situations we study. While the ego-vehicle is driving, we demonstrate positive results on acoustic detection, still achieving an accuracy of 0.84 within one environment type. We further study failure cases across environments to identify future research directions.Comment: Accepted to IEEE Robotics & Automation Letters (2021), DOI: 10.1109/LRA.2021.3062254. Code, Data & Video: https://github.com/tudelft-iv/occluded_vehicle_acoustic_detectio

Transformer-Based Neural Surrogate for Link-Level Path Loss Prediction from Variable-Sized Maps

Estimating path loss for a transmitter-receiver location is key to many use-cases including network planning and handover. Machine learning has become a popular tool to predict wireless channel properties based on map data. In this work, we present a transformer-based neural network architecture that enables predicting link-level properties from maps of various dimensions and from sparse measurements. The map contains information about buildings and foliage. The transformer model attends to the regions that are relevant for path loss prediction and, therefore, scales efficiently to maps of different size. Further, our approach works with continuous transmitter and receiver coordinates without relying on discretization. In experiments, we show that the proposed model is able to efficiently learn dominant path losses from sparse training data and generalizes well when tested on novel maps.Comment: Accepted at IEEE GLOBECOM 2023, v2: Changed license on arxi

Magnetoresistance, Micromagnetism, and Domain Wall Scattering in Epitaxial hcp Co Films

Large negative magnetoresistance (MR) observed in transport measurements of hcp Co films with stripe domains were recently reported and interpreted in terms of a novel domain wall (DW) scattering mechanism. Here detailed MR measurements, magnetic force microscopy, and micromagnetic calculations are combined to elucidate the origin of MR in this material. The large negative room temperature MR reported previously is shown to be due to ferromagnetic resistivity anisotropy. Measurements of the resistivity for currents parallel (CIW) and perpendicular to DWs (CPW) have been conducted as a function of temperature. Low temperature results show that any intrinsic effect of DWs scattering on MR of this material is very small compared to the anisotropic MR.Comment: 5 pages, 5 Figures, submitted to PR

The FLUXNET2015 dataset and the ONEFlux processing pipeline for eddy covariance data

The FLUXNET2015 dataset provides ecosystem-scale data on CO2, water, and energy exchange between the biosphere and the atmosphere, and other meteorological and biological measurements, from 212 sites around the globe (over 1500 site-years, up to and including year 2014). These sites, independently managed and operated, voluntarily contributed their data to create global datasets. Data were quality controlled and processed using uniform methods, to improve consistency and intercomparability across sites. The dataset is already being used in a number of applications, including ecophysiology studies, remote sensing studies, and development of ecosystem and Earth system models. FLUXNET2015 includes derived-data products, such as gap-filled time series, ecosystem respiration and photosynthetic uptake estimates, estimation of uncertainties, and metadata about the measurements, presented for the first time in this paper. In addition, 206 of these sites are for the first time distributed under a Creative Commons (CC-BY 4.0) license. This paper details this enhanced dataset and the processing methods, now made available as open-source codes, making the dataset more accessible, transparent, and reproducible.Peer reviewe

Magnetoresistance in an amorphous exchange-coupled bilayer

International audienceThe effect of a magnetic domain wall on the electronic transport in disordered materials is studied in an exchange-coupled amorphous Gd 40 Fe 60 / Gd 10 Fe 90 bilayer. In this amorphous system, the size and the shape of an interfacial domain wall is controlled by an external magnetic field. Current-in-plane transport measurements are performed on single GdFe layers, Gd 40 Fe 60 / Gd 10 Fe 90 bilayer, and on a Gd 40 Fe 60 / Si/ Gd 10 Fe 90 trilayer where the Si layer prevents the formation of the interfacial magnetic domain wall. Different contributions to the resistance are evidenced. In all types of samples, a linear positive magnetoresistance contribution is observed at high field which can be linked to the amorphous structure of the GdFe alloys. The comparison between the bilayer and the trilayer allows to eliminate this contribution and evidences that anisotropic mag-netoresistance is the main effect induced by the interfacial domain wall. Beyond the anisotropic magnetore-sistance signal, a supplementary negative magnetoresistance is evidenced. The origin of this effect is discussed qualitatively using previous theoretical predictions on magnetotransport through a magnetic domain wall in disordered metals

Extraordinary Hall effect based magnetic logic applications

International audienceExtraordinary Hall Effect (EHE) based original concepts of a reconfigurable logic gate and a multibitlogic comparator are presented. They exploit the EHE voltage that develops on cross cells connectedin series that has no size limitation down to the nanometer scale. Experimental demonstrationsare performed on both micro- and nanometer lateral size crosses made of ferrimagnetic TbCo alloy.The simplicity of the device architecture and its robustness make it advantageous when comparedwith existing systems