Sim2real and Digital Twins in Autonomous Driving: A Survey

Safety and cost are two important concerns for the development of autonomous driving technologies. From the academic research to commercial applications of autonomous driving vehicles, sufficient simulation and real world testing are required. In general, a large scale of testing in simulation environment is conducted and then the learned driving knowledge is transferred to the real world, so how to adapt driving knowledge learned in simulation to reality becomes a critical issue. However, the virtual simulation world differs from the real world in many aspects such as lighting, textures, vehicle dynamics, and agents' behaviors, etc., which makes it difficult to bridge the gap between the virtual and real worlds. This gap is commonly referred to as the reality gap (RG). In recent years, researchers have explored various approaches to address the reality gap issue, which can be broadly classified into two categories: transferring knowledge from simulation to reality (sim2real) and learning in digital twins (DTs). In this paper, we consider the solutions through the sim2real and DTs technologies, and review important applications and innovations in the field of autonomous driving. Meanwhile, we show the state-of-the-arts from the views of algorithms, models, and simulators, and elaborate the development process from sim2real to DTs. The presentation also illustrates the far-reaching effects of the development of sim2real and DTs in autonomous driving

FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning

Semi-Supervised Learning (SSL) has been an effective way to leverage abundant unlabeled data with extremely scarce labeled data. However, most SSL methods are commonly based on instance-wise consistency between different data transformations. Therefore, the label guidance on labeled data is hard to be propagated to unlabeled data. Consequently, the learning process on labeled data is much faster than on unlabeled data which is likely to fall into a local minima that does not favor unlabeled data, leading to sub-optimal generalization performance. In this paper, we propose FlatMatch which minimizes a cross-sharpness measure to ensure consistent learning performance between the two datasets. Specifically, we increase the empirical risk on labeled data to obtain a worst-case model which is a failure case that needs to be enhanced. Then, by leveraging the richness of unlabeled data, we penalize the prediction difference (i.e., cross-sharpness) between the worst-case model and the original model so that the learning direction is beneficial to generalization on unlabeled data. Therefore, we can calibrate the learning process without being limited to insufficient label information. As a result, the mismatched learning performance can be mitigated, further enabling the effective exploitation of unlabeled data and improving SSL performance. Through comprehensive validation, we show FlatMatch achieves state-of-the-art results in many SSL settings.Comment: NeurIPS 202

Visual Tracking by Sampling in Part Space

In this paper, we present a novel part-based visual tracking method from the perspective of probability sampling. Specifically, we represent the target by a part space with two online learned probabilities to capture the structure of the target. The proposal distribution memorizes the historical performance of different parts, and it is used for the first round of part selection. The acceptance probability validates the specific tracking stability of each part in a frame, and it determines whether to accept its vote or to reject it. By doing this, we transform the complex online part selection problem into a probability learning one, which is easier to tackle. The observation model of each part is constructed by an improved supervised descent method and is learned in an incremental manner. Experimental results on two benchmarks demonstrate the competitive performance of our tracker against state-of-the-art methods

Poly[[tri-μ3-hydroxido-tris­(μ4-pyridine-2,5-dicarboxyl­ato)trineodymium(III)] monohydrate]

In the title compound, {[Nd3(C7H3NO4)3(OH)3]·H2O}n, the NdIII atom is eight-coordinated by the three O atoms of three asymmetrically μ3-bridging hydroxide groups, by four carboxyl­ate O atoms of four different pyridine-2,5-dicarboxyl­ate (2,5-pydc) ligands, and by the N atom of a 2,5-pydc ligand. Six Nd atoms are connected by six hydroxide groups, forming an [Nd6(μ3-OH)6] cluster unit of symmetry -3 and a slightly compressed octa­hedral geometry. Adjacent [Nd6(μ3-OH)6] clusters are connected by the 2,5-pydc ligands, via O and N atoms, forming chains along the c axis. The remaining O atoms of the 2,5-pydc ligands link these chains into a three-dimensional framework. A disordered water molecule, located on a threefold rotation axis at the opposite side of the [Nd6(μ3-OH)6] cluster and exposed to each of the three Nd atoms, completes the structure

Precision calculations of Bd,s→π,KB_{d, s} \to \pi, K decay form factors in soft-collinear effective theory

We improve QCD calculations of the $B_{d, s} \to \pi, K$ form factors at large hadronic recoil by implementing the next-to-leading-logarithmic resummation for the obtained leading-power light-cone sum rules in the soft-collinear effective theory (SCET) framework. Additionally, we endeavour to investigate a variety of the subleading-power contributions to these heavy-to-light form factors at ${\cal O}(\alpha_s^0)$, by including the higher-order terms in the heavy-quark expansion of the hard-collinear quark propagator, by evaluating the desired effective matrix element of the next-to-leading-order term in the ${\rm SCET_{I}}$ representation of the weak transition current, by taking into account the off-light-cone contributions of the two-body heavy-quark effective theory matrix elements as well as the three-particle higher-twist corrections from the subleading bottom-meson light-cone distribution amplitudes, and by computing the twist-five and twist-six four-body higher-twist effects with the aid of the factorization approximation. Having at our disposal the SCET sum rules for the exclusive $B$-meson decay form factors, we further explore in detail numerical implications of the newly computed subleading-power corrections by employing the three-parameter model for both the leading-twist and higher-twist $B$-meson distribution amplitudes. Taking advantage of the customary Bourrely-Caprini-Lellouch parametrization for the semileptonic $B_{d, s} \to \pi, K$ form factors, we then determine the correlated numerical results for the interesting series coefficients, by carrying out the simultaneous fit of the exclusive $B$-meson decay form factors to both the achieved SCET sum rule predictions and the available lattice QCD results.Comment: 74 pages, 15 figure

Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization

Out-of-Distribution (OOD) Generalization aims to learn robust models that generalize well to various environments without fitting to distribution-specific features. Recent studies based on Lottery Ticket Hypothesis (LTH) address this problem by minimizing the learning target to find some of the parameters that are critical to the task. However, in OOD problems, such solutions are suboptimal as the learning task contains severe distribution noises, which can mislead the optimization process. Therefore, apart from finding the task-related parameters (i.e., invariant parameters), we propose Exploring Variant parameters for Invariant Learning (EVIL) which also leverages the distribution knowledge to find the parameters that are sensitive to distribution shift (i.e., variant parameters). Once the variant parameters are left out of invariant learning, a robust subnetwork that is resistant to distribution shift can be found. Additionally, the parameters that are relatively stable across distributions can be considered invariant ones to improve invariant learning. By fully exploring both variant and invariant parameters, our EVIL can effectively identify a robust subnetwork to improve OOD generalization. In extensive experiments on integrated testbed: DomainBed, EVIL can effectively and efficiently enhance many popular methods, such as ERM, IRM, SAM, etc.Comment: 27 pages, 9 figure

Discriminative tracking using tensor pooling

How to effectively organize local descriptors to build a global representation has a critical impact on the performance of vision tasks. Recently, local sparse representation has been successfully applied to visual tracking, owing to its discriminative nature and robustness against local noise and partial occlusions. Local sparse codes computed with a template actually form a three-order tensor according to their original layout, although most existing pooling operators convert the codes to a vector by concatenating or computing statistics on them. We argue that, compared to pooling vectors, the tensor form could deliver more intrinsic structural information for the target appearance, and can also avoid high dimensionality learning problems suffered in concatenation-based pooling methods. Therefore, in this paper, we propose to represent target templates and candidates directly with sparse coding tensors, and build the appearance model by incrementally learning on these tensors. We propose a discriminative framework to further improve robustness of our method against drifting and environmental noise. Experiments on a recent comprehensive benchmark indicate that our method performs better than state-of-the-art trackers

Analysis of carrier injection under high temperature AC operation in top gate IGZO TFTs

Abstract– With the development of high-quality displays, metal oxides gradually become a popular active layer in TFTs [1]. In this work, InGaZnO thin film transistors with double-layer oxide are investigated. The oxide layer is divided into top and bottom layers. We improve the characteristics and reliability of the device through the design of double-layer oxide stack structure. The bottom oxide layer is deposited with a lower SiH4 flow rate, and the top oxide layer is deposited with a higher SiH4 flow rate. By increasing the SiH4 flow rate of the top oxide layer, two effects can be achieved. Firstly, it is beneficial for speeding up the film deposition process. Furthermore, the hydrogen residue passivates the dangling bonds in the oxide layer and increases the bonding amount of silanol groups, SiO-H, and achieve hydrogen channel doping [2]. By modulating the SiH4 flow rate of the top oxide layer, the basic characteristics of the devices and the reliability under alternating current (AC) operation are improved. In this work, we use three waveform types of switch process to analyze the degradation under AC stress, and the physic mechanism is proposed subsequently [3-4]. After AC stress, the top oxide layer with higher SiH4 flow rate has a smaller threshold voltage right shift, and the reliability is significantly improved. Please click Download on the upper right corner to see the full abstract