Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling

Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds

Premiers états du 15F étudiés par diffusion élastique résonante

Cette thèse s inscrit dans le cadre de l étude des noyaux déficients en neutrons situés au dela de la drip-line proton. Afin d étudier le noyau de 15F, une expérience de diffusion élastique résonante d un faisceau d 14O sur une cible de protons, 14O(p,14O)p, a été réalisée au GANIL avec un faisceau post-accéléré SPIRAL. La fonction d excitation du 15F a été mesurée jusqu a 5.6MeV d énergie de résonance et a révélé la présence de trois états résonants, dont un état étroit de parité négative observé pour la première fois via la diffusion élastique résonante. Cet état a éte analysé comme disposant de deux voies de décroissance, l une correspondant à l émission proton et l autre interprétée comme une émission (p-gamma) procédant par un état virtuel de l 14O. L interprétation des résultats en terme de structure du noyau et l existence d une décroissance deux proton ou (p-gamma) est discutée.This thesis describes a study of a light proton-rich nuclei located beyond drip-line :15F. It was populated using a resonant elastic scattering reaction in inverse kinematic 14O(p,14O)p. A 14O beam was sent on a thick polyprophylene target. This experiment was done at GANIL using a post-accelerated radioactive beam from SPIRAL facility. Excitation fonction was measured until 5.6MeV resonance energy and showed three resonant states. One of them was a negative parity state, observed for the first time. Two decay channels were found for to this state, one corresponding to the proton-gamma decay, and the other suggested as a (p) decay via a virtual state of 14O. Results are discussed in the framework of nuclear structure and presence of two-protons or (p-gamma) decay.CAEN-BU Sciences et STAPS (141182103) / SudocSTRASBOURG-Bib.Central Recherche (674822133) / SudocSudocFranceF

Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling

Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds

Conformal Prediction for Trustworthy Detection of Railway Signals

We present an application of conformal prediction, a form of uncertainty quantification with guarantees, to the detection of railway signals. State-of-the-art architectures are tested and the most promising one undergoes the process of conformalization, where a correction is applied to the predicted bounding boxes (i.e. to their height and width) such that they comply with a predefined probability of success. We work with a novel exploratory dataset of images taken from the perspective of a train operator, as a first step to build and validate future trustworthy machine learning models for the detection of railway signals

Conformal Prediction for Trustworthy Detection of Railway Signals

We present an application of conformal prediction, a form of uncertainty quantification with guarantees, to the detection of railway signals. State-of-the-art architectures are tested and the most promising one undergoes the process of conformalization, where a correction is applied to the predicted bounding boxes (i.e. to their height and width) such that they comply with a predefined probability of success. We work with a novel exploratory dataset of images taken from the perspective of a train operator, as a first step to build and validate future trustworthy machine learning models for the detection of railway signals

Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling

Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds

Toward the certification of safety-related systems using ML techniques: the ACAS-Xu experience

International audienceIn the context of the use of Machine Learning (ML) techniques in the development of safety-critical applications for both airborne and ground aeronautical products, this paper proposes elements of reasoning for a conformity to the future industrial standard. Indeed, this contribution is based on the EUROCAE WG-114/SAE G-34 ongoing standardization work that will produce the guidance to support the future certification/approval objectives. The proposed argumentation is structured using assurance case patterns that will support the demonstration of compliance with assurance objectives of the new standard. At last, these patterns are applied to the ACAS-Xu use case to contribute to a future conformity demonstration using evidences from ML development process outputs. Disclaimer: This paper is based on the EUROCAE WG-114/SAE G-34 standardization results at the time of the writing. Though some of the authors are active members of the working group, it is a free interpretation of the current draft work and only reflects the authors' view. As the working group has not published any released outcomes yet, some parts of the described argumentation may have to be modified in the future to conform to the final standard objectives

Object Detection With Probabilistic Guarantees: a Conformal Prediction Approach

This preprint has not undergone peer review or any post-submission improvements or corrections. The Version of Record of this contribution will appear in SAFECOMP 2022, LNCS 13415 proceedings.International audienceProviding reliable uncertainty quantification for complex visual tasks such as object detection is of utmost importance for safety-critical applications such as autonomous driving, tumor detection, etc. Conformal prediction methods offer simple yet practical means to build uncertainty estimations that come with probabilistic guarantees. In this paper we apply such methods to the task of object localization and illustrate our analysis on a pedestrian detection use-case. We highlight both theoretical and practical implications of our analysis

Object Detection With Probabilistic Guarantees: a Conformal Prediction Approach

This preprint has not undergone peer review or any post-submission improvements or corrections. The Version of Record of this contribution will appear in SAFECOMP 2022, LNCS 13415 proceedings.International audienceProviding reliable uncertainty quantification for complex visual tasks such as object detection is of utmost importance for safety-critical applications such as autonomous driving, tumor detection, etc. Conformal prediction methods offer simple yet practical means to build uncertainty estimations that come with probabilistic guarantees. In this paper we apply such methods to the task of object localization and illustrate our analysis on a pedestrian detection use-case. We highlight both theoretical and practical implications of our analysis