6 research outputs found
Driving through the Concept Gridlock: Unraveling Explainability Bottlenecks in Automated Driving
Concept bottleneck models have been successfully used for explainable machine
learning by encoding information within the model with a set of human-defined
concepts. In the context of human-assisted or autonomous driving,
explainability models can help user acceptance and understanding of decisions
made by the autonomous vehicle, which can be used to rationalize and explain
driver or vehicle behavior. We propose a new approach using concept bottlenecks
as visual features for control command predictions and explanations of user and
vehicle behavior. We learn a human-understandable concept layer that we use to
explain sequential driving scenes while learning vehicle control commands. This
approach can then be used to determine whether a change in a preferred gap or
steering commands from a human (or autonomous vehicle) is led by an external
stimulus or change in preferences. We achieve competitive performance to latent
visual features while gaining interpretability within our model setup
SpecTracle: Wearable Facial Motion Tracking from Unobtrusive Peripheral Cameras
Facial motion tracking in head-mounted displays (HMD) has the potential to
enable immersive "face-to-face" interaction in a virtual environment. However,
current works on facial tracking are not suitable for unobtrusive augmented
reality (AR) glasses or do not have the ability to track arbitrary facial
movements. In this work, we demonstrate a novel system called SpecTracle that
tracks a user's facial motions using two wide-angle cameras mounted right next
to the visor of a Hololens. Avoiding the usage of cameras extended in front of
the face, our system greatly improves the feasibility to integrate full-face
tracking into a low-profile form factor. We also demonstrate that a neural
network-based model processing the wide-angle cameras can run in real-time at
24 frames per second (fps) on a mobile GPU and track independent facial
movement for different parts of the face with a user-independent model. Using a
short personalized calibration, the system improves its tracking performance by
42.3% compared to the user-independent model
Comparing Apples to Apples: Generating Aspect-Aware Comparative Sentences from User Review
It is time-consuming to find the best product among many similar
alternatives. Comparative sentences can help to contrast one item from others
in a way that highlights important features of an item that stand out. Given
reviews of one or multiple items and relevant item features, we generate
comparative review sentences to aid users to find the best fit. Specifically,
our model consists of three successive components in a transformer: (i) an item
encoding module to encode an item for comparison, (ii) a comparison generation
module that generates comparative sentences in an autoregressive manner, (iii)
a novel decoding method for user personalization. We show that our pipeline
generates fluent and diverse comparative sentences. We run experiments on the
relevance and fidelity of our generated sentences in a human evaluation study
and find that our algorithm creates comparative review sentences that are
relevant and truthful
Recommended from our members
Augmenting Telepostpartum Care With Vision-Based Detection of Breastfeeding-Related Conditions: Algorithm Development and Validation.
Breastfeeding benefits both the mother and infant and is a topic of attention in public health. After childbirth, untreated medical conditions or lack of support lead many mothers to discontinue breastfeeding. For instance, nipple damage and mastitis affect 80% and 20% of US mothers, respectively. Lactation consultants (LCs) help mothers with breastfeeding, providing in-person, remote, and hybrid lactation support. LCs guide, encourage, and find ways for mothers to have a better experience breastfeeding. Current telehealth services help mothers seek LCs for breastfeeding support, where images help them identify and address many issues. Due to the disproportional ratio of LCs and mothers in need, these professionals are often overloaded and burned out.This study aims to investigate the effectiveness of 5 distinct convolutional neural networks in detecting healthy lactating breasts and 6 breastfeeding-related issues by only using red, green, and blue images. Our goal was to assess the applicability of this algorithm as an auxiliary resource for LCs to identify painful breast conditions quickly, better manage their patients through triage, respond promptly to patient needs, and enhance the overall experience and care for breastfeeding mothers.We evaluated the potential for 5 classification models to detect breastfeeding-related conditions using 1078 breast and nipple images gathered from web-based and physical educational resources. We used the convolutional neural networks Resnet50, Visual Geometry Group model with 16 layers (VGG16), InceptionV3, EfficientNetV2, and DenseNet169 to classify the images across 7 classes: healthy, abscess, mastitis, nipple blebs, dermatosis, engorgement, and nipple damage by improper feeding or misuse of breast pumps. We also evaluated the models' ability to distinguish between healthy and unhealthy images. We present an analysis of the classification challenges, identifying image traits that may confound the detection model.The best model achieves an average area under the receiver operating characteristic curve of 0.93 for all conditions after data augmentation for multiclass classification. For binary classification, we achieved, with the best model, an average area under the curve of 0.96 for all conditions after data augmentation. Several factors contributed to the misclassification of images, including similar visual features in the conditions that precede other conditions (such as the mastitis spectrum disorder), partially covered breasts or nipples, and images depicting multiple conditions in the same breast.This vision-based automated detection technique offers an opportunity to enhance postpartum care for mothers and can potentially help alleviate the workload of LCs by expediting decision-making processes