3,653 research outputs found
Crowd-Certain: Label Aggregation in Crowdsourced and Ensemble Learning Classification
Crowdsourcing systems have been used to accumulate massive amounts of labeled
data for applications such as computer vision and natural language processing.
However, because crowdsourced labeling is inherently dynamic and uncertain,
developing a technique that can work in most situations is extremely
challenging. In this paper, we introduce Crowd-Certain, a novel approach for
label aggregation in crowdsourced and ensemble learning classification tasks
that offers improved performance and computational efficiency for different
numbers of annotators and a variety of datasets. The proposed method uses the
consistency of the annotators versus a trained classifier to determine a
reliability score for each annotator. Furthermore, Crowd-Certain leverages
predicted probabilities, enabling the reuse of trained classifiers on future
sample data, thereby eliminating the need for recurrent simulation processes
inherent in existing methods. We extensively evaluated our approach against ten
existing techniques across ten different datasets, each labeled by varying
numbers of annotators. The findings demonstrate that Crowd-Certain outperforms
the existing methods (Tao, Sheng, KOS, MACE, MajorityVote, MMSR, Wawa,
Zero-Based Skill, GLAD, and Dawid Skene), in nearly all scenarios, delivering
higher average accuracy, F1 scores, and AUC rates. Additionally, we introduce a
variation of two existing confidence score measurement techniques. Finally we
evaluate these two confidence score techniques using two evaluation metrics:
Expected Calibration Error (ECE) and Brier Score Loss. Our results show that
Crowd-Certain achieves higher Brier Score, and lower ECE across the majority of
the examined datasets, suggesting better calibrated results.Comment: 49 pages, 5 figure
Tricuspid annular plane systolic excursion (TAPSE) revisited using CMR
This observational pilot project was performed as background to eventually create a rapid, automated and accurate assessment of RV systolic function in variable clinical subgroups. We propose new parameters that characterize the global systolic function of the right ventricle with a simple linear measurement
Design of Robust Policies for Uncertain Natural Resource Systems: Application to the Classic Gordon-Schaefer Fishery Model
Counterfactual Explanations via Locally-guided Sequential Algorithmic Recourse
Counterfactuals operationalised through algorithmic recourse have become a
powerful tool to make artificial intelligence systems explainable.
Conceptually, given an individual classified as y -- the factual -- we seek
actions such that their prediction becomes the desired class y' -- the
counterfactual. This process offers algorithmic recourse that is (1) easy to
customise and interpret, and (2) directly aligned with the goals of each
individual. However, the properties of a "good" counterfactual are still
largely debated; it remains an open challenge to effectively locate a
counterfactual along with its corresponding recourse. Some strategies use
gradient-driven methods, but these offer no guarantees on the feasibility of
the recourse and are open to adversarial attacks on carefully created
manifolds. This can lead to unfairness and lack of robustness. Other methods
are data-driven, which mostly addresses the feasibility problem at the expense
of privacy, security and secrecy as they require access to the entire training
data set. Here, we introduce LocalFACE, a model-agnostic technique that
composes feasible and actionable counterfactual explanations using
locally-acquired information at each step of the algorithmic recourse. Our
explainer preserves the privacy of users by only leveraging data that it
specifically requires to construct actionable algorithmic recourse, and
protects the model by offering transparency solely in the regions deemed
necessary for the intervention.Comment: 7 pages, 5 figures, 3 appendix page
Recommended from our members
Strategies for Using a Spatial Method to Promote Active Learning of Probability Concepts
We developed and tested strategies for using spatial representations to help students understand core probability concepts, including the multiplication rule for computing a joint probability from a marginal and conditional probability, interpreting an odds value as the ratio of two probabilities, and Bayesian inference. The general goal of these strategies is to promote active learning by introducing concepts in an intuitive spatial format and then encouraging students to try to discover the explicit equations associated with the spatial representations. We assessed the viability of the proposed active-learning approach with two exercises that tested undergraduates’ ability to specify mathematical equations after learning to use the spatial solution method. A majority of students succeeded in independently discovering fundamental mathematical concepts underlying probabilistic reasoning. For example, in the second exercise, 76% of students correctly multiplied marginal and conditional probabilities to find joint probabilities, 86% correctly divided joint probabilities to get an odds value, and 69% did both to achieve full Bayesian inference. Thus, we conclude that the spatial method is an effective way to promote active learning of probability equations
International Space Station Electric Power System Performance Code-SPACE
The System Power Analysis for Capability Evaluation (SPACE) software analyzes and predicts the minute-by-minute state of the International Space Station (ISS) electrical power system (EPS) for upcoming missions as well as EPS power generation capacity as a function of ISS configuration and orbital conditions. In order to complete the Certification of Flight Readiness (CoFR) process in which the mission is certified for flight each ISS System must thoroughly assess every proposed mission to verify that the system will support the planned mission operations; SPACE is the sole tool used to conduct these assessments for the power system capability. SPACE is an integrated power system model that incorporates a variety of modules tied together with integration routines and graphical output. The modules include orbit mechanics, solar array pointing/shadowing/thermal and electrical, battery performance, and power management and distribution performance. These modules are tightly integrated within a flexible architecture featuring data-file-driven configurations, source- or load-driven operation, and event scripting. SPACE also predicts the amount of power available for a given system configuration, spacecraft orientation, solar-array-pointing conditions, orbit, and the like. In the source-driven mode, the model must assure that energy balance is achieved, meaning that energy removed from the batteries must be restored (or balanced) each and every orbit. This entails an optimization scheme to ensure that energy balance is maintained without violating any other constraints
Deep learning classification of chest x-ray images
We propose a deep learning based method for classification of commonly
occurring pathologies in chest X-ray images. The vast number of publicly
available chest X-ray images provides the data necessary for successfully
employing deep learning methodologies to reduce the misdiagnosis of thoracic
diseases. We applied our method to the classification of two example
pathologies, pulmonary nodules and cardiomegaly, and we compared the
performance of our method to three existing methods. The results show an
improvement in AUC for detection of nodules and cardiomegaly compared to the
existing methods.Comment: 4 pages, 4 figures, 2 tables, conference , SSIAI 202
H7N9 influenza A virus transmission in a multispecies barnyard model
Influenza A viruses are a diverse group of pathogens that have been responsible for millions of human and avian deaths throughout history. Here, we illustrate the transmission potential of H7N9 influenza A virus between Coturnix quail (Coturnix sp.), domestic ducks (Anas platyrhynchos domesticus), chickens (Gallus gallus domesticus), and house sparrows (Passer domesticus) co-housed in an artificial barnyard setting. In each of four replicates, individuals from a single species were infected with the virus. Quail shed virus orally and were a source of infection for both chickens and ducks. Infected chickens transmitted the virus to quail but not to ducks or house sparrows. Infected ducks transmitted to chickens, resulting in seroconversion without viral shedding. House sparrows did not shed virus sufficiently to transmit to other species. These results demonstrate that onward transmission varies by index species, and that gallinaceous birds are more likely to maintain H7N9 than ducks or passerines
- …