19,808 research outputs found
Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure
As machine learning systems move from computer-science laboratories into the
open world, their accountability becomes a high priority problem.
Accountability requires deep understanding of system behavior and its failures.
Current evaluation methods such as single-score error metrics and confusion
matrices provide aggregate views of system performance that hide important
shortcomings. Understanding details about failures is important for identifying
pathways for refinement, communicating the reliability of systems in different
settings, and for specifying appropriate human oversight and engagement.
Characterization of failures and shortcomings is particularly complex for
systems composed of multiple machine learned components. For such systems,
existing evaluation methods have limited expressiveness in describing and
explaining the relationship among input content, the internal states of system
components, and final output quality. We present Pandora, a set of hybrid
human-machine methods and tools for describing and explaining system failures.
Pandora leverages both human and system-generated observations to summarize
conditions of system malfunction with respect to the input content and system
architecture. We share results of a case study with a machine learning pipeline
for image captioning that show how detailed performance views can be beneficial
for analysis and debugging
Learning Single-Image Depth from Videos using Quality Assessment Networks
Depth estimation from a single image in the wild remains a challenging
problem. One main obstacle is the lack of high-quality training data for images
in the wild. In this paper we propose a method to automatically generate such
data through Structure-from-Motion (SfM) on Internet videos. The core of this
method is a Quality Assessment Network that identifies high-quality
reconstructions obtained from SfM. Using this method, we collect single-view
depth training data from a large number of YouTube videos and construct a new
dataset called YouTube3D. Experiments show that YouTube3D is useful in training
depth estimation networks and advances the state of the art of single-view
depth estimation in the wild
Prognostics and health management for maintenance practitioners - Review, implementation and tools evaluation.
In literature, prognostics and health management (PHM) systems have been studied by many researchers from many different engineering fields to increase system reliability, availability, safety and to reduce the maintenance cost of engineering assets. Many works conducted in PHM research concentrate on designing robust and accurate models to assess the health state of components for particular applications to support decision making. Models which involve mathematical interpretations, assumptions and approximations make PHM hard to understand and implement in real world applications, especially by maintenance practitioners in industry. Prior knowledge to implement PHM in complex systems is crucial to building highly reliable systems. To fill this gap and motivate industry practitioners, this paper attempts to provide a comprehensive review on PHM domain and discusses important issues on uncertainty quantification, implementation aspects next to prognostics feature and tool evaluation. In this paper, PHM implementation steps consists of; (1) critical component analysis, (2) appropriate sensor selection for condition monitoring (CM), (3) prognostics feature evaluation under data analysis and (4) prognostics methodology and tool evaluation matrices derived from PHM literature. Besides PHM implementation aspects, this paper also reviews previous and on-going research in high-speed train bogies to highlight problems faced in train industry and emphasize the significance of PHM for further investigations
- …