Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure

As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggregate views of system performance that hide important shortcomings. Understanding details about failures is important for identifying pathways for refinement, communicating the reliability of systems in different settings, and for specifying appropriate human oversight and engagement. Characterization of failures and shortcomings is particularly complex for systems composed of multiple machine learned components. For such systems, existing evaluation methods have limited expressiveness in describing and explaining the relationship among input content, the internal states of system components, and final output quality. We present Pandora, a set of hybrid human-machine methods and tools for describing and explaining system failures. Pandora leverages both human and system-generated observations to summarize conditions of system malfunction with respect to the input content and system architecture. We share results of a case study with a machine learning pipeline for image captioning that show how detailed performance views can be beneficial for analysis and debugging

DYNAMIC COMPLEMENTARITY IN EXPORT PROMOTION: THE MARKET ACCESS PROGRAM IN FRUITS AND VEGETABLES

Government-supported promotion in foreign markets may justified when market failures exist, such as spillover externalities, where promotion of one commodity positively influences exports of another, or when market uncertainties cause planning horizons to be shorter than the persistent effects of promotion. A dynamic model of U.S. apple, almond, grape, and wine export supply is developed to test for these market failures. Promotion is viewed as an investment in establishing and maintaining a productÂ’'s image. Evidence supporting the existence of each market failure is found. Exporters and program administrators may fail to account for them in export promotion planning.International Relations/Trade,

Accurate and automatic NOAA-AVHRR image navigation using a global contour matching approach

The problem of precise and automatic AVHRR image navigation is tractable in theory, but has proved to be somewhat difficult in practice. The authors' work has been motivated by the need for a fully automatic and operational navigation system capable of geo-referencing NOAA-AVHRR images with high accuracy and without operator supervision. The proposed method is based on the simultaneous use of an orbital model and a contour matching approach. This last process, relying on an affine transformation model, is used to correct the errors caused by inaccuracies in orbit modeling, nonzero value for the spacecraft's roll, pitch and yaw, errors due to inaccuracies in the satellite positioning and failures in the satellite internal clock. The automatic global contour matching process is summarized as follows: i) Estimation of the gradient energy map (edges) in the sensed image and detection of the cloudless (reliable) areas in this map. ii) Initialization of the affine model parameters by minimizing the Euclidean distance between the reference and sensed images objects. iii) Simultaneous optimization of all reference image contours on the sensed image by energy minimization in the domain of the global transformation parameters. The process is iterated in a hierarchical way, reducing the parameter searching space at each iteration. The proposed image navigation algorithm has proved to be capable of geo-referencing a satellite image within 1 pixel.Peer ReviewedPostprint (published version