Predictable Artificial Intelligence

We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key indicators of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems, and thus should be prioritised over performance. While distinctive from other areas of technical and non-technical AI research, the questions, hypotheses and challenges relevant to Predictable AI were yet to be clearly described. This paper aims to elucidate them, calls for identifying paths towards AI predictability and outlines the potential impact of this emergent field.Comment: 11 pages excluding references, 4 figures, and 2 tables. Paper Under Revie

Further Details on Examining Adversarial Evaluation: Role of Difficulty

Adversarial benchmark construction, where harder instances challenge new generations of AI systems, is becoming the norm. While this approach may lead to better machine learning models ---on average and for the new \mbox{benchmark---,} it is unclear how these models behave on the original distribution. Two opposing effects are intertwined here. On the one hand, the adversarial benchmark has a higher proportion of difficult instances, with lower expected performance. On the other hand, models trained on the adversarial benchmark may improve on these difficult instances (but may also neglect some easy ones). To disentangle these two effects we can control for difficulty, showing that we can recover the performance on the original distribution, provided the harder instances were obtained from this distribution in the first place. We show this difficulty-aware rectification works in practice, through a series of experiments with several benchmark construction schemas and the use of a populational difficulty metric. As a take-away message, instead of distributional averages we recommend using difficulty-conditioned characteristic curves when evaluating models built with adversarial benchmarks.We thank the anonymous reviewers for their comments. This work was funded by valgrAI, the Norwegian Research Council grant 329745 Machine Teaching for Explainable AI, the Future of Life Institute, FLI, under grant RFP2-152, the EU (FEDER) and Spanish grant RTI2018-094403-B-C32 funded by MCIN/AEI/10.13039/501100011033 and by CIPROM/2022/6 funded by Generalitat Valenciana, EU’s Horizon 2020 research and innovation programme under grant agreement No. 952215 (TAILOR), US DARPA HR00112120007 (RECoG-AI) and Spanish grant PID2021-122830OB-C42 (SFERA) funded by MCIN/AEI/10.13039/501100011033 and "ERDF A way of making Europe" In compliance with the recommendations of the Science paper about reporting of evaluation results in AI [3], we include all the results at the instance levelMehrbakhsh, B.; Martínez-Plumed, F.; Hernández-Orallo, J. (2023). Further Details on Examining Adversarial Evaluation: Role of Difficulty. http://hdl.handle.net/10251/19568

Determinants of environmental, financial, and social sustainable performance of manufacturing SMEs in Malaysia

Discovering the determinants of firm sustainable performance from the Small and Medium-sized Enterprises (SMEs) perspectives is essential. However, few studies have empirically examined all three environmental, financial, and social pillars of sustainable performance into a single research framework in the context of emerging economies like Malaysia. Drawing on the resource-based view and institutional theory, this study identified the determinants of environment, financial, and social sustainable performance of manufacturing SMEs. Data was collected from 209 Malaysian manufacturing firms. A hybrid approach of structural equation modeling (SEM) - artificial neural network (ANN) was used to assess the hypotheses and predict the level of their importance toward sustainable performance. Results showed that green entrepreneurial orientation, green innovation, leadership commitment, stakeholder pressure, and market orientation positively and significantly influenced social performance. Environmental performance was predicted by green entrepreneurial orientation, green innovation, leadership commitment, and market orientation. Green entrepreneurial orientation and market orientation demonstrated a positive influence on financial performance. Results of ANN showed that leadership commitment is the most significant factor influencing environmental and social performance while green entrepreneurial orientation is the first ranked factor predicting financial performance. These findings extend the knowledge by shedding light on the determinants of SMEs\u27 sustainable performance. The study enables SMEs to take proper actions in response to sustainability development. Besides, the findings assist practitioners and policymakers in setting effective plans by giving more attention to leadership commitment and green entrepreneurial orientation as the most significant determinants