No "zero-shot" without exponential data: pretraining concept frequency determines multimodal model performance

Web-crawled pretraining datasets underlie the impressive “zero-shot” evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for image generation. However, it is unclear how meaningful the notion of “zero-shot” generalization is for such multimodal models, as it is not known to what extent their pretraining datasets encompass the downstream concepts targeted for during “zero-shot” evaluation. In this work, we ask: How is the performance of multimodal models on downstream concepts influenced by the frequency of these concepts in their pretraining datasets? We comprehensively investigate this question across 34 models and five standard pretraining datasets (CC-3M, CC-12M, YFCC-15M, LAION-400M, LAION-Aesthetics), generating over 300GB of data artifacts. We consistently find that, far from exhibiting “zero-shot” generalization, multimodal models require exponentially more data to achieve linear improvements in downstream “zero-shot” performance, following a sample inefficient log-linear scaling trend. This trend persists even when controlling for sample-level similarity between pretraining and downstream datasets [79], and testing on purely synthetic data distributions [51]. Furthermore, upon benchmarking models on long-tailed data sampled based on our analysis, we demonstrate that multimodal models across the board perform poorly. We contribute this long-tail test set as the Let it Wag! benchmark to further research in this direction. Taken together, our study reveals an exponential need for training data which implies that the key to “zero-shot” generalization capabilities under large-scale training paradigms remains to be found

Incentives to Create Under a Lifetime-Plus-Years Copyright Duration: Lessons from a Behavioral Economic Analysis for Eldred v. Ashcroft

In this Article, we highlight for the first time some of the significant but hitherto unrecognized behavioral effects of copyright law on individuals\u27 incentives to create and then examine the implications of our findings for the constitutional analysis of Eldred v. Ashcroft. We show that behavioral biases - namely, individuals\u27 optimistic bias regarding their future longevity and their subadditive judgments in circumstances resembling the extant rule of copyright duration - explain the otherwise puzzling lifetime-plus-years basis for copyright protection given to individual authors, and reveal how this regime provides superior incentives to create. Thus, insofar as the provision of increased incentives to individual authors is socially desirable, a lifetime-plus-years rule is a more effective legal means of accomplishing this goal than a rule based on a fixed term of years of a comparable expected duration. We also find, however, that the behavioral efficacy of a lifetime-plus-years regime does not apply to the Copyright Term Extension Act (CTEA), which merely extends the years component of an already existing lifetime-plus-years rule. Drawing on empirical findings on intertemporal choice, as well as our preceding analysis of the lifetime-plus-years regime and our own experimental tests, we determine that the CTEA\u27s prospective extension provides negligible additional incentives to individual authors. We conclude the extension is unjustified on incentive-provision grounds, a finding of relevance to the Court\u27s determination in Eldred v. Ashcroft of the constitutionality of the CTEA under the Copyright Clause

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Involving users in the design process: the role of product representations in co-designing

Allowing users to be part of shaping change in new product development can contribute to more successful products. Advances in recent years in digital product representations (such as CAD and rapid prototyping) can potentially offer economic and time-saving benefits to this process. The research in this thesis has generated guidelines to support co-designing activity by exploring the issues of user involvement in the design process, paying particular attention to the use of digital (computer-based) and non-digital product representations to facilitate understanding and communication. The guidelines emerged through empirical research. The first stage of the research explored users' perceptions of physical and emotional product properties through digital and rapid prototyped representations: initial guidelines for Including product representations in co-designing were generated. An Interview study was then conducted to examine the wider issues of user involvement in designing and the use of digital and non-digital product representations from the standpoint of ten practicing - designers. Challenges and barriers to user Involvement were perceived but designers were open-minded to the Idea of digital co-designing. In parallel an audit was undertaken to evaluate product representation technologies for their ability to facilitate co-designing: traditional non-digital methods of sketching and hand-made models were used to develop criteria for this benchmarking. Limitations were found with existing technology and it was apparent that traditional methods (e. g. hand-drawn sketches and models) were better able to facilitate co-designing at this time than digital methods. These findings led to recommendations for future co-designing tools. Co-designing processes were then explored through six practical studies conducted with individuals and small groups of users. Users experimented with designing and making improved handles for a small gardening tool through sketching and day modelling. Design concepts were then taken further into digital media, through 3D scanning, digital CAD images and rapid prototyping and presented back to users for evaluation. Co-designing was also explored through a commercial context with an international packaging manufacturer. Ten users communicated design ideas for improved packaging by triangulation of notes, sketches, discussion and modelling activity. This produced user-led design criteria and commercially valuable concept designs. Important insights were gained into how codesigning may be facilitated within a commercial context and the experiences of the stakeholders. Several pertinent ethical issues such as ownership of ideas, incentives and rewards for user involvement were raised. The thesis concludes with guidelines and recommendations for co-designing, particularly regarding the role of product representations

Gaining Insight into Determinants of Physical Activity using Bayesian Network Learning

Contains fulltext : 228326pre.pdf (preprint version ) (Open Access) Contains fulltext : 228326pub.pdf (publisher's version ) (Open Access)BNAIC/BeneLearn 202

CLiFF Notes: Research in the Language Information and Computation Laboratory of The University of Pennsylvania

This report takes its name from the Computational Linguistics Feedback Forum (CLIFF), an informal discussion group for students and faculty. However the scope of the research covered in this report is broader than the title might suggest; this is the yearly report of the LINC Lab, the Language, Information and Computation Laboratory of the University of Pennsylvania. It may at first be hard to see the threads that bind together the work presented here, work by faculty, graduate students and postdocs in the Computer Science, Psychology, and Linguistics Departments, and the Institute for Research in Cognitive Science. It includes prototypical Natural Language fields such as: Combinatorial Categorial Grammars, Tree Adjoining Grammars, syntactic parsing and the syntax-semantics interface; but it extends to statistical methods, plan inference, instruction understanding, intonation, causal reasoning, free word order languages, geometric reasoning, medical informatics, connectionism, and language acquisition. With 48 individual contributors and six projects represented, this is the largest LINC Lab collection to date, and the most diverse

A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence

A number of algorithms in the field of artificial intelligence offer poorly interpretable decisions. To disclose the reasoning behind such algorithms, their output can be explained by means of socalled evidence-based (or factual) explanations. Alternatively, contrastive and counterfactual explanations justify why the output of the algorithms is not any different and how it could be changed, respectively. It is of crucial importance to bridge the gap between theoretical approaches to contrastive and counterfactual explanation and the corresponding computational frameworks. In this work we conduct a systematic literature review which provides readers with a thorough and reproducible analysis of the interdisciplinary research field under study. We first examine theoretical foundations of contrastive and counterfactual accounts of explanation. Then, we report the state-of-the-art computational frameworks for contrastive and counterfactual explanation generation. In addition, we analyze how grounded such frameworks are on the insights from the inspected theoretical approaches. As a result, we highlight a variety of properties of the approaches under study and reveal a number of shortcomings thereof. Moreover, we define a taxonomy regarding both theoretical and practical approaches to contrastive and counterfactual explanation.S