Beyond Particular Problem Instances: How to Create Meaningful and Generalizable Results

Computational intelligence methods have gained importance in several real-world domains such as process optimization, system identification, data mining, or statistical quality control. Tools are missing, which determine the applicability of computational intelligence methods in these application domains in an objective manner. Statistics provide methods for comparing algorithms on certain data sets. In the past, several test suites were presented and considered as state of the art. However, there are several drawbacks of these test suites, namely: (i) problem instances are somehow artificial and have no direct link to real-world settings; (ii) since there is a fixed number of test instances, algorithms can be fitted or tuned to this specific and very limited set of test functions; (iii) statistical tools for comparisons of several algorithms on several test problem instances are relatively complex and not easily to analyze. We propose a methodology to overcome these difficulties. It is based on standard ideas from statistics: analysis of variance and its extension to mixed models. This paper combines essential ideas from two approaches: problem generation and statistical analysis of computer experiments

Comparing Data Quality and Cost from Three Modes of On-Board Transit Passenger Surveys

This report presents the findings from a research project investigating the relative data quality and administration costs for three different modes of surveying bus passengers that produce results generalizable to the full passenger population. The three modes, all of which used survey methods distributed or administered onboard the transit vehicle, were: self-complete paper surveys, self-complete online surveys, and interviewer-assisted tablet-based surveys. Results from this study indicate several implications for practitioners choosing a survey mode. First, and most importantly, the analysis reinforces the point that there is no single, best survey mode. The choice of mode must depend on an agency’s priorities for what questions most need to be answered, what population groups are most important to represent, and exactly how the agency chooses to define concepts like a “complete” survey or a “usable” address. Findings suggest several general recommendations for current survey practice: (1) online surveys administered via an invitation distributed on the transit vehicle are not a good option; (2) old-fashioned, low-tech paper survey may still be the best option for many bus passenger surveys; (3) changes in survey results that accompany changes in survey methods should be interpreted with caution; and (4) using a new survey method, especially one relying on more complex technologies, may create unexpected glitches

Can We Build Behavioral Game Theory?

The way economists and other social scientists model how people make interdependent decisions is through the theory of games. Psychologists and behavioral economists, however, have established many deviations from the predictions of game theory. In response to these findings, a broad movement has arisen to salvage the core of game theory. Extant models of interdependent decision-making try to improve their explanatory domain by adding some corrective terms or limits. We will make the argument that this approach is misguided. For this approach to work, the deviations would have to be consistent. Drawing in part on our experimental results, we will argue that deviations from classical models are not consistent for any individual from one task to the next or between individuals for the same task. In turn, the problem of finding an equilibrium strategy is not easier but rather is exponentially more difficult. It does not seem that game theory can be repaired by adding corrective terms (such as consideration of personal characteristics, social norms, heuristic or bias terms, or cognitive limits on choice and learning). In what follows, we describe new methods for investigating interdependent decision-making. Our experimental results show that people do not choose consistently, do not hold consistent beliefs, and do not in general align actions and beliefs. We will show that experimental choices are inconsistent in ways that prevent us from drawing general characterizations of an individual’s choices or beliefs or of the general population\u27s choices and beliefs. A general behavioral game theory seems a distant and, at present, unfulfilled hope

Against Game Theory

People make choices. Often, the outcome depends on choices other people make. What mental steps do people go through when making such choices? Game theory, the most influential model of choice in economics and the social sciences, offers an answer, one based on games of strategy such as chess and checkers: the chooser considers the choices that others will make and makes a choice that will lead to a better outcome for the chooser, given all those choices by other people. It is universally established in the social sciences that classical game theory (even when heavily modified) is bad at predicting behavior. But instead of abandoning classical game theory, those in the social sciences have mounted a rescue operation under the name of “behavioral game theory.” Its main tool is to propose systematic deviations from the predictions of game theory, deviations that arise from character type, for example. Other deviations purportedly come from cognitive overload or limitations. The fundamental idea of behavioral game theory is that, if we know the deviations, then we can correct our predictions accordingly, and so get it right. There are two problems with this rescue operation, each of them is fatal. (1) For a chooser, contemplating the range of possible deviations, as there are many dozens, actually makes it exponentially harder to figure out a path to an outcome. This makes the theoretical models useless for modeling human thought or human behavior in general. (2) Modeling deviations are helpful only if the deviations are consistent, so that scientists (and indeed decision makers) can make predictions about future choices on the basis of past choices. But the deviations are not consistent. In general, deviations from classical models are not consistent for any individual from one task to the next or between individuals for the same task. In addition, people’s beliefs are in general not consistent with their choices. Accordingly, all hope is hollow that we can construct a general behavioral game theory. What can replace it? We survey some of the emerging candidates

Too Trivial To Test? An Inverse View on Defect Prediction to Identify Methods with Low Fault Risk

Background. Test resources are usually limited and therefore it is often not possible to completely test an application before a release. To cope with the problem of scarce resources, development teams can apply defect prediction to identify fault-prone code regions. However, defect prediction tends to low precision in cross-project prediction scenarios. Aims. We take an inverse view on defect prediction and aim to identify methods that can be deferred when testing because they contain hardly any faults due to their code being "trivial". We expect that characteristics of such methods might be project-independent, so that our approach could improve cross-project predictions. Method. We compute code metrics and apply association rule mining to create rules for identifying methods with low fault risk. We conduct an empirical study to assess our approach with six Java open-source projects containing precise fault data at the method level. Results. Our results show that inverse defect prediction can identify approx. 32-44% of the methods of a project to have a low fault risk; on average, they are about six times less likely to contain a fault than other methods. In cross-project predictions with larger, more diversified training sets, identified methods are even eleven times less likely to contain a fault. Conclusions. Inverse defect prediction supports the efficient allocation of test resources by identifying methods that can be treated with less priority in testing activities and is well applicable in cross-project prediction scenarios.Comment: Submitted to PeerJ C

Research versus practice in quality improvement? Understanding how we can bridge the gap

The gap between implementers and researchers of quality improvement (QI) has hampered the degree and speed of change needed to reduce avoidable suffering and harm in health care. Underlying causes of this gap include differences in goals and incentives, preferred methodologies, level and types of evidence prioritized and targeted audiences. The Salzburg Global Seminar on 'Better Health Care: How do we learn about improvement?' brought together researchers, policy makers, funders, implementers, evaluators from low-, middle- and high-income countries to explore how to increase the impact of QI. In this paper, we describe some of the reasons for this gap and offer suggestions to better bridge the chasm between researchers and implementers. Effectively bridging this gap can increase the generalizability of QI interventions, accelerate the spread of effective approaches while also strengthening the local work of implementers. Increasing the effectiveness of research and work in the field will support the knowledge translation needed to achieve quality Universal Health Coverage and the Sustainable Development Goals.Fil: Hirschhorn, Lisa R.. Northwestern University; Estados UnidosFil: Ramaswamy, Rohit. University of North Carolina; Estados UnidosFil: Devnani, Mahesh. Post Graduate Institute of Medical Education & Research; IndiaFil: Wandersman, Abraham. University Of South Carolina; Estados UnidosFil: Simpson, Lisa A.. Academy Health; Estados UnidosFil: Garcia Elorrio, Ezequiel. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentina. Instituto de Efectividad Clínica y Sanitaria; Argentin

Path Ranking with Attention to Type Hierarchies

The objective of the knowledge base completion problem is to infer missing information from existing facts in a knowledge base. Prior work has demonstrated the effectiveness of path-ranking based methods, which solve the problem by discovering observable patterns in knowledge graphs, consisting of nodes representing entities and edges representing relations. However, these patterns either lack accuracy because they rely solely on relations or cannot easily generalize due to the direct use of specific entity information. We introduce Attentive Path Ranking, a novel path pattern representation that leverages type hierarchies of entities to both avoid ambiguity and maintain generalization. Then, we present an end-to-end trained attention-based RNN model to discover the new path patterns from data. Experiments conducted on benchmark knowledge base completion datasets WN18RR and FB15k-237 demonstrate that the proposed model outperforms existing methods on the fact prediction task by statistically significant margins of 26% and 10%, respectively. Furthermore, quantitative and qualitative analyses show that the path patterns balance between generalization and discrimination.Comment: Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20

Islam: región MENA y métodos de investigación

The distinction between normative and objective knowledge and how social scientist imagine that their research is solely built on objectivity is currently being challenged especially in the political science field. If we take culture as an example and more specifically the question of identity and identity politics in the Middle East, we will find that the current modus operandi in political science research is distancing itself from objective knowledge because of the increased focus in the field on quantification. Whether one analyzes the work of Telhami on Identity in the Middle East, or Lynch’s “The Arab Uprisings Explained”, one will find that they all reflect on the academic conundrum that the field is facing.La distinción entre el conocimiento normativo y el conocimiento objetivo, así como el modo en que los científicos sociales imaginan su investigación en términos o no de objetividad, es algo que viene discutiéndose actualmente, especialmente en el campo de la ciencia política. Considerando la cultura y, más específicamente, la cuestión de la identidad y las políticas de identidad en Oriente Medio, encontramos que el modus operandi actual en la investigación de la ciencia política se está alejando del conocimiento objetivo debido al mayor enfoque en el campo de la cuantificación. Si uno analiza el trabajo de Telhami sobre Identidad en Oriente Medio, o bien las razones de los llamados “levantamientos” árabes de Lynch, encontrará que todos ellos cuestionan el rango académico del campo de estudio

Using virtual worlds for online role-play

The paper explores the use of virtual worlds to support online role-play as a collaborative activity. This paper describes some of the challenges involved in building online role-play environments in a virtual world and presents some of the ideas being explored by the project in the role-play applications being developed. Finally we explore how this can be used within the context of immersive education and 3D collaborative environments