Elicitation for Aggregation

We study the problem of eliciting and aggregating probabilistic information from multiple agents. In order to successfully aggregate the predictions of agents, the principal needs to elicit some notion of confidence from agents, capturing how much experience or knowledge led to their predictions. To formalize this, we consider a principal who wishes to elicit predictions about a random variable from a group of Bayesian agents, each of whom have privately observed some independent samples of the random variable, and hopes to aggregate the predictions as if she had directly observed the samples of all agents. Leveraging techniques from Bayesian statistics, we represent confidence as the number of samples an agent has observed, which is quantified by a hyperparameter from a conjugate family of prior distributions. This then allows us to show that if the principal has access to a few samples, she can achieve her aggregation goal by eliciting predictions from agents using proper scoring rules. In particular, if she has access to one sample, she can successfully aggregate the agents' predictions if and only if every posterior predictive distribution corresponds to a unique value of the hyperparameter. Furthermore, this uniqueness holds for many common distributions of interest. When this uniqueness property does not hold, we construct a novel and intuitive mechanism where a principal with two samples can elicit and optimally aggregate the agents' predictions.Engineering and Applied Science

Elicitation and Aggregation of Crowd Information

This thesis addresses challenges in elicitation and aggregation of crowd information for settings where an information collector, called center, has a limited knowledge about information providers, called agents. Each agent is assumed to have noisy private information that brings a high information gain to the center when it is aggregated with the private information of other agents. We address two particular issues in eliciting crowd information: 1) how to incentivize agents to participate and provide accurate data; 2) how to aggregate crowd information so that the negative impact of agents who provide low quality information is bounded. We examine three different information elicitation settings. In the first elicitation setting, agents report their observations regarding a single phenomenon that represents an abstraction of a crowdsourcing task. The center itself does not observe the phenomenon, so it rewards agents by comparing their reports. Clearly, a rational agent bases her reporting strategy on what she believes about other agents, called peers. We prove that, in general, no payment mechanism can achieve strict properness (i.e., adopt truthful reporting as a strict equilibrium strategy) if agents only report their observations, even if they share a common belief system. This motivates the use of payment mechanisms that are based on an additional report. We show that a general payment mechanism cannot have a simple structure, often adopted by prior work, and that in the limit case, when observations can take real values, agents are constrained to share a common belief system. Furthermore, we develop several payment mechanisms for the elicitation of non-binary observations. In the second elicitation setting, a group of agents observes multiple a priori similar phenomena. Due to the a priori similarity condition, the setting represents a refinement of the former setting and enables one to achieve stronger incentive properties without requiring additional reports or constraining agents to share a common belief system. We extend the existing mechanisms to allow non-binary observations by constructing strongly truthful mechanisms (i.e., mechanisms in which truthful reporting is the highest-paying equilibrium) for different types of agents' population. In the third elicitation setting, agents observe a time evolving phenomenon, and a few of them, whose identity is known, are trusted to report truthful observations. The existence of trusted agents makes this setting much more stringent than the previous ones. We show that, in the context of online information aggregation, one can not only incentivize agents to provide informative reports, but also limit the effectiveness of malicious agents who deliberately misreport. To do so, we construct a reputation system that puts a bound on the negative impact that any misreporting strategy can have on the learned aggregate. Finally, we experimentally verify the effectiveness of novel elicitation mechanisms in community sensing simulation testbeds and a peer grading experiment

Algorithmic Bayesian Epistemology

One aspect of the algorithmic lens in theoretical computer science is a view on other scientific disciplines that focuses on satisfactory solutions that adhere to real-world constraints, as opposed to solutions that would be optimal ignoring such constraints. The algorithmic lens has provided a unique and important perspective on many academic fields, including molecular biology, ecology, neuroscience, quantum physics, economics, and social science. This thesis applies the algorithmic lens to Bayesian epistemology. Traditional Bayesian epistemology provides a comprehensive framework for how an individual's beliefs should evolve upon receiving new information. However, these methods typically assume an exhaustive model of such information, including the correlation structure between different pieces of evidence. In reality, individuals might lack such an exhaustive model, while still needing to form beliefs. Beyond such informational constraints, an individual may be bounded by limited computation, or by limited communication with agents that have access to information, or by the strategic behavior of such agents. Even when these restrictions prevent the formation of a *perfectly* accurate belief, arriving at a *reasonably* accurate belief remains crucial. In this thesis, we establish fundamental possibility and impossibility results about belief formation under a variety of restrictions, and lay the groundwork for further exploration

Advancing sustainable nanotechnology with multiple criteria decision aiding

Nanotechnology is currently emerging as the next industrial revolution. It enables the production of goods (i.e. nanoproducts, NPs) with enhanced functionalities, which have nonetheless caused mounting concerns about the potential implications they can have on the environment, economy and society. This thesis employs Multiple Criteria Decision Aiding (MCDA), one form of decision support, to aid the sustainable development of nanotechnology. The first original contribution of this doctoral research is the development of a framework of sustainability assessment criteria for NPs, through a three-phase procedure based on the MCDA process, including a literature review, a pilot and a main survey. It lead to a comprehensive framework of 68 criteria, ranked according to their relative importance, allocated to six main domain areas: (i) economic performance; (ii) environmental impacts; (iii) environmental risk assessment; (iv) human health risk assessment; (v) social implications; and (vi) technical performance. All the criteria are reliable and can be used in real case studies to increase the knowledge about the sustainability of NPs. The second original contribution presented in this thesis is a robust model (DRSA-based model) based on green chemistry principles implementation for the classification of synthesis processes of nanomaterials in preference-ordered classes. This tool was developed through knowledge elicitation techniques based on coconstructive MCDA with the collaboration of two experts (the decision makers) in synthesis of nanomaterials. The robustness of the ensuing model was assessed (and confirmed) by means of another model developed ad hoc (ELECTRE-based model), structured on an MCDA method implementing a stochastic multiple criteria classification strategy. The results confirm that MCDA is an effective decision support approach to foster sustainable development of nanotechnology, providing that the analysts who apply it take these considerations into account. They must ensure that (1) multidisciplinary teams are created to perform comprehensive and credible sustainability evaluations; (2) problem structuring and model construction are as important as (if not more important) than the results (i.e. decision recommendations) themselves; (3) identification of the appropriate MCDA method depends on the problem at hand and not vice-versa; and (4) the credibility of the decision recommendations is subject to the preferences of the decision-makers. If these considerations are accounted for, the possibility of advancing nanotechnology on a sustainable path is very concrete and realistic

Preference Elicitation for Aggregation Models based

I would like to show my appreciation to all the people who made this thesis possible and an unforgettable experience for me. First of all, I would like to express my sincere gratitude to my advisor Prof.Vincent Mousseau for accepting me to carry on this research and guiding me to the field of MCDA. I benefited a lot from his continuous support during my three years Ph.D study. Without his encouragement when I was facing difficulties, I wouldn’t have completed the thesis. Besides the scientific knowledge, I also learnt his positive and optimistic attitude in concentrating on the bright side of things, which I believe is not only important for the complete of the present thesis, but also for my future life. Besides my advisor, I would like to thank the rest of my thesis committee, with whom I feel so lucky to have the chance to interact. My deeply thanks goes to Prof. Alexis Tsoukiàs. Without him, I wouldn’t have the chance to pursue my Phd in France, which changed my entire life. I am grateful to Prof. Alexis Tsoukiàs also for providing financial support to attend the 2nd Algorithmic Decison Theory Conference. I worked with Prof. Marc Pirlot for the elicitation problems of ELECTRE TRI with optimistic rule. His insightful perspectives are always valuable and helpful, and it was very pleasant to work with him because of his sense of humor an

Preference Elicitation for Aggregation Models based on Reference Points : Algorithms and Procedures

L’Aide Multicritère à la Décision (AMCD) vise à aider un décideur (DM) confronté à un problème de décision impliquant plusieurs objectifs contradictoires. Les préférences du DM jouent un rôle important au sein du processus d'aide à la décision, puisque les recommandations ne sont pertinentes et acceptables que si le système de valeurs du DM est pris en considération. Un outil d'élicitation des préférences est donc nécessaire pour aider l'analyste à intégrer les préférences du DM de façon appropriée dans les modèles de décision. Nous sommes intéressés par le développement d’outils d'élicitation des préférences pour deux modèles d'agrégation basés sur des points de référence à savoir Electre Tri et une méthode de Rangement basé sur des Points de Référence multiples (RPM). Tout d'abord, nous considérons Electre Tri en utilisant la règle d’affectation optimiste. Nous proposons un outil d'élicitation des préférences, qui infère les paramètres de préférence de ce modèle à partir d’exemples d’affectation du DM, et analyse également la robustesse des affectations résultant de la nature imprécise de l'information préférentiel. En second lieu, un outil d'élicitation des préférences est développé pour le problème de sélection de portefeuille formulée comme des problèmes de tri contraint en utilisant Electre Tri. Les préférences du DM à la fois au niveau individuel et au niveau du portefeuille sont considérés pour infère le modèle Electre Tri. Le modèle élicité évalue intrinsèquement les individus et sélectionne simultanément un portefeuille satisfaisant comme un groupe. Troisièmement, nous nous intéressons à l’élicitation des préférences pour le modèle RPM, qui détermine un pré-ordre comparant des alternatives avec des points de référence. Nous proposons un outil qui infère un modèle RPM parcimonieux à partir de comparaisons par paires du DM. Enfin, trois web services implémentent des outils d'élicitation des préférences pour Electre Tri et ont été intégrées au logiciel de Decision Deck. Les outils d’élicitation des préférences proposés consistent en des algorithmes qui résolvent des programmes linéaires en nombres mixtes. Des expériences numériques approfondies ont été réalisées pour étudier la performance et le comportement des outils d'élicitation proposées. Ces expériences éclairent sur l’applicabilité pratique de ces outils. De plus, les outils ont été appliqués avec succès à trois cas.Multiple Criteria Decision Aid (MCDA) aims at supporting decision makers (DM) facing decisions involving several conflicting objectives. DM's preferences play a key role in the decision aiding process, since the recommendations are meaningful and acceptable only if the DM's values are taken into consideration. A preference elicitation tool is therefore necessary to help the analyst to incorporate appropriately the DM's preferences in the decision models. We are interested in developing preference elicitation tools for two aggregation models based on reference points, namely Electre Tri and a new Ranking method based on Multiple reference Points (RMP). Firstly, we consider Electre Tri using the optimistic assignment rule. We propose a preference elicitation tool which elicits the preference parameters of the model from assignment examples provided by the DM, and also analyzes the robustness of the assignments related to the imprecise nature of the preference information. Secondly, a preference elicitation tool is developed for portfolio selection problems. These problems are formulated as constrained sorting problems using Electre Tri. The DM's preferences both at the individual and portfolio level are considered to elicit the Electre Tri model. The elicited model evaluates intrinsically the individuals and simultaneously selects a satisfactory portfolio as a group. Thirdly, we are interested in preference elicitation for RMP model, which determines a weak order by comparing alternatives with reference points. A preference elicitation tool is provided which infers a parsimonious RMP model from the DM's pairwise comparisons. Lastly, three web services implementing the preference elicitation tools for Electre Tri have been developed and integrated to Decision Deck software. The proposed preference elicitation tools consist of algorithms solving mixed integer programs. Extensive numerical experiments have been performed to study the performance and behavior of the proposed elicitation tools to give insights into their applicability in practice. Moreover, the tools have been successfully applied to three real-world cases

Preference Elicitation for Aggregation Models based on Reference Points (Algorithms and Procedures)

L Aide Multicritère à la Décision (AMCD) vise à aider un décideur (DM) confronté à un problème de décision impliquant plusieurs objectifs contradictoires. Les préférences du DM jouent un rôle important au sein du processus d'aide à la décision, puisque les recommandations ne sont pertinentes et acceptables que si le système de valeurs du DM est pris en considération. Un outil d'élicitation des préférences est donc nécessaire pour aider l'analyste à intégrer les préférences du DM de façon appropriée dans les modèles de décision. Nous sommes intéressés par le développement d outils d'élicitation des préférences pour deux modèles d'agrégation basés sur des points de référence à savoir Electre Tri et une méthode de Rangement basé sur des Points de Référence multiples (RPM). Tout d'abord, nous considérons Electre Tri en utilisant la règle d affectation optimiste. Nous proposons un outil d'élicitation des préférences, qui infère les paramètres de préférence de ce modèle à partir d exemples d affectation du DM, et analyse également la robustesse des affectations résultant de la nature imprécise de l'information préférentiel. En second lieu, un outil d'élicitation des préférences est développé pour le problème de sélection de portefeuille formulée comme des problèmes de tri contraint en utilisant Electre Tri. Les préférences du DM à la fois au niveau individuel et au niveau du portefeuille sont considérés pour infère le modèle Electre Tri. Le modèle élicité évalue intrinsèquement les individus et sélectionne simultanément un portefeuille satisfaisant comme un groupe. Troisièmement, nous nous intéressons à l élicitation des préférences pour le modèle RPM, qui détermine un pré-ordre comparant des alternatives avec des points de référence. Nous proposons un outil qui infère un modèle RPM parcimonieux à partir de comparaisons par paires du DM. Enfin, trois web services implémentent des outils d'élicitation des préférences pour Electre Tri et ont été intégrées au logiciel de Decision Deck. Les outils d élicitation des préférences proposés consistent en des algorithmes qui résolvent des programmes linéaires en nombres mixtes. Des expériences numériques approfondies ont été réalisées pour étudier la performance et le comportement des outils d'élicitation proposées. Ces expériences éclairent sur l applicabilité pratique de ces outils. De plus, les outils ont été appliqués avec succès à trois cas.Multiple Criteria Decision Aid (MCDA) aims at supporting decision makers (DM) facing decisions involving several conflicting objectives. DM's preferences play a key role in the decision aiding process, since the recommendations are meaningful and acceptable only if the DM's values are taken into consideration. A preference elicitation tool is therefore necessary to help the analyst to incorporate appropriately the DM's preferences in the decision models. We are interested in developing preference elicitation tools for two aggregation models based on reference points, namely Electre Tri and a new Ranking method based on Multiple reference Points (RMP). Firstly, we consider Electre Tri using the optimistic assignment rule. We propose a preference elicitation tool which elicits the preference parameters of the model from assignment examples provided by the DM, and also analyzes the robustness of the assignments related to the imprecise nature of the preference information. Secondly, a preference elicitation tool is developed for portfolio selection problems. These problems are formulated as constrained sorting problems using Electre Tri. The DM's preferences both at the individual and portfolio level are considered to elicit the Electre Tri model. The elicited model evaluates intrinsically the individuals and simultaneously selects a satisfactory portfolio as a group. Thirdly, we are interested in preference elicitation for RMP model, which determines a weak order by comparing alternatives with reference points. A preference elicitation tool is provided which infers a parsimonious RMP model from the DM's pairwise comparisons. Lastly, three web services implementing the preference elicitation tools for Electre Tri have been developed and integrated to Decision Deck software. The proposed preference elicitation tools consist of algorithms solving mixed integer programs. Extensive numerical experiments have been performed to study the performance and behavior of the proposed elicitation tools to give insights into their applicability in practice. Moreover, the tools have been successfully applied to three real-world cases.CHATENAY MALABRY-Ecole centrale (920192301) / SudocSudocFranceF