834,708 research outputs found

    Sequential Decision Making with Strategic Agents and Limited Feedback

    Get PDF
    Sequential decision-making is a natural model for machine learning applications where the learner must make online decisions in real time and simultaneously learn from the sequential data to make better decisions in the future. Classical work has focused on variants of the problem based on the data distribution being either stochastic or adversarial, or based on the feedback available to the learner’s decisions which could be either partial or complete. With the rapid rise of large online markets, sequential learning methods have increasingly been deployed in complex multi-agent systems where agents may behave strategically to optimize for their own personal objectives. This has added a new dimension to the sequential decision-making problem where the learner must account for the strategic behavior of the agents it is learning from who might want to steer its future decisions in their favor. This thesis aims to design effective online decision-making algorithms from the point of view of the system designers aiming to learn in environments with strategic agents and limited feedback and the strategic agents seeking to optimize personal objectives. In the first part of the thesis, we focus on repeated auctions and design mechanisms where the auctioneer can effectively learn in the presence of strategic bidders, and conversely, address how agents can bid in repeated auctions or use data-poisoning attacks to maximize their own objectives. In the second part, we consider an online learning setting where feedback about the learner’s decisions is expensive to obtain. We introduce an online learning algorithm inspired by techniques from active learning that can fast forward a small fraction of more informative examples ahead in the queue. This allows the learner to obtain the same performance as the optimal online algorithm but only by querying feedback on a very small fraction of points. Finally, in the third part of the thesis, we consider a new learning objective for stochastic multi-arm bandits that promotes merit-based fairness in opportunity for individuals and groups.Ph.D

    How and Why Decision Models Influence Marketing Resource Allocations

    Get PDF
    We study how and why model-based Decision Support Systems (DSSs) influence managerial decision making, in the context of marketing budgeting and resource allocation. We consider several questions: (1) What does it mean for a DSS to be "good?"; (2) What is the relationship between an anchor or reference condition, DSS-supported recommendation and decision quality? (3) How does a DSS influence the decision process, and how does the process influence outcomes? (4) Is the effect of the DSS on the decision process and outcome robust, or context specific? We test hypotheses about the effects of DSSs in a controlled experiment with two award winning DSSs and find that, (1) DSSs improve users' objective decision outcomes (an index of likely realized revenue or profit); (2) DSS users often do not report enhanced subjective perceptions of outcomes; (3) DSSs, that provide feedback in the form of specific recommendations and their associated projected benefits had a stronger effect both on the decision making process and on the outcomes.Our results suggest that although managers actually achieve improved outcomes from DSS use, they may not perceive that the DSS has improved the outcomes. Therefore, there may be limited interest in managerial uses of DSSs, unless they are designed to: (1) encourage discussion (e.g., by providing explanations and support for the recommendations), (2) provide feedback to users on likely marketplace results, and (3) help reduce the perceived complexity of the problem so that managers will consider more alternatives and invest more cognitive effort in searching for improved outcomes.marketing models;resource allocation;DSS;decision process;decision quality

    Take it or leave it: Experimental evidence on the effect of time-limited offers on consumer behaviour

    Get PDF
    Making time-limited offers is a common retail pricing strategy. Economic theory implies that such offers inhibit price search, making markets less competitive. We investigate experimentally whether this effect is intensified by behavioural factors – specifically, feedback-conditional regret, reduced decision quality due to time constraints, and aversion to small-scale risk. Participants choose from a sequence of alternative price offers, one of which might be time-limited, under various conditions. These price search problems were matched with equivalent, time-unconstrained binary choices between lotteries. We find no evidence of regret effects. Surprisingly, time-limited offers are more likely to be chosen when the time available for decision-making is longer. Overall, individuals show aversion to small-scale risk; this is stronger in price search than lottery choice. Allowing for this, choices in the two types of task tend to be mutually consistent at the individual level, even when decision-making is subject to tight time constraints

    Agent Based Approaches to Engineering Autonomous Space Software

    Full text link
    Current approaches to the engineering of space software such as satellite control systems are based around the development of feedback controllers using packages such as MatLab's Simulink toolbox. These provide powerful tools for engineering real time systems that adapt to changes in the environment but are limited when the controller itself needs to be adapted. We are investigating ways in which ideas from temporal logics and agent programming can be integrated with the use of such control systems to provide a more powerful layer of autonomous decision making. This paper will discuss our initial approaches to the engineering of such systems.Comment: 3 pages, 1 Figure, Formal Methods in Aerospac

    Chasing Ghosts: Competing with Stateful Policies

    Full text link
    We consider sequential decision making in a setting where regret is measured with respect to a set of stateful reference policies, and feedback is limited to observing the rewards of the actions performed (the so called "bandit" setting). If either the reference policies are stateless rather than stateful, or the feedback includes the rewards of all actions (the so called "expert" setting), previous work shows that the optimal regret grows like Θ(T)\Theta(\sqrt{T}) in terms of the number of decision rounds TT. The difficulty in our setting is that the decision maker unavoidably loses track of the internal states of the reference policies, and thus cannot reliably attribute rewards observed in a certain round to any of the reference policies. In fact, in this setting it is impossible for the algorithm to estimate which policy gives the highest (or even approximately highest) total reward. Nevertheless, we design an algorithm that achieves expected regret that is sublinear in TT, of the form O(T/log1/4T)O( T/\log^{1/4}{T}). Our algorithm is based on a certain local repetition lemma that may be of independent interest. We also show that no algorithm can guarantee expected regret better than O(T/log3/2T)O( T/\log^{3/2} T)

    Using Audio Augmented Reality to Support Decision Making

    Get PDF
    Audio augmented reality (AR) is becoming increasingly popular as a medium for communicating additional layers of information about a user’s environment. This extended abstract reports the preliminary results of a study examining the impact of an audio AR enabled mobile decision aid, FoodGlance, on users’ decision quality as well as their overall experience of the decision-making process. Research shows that people often find nutrition facts labels confusing and difficult to use [1]. FoodGlance was designed to make it easier for people to choose healthy products based on their dietary needs and preferences. FoodGlance uses OCR technology to extract textual information from nutrition fact labels and then translates it into simple feedback [2]. In this study, the feedback was provided in three different ways: audio AR, visual, and the combination of the two. Audio AR was provided via Bose glasses. Visual feedback, consistent with our previous study, was displayed as a pop-up on the smartphone showing thumbs up (good amount) or thumps down (bad amount) for sugar, fat, and protein separately [2]. For this study we used the FDA’s 5-20 percent daily value rule to design feedback. The feedback engine in FoodGlance, however, is not limited to this rule and can be modified by users

    Interventions to improve obstetric emergency referral decision making, communication and feedback between health facilities in sub-Saharan Africa: a systematic review

    Get PDF
    Objective: The objective of the study was to review the evidence on interventions to improve obstetric emergency referral decision making, communication and feedback between health facilities in sub-Saharan Africa (SSA). Methods: A systematic search of PubMed, Embase, Cochrane Register and CINAHL Plus was conducted to identify studies on obstetric emergency referral in SSA. Studies were included based on pre-defined eligibility criteria. Details of reported referral interventions were extracted and categorised. The Joanna Biggs Institute Critical Appraisal checklists were used for quality assessment of included studies. A formal narrative synthesis approach was used to summarise findings guided by the WHO's referral system flow. Results: A total of 14 studies were included, with seven deemed high quality. Overall, 7 studies reported referral decision-making interventions including training programmes for health facility and community health workers, use of a triage checklist and focused obstetric ultrasound, which resulted in improved knowledge and practice of recognising danger signs for referral. 9 studies reported on referral communication using mobile phones and referral letters/notes, resulting in increased communication between facilities despite telecommunication network failures. Referral decision making and communication interventions achieved a perceived reduction in maternal mortality. 2 studies focused on referral feedback, which improved collaboration between health facilities. Conclusion: There is limited evidence on how well referral interventions work in sub-Saharan Africa, and limited consensus regarding the framework underpinning the expected change. This review has led to the proposition of a logic model that can serve as the base for future evaluations which robustly expose the (in)efficiency of referral interventions

    Powder fever and its impact on decision-making in avalanche terrain

    Get PDF
    We examined the effect of emotions, associated with “powder fever”, on decision-making in avalanche terrain. Background: Skiing in avalanche terrain is a voluntary activity that exposes the participant to potentially fatal risk. Impaired decision-making in this context can therefore have devastating results, often with limited prior corrective feedback and learning opportunities. Previous research has suggested that arousal caused by emotions affects risk assessment and intentions to engage in risky behavior. We propose that powder fever may induce similar responses. Methods: We used the following two experimental methods: laboratory studies with visual visceral stimuli (ski movies) and a field study with real stimuli (skiing exciting terrain). We evaluated the effect of emotions on attention, risk assessment, and willingness to expose oneself and others to risk. Results: Both the laboratory studies and the field study showed that skiing-related stimuli had a relatively strong effect on reported emotions. However, we found very few significant effects on decision-making or assessment of risk. Conclusions: Skiing activities make people happier. However, despite the clear parallels to sexual arousal, powder fever does not appear to significantly impair decision-making in our study. More research on the effects of powder fewer on milder forms of risk-taking behavior is needed
    corecore