Reward-Rate Maximization in Sequential-Identification Under a Stochastic Deadline

Cataloged from PDF version of article.Any intelligent system performing evidence-based decision making under time pressure must negotiate a speed-accuracy trade-off. In computer science and engineering, this is typically modeled as minimizing a Bayes-risk functional that is a linear combination of expected decision delay and expected terminal decision loss. In neuroscience and psychology, however, it is often modeled as maximizing the long-term reward rate, or the ratio of expected terminal reward and expected decision delay. The two approaches have opposing advantages and disadvantages. While Bayes-risk minimization can be solved with powerful dynamic programming techniques unlike reward-rate maximization, it also requires the explicit specification of the relative costs of decision delay and error, which is obviated by reward-rate maximization. Here, we demonstrate that, for a large class of sequential multihypothesis identification problems under a stochastic deadline, the reward-rate maximization is equivalent to a special case of Bayes-risk minimization, in which the optimal policy that attains the minimal risk when the unit sampling cost is exactly the maximal reward rate is also the policy that attains maximal reward rate. We show that the maximum reward rate is the unique unit sampling cost for which the expected total observation cost and expected terminal reward break even under every Bayes-risk optimal decision rule. This interplay between reward-rate maximization and Bayesrisk minimization formulations allows us to show that maximum reward rate is always attained. We can compute the policy that maximizes reward rate by solving an inverse Bayes-risk minimization problem, whereby we know the Bayes risk of the optimal policy and need to find the associated unit sampling cost parameter. Leveraging this equivalence, we derive an iterative dynamic programming procedure for solving the reward-rate maximization problem exponentially fast, thus incorporating the advantages of both the reward-rate maximization and Bayes-risk minimization formulations. As an illustration, we will apply the procedure to a two-hypothesis identification example

Dynamic bidding strategies in search-based advertising

Cataloged from PDF version of article.Search-based advertising allows the advertisers to run special campaigns targeted to different groups of potential consumers at low costs. Google, Yahoo and Microsoft advertising programs allow the advertisers to bid for an ad position on the result page of a user's query when the user searches for a keyword that the advertiser relates to its products or services. The expected revenue generated by the ad depends on the ad position, and the ad positions of the advertisers are concurrently determined after an instantaneous auction based on the bids of the advertisers. The advertisers are charged only when their ads are clicked by the users. To avoid excessive ad expenditures due to sudden surges in the keyword-search activities, each advertiser reserves a fixed finite daily budget, and the ads are not shown in the remainder of the day when the budget is depleted. Arrival times of keyword-search instances, ad positions, ad selections, and sales generated by the ads are random. Therefore, an advertiser faces a dynamic stochastic total net revenue optimization problem subject to a strict budget constraint. Here we formulate and solve this problem using dynamic programming. We show that there is always an optimal dynamic bidding policy. We describe an iterative numerical approximation algorithm that uniformly converges to the optimal solution at an exponential rate of the number of iterations. We illustrate the algorithm on numerical examples. Because dynamic programing calculations of the optimal bidding policies are computationally demanding, we also propose both static and dynamic alternative bidding policies. We numerically compare the performances of optimal and alternative bidding policies by systematically changing each input parameter. The relative percentage total net revenue losses of the alternative bidding policies increases with the budget loading, but were never more than 3.5 % of maximum expected total net revenue. The best alternative to the optimal bidding policy turned out to be a static greedy bidding policy. Finally, statistical estimation of the model parameters is visited

Wiener disorder problem with observations at fixed discrete time epochs

Suppose that a Wiener process gains a known drift rate at some unobservable disorder time with some zero-modified exponential distribution. The process is observed only at known fixed discrete time epochs, which may not always be spaced in equal distances. The problem is to detect the disorder time as quickly as possible by means of an alarm that depends only on the observations of Wiener process at those discrete time epochs. We show that Bayes optimal alarm times, which minimize expected total cost of frequent false alarms and detection delay time, always exist. Optimal alarms may in general sound between observation times and when the space-time process of the odds that disorder happened in the past hits a set with a nontrivial boundary. The optimal stopping boundary is piecewise-continuous and explodes as time approaches from left to each observation time. On each observation interval, if the boundary is not strictly increasing everywhere, then it irst decreases and then increases. It is strictly monotone wherever it does not vanish. Its decreasing portion always coincides with some explicit function. We develop numerical algorithms to calculate nearly optimal detection algorithms and their Bayes risks, and we illustrate their use on numerical examples. The solution of Wiener disorder problem with discretely spaced observation times will help reduce risks and costs associated with disease outbreak and production quality control, where the observations are often collected and/or inspected periodically. © 2010 INFORMS

Asymptotically optimal Bayesian sequential change detection and identification rules

Cataloged from PDF version of article.We study the joint problem of sequential change detection and multiple hypothesis testing. Suppose that the common distribution of a sequence of i.i.d. random variables changes suddenly at some unobservable time to one of finitely many distinct alternatives, and one needs to both detect and identify the change at the earliest possible time. We propose computationally efficient sequential decision rules that are asymptotically either Bayesoptimal or optimal in a Bayesian fixed-error-probability formulation, as the unit detection delay cost or the misdiagnosis and false alarm probabilities go to zero, respectively. Numerical examples are provided to verify the asymptotic optimality and the speed of convergence

Compound poisson disorder problems with nonlinear detection delay penalty cost functions

The quickest detection of the unknown and unobservable disorder time, when the arrival rate and mark distribution of a compound Poisson process suddenly changes, is formulated in a Bayesian setting, where the detection delay penalty is a general smooth function of the detection delay time. Under suitable conditions, the problem is shown to be equivalent to the optimal stopping of a finite-dimensional piecewise-deterministic strongly Markov sufficient statistic. The solution of the optimal stopping problem is described in detail for the compound Poisson disorder problem with polynomial detection delay penalty function of arbitrary but fixed degree. The results are illustrated for the case of the quadratic detection delay penalty function. © Taylor & Francis Group, LLC

Sequential sensor installation for Wiener disorder detection

We consider a centralized multisensor online quickest disorder detection problem where the observation from each sensor is a Wiener process gaining a constant drift at a common unobservable disorder time. The objective is to detect the disorder time as quickly as possible with small probability of false alarms. Unlike the earlier work on multisensor change detection problems, we assume that the observer can apply a sequential sensor installation policy. At any time before a disorder alarm is raised, the observer can install new sensors to collect additional signals. The sensors are statistically identical, and there is a fixed installation cost per sensor. We propose a Bayesian formulation of the problem. We identify an optimal policy consisting of a sequential sensor installation strategy and an alarm time, which minimize a linear Bayes risk of detection delay, false alarm, and new sensor installations. We also provide a numerical algorithm and illustrate it on examples. Our numerical examples show that significant reduction in the Bayes risk can be attained compared to the case where we apply a static sensor policy only. In some examples, the optimal sequential sensor installation policy starts with 30% less number of sensors than the optimal static sensor installation policy and the total percentage savings reach to 12%. © 2016 INFORMS

Asymptotically optimal Bayesian sequential change detection and identification rules

We study the joint problem of sequential change detection and multiple hypothesis testing. Suppose that the common distribution of a sequence of i.i.d. random variables changes suddenly at some unobservable time to one of finitely many distinct alternatives, and one needs to both detect and identify the change at the earliest possible time. We propose computationally efficient sequential decision rules that are asymptotically either Bayes-optimal or optimal in a Bayesian fixed-error-probability formulation, as the unit detection delay cost or the misdiagnosis and false alarm probabilities go to zero, respectively. Numerical examples are provided to verify the asymptotic optimality and the speed of convergence. © 2012 Springer Science+Business Media, LLC

Artificial neural network modeling and simulation of in-vitro nanoparticle-cell interactions

In this research a prediction model for the cellular uptake efficiency of nanoparticles (NPs), which is the rate that NPs adhere to a cell surface or enter a cell, is investigated via an artificial neural network (ANN) method. An appropriate mathematical model for the prediction of the cellular uptake rate of NPs will significantly reduce the number of time-consuming experiments to determine which of the thousands of possible variables have an impact on NP uptake rate. Moreover, this study constitutes a basis for targeted drug delivery and cell-level detection, treatment and diagnosis of existing pathologies through simulating NP-cell interactions. Accordingly, this study will accelerate nanomedicine research. Our research focuses on building a proper ANN model based on a multilayered feed-forward back-propagation algorithm that depends on NP type, size, surface charge, concentration and time for prediction of cellular uptake efficiency. The NP types for in-vitro NP-healthy cell interaction analysis are polymethyl methacrylate (PMMA), silica and polylactic acid (PLA), all of whose shapes are spheres. The proposed ANN model has been developed on MATLAB Programming Language by optimizing a number of hidden layers (HLs), node numbers and training functions. The datasets are obtained from in-vitro NP-cell interaction experiments conducted by Nanomedicine and Advanced Technology Research Center. The dispersion characteristics and cell interactions with different NPs in organisms are explored using an optimal ANN prediction model. Simulating the possible interactions of targeted NPs with cells via an ANN model will be faster and cheaper compared to the excessive experimentation currently necessary. Copyright © 2014 American Scientific Publishers All rights reserved

Analysis of the in vitro nanoparticle–cell interactions via a smoothing-splines mixed-effects model

A mixed-effects statistical model has been developed to understand the nanoparticle (NP)–cell interactions and predict the rate of cellular uptake of NPs. NP–cell interactions are crucial for targeted drug delivery systems, cell-level diagnosis, and cancer treatment. The cellular uptake of NPs depends on the size, charge, chemical structure, and concentration of NPs, and the incubation time. The vast number of combinations of these variable values disallows a comprehensive experimental study of NP–cell interactions. A mathematical model can, however, generalize the findings from a limited number of carefully designed experiments and can be used for the simulation of NP uptake rates, to design, plan, and compare alternative treatment options. We propose a mathematical model based on the data obtained from in vitro interactions of NP–healthy cells, through experiments conducted at the Nanomedicine and Advanced Technologies Research Center in Turkey. The proposed model predicts the cellular uptake rate of silica, polymethyl methacrylate, and polylactic acid NPs, given the incubation time, size, charge and concentration of NPs. This study implements the mixed-model methodology in the field of nanomedicine for the first time, and is the first mathematical model that predicts the rate of cellular uptake of NPs based on sound statistical principles. Our model provides a cost-effective tool for researchers developing targeted drug delivery systems. © 2015 Informa Healthcare USA, Inc

Optimal entry to an irreversible investment plan with non convex costs

A problem of optimally purchasing electricity at a real-valued spot price (that is, allowing negative prices) has been recently addressed in De Angelis et al. (SIAM J Control Optim 53(3), 1199–1223, 2015). The problem can be considered one of irreversible investment with a cost function which is non convex with respect to the control variable. In this paper we study optimal entry into the investment plan. The optimal entry policy can have an irregular boundary, with a kinked shape