171 research outputs found
Some comments on preference order dynamic programming models
AbstractA simple deterministic dynamic programming model is used as a general framework for the analysis of stochastic versions of three classical optimization problems: knapsack, traveling salesperson, and assembly line balancing problems. It is shown that this model can provide an alternative to the preference order models proposed for these problems. Counterexample to the optimality of the preference order models are presented
Dynamic programming: An interactive approach
AbstractAn interactive approach to the formulation, modeling, analysis, and solution of discrete deterministic dynamic programming problems is presented. The approach utilizes APL both as the mathematical and the programming language. The interactive capabilities of APL and the simple one-to-one correspondence between the programming and the mathematical language provide an extremely convenient environment for dynamic programming investigations in general and for teaching/learning purposes in particular. The approach is illustrated by a simple model and a numerical example
The Art and Science of Modeling Decision-Making Under Severe Uncertainty
For obvious reasons, models for decision-making under severe uncertainty are austere. Simply put, there is precious little to work with under these conditions. This fact highlights the great importance of utilizing in such cases the ingredients of the mathematical model to the fullest extent, which in turn brings under the spotlight the art of mathematical modeling. In this discussion we examine some of the subtle considerations that are called for in the mathematical modeling of decision-making under severe uncertainty in general, and worst-case analysis in particular. As a case study we discuss the lessons learnt on this front from the Info-Gap experience
Multiple-copy state discrimination: Thinking globally, acting locally
We theoretically investigate schemes to discriminate between two
nonorthogonal quantum states given multiple copies. We consider a number of
state discrimination schemes as applied to nonorthogonal, mixed states of a
qubit. In particular, we examine the difference that local and global
optimization of local measurements makes to the probability of obtaining an
erroneous result, in the regime of finite numbers of copies , and in the
asymptotic limit as . Five schemes are considered:
optimal collective measurements over all copies, locally optimal local
measurements in a fixed single-qubit measurement basis, globally optimal fixed
local measurements, locally optimal adaptive local measurements, and globally
optimal adaptive local measurements. Here, adaptive measurements are those for
which the measurement basis can depend on prior measurement results. For each
of these measurement schemes we determine the probability of error (for finite
) and scaling of this error in the asymptotic limit. In the asymptotic
limit, adaptive schemes have no advantage over the optimal fixed local scheme,
and except for states with less than 2% mixture, the most naive scheme (locally
optimal fixed local measurements) is as good as any noncollective scheme. For
finite , however, the most sophisticated local scheme (globally optimal
adaptive local measurements) is better than any other noncollective scheme, for
any degree of mixture.Comment: 11 pages, 14 figure
Multi-radio access network assignment using dynamic programming
This paper addresses the formulation of an optimisation problem that assigns a user to a multi-network base station for rural broadband access. The base station in this work is a fully off-grid—powered by renewable energy—system with a wireless backhaul link. The solution proposed in this paper relies on a dynamic programming approach, implementing a cost function that balances power consumption and quality of service. The cost is then aggregated using penalties based on the energy harvested and battery charge. The implemented algorithm is demonstrated (in simulations) able to adapt the user assignment to the network load and energy production
New prioritized value iteration for Markov decision processes
The problem of solving large Markov decision processes accurately and quickly is challenging. Since the computational effort incurred is considerable, current research focuses on finding superior acceleration techniques. For instance, the convergence properties of current solution methods depend, to a great extent, on the order of backup operations. On one hand, algorithms such as topological sorting are able to find good orderings but their overhead is usually high. On the other hand, shortest path methods, such as Dijkstra's algorithm which is based on priority queues, have been applied successfully to the solution of deterministic shortest-path Markov decision processes. Here, we propose an improved value iteration algorithm based on Dijkstra's algorithm for solving shortest path Markov decision processes. The experimental results on a stochastic shortest-path problem show the feasibility of our approach. © Springer Science+Business Media B.V. 2011.García Hernández, MDG.; Ruiz Pinales, J.; Onaindia De La Rivaherrera, E.; Aviña Cervantes, JG.; Ledesma Orozco, S.; Alvarado Mendez, E.; Reyes Ballesteros, A. (2012). New prioritized value iteration for Markov decision processes. Artificial Intelligence Review. 37(2):157-167. doi:10.1007/s10462-011-9224-zS157167372Agrawal S, Roth D (2002) Learning a sparse representation for object detection. In: Proceedings of the 7th European conference on computer vision. Copenhagen, Denmark, pp 1–15Bellman RE (1954) The theory of dynamic programming. Bull Amer Math Soc 60: 503–516Bellman RE (1957) Dynamic programming. Princeton University Press, New JerseyBertsekas DP (1995) Dynamic programming and optimal control. Athena Scientific, MassachusettsBhuma K, Goldsmith J (2003) Bidirectional LAO* algorithm. In: Proceedings of indian international conferences on artificial intelligence. p 980–992Blackwell D (1965) Discounted dynamic programming. Ann Math Stat 36: 226–235Bonet B, Geffner H (2003a) Faster heuristic search algorithms for planning with uncertainty and full feedback. In: Proceedings of the 18th international joint conference on artificial intelligence. Morgan Kaufmann, Acapulco, México, pp 1233–1238Bonet B, Geffner H (2003b) Labeled RTDP: improving the convergence of real-time dynamic programming. In: Proceedings of the international conference on automated planning and scheduling. Trento, Italy, pp 12–21Bonet B, Geffner H (2006) Learning depth-first search: a unified approach to heuristic search in deterministic and non-deterministic settings and its application to MDP. In: Proceedings of the 16th international conference on automated planning and scheduling. Cumbria, UKBoutilier C, Dean T, Hanks S (1999) Decision-theoretic planning: structural assumptions and computational leverage. J Artif Intell Res 11: 1–94Chang I, Soo H (2007) Simulation-based algorithms for Markov decision processes Communications and control engineering. Springer, LondonDai P, Goldsmith J (2007a) Faster dynamic programming for Markov decision processes. Technical report. Doctoral consortium, department of computer science and engineering. University of WashingtonDai P, Goldsmith J (2007b) Topological value iteration algorithm for Markov decision processes. In: Proceedings of the 20th international joint conference on artificial intelligence. Hyderabad, India, pp 1860–1865Dai P, Hansen EA (2007c) Prioritizing bellman backups without a priority queue. In: Proceedings of the 17th international conference on automated planning and scheduling, association for the advancement of artificial intelligence. Rhode Island, USA, pp 113–119Dibangoye JS, Chaib-draa B, Mouaddib A (2008) A Novel prioritization technique for solving Markov decision processes. In: Proceedings of the 21st international FLAIRS (The Florida Artificial Intelligence Research Society) conference, association for the advancement of artificial intelligence. Florida, USAFerguson D, Stentz A (2004) Focused propagation of MDPs for path planning. In: Proceedings of the 16th IEEE international conference on tools with artificial intelligence. pp 310–317Hansen EA, Zilberstein S (2001) LAO: a heuristic search algorithm that finds solutions with loops. Artif Intell 129: 35–62Hinderer K, Waldmann KH (2003) The critical discount factor for finite Markovian decision processes with an absorbing set. Math Methods Oper Res 57: 1–19Li L (2009) A unifying framework for computational reinforcement learning theory. PhD Thesis. The state university of New Jersey, New Brunswick. NJLittman ML, Dean TL, Kaelbling LP (1995) On the complexity of solving Markov decision problems.In: Proceedings of the 11th international conference on uncertainty in artificial intelligence. Montreal, Quebec pp 394–402McMahan HB, Gordon G (2005a) Fast exact planning in Markov decision processes. In: Proceedings of the 15th international conference on automated planning and scheduling. Monterey, CA, USAMcMahan HB, Gordon G (2005b) Generalizing Dijkstra’s algorithm and gaussian elimination for solving MDPs. Technical report, Carnegie Mellon University, PittsburghMeuleau N, Brafman R, Benazera E (2006) Stochastic over-subscription planning using hierarchies of MDPs. In: Proceedings of the 16th international conference on automated planning and scheduling. Cumbria, UK, pp 121–130Moore A, Atkeson C (1993) Prioritized sweeping: reinforcement learning with less data and less real time. Mach Learn 13: 103–130Puterman ML (1994) Markov decision processes. Wiley Editors, New YorkPuterman ML (2005) Markov decision processes. Wiley Inter Science Editors, New YorkRussell S (2005) Artificial intelligence: a modern approach. Making complex decisions (Ch-17), 2nd edn. Pearson Prentice Hill Ed., USAShani G, Brafman R, Shimony S (2008) Prioritizing point-based POMDP solvers. IEEE Trans Syst Man Cybern 38(6): 1592–1605Sniedovich M (2006) Dijkstra’s algorithm revisited: the dynamic programming connexion. Control Cybern 35: 599–620Sniedovich M (2010) Dynamic programming: foundations and principles, 2nd edn. Pure and Applied Mathematics Series, UKTijms HC (2003) A first course in stochastic models. Discrete-time Markov decision processes (Ch-6). Wiley Editors, UKVanderbei RJ (1996) Optimal sailing strategies. Statistics and operations research program, University of Princeton, USA ( http://www.orfe.princeton.edu/~rvdb/sail/sail.html )Vanderbei RJ (2008) Linear programming: foundations and extensions, 3rd edn. Springer, New YorkWingate D, Seppi KD (2005) Prioritization methods for accelerating MDP solvers. J Mach Learn Res 6: 851–88
Towards Machine Wald
The past century has seen a steady increase in the need of estimating and
predicting complex systems and making (possibly critical) decisions with
limited information. Although computers have made possible the numerical
evaluation of sophisticated statistical models, these models are still designed
\emph{by humans} because there is currently no known recipe or algorithm for
dividing the design of a statistical model into a sequence of arithmetic
operations. Indeed enabling computers to \emph{think} as \emph{humans} have the
ability to do when faced with uncertainty is challenging in several major ways:
(1) Finding optimal statistical models remains to be formulated as a well posed
problem when information on the system of interest is incomplete and comes in
the form of a complex combination of sample data, partial knowledge of
constitutive relations and a limited description of the distribution of input
random variables. (2) The space of admissible scenarios along with the space of
relevant information, assumptions, and/or beliefs, tend to be infinite
dimensional, whereas calculus on a computer is necessarily discrete and finite.
With this purpose, this paper explores the foundations of a rigorous framework
for the scientific computation of optimal statistical estimators/models and
reviews their connections with Decision Theory, Machine Learning, Bayesian
Inference, Stochastic Optimization, Robust Optimization, Optimal Uncertainty
Quantification and Information Based Complexity.Comment: 37 page
Fast alignment of fragmentation trees
Motivation: Mass spectrometry allows sensitive, automated and high-throughput analysis of small molecules such as metabolites. One major bottleneck in metabolomics is the identification of ‘unknown’ small molecules not in any database. Recently, fragmentation tree alignments have been introduced for the automated comparison of the fragmentation patterns of small molecules. Fragmentation pattern similarities are strongly correlated with the chemical similarity of the molecules, and allow us to cluster compounds based solely on their fragmentation patterns
Oxy-fuel combustion of coal and biomass blends
The ignition temperature, burnout and NO emissions of blends of a semi-anthracite and a high-volatile bituminous coal with 10 and 20 wt.% of olive waste were studied under oxy-fuel combustion conditions in an entrained flow reactor (EFR). The results obtained under several oxy-fuel atmospheres (21%O2–79%CO2, 30%O2–70%CO2 and 35%O2–65%CO2) were compared with those attained in air. The results indicated that replacing N2 by CO2 in the combustion atmosphere with 21% of O2 caused an increase in the temperature of ignition and a decrease in the burnout value. When the O2 concentration was increased to 30 and 35%, the temperature of ignition was lower and the burnout value was higher than in air conditions. A significant reduction in ignition temperature and a slight increase in the burnout value was observed after the addition of biomass, this trend becoming more noticeable as the biomass concentration was increased. The emissions of NO during oxy-fuel combustion were lower than under air-firing. However, they remained similar under all the oxy-fuel atmospheres with increasing O2 concentrations. Emissions of NO were significantly reduced by the addition of biomass to the bituminous coal, although this effect was less noticeable in the case of the semi-anthracite.This work was carried out with financial support from the Spanish MICINN (Project
PS-120000-2005-2) co-financed by the European Regional Development Fund. M.V.G.
and L.A. acknowledge funding from the CSIC JAE-Doc and CSIC JAE-Pre programs,
respectively, co-financed by the European Social Fund. J.R. acknowledges funding
from the Government of the Principado de Asturias (Severo Ochoa program).Peer reviewe
- …