Q-learning with censored data

We develop methodology for a multistage decision problem with flexible number of stages in which the rewards are survival times that are subject to censoring. We present a novel Q-learning algorithm that is adjusted for censored data and allows a flexible number of stages. We provide finite sample bounds on the generalization error of the policy learned by the algorithm, and show that when the optimal Q-function belongs to the approximation space, the expected survival time for policies obtained by the algorithm converges to that of the optimal policy. We simulate a multistage clinical trial with flexible number of stages and apply the proposed censored-Q-learning algorithm to find individualized treatment regimens. The methodology presented in this paper has implications in the design of personalized medicine trials in cancer and in other life-threatening diseases.Comment: Published in at http://dx.doi.org/10.1214/12-AOS968 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Fluctuation-dissipation relations in driven dissipative systems

Exact theoretical results for the violation of time dependent fluctuation-dissipation relations in driven dissipative systems are presented. The ratio of correlation to delayed response in the stochastic model introduced in [Phys. Rev. Lett. 93, 240601 (2004)] is shown to depend on measurement time. The fluctuation temperature defined by this ratio differs both from the temperature of the environment performing the driving, and from other effective temperatures of the system, such as the average energy (or "granular temperature"). General explanations are given for the time independence of fluctuation temperature for simple measurements or long measurement times.Comment: Author name changed; Clarifications made (mostly in introduction); References adde

Isolated Non-Equilibrium Systems in Contact

We investigate a solvable model for energy conserving non-equilibrium steady states. The time-reversal asymmetry of the dynamics leads to the violation of detailed balance and to ergodicity breaking, as manifested by the presence of dynamically inaccessible states. Two such systems in contact do not reach the same effective temperature if standard definitions are used. However, we identify the effective temperature that controls energy flow. Although this operational temperature does reach a common value upon contact, the total entropy of the joint system can decrease.Comment: 4 pages, 3 figure

Higher Degree Erdos-Ginzburg-Ziv Constants

We generalize the notion of Erd\H{o}s-Ginzburg-Ziv constants -- along the same lines we generalized in earlier work the notion of Davenport constants -- to a ``higher degree" and obtain various lower and upper bounds. These bounds are sometimes exact as is the case for certain finite commutative rings of prime power cardinality. We also consider to what extent a theorem due independently to W.D.~Gao and the first author that relates these two parameters extends to this higher degree setting. Two simple examples that capture the essence of these higher degree Erd\H{o}s-Ginzburg-Ziv constants are the following. 1) Let $\nu_p(m)$ denote the $p-$adic valuation of the integer $m$. Suppose we have integers $t | {m \choose 2}$ and $n=t+2^{\nu_2(m)}$, then every sequence $S$ over ${\mathbb Z}_2$ of length $|S| \geq n$ contains a subsequence $S'$ of length $t$ for which $\sum_{a_{i_1},\ldots, a_{i_m} \in S'} a_{i_1}\cdots a_{i_m} \equiv 0 \pmod{2}$, and this is sharp. 2) Suppose $k=3^{\alpha}$ for some integer $\alpha \geq 2$. Then every sequence $S$ over ${\mathbb Z}_3$ of length $|S| \geq k+6$ contains a subsequence $S'$ of length $k$ for which $\sum_{a_h, a_i, a_j \in S'} a_ha_ia_j \equiv 0 \pmod{3}$. These examples illustrate that if a sequence of elements from a finite commutative ring is long enough, certain symmetric expressions (symmetric polynomials) have to vanish on the elements of a subsequence of prescribed length. The Erd\H{o}s-Ginzburg-Ziv Theorem is just the case where a sequence of length $2n-1$ over ${\mathbb Z}_n$ contains a subsequence $S'=(a_1, \ldots, a_n)$ of length $n$ that vanishes when substituted in the linear symmetric polynomial $a_1+\cdots+a_n.

Interference in Bohmian Mechanics with Complex Action

In recent years, intensive effort has gone into developing numerical tools for exact quantum mechanical calculations that are based on Bohmian mechanics. As part of this effort we have recently developed as alternative formulation of Bohmian mechanics in which the quantum action, S, is taken to be complex [JCP {125}, 231103 (2006)]. In the alternative formulation there is a significant reduction in the magnitude of the quantum force as compared with the conventional Bohmian formulation, at the price of propagating complex trajectories. In this paper we show that Bohmian mechanics with complex action is able to overcome the main computational limitation of conventional Bohmian methods -- the propagation of wavefunctions once nodes set in. In the vicinity of nodes, the quantum force in conventional Bohmian formulations exhibits rapid oscillations that pose severe difficulties for existing numerical schemes. We show that within complex Bohmian mechanics, multiple complex initial conditions can lead to the same real final position, allowing for the description of nodes as a sum of the contribution from two or more crossing trajectories. The idea is illustrated on the reflection amplitude from a one-dimensional Eckart barrier. We believe that trajectory crossing, although in contradiction to the conventional Bohmian trajectory interpretation, provides an important new tool for dealing with the nodal problem in Bohmian methods

Systems Analysis from a Qualitative Perspective: An Emerging Skills Set For Information Systems Professionals

“Effective systems analysis is at the core of the design, development and operation of a modern information system. As part of their analysis and design work, information technology (IT) professionals are called upon to interview clients, observe daily operations and interpret and evaluate existing or proposed solutions. These practitioners must understand and situate themselves in the context of multiple stakeholder organizations and remain cognizant of organizational goals. Unfortunately many of these interaction skills, critical to effective application development and delivery, are not taught in a university setting, Fortunately, many of these needed skills are the focus of effective qualitative research.