Effective Warm Start for the Online Actor-Critic Reinforcement Learning based mHealth Intervention

Online reinforcement learning (RL) is increasingly popular for the personalized mobile health (mHealth) intervention. It is able to personalize the type and dose of interventions according to user's ongoing statuses and changing needs. However, at the beginning of online learning, there are usually too few samples to support the RL updating, which leads to poor performances. A delay in good performance of the online learning algorithms can be especially detrimental in the mHealth, where users tend to quickly disengage with the mHealth app. To address this problem, we propose a new online RL methodology that focuses on an effective warm start. The main idea is to make full use of the data accumulated and the decision rule achieved in a former study. As a result, we can greatly enrich the data size at the beginning of online learning in our method. Such case accelerates the online learning process for new users to achieve good performances not only at the beginning of online learning but also through the whole online learning process. Besides, we use the decision rules achieved in a previous study to initialize the parameter in our online RL model for new users. It provides a good initialization for the proposed online RL algorithm. Experiment results show that promising improvements have been achieved by our method compared with the state-of-the-art method

Second-order asymptotics for convolution of distributions with light tails

In this paper, asymptotic behavior of convolution of distributions belonging to two subclasses of distributions with exponential tails are considered, respectively. The precise second-order tail asymptotics of the convolutions are derived under the condition of second-order regular variation.Comment: 22 page

Asymptotics and statistical inferences on independent and non-identically distributed bivariate Gaussian triangular arrays

In this paper, we establish the first and the second-order asymptotics of distributions of normalized maxima of independent and non-identically distributed bivariate Gaussian triangular arrays, where each vector of the $n$th row follows from a bivariate Gaussian distribution with correlation coefficient being a monotone continuous function of $i/n$. Furthermore, parametric inference for this unknown function is studied. Some simulation study and real data sets analysis are also presented.Comment: 19 pages, 8 figure

Dynamic Bivariate Normal Copula

Normal copula with a correlation coefficient between $-1$ and $1$ is tail independent and so it severely underestimates extreme probabilities. By letting the correlation coefficient in a normal copula depend on the sample size, H\"usler and Reiss (1989) showed that the tail can become asymptotically dependent. In this paper, we extend this result by deriving the limit of the normalized maximum of $n$ independent observations, where the $i$-th observation follows from a normal copula with its correlation coefficient being either a parametric or a nonparametric function of $i/n$. Furthermore, both parametric and nonparametric inference for this unknown function are studied, which can be employed to test the condition in H\"usler and Reiss (1989). A simulation study and real data analysis are presented too.Comment: 22pages, 4 figure

Effective field theory approach to light propagation in an external magnetic field

The recent PVLAS experiment observed the rotation of polarization and the ellipticity when a linearly polarized laser beam passes through a transverse magnetic field. The phenomenon cannot be explained in the conventional QED. We attempt to accommodate the result by employing an effective theory for the electromagnetic field alone. No new particles with a mass of order the laser frequency or below are assumed. To quartic terms in the field strength, a parity-violating term appears besides the two ordinary terms. The rotation of polarization and ellipticity are computed for parity asymmetric and symmetric experimental set-ups. While rotation occurs in an ideal asymmetric case and has the same magnitude as the ellipticity, it disappears in a symmetric set-up like PVLAS. This would mean that we have to appeal to some low-mass new particles with nontrivial interactions with photons to understand the PVLAS result.Comment: 10 pages, no figure

Higher-order expansions of extremes from mixed skew-t distribution

In this paper, we study the asymptotic behaviors of the extreme of mixed skew-t distribution. We considered limits on distribution and density of maximum of mixed skew-t distribution under linear and power normalization, and further derived their higher-order expansions, respectively. Examples are given to support our findings.Comment: 24 pages,8 figure

Rates of convergence of extremes from skew normal samples

For a skew normal random sequence, convergence rates of the distribution of its partial maximum to the Gumbel extreme value distribution are derived. The asymptotic expansion of the distribution of the normalized maximum is given under an optimal choice of norming constants. We find that the optimal convergence rate of the normalized maximum to the Gumbel extreme value distribution is proportional to $1/\log n$

The stratified micro-randomized trial design: sample size considerations for testing nested causal effects of time-varying treatments

Technological advancements in the field of mobile devices and wearable sensors have helped overcome obstacles in the delivery of care, making it possible to deliver behavioral treatments anytime and anywhere. Increasingly the delivery of these treatments is triggered by predictions of risk or engagement which may have been impacted by prior treatments. Furthermore the treatments are often designed to have an impact on individuals over a span of time during which subsequent treatments may be provided. Here we discuss our work on the design of a mobile health smoking cessation experimental study in which two challenges arose. First the randomizations to treatment should occur at times of stress and second the outcome of interest accrues over a period that may include subsequent treatment. To address these challenges we develop the "stratified micro-randomized trial," in which each individual is randomized among treatments at times determined by predictions constructed from outcomes to prior treatment and with randomization probabilities depending on these outcomes. We define both conditional and marginal proximal treatment effects. Depending on the scientific goal these effects may be defined over a period of time during which subsequent treatments may be provided. We develop a primary analysis method and associated sample size formulae for testing these effects

Sample Size Calculations for Micro-randomized Trials in mHealth

The use and development of mobile interventions are experiencing rapid growth. In "just-in-time" mobile interventions, treatments are provided via a mobile device and they are intended to help an individual make healthy decisions "in the moment," and thus have a proximal, near future impact. Currently the development of mobile interventions is proceeding at a much faster pace than that of associated data science methods. A first step toward developing data-based methods is to provide an experimental design for testing the proximal effects of these just-in-time treatments. In this paper, we propose a "micro-randomized" trial design for this purpose. In a micro-randomized trial, treatments are sequentially randomized throughout the conduct of the study, with the result that each participant may be randomized at the 100s or 1000s of occasions at which a treatment might be provided. Further, we develop a test statistic for assessing the proximal effect of a treatment as well as an associated sample size calculator. We conduct simulation evaluations of the sample size calculator in various settings. Rules of thumb that might be used in designing a micro-randomized trial are discussed. This work is motivated by our collaboration on the HeartSteps mobile application designed to increase physical activity.Comment: 29 pages, 5 figures, 18 table

Long-distance continuous-variable quantum key distribution using non-Gaussian state-discrimination detection

We propose a long-distance continuous-variable quantum key distribution (CVQKD) with four-state protocol using non-Gaussian state-discrimination detection. A photon subtraction operation, which is deployed at the transmitter, is used for splitting the signal required for generating the non-Gaussian operation to lengthen the maximum transmission distance of CVQKD. Whereby an improved state-discrimination detector, which can be deemed as an optimized quantum measurement that allows the discrimination of nonorthogonal coherent states beating the standard quantum limit, is applied at the receiver to codetermine the measurement result with conventional coherent detector. By tactfully exploiting multiplexing technique, the resulting signals can be simultaneously transmitted through an untrusted quantum channel, and subsequently sent to the state-discrimination detector and coherent detector respectively. Security analysis shows that the proposed scheme can lengthen the maximum transmission distance up to hundreds of kilometers. Furthermore, by taking finite-size effect and composable security into account we obtain the tightest bound of the secure distance, which is more practical than that obtained in the asymptotic limit.Comment: 13 pages, 9 figure