Functional generalized autoregressive conditional heteroskedasticity

Heteroskedasticity is a common feature of financial time series and is commonly addressed in the model building process through the use of ARCH and GARCH processes. More recently multivariate variants of these processes have been in the focus of research with attention given to methods seeking an efficient and economic estimation of a large number of model parameters. Due to the need for estimation of many parameters, however, these models may not be suitable for modeling now prevalent high-frequency volatility data. One potentially useful way to bypass these issues is to take a functional approach. In this paper, theory is developed for a new functional version of the generalized autoregressive conditionally heteroskedastic process, termed fGARCH. The main results are concerned with the structure of the fGARCH(1,1) process, providing criteria for the existence of a strictly stationary solutions both in the space of square-integrable and continuous functions. An estimation procedure is introduced and its consistency verified. A small empirical study highlights potential applications to intraday volatility estimation

PAC-Bayesian Treatment Allocation Under Budget Constraints

This paper considers the estimation of treatment assignment rules when the policy maker faces a general budget or resource constraint. Utilizing the PAC-Bayesian framework, we propose new treatment assignment rules that allow for flexible notions of treatment outcome, treatment cost, and a budget constraint. For example, the constraint setting allows for cost-savings, when the costs of non-treatment exceed those of treatment for a subpopulation, to be factored into the budget. It also accommodates simpler settings, such as quantity constraints, and doesn't require outcome responses and costs to have the same unit of measurement. Importantly, the approach accounts for settings where budget or resource limitations may preclude treating all that can benefit, where costs may vary with individual characteristics, and where there may be uncertainty regarding the cost of treatment rules of interest. Despite the nomenclature, our theoretical analysis examines frequentist properties of the proposed rules. For stochastic rules that typically approach budget-penalized empirical welfare maximizing policies in larger samples, we derive non-asymptotic generalization bounds for the target population costs and sharp oracle-type inequalities that compare the rules' welfare regret to that of optimal policies in relevant budget categories. A closely related, non-stochastic, model aggregation treatment assignment rule is shown to inherit desirable attributes.Comment: 70 pages, 7 figure

Speeding up deep neural architecture search for wearable activity recognition with early prediction of converged performance

Neural architecture search (NAS) has the potential to uncover more performant networks for human activity recognition from wearable sensor data. However, a naive evaluation of the search space is computationally expensive. We introduce neural regression methods for predicting the converged performance of a deep neural network (DNN) using validation performance in early epochs and topological and computational statistics. Our approach shows a significant improvement in predicting converged testing performance over a naive approach taking the ranking of the DNNs at an early epoch as an indication of their ranking on convergence. We apply this to the optimization of the convolutional feature extractor of an LSTM recurrent network using NAS with deep Q-learning, optimizing the kernel size, number of kernels, number of layers, and the connections between layers, allowing for arbitrary skip connections and dimensionality reduction with pooling layers. We find architectures which achieve up to 4% better F1 score on the recognition of gestures in the Opportunity dataset than our implementation of DeepConvLSTM and 0.8% better F1 score than our implementation of state-of-the-art model Attend and Discriminate, while reducing the search time by more than 90% over a random search. This opens the way to rapidly search for well-performing dataset-specific architectures. We describe the computational implementation of the system (software frameworks, computing resources) to enable replication of this work. Finally, we lay out several future research directions for NAS which the community may pursue to address ongoing challenges in human activity recognition, such as optimizing architectures to minimize power, minimize sensor usage, or minimize training data needs

Fast deep neural architecture search for wearable activity recognition by early prediction of converged performance

Neural Architecture Search (NAS) has the potential to uncover more performant networks for wearable activity recognition, but a naive evaluation of the search space is computationally expensive. We introduce neural regression methods for predicting the converged performance of a Deep Neural Network (DNN) using validation performance in early epochs and topological and computational statistics. Our approach shows a significant improvement in predicting converged testing performance. We apply this to the optimisation of the convolutional feature extractor of an LSTM recurrent network using NAS with deep Q-learning, optimising the kernel size, number of kernels, number of layers and the connections between layers, allowing for arbitrary skip connections and dimensionality reduction with pooling layers. We find architectures which achieve up to 4% better F1 score on the recognition of gestures in the Opportunity dataset than our implementation of the state of the art model DeepConvLSTM, while reducing the search time by >90% over a random search. This opens the way to rapidly search for well performing dataset-specific architectures

Detecting Freezing of Gait with earables trained from VR motion capture data

Freezing of Gait (FoG) is a common disabling motor symptom in Parkinson’s Disease (PD). Auditory cueing provided when FoG is detected can help mitigate the condition, for which earables are potentially well suited as they are capable of motion sensing and audio feedback. However, there are no studies so far on FoG detection at the ear. Immersive Virtual Reality (VR) combined with video-based full-body motion capture has been increasingly used to run FoG studies in the medical community. While there are motion capture datasets collected in such an environment, there are no datasets collected from IMU placed at the ear. In this paper, we show how to transfer such motion capture datasets to IMU domain and evaluate the capability of FoG detection from ear position in an immersive VR environment. Using a dataset of 6 PD patients, we compare machine learning-based FoG detection applied to the motion capture data and the virtual IMU. We have achieved an average sensitivity of 80.3% and an average specificity of 87.6% on FoG detection using the virtual earable IMU, which indicates the potential of FoG detection at the ear. This study is a step toward user-studies with earables in the VR setup, prior to conducting research in over-ground walking and everyday life

Essays in Econometrics

Each chapter of this dissertation examines a different econometric problem of interest and proposes a new approach to the data analysis problem at hand. The chapter titles may give the impression that some of these topics lie in disparate areas of focus. The topic and approach of Chapter 3, for example, shares less commonalities with Chapters 1 and 2 than the first two chapters share with one another. A connecting theme between all three chapters is the combination of foundational problems with modern data or methodologies designed to accommodate modern data analysis techniques. In the first two chapters, the PAC-Bayesian analytical framework, which has developed alongside the growth of machine learning applications, drives analyses of more traditional problems involving binary decision and individual treatment rules. In Chapter 1, this facilitates the derivation of new individual treatment rule estimators in the setting where a policy maker faces a general budget or resource constraint. In Chapter 2, this suggests new decision rules when a policy maker has a general utility function over payoffs that may have asymmetries and vary with covariates relevant to the decision problem. In each case the rules possess desirable theoretical properties, perform competitively against state-of-the-art alternatives, and have additional advantages in terms of applicability, estimation options, and modeling flexibility. Chapter 3 considers hypothesis testing in linear regressions when observations may be sampled at short time intervals. Whereas monthly or even quarterly observations were once ubiquitous in time series regression applications, it is becoming more common to have weekly, daily or even intraday observations. However, higher frequency data can pose challenges for classical inference procedures. F tests are proposed that utilize series long run variance estimation. Under reasonable discrete-time or continuous-time settings, the procedures yield valid inference so that the proposed hypothesis tests are robust to the sampling interval available to the practitioner. The tests have competitive size and power properties against the limited set of alternatives in a simulation study. Finally, an empirical example examining a relationship between interest rates associated with shorter and longer duration bonds illustrates the usefulness of the procedure

CausalBatch: solving complexity/performance tradeoffs for deep convolutional and LSTM networks for wearable activity recognition

Deep neural networks consisting of a combination of convolutional feature extractor layers and Long Short Term Memory (LSTM) recurrent layers are widely used models for activity recognition from wearable sensors - -referred to as DeepConvLSTM architectures hereafter. However, the subtleties of training these models on sequential time series data is not often discussed in the literature. Continuous sensor data must be segmented into temporal 'windows', and fed through the network to produce a loss which is used to update the parameters of the network. If trained naively using batches of randomly selected data as commonly reported, then the temporal horizon (the maximum delay at which input samples can effect the output of the model) of the network is limited to the length of the window. An alternative approach, which we will call CausalBatch training, is to construct batches deliberately such that each consecutive batch contains windows which are contiguous in time with the windows of the previous batch, with only the first batch in the CausalBatch consisting of randomly selected windows. After a given number of consecutive batches (referred to as the CausalBatch duration t), the LSTM states are reset, new random starting points are chosen from the dataset and a new CausalBatch is started. This approach allows us to increase the temporal horizon of the network without increasing the window size, which enables networks to learn data dependencies on a longer timescale without increasing computational complexity. We evaluate these two approaches on the Opportunity dataset. We find that using the CausalBatch method we can reduce the training time of DeepConvLSTM by up to 90%, while increasing the user-independent accuracy by up to 6.3% and the class weighted F1 score by up to 5.9% compared to the same model trained by random batch training with the best performing choice of window size for the latter. Compared to the same model trained using the same window length, and therefore the same computational complexity and almost identical training time, we observe an 8.4% increase in accuracy and 14.3% increase in weighted F1 score. We provide the source code for all experiments as well as a Pytorch reference implementation of DeepConvLSTM in a public github repository

Asymptotic F test in Regressions with Observations Collected at High Frequency over Long Span

This paper proposes tests of linear hypotheses when the variables may be continuous-time processes with observations collected at a high sampling frequency over a long span. Utilizing series long run variance (LRV) estimation in place of the traditional kernel LRV estimation, we develop easy-to-implement and more accurate F tests in both stationary and nonstationary environments. The nonstationary environment accommodates endogenous regressors that are general semimartinglales. The F tests can be implemented in exactly the same way as in the usual discrete-time setting. The F tests are, therefore, robust to the continuous-time or discrete-time nature of the data. Simulations demonstrate the improved size accuracy and competitive power of the F tests relative to existing continuous-time testing procedures and their improved versions. The F tests are of practical interest as recent work by Chang et al. (2018) demonstrates that traditional inference methods can become invalid and produce spurious results when continuous-time processes are observed on finer grids over a long span

Asymptotic F test in Regressions with Observations Collected at High Frequency over Long Span

This paper proposes tests of linear hypotheses when the variables may be continuous-time processes with observations collected at a high sampling frequency over a long span. Utilizing series long run variance (LRV) estimation in place of the traditional kernel LRV estimation, we develop easy-to-implement and more accurate F tests in both stationary and nonstationary environments. The nonstationary environment accommodates endogenous regressors that are general semimartinglales. The F tests can be implemented in exactly the same way as in the usual discrete-time setting. The F tests are, therefore, robust to the continuous-time or discrete-time nature of the data. Simulations demonstrate the improved size accuracy and competitive power of the F tests relative to existing continuous-time testing procedures and their improved versions. The F tests are of practical interest as recent work by Chang et al. (2018) demonstrates that traditional inference methods can become invalid and produce spurious results when continuous-time processes are observed on finer grids over a long span