A Stochastic Tensor Method for Non-convex Optimization

We present a stochastic optimization method that uses a fourth-order regularized model to find local minima of smooth and potentially non-convex objective functions with a finite-sum structure. This algorithm uses sub-sampled derivatives instead of exact quantities. The proposed approach is shown to find an $(\epsilon_1,\epsilon_2,\epsilon_3)$-third-order critical point in at most \bigO\left(\max\left(\epsilon_1^{-4/3}, \epsilon_2^{-2}, \epsilon_3^{-4}\right)\right) iterations, thereby matching the rate of deterministic approaches. In order to prove this result, we derive a novel tensor concentration inequality for sums of tensors of any order that makes explicit use of the finite-sum structure of the objective function

Effects of continuous positive airway pressure therapy on glucose metabolism in patients with obstructive sleep apnoea and type 2 diabetes: a systematic review and meta-analysis

Obstructive sleep apnoea is a highly prevalent chronic disorder and has been shown to be associated with disturbed glucose metabolism and type 2 diabetes. However, the evidence from individual clinical trials on the effect of continuous positive airway pressure (CPAP) treatment on glycaemic control in patients with co-existing obstructive sleep apnoea and type 2 diabetes remains controversial. A systematic review of randomised controlled trials assessing the effect of CPAP on glycaemic control in patients with obstructive sleep apnoea and type 2 diabetes was conducted using the databases MEDLINE, Embase, Cochrane and Scopus up to December 2022. Meta-analysis using a random-effect model was performed for outcomes that were reported in at least two randomised controlled trials. From 3031 records screened, 11 RCTs with a total of 964 patients were included for analysis. CPAP treatment led to a significant reduction in haemoglobin A1c (HbA1c) (mean difference −0.24%, 95% CI −0.43– −0.06%, p=0.001) compared to inactive control groups. Meta-regression showed a significant association between reduction in HbA1c and hours of nightly CPAP usage. CPAP therapy seems to significantly improve HbA1c and thus long-term glycaemic control in patients with type 2 diabetes and obstructive sleep apnoea. The amount of improvement is dependent on the hours of usage of CPAP and thus optimal adherence to CPAP should be a primary goal in these patients

Batch Normalization Provably Avoids Rank Collapse for Randomly Initialised Deep Networks

Randomly initialized neural networks are known to become harder to train with increasing depth, unless architectural enhancements like residual connections and batch normalization are used. We here investigate this phenomenon by revisiting the connection between random initialization in deep networks and spectral instabilities in products of random matrices. Given the rich literature on random matrices, it is not surprising to find that the rank of the intermediate representations in unnormalized networks collapses quickly with depth. In this work we highlight the fact that batch normalization is an effective strategy to avoid rank collapse for both linear and ReLU networks. Leveraging tools from Markov chain theory, we derive a meaningful lower rank bound in deep linear networks. Empirically, we also demonstrate that this rank robustness generalizes to ReLU nets. Finally, we conduct an extensive set of experiments on real-world data sets, which confirm that rank stability is indeed a crucial condition for training modern-day deep neural architectures

Synthesizing Speech from Intracranial Depth Electrodes using an Encoder-Decoder Framework

Speech Neuroprostheses have the potential to enable communication for people with dysarthria or anarthria. Recent advances have demonstrated high-quality text decoding and speech synthesis from electrocorticographic grids placed on the cortical surface. Here, we investigate a less invasive measurement modality in three participants, namely stereotactic EEG (sEEG) that provides sparse sampling from multiple brain regions, including subcortical regions. To evaluate whether sEEG can also be used to synthesize high-quality audio from neural recordings, we employ a recurrent encoder-decoder model based on modern deep learning methods. We find that speech can indeed be reconstructed with correlations up to 0.8 from these minimally invasive recordings, despite limited amounts of training data