On the optimality of misspecified spectral algorithms

In the misspecified spectral algorithms problem, researchers usually assume the underground true function $f_{\rho}^{*} \in [\mathcal{H}]^{s}$, a less-smooth interpolation space of a reproducing kernel Hilbert space (RKHS) $\mathcal{H}$ for some $s\in (0,1)$. The existing minimax optimal results require $\|f_{\rho}^{*}\|_{L^{\infty}} \alpha_{0}$ where $\alpha_{0}\in (0,1)$ is the embedding index, a constant depending on $\mathcal{H}$. Whether the spectral algorithms are optimal for all $s\in (0,1)$ is an outstanding problem lasting for years. In this paper, we show that spectral algorithms are minimax optimal for any $\alpha_{0}-\frac{1}{\beta} < s < 1$, where $\beta$ is the eigenvalue decay rate of $\mathcal{H}$. We also give several classes of RKHSs whose embedding index satisfies $\alpha_0 = \frac{1}{\beta}$. Thus, the spectral algorithms are minimax optimal for all $s\in (0,1)$ on these RKHSs.Comment: 48 pages, 2 figure

Unified Near-field and Far-field Localization with Holographic MIMO

Localization which uses holographic multiple input multiple output surface such as reconfigurable intelligent surface (RIS) has gained increasing attention due to its ability to accurately localize users in non-line-of-sight conditions. However, existing RIS-enabled localization methods assume the users at either the near-field (NF) or the far-field (FF) region, which results in high complexity or low localization accuracy, respectively, when they are applied in the whole area. In this paper, a unified NF and FF localization method is proposed for the RIS-enabled localization system to overcome the above issue. Specifically, the NF and FF regions are both divided into grids. The RIS reflects the signals from the user to the base station~(BS), and then the BS uses the received signals to determine the grid where the user is located. Compared with existing NF- or FF-only schemes, the design of the location estimation method and the RIS phase shift optimization algorithm is more challenging because they are based on a hybrid NF and FF model. To tackle these challenges, we formulate the optimization problems for location estimation and RIS phase shifts, and design two algorithms to effectively solve the formulated problems, respectively. The effectiveness of the proposed method is verified through simulations

Kernel interpolation generalizes poorly

One of the most interesting problems in the recent renaissance of the studies in kernel regression might be whether the kernel interpolation can generalize well, since it may help us understand the `benign overfitting henomenon' reported in the literature on deep networks. In this paper, under mild conditions, we show that for any $\varepsilon>0$, the generalization error of kernel interpolation is lower bounded by $\Omega(n^{-\varepsilon})$. In other words, the kernel interpolation generalizes poorly for a large class of kernels. As a direct corollary, we can show that overfitted wide neural networks defined on sphere generalize poorly

Holographic Integrated Sensing and Communications: Principles, Technology, and Implementation

Integrated sensing and communication (ISAC) has attracted much attention as a promising approach to alleviate spectrum congestion. However, traditional ISAC systems rely on phased arrays to provide high spatial diversity, where enormous power-consuming components such as phase shifters are used, leading to the high power consumption of the system. In this article, we introduce holographic ISAC, a new paradigm to enable high spatial diversity with low power consumption by using reconfigurable holographic surfaces (RHSs), which is an innovative type of planar antenna with densely deployed metamaterial elements. We first introduce the hardware structure and working principle of the RHS and then propose a novel holographic beamforming scheme for ISAC. Moreover, we build an RHS-enabled hardware prototype for ISAC and evaluate the system performance in the built prototype. Simulation and experimental results verify the feasibility of holographic ISAC and reveal the great potential of the RHS for reducing power consumption. Furthermore, future research directions and key challenges related to holographic ISAC are discussed

On the Asymptotic Learning Curves of Kernel Ridge Regression under Power-law Decay

The widely observed 'benign overfitting phenomenon' in the neural network literature raises the challenge to the 'bias-variance trade-off' doctrine in the statistical learning theory. Since the generalization ability of the 'lazy trained' over-parametrized neural network can be well approximated by that of the neural tangent kernel regression, the curve of the excess risk (namely, the learning curve) of kernel ridge regression attracts increasing attention recently. However, most recent arguments on the learning curve are heuristic and are based on the 'Gaussian design' assumption. In this paper, under mild and more realistic assumptions, we rigorously provide a full characterization of the learning curve: elaborating the effect and the interplay of the choice of the regularization parameter, the source condition and the noise. In particular, our results suggest that the 'benign overfitting phenomenon' exists in very wide neural networks only when the noise level is small

RECOST: External Knowledge Guided Data-efficient Instruction Tuning

In the current landscape of large language models (LLMs), the process of instruction tuning serves as an essential step. Considering the high computing power overhead, data-efficient instruction tuning was proposed to reduce the training data size in this process, aiming at selecting high-quality instructional data. Nevertheless, we argue that most current data-efficient instruction-tuning methods are highly dependent on the quality of the original instruction-tuning dataset. When it comes to datasets synthesized by LLMs, a common scenario in this field, dirty samples will even be selected with a higher probability than other samples. To address these challenges, we utilized external knowledge (relevant examples or paragraphs) to evaluate those samples synthesized by LLMs with an in-context-based relative predictive entropy. Based on the new metric, we proposed a framework, dubbed as \textbf{RECOST}, which integrates external-knowledge-base re-ranking and diversity-consistent sampling into a single pipeline. Through extensive experiments on several synthetic datasets (Alpaca and Alpaca-gpt4), we demonstrate the effectiveness of our method and achieve even better results with only \textbf{1\%} of the full dataset

Machine Learning Solution to Organ-At-Risk Segmentation for Radiation Treatment Planning

In the treatment of cancer using ionizing radiation, it is important to design a treatment plan such that dose to normal, healthy organs is sufficiently low. Today, segmentation requires a trained human to carefully outline, or segment, organs on each slice of a treatment planning computed tomography (CT) scan but it is laborious, time-consuming, and contains intra- and inter-rater variability. Currently, existing clinical automation technology relies on atlas-based automation, which has limited segmentation accuracy. Thus the auto-segmentations require post process editing by an expert. In this paper, we propose a machine learning solution that shortens the segmentation time of organs-at-risk (OARs) in the thoracic cavity. The overall system will include preprocessing, model processing, and postprocessing steps to make the system easily integratable into the radiotherapy planning process. For our model, we chose to use a 3D deep convolutional neural network with a U-net based architecture because this machine learning strategy takes into account local spatial relationships, will restore the original image resolution and has been utilized in image segmentation, especially in medical image analysis. Training and testing were done with a 60 patient dataset of thoracic CT scans from the AAPM 2017 Grand Challenge. To assess and improve our system we calculated accuracy metrics (Dice similarity coefficient (DSC), mean surface distance (MSD)) and compared our model’s segmentation performance to that of an expert and the top two performing machine learning methods of the challenge. We explored using preprocessing steps such as cropping and image enhancement to improve the model segmentation accuracy. Our final model was able to segment the lungs as accurately as a dosimetrist and the heart and spinal cord within acceptable DSC ranges. All DSC values of the OARs from our method were as accurate as other machine learning methods. The DSC for the esophagus was below tolerable error for radiotherapy planning, but our mean surface distance was superior to other auto-segmentation methods. We were successful in significantly reducing manual segmentation time by developing a machine learning system. Though our approach still necessitates a single preparatory step of manually cropping anatomical regions to isolate segmentation volume, a general hospital technician could complete this task which removes the need of an expert for one time-consuming step of radiotherapy planning. Implementation of our methods to provide radiotherapy in lower-middle income countries brings us closer to accessibility of treatment for a wider population

Reconfigurable Holographic Surface Aided Wireless Simultaneous Localization and Mapping

As a crucial facilitator of future autonomous driving applications, wireless simultaneous localization and mapping (SLAM) has drawn growing attention recently. However, the accuracy of existing wireless SLAM schemes is limited because the antenna gain is constrained given the cost budget due to the expensive hardware components such as phase arrays. To address this issue, we propose a reconfigurable holographic surface (RHS)-aided SLAM system in this paper. The RHS is a novel type of low-cost antenna that can cut down the hardware cost by replacing phased arrays in conventional SLAM systems. However, compared with a phased array where the phase shifts of parallelfed signals are adjusted, the RHS exhibits a different radiation model because its amplitude-controlled radiation elements are series-fed by surface waves, implying that traditional schemes cannot be applied directly. To address this challenge, we propose an RHS-aided beam steering method for sensing the surrounding environment and design the corresponding SLAM algorithm. Simulation results show that the proposed scheme can achieve more than there times the localization accuracy that traditional wireless SLAM with the same cost achieves

Multi-target Detection for Reconfigurable Holographic Surfaces Enabled Radar

Multi-target detection is one of the primary tasks in radar-based localization and sensing, typically built on phased array antennas. However, the bulky hardware in the phased array restricts its potential for enhancing detection accuracy, since the cost and power of the phased array can become unaffordable as its physical aperture scales up to pursue higher beam shaping capabilities. To resolve this issue, we propose a radar system enabled by reconfigurable holographic surfaces (RHSs), a novel meta-surface antenna composed of meta-material elements with cost-effective and power-efficient hardware, which performs multi-target detection in an adaptive manner. Different from the phase-control structure in the phased array, the RHS is able to apply beamforming by controlling the radiation amplitudes of its elements. Consequently, traditional beamforming schemes designed for phased arrays cannot be directly applied to RHSs due to this structural difference. To tackle this challenge, a waveform and amplitude optimization algorithm (WAOA) is designed to jointly optimize the radar waveform and RHS amplitudes in order to improve the detection accuracy. Simulation results reveal that the proposed RHS-enabled radar increases the probability of detection by 0.13 compared to phased array radars when six iterations of adaptive detection are performed given the same hardware cost