A Unifying Variational Perspective on Some Fundamental Information Theoretic Inequalities

This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher information (Cram\'er-Rao inequality), worst additive noise lemma, entropy power inequality (EPI), and extremal entropy inequality (EEI) are interpreted as functional problems and proved within the framework of calculus of variations. Several applications and possible extensions of the proposed results are briefly mentioned

Learning How to Demodulate from Few Pilots via Meta-Learning

Consider an Internet-of-Things (IoT) scenario in which devices transmit sporadically using short packets with few pilot symbols. Each device transmits over a fading channel and is characterized by an amplifier with a unique non-linear transfer function. The number of pilots is generally insufficient to obtain an accurate estimate of the end-to-end channel, which includes the effects of fading and of the amplifier's distortion. This paper proposes to tackle this problem using meta-learning. Accordingly, pilots from previous IoT transmissions are used as meta-training in order to learn a demodulator that is able to quickly adapt to new end-to-end channel conditions from few pilots. Numerical results validate the advantages of the approach as compared to training schemes that either do not leverage prior transmissions or apply a standard learning algorithm on previously received data

The Very Dark Side of Internal Capital Markets: Evidence from Diversified Business Groups in Korea

This paper examines the capital allocation within Korean chaebol firms during the period from 1991 to 2000. We find strong evidence that, during the pre-Asian financial crisis period in the early 1990's, poorly performing firms with less investment opportunities invest more than well-performing firms with better growth opportunities. We also find the evidence of cross-subsidization among firms in the same chaebol group during the pre-crisis period. It appears that the existence of the "dark" side of internal capital markets explains most part of this striking phenomenon where "tunneling" practice has been common during the pre-crisis period. However, the inefficient capital allocation seems to disappear after the crisis as banks gain more power and market disciplines inefficient chaebol firms.

Quantum Conformal Prediction for Reliable Uncertainty Quantification in Quantum Machine Learning

In this work, we aim at augmenting the decisions output by quantum models with "error bars" that provide finite-sample coverage guarantees. Quantum models implement implicit probabilistic predictors that produce multiple random decisions for each input through measurement shots. Randomness arises not only from the inherent stochasticity of quantum measurements, but also from quantum gate noise and quantum measurement noise caused by noisy hardware. Furthermore, quantum noise may be correlated across shots and it may present drifts in time. This paper proposes to leverage such randomness to define prediction sets for both classification and regression that provably capture the uncertainty of the model. The approach builds on probabilistic conformal prediction (PCP), while accounting for the unique features of quantum models. Among the key technical innovations, we introduce a new general class of non-conformity scores that address the presence of quantum noise, including possible drifts. Experimental results, using both simulators and current quantum computers, confirm the theoretical calibration guarantees of the proposed framework.Comment: added detailed discussion on quantum hardware nois

Agreeing to Stop: Reliable Latency-Adaptive Decision Making via Ensembles of Spiking Neural Networks

Spiking neural networks (SNNs) are recurrent models that can leverage sparsity in input time series to efficiently carry out tasks such as classification. Additional efficiency gains can be obtained if decisions are taken as early as possible as a function of the complexity of the input time series. The decision on when to stop inference and produce a decision must rely on an estimate of the current accuracy of the decision. Prior work demonstrated the use of conformal prediction (CP) as a principled way to quantify uncertainty and support adaptive-latency decisions in SNNs. In this paper, we propose to enhance the uncertainty quantification capabilities of SNNs by implementing ensemble models for the purpose of improving the reliability of stopping decisions. Intuitively, an ensemble of multiple models can decide when to stop more reliably by selecting times at which most models agree that the current accuracy level is sufficient. The proposed method relies on different forms of information pooling from ensemble models, and offers theoretical reliability guarantees. We specifically show that variational inference-based ensembles with p-variable pooling significantly reduce the average latency of state-of-the-art methods, while maintaining reliability guarantees.Comment: Under revie

Advancing the Functionality and Wearability of Robotic Hand Orthoses Towards Activities of Daily Living in Stroke Patients

Post stroke rehabilitation is effective when a large number of motor repetitions are provided to patients. However, conventional physical therapy or traditional desktop-size robot aided rehabilitation do not provide sufficient number of repetitions due to cost and logistical barriers. Our vision is to realize a wearable and functional hand orthosis that could be used outside of controlled, clinical settings, thus allowing for more training repetitions. Furthermore, if such a device can prove effective for Activities of Daily Living (ADLs) while actively worn, this can incentivize patients to increase its use, further enhancing rehabilitative effects. However, in order to provide such clinical benefits, the device must be completely wearable without obtrusive features, and intuitive to control even for non-experts. In this thesis, we thus focus on wearability, functionality, and intuitive intent detection technology for a novel hand robot, and assess its performance when used both as a rehabilitative device and an assistive tool. A fully wearable device must deliver meaningful manipulation capability in small and lightweight package. In this context, we investigate the capability of single-actuator devices to assist whole hand movement patterns through a network of exotendons. Our prototypes combine a single linear actuator (mounted on a forearm splint) with a network of exotendons (routed on the surface of a soft glove). We investigate two possible tendon network configurations: one that produces full finger extension (overcoming flexor spasticity) and one that combines proximal flexion with distal extension at each finger. In experiments with stroke survivors, we measure the force levels needed to overcome various levels of spasticity and to open the hand for grasping using the first of these configurations, and qualitatively demonstrate the ability to execute fingertip grasps using the second. Our results support the feasibility of developing future wearable devices able to assist a range of manipulation tasks. In order to further improve the wearability of the device, we propose two designs that provide effective force transmission by increasing moment arms around finger joints. We evaluate the designs with geometric models and experiment using a 3D-printed artificial finger to find force and joint angle characteristics of the suggested structures. We also perform clinical tests with stroke patients to demonstrate the feasibility of the designs. The testing supports the hypothesis that the proposed designs efficiently elicit extension of the digits in patients with spasticity as compared to existing baselines. With the suggested transmission designs, the device can deliver sufficient extension force even when the users have increased muscle tone due to fatigue. The vision of an orthotic device used for ADLs can only be realized if the patients are able to operate the device themselves. However, the field is generally lacking effective methods by which the user can operate the device: such controls must be effective, intuitive, and robust to the wide range of possible impairment patterns. The variety of encountered upper limb impairment patterns in stroke patients means that a single sensing modality, such as electromyography, might not be sufficient to enable controls for a broad range of users. To address this significant gap, we introduce a multimodal sensing and interaction paradigm for an active hand orthosis. In our proof-of-concept implementation, EMG is complemented by other sensing modalities, such as finger bend and contact pressure sensors. We propose multimodal interaction methods that utilize this sensory data as input, and show they can enable tasks for stroke survivors who exhibit different impairment patterns. We then assess the performance of the robotic orthosis for two possible roles: as a therapeutic tool that facilitates device mediated hand exercises to recover neuromuscular function, or as an assistive device for use in everyday activities to aid functional use of the hand. 11 chronic stroke (> 2 years) patients with moderate muscle tone (Modified Ashworth Scale ≤ 2 in upper extremity) engage in a month-long training protocol using the orthosis. Individuals are evaluated using standardized outcome measures, both with and without orthosis assistance. The results highlight the potential for wearable and user-driven robotic hand orthoses to extend the use and training of the affected upper limb after stroke. The advances proposed in this thesis have the potential to enable robotic based hand rehabilitation during daily activities (as opposed to isolated hand exercises with limited upper limb engagement) and over extended periods of time, even in a patient’s home environment. Numerous challenges must still be overcome in order to achieve this vision, related to design (compact devices with easier donning/doffing), control (robust yet intuitive intent inferral), and effectiveness (improved functionality in a wider range of metrics). However, if these challenges can be addressed, wearable robotic devices have the potential to greatly extend the use and training of the affected upper limb after stroke, and help improve the quality of life for a large patient population

Interpretable Prototype-based Graph Information Bottleneck

The success of Graph Neural Networks (GNNs) has led to a need for understanding their decision-making process and providing explanations for their predictions, which has given rise to explainable AI (XAI) that offers transparent explanations for black-box models. Recently, the use of prototypes has successfully improved the explainability of models by learning prototypes to imply training graphs that affect the prediction. However, these approaches tend to provide prototypes with excessive information from the entire graph, leading to the exclusion of key substructures or the inclusion of irrelevant substructures, which can limit both the interpretability and the performance of the model in downstream tasks. In this work, we propose a novel framework of explainable GNNs, called interpretable Prototype-based Graph Information Bottleneck (PGIB) that incorporates prototype learning within the information bottleneck framework to provide prototypes with the key subgraph from the input graph that is important for the model prediction. This is the first work that incorporates prototype learning into the process of identifying the key subgraphs that have a critical impact on the prediction performance. Extensive experiments, including qualitative analysis, demonstrate that PGIB outperforms state-of-the-art methods in terms of both prediction performance and explainability.Comment: NeurIPS 202