Online Sensitivity Optimization in Differentially Private Learning

Training differentially private machine learning models requires constraining an individual’s contribution to the optimization process. This is achieved by clipping the 2-norm of their gradient at a predetermined threshold prior to averaging and batch sanitization. This selection adversely influences optimization in two opposing ways: it either exacerbates the bias due to excessive clipping at lower values, or augments sanitization noise at higher values. The choice significantly hinges on factors such as the dataset, model architecture, and even varies within the same optimization, demanding meticulous tuning usually accomplished through a grid search. In order to circumvent the privacy expenses incurred in hyperparameter tuning, we present a novel approach to dynamically optimize the clipping threshold. We treat this threshold as an additional learnable parameter, establishing a clean relationship between the threshold and the cost function. This allows us to optimize the former with gradient descent, with minimal repercussions on the overall privacy analysis. Our method is thoroughly assessed against alternative fixed and adaptive strategies across diverse datasets, tasks, model dimensions, and privacy levels. Our results indicate that it performs comparably or better in the evaluated scenarios, given the same privacy requirements

Advances in Kriging-Based Autonomous X-Ray Scattering Experiments.

Autonomous experimentation is an emerging paradigm for scientific discovery, wherein measurement instruments are augmented with decision-making algorithms, allowing them to autonomously explore parameter spaces of interest. We have recently demonstrated a generalized approach to autonomous experimental control, based on generating a surrogate model to interpolate experimental data, and a corresponding uncertainty model, which are computed using a Gaussian process regression known as ordinary Kriging (OK). We demonstrated the successful application of this method to exploring materials science problems using x-ray scattering measurements at a synchrotron beamline. Here, we report several improvements to this methodology that overcome limitations of traditional Kriging methods. The variogram underlying OK is global and thus insensitive to local data variation. We augment the Kriging variance with model-based measures, for instance providing local sensitivity by including the gradient of the surrogate model. As with most statistical regression methods, OK minimizes the number of measurements required to achieve a particular model quality. However, in practice this may not be the most stringent experimental constraint; e.g. the goal may instead be to minimize experiment duration or material usage. We define an adaptive cost function, allowing the autonomous method to balance information gain against measured experimental cost. We provide synthetic and experimental demonstrations, validating that this improved algorithm yields more efficient autonomous data collection

Online Sensitivity Optimization in Differentially Private Learning

Training differentially private machine learning models requires constraining an individual's contribution to the optimization process. This is achieved by clipping the $2$-norm of their gradient at a predetermined threshold prior to averaging and batch sanitization. This selection adversely influences optimization in two opposing ways: it either exacerbates the bias due to excessive clipping at lower values, or augments sanitization noise at higher values. The choice significantly hinges on factors such as the dataset, model architecture, and even varies within the same optimization, demanding meticulous tuning usually accomplished through a grid search. In order to circumvent the privacy expenses incurred in hyperparameter tuning, we present a novel approach to dynamically optimize the clipping threshold. We treat this threshold as an additional learnable parameter, establishing a clean relationship between the threshold and the cost function. This allows us to optimize the former with gradient descent, with minimal repercussions on the overall privacy analysis. Our method is thoroughly assessed against alternative fixed and adaptive strategies across diverse datasets, tasks, model dimensions, and privacy levels. Our results indicate that it performs comparably or better in the evaluated scenarios, given the same privacy requirements

Scalable Approach to Uncertainty Quantification and Robust Design of Interconnected Dynamical Systems

Development of robust dynamical systems and networks such as autonomous aircraft systems capable of accomplishing complex missions faces challenges due to the dynamically evolving uncertainties coming from model uncertainties, necessity to operate in a hostile cluttered urban environment, and the distributed and dynamic nature of the communication and computation resources. Model-based robust design is difficult because of the complexity of the hybrid dynamic models including continuous vehicle dynamics, the discrete models of computations and communications, and the size of the problem. We will overview recent advances in methodology and tools to model, analyze, and design robust autonomous aerospace systems operating in uncertain environment, with stress on efficient uncertainty quantification and robust design using the case studies of the mission including model-based target tracking and search, and trajectory planning in uncertain urban environment. To show that the methodology is generally applicable to uncertain dynamical systems, we will also show examples of application of the new methods to efficient uncertainty quantification of energy usage in buildings, and stability assessment of interconnected power networks

Limits on additional planetary companions to OGLE-2005-BLG-390L

We investigate constraints on additional planets orbiting the distant M-dwarf star OGLE-2005-BLG-390L, around which photometric microlensing data has revealed the existence of the sub-Neptune-mass planet OGLE-2005-BLG-390Lb. We specifically aim to study potential Jovian companions and compare our findings with predictions from core-accretion and disc-instability models of planet formation. We also obtain an estimate of the detection probability for sub-Neptune mass planets similar to OGLE-2005-BLG-390Lb using a simplified simulation of a microlensing experiment. We compute the efficiency of our photometric data for detecting additional planets around OGLE-2005-BLG-390L, as a function of the microlensing model parameters and convert it into a function of the orbital axis and planet mass by means of an adopted model of the Milky Way. We find that more than 50 % of potential planets with a mass in excess of 1 M_J between 1.1 and 2.3 AU around OGLE-2005-BLG-390L would have revealed their existence, whereas for gas giants above 3 M_J in orbits between 1.5 and 2.2 AU, the detection efficiency reaches 70 %; however, no such companion was observed. Our photometric microlensing data therefore do not contradict the existence of gas giant planets at any separation orbiting OGLE-2005-BLG-390L. Furthermore we find a detection probability for an OGLE-2005-BLG-390Lb-like planet of around 2-5 %. In agreement with current planet formation theories, this quantitatively supports the prediction that sub-Neptune mass planets are common around low-mass stars.Comment: 10 pages, 4 figures, accepted by A&

Modular Supply Network Optimization of Renewable Ammonia and Methanol Co-production

To reduce the use of fossil fuels and other carbonaceous fuels, renewable energy sources such as solar, wind, geothermal energy have been suggested to be promising alternative energy that guarantee sustainable and clean environment. However, the availability of renewable energy has been limited due to its dependence on weather and geographical location. This challenge is intended to be solved by the utilization of the renewable energy in the production of chemical energy carriers. Hydrogen has been proposed as a potential renewable energy carrier, however, its chemical instability and high liquefaction energy makes researchers seek for other alternative energy carriers. Ammonia and methanol can serve as promising alternative energy carriers due to their chemical stability at room temperature, low liquefaction energy, high energy value. The co-production of these high energy dense energy carriers offers economic and environmental advantages since their synthesis involve the direct utilization of CO2 and common unit operations. This problem report aims to review the optimization of the co-production of methanol and ammonia from renewable energy. Form this review, research challenges and opportunities are identified in the following areas: (i) optimization of methanol and ammonia co-production under renewable and demand uncertainty, (ii) impacts of the modular exponent on the feasibility of co-production of ammonia and methanol, and (iii) development of modern computational tools for systems-based analysis

Neutral interstellar He parameters in front of the heliosphere 1994--2007

Analysis of IBEX measurements of neutral interstellar He flux brought the inflow velocity vector different from the results of earlier analysis of observations from GAS/Ulysses. Recapitulation of results on the helium inflow direction from the past ~40 years suggested that the inflow direction may be changing with time. We reanalyze the old Ulysses data and reprocess them to increase the accuracy of the instrument pointing to investigate if the GAS observations support the hypothesis that the interstellar helium inflow direction is changing. We employ a similar analysis method as in the analysis of the IBEX data. We seek a parameter set that minimizes reduced chi-squared, using the Warsaw Test Particle Model for the interstellar He flux at Ulysses with a state of the art model of neutral He ionization in the heliosphere, and precisely reproducing the observation conditions. We also propose a supplementary method of constraining the parameters based on cross-correlations of parameters obtained from analysis of carefully selected subsets of data. We find that the ecliptic longitude and speed of interstellar He are in a very good agreement with the values reported in the original GAS analysis. We find, however, that the temperature is markedly higher. The 3-seasons optimum parameter set is lambda = 255.3, beta = 6, v = 26.0 km/s, T = 7500 K. We find no evidence that it is varying with time, but the uncertainty range is larger than originally reported. The originally-derived parameters of interstellar He from GAS are in good agreement with presently derived, except for the temperature, which seems to be appreciably higher, in good agreement with interstellar absorption line results. While the results of the present analysis are in marginal agreement with the earlier reported results from IBEX, the most likely values from the two analyses differ for reasons that are still not understood.Comment: submitted for publication in Astronomy & Astrophysic