Scaling Up Differentially Private LASSO Regularized Logistic Regression via Faster Frank-Wolfe Iterations

To the best of our knowledge, there are no methods today for training differentially private regression models on sparse input data. To remedy this, we adapt the Frank-Wolfe algorithm for $L_1$ penalized linear regression to be aware of sparse inputs and to use them effectively. In doing so, we reduce the training time of the algorithm from $\mathcal{O}( T D S + T N S)$ to $\mathcal{O}(N S + T \sqrt{D} \log{D} + T S^2)$, where $T$ is the number of iterations and a sparsity rate $S$ of a dataset with $N$ rows and $D$ features. Our results demonstrate that this procedure can reduce runtime by a factor of up to $2,200\times$, depending on the value of the privacy parameter $\epsilon$ and the sparsity of the dataset.Comment: To appear in the 37th Conference on Neural Information Processing Systems (NeurIPS 2023

동력원을 고려한 교통망에서 에너지 최적화를 위한 링크 시계열로 이산화 된 동적 교통 배정 연구

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 기계항공공학부, 2020. 8. 차석원.Vehicle that provides convenience for mobility has been studied for more than 100 years. Recently, there has been a lot of research on the performance of a single-vehicle and interaction between other cars. For example, research on technologies such as vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and autonomous driver assistant system (ADAS) is actively studied. This change also extends the scope of the study, from a single vehicle to a vehicle fleet, and from micro-traffic to macro-traffic. In the case of vehicles subject to the main experiment, it is classified into internal combustion engine vehicle (ICEV), hybrid electric vehicle (HEV), electric vehicle (EV), and fuel-cell electric vehicle (FCEV) according to the electrification of the powertrain. Also, it can be divided into different categories depending on whether autonomous driving and communication are possible. This study focused on expanding the fuel consumption of vehicles, which has affected environmental pollution for a long time, to the transportation network level. Of course, these researches have been studied for more than a decade, but recent optimization studies using various powertrains have been hard to find. In particular, I decided to build a system that reflects the energy superiority of each road, based on the tendency to consume fuel by road type according to the powertrain. For several decades, the study of arranging the traffic situation of vehicles and determining the route of each vehicle has been mainly applied to traffic allocation for road planning, such as road construction. Therefore, the main content was to predict users' choices and to study from a macro perspective in hours or days. However, in the near future, it is expected to be able to control the route of vehicles in a specific unit of a transportation network, so based on these assumptions, researchers conducted many researches to optimize energy in the transportation network. Many studies on fuel consumption have advanced, but it is hard to find a study of many vehicles consisting of various powertrains. The main reason is that the fuel consumption itself is difficult to predict and calculate, and there is a significant variation for each vehicle. In this study, the average value of each variable for energy consumption was predicted using Vehicle Specific Power (VSP). It used to calculate the fuel consumption that matches the powertrain by each vehicle. Data on fuel consumption were taken from Autonomie, a forward simulator provided by Argonne National Laboratory in the United States. Based on the relationship between the simulated fuel consumption and the VSP as a variable, the deviation was optimized with Newton's method. However, after energy optimization, different vehicles have different travel times, resulting in wasted time due to relative superiority about the fuel consumption, which is a problem in terms of fairness for drivers. Therefore, based on the traffic time of each road, the first principle of Wardrop was applied to optimize the allocation of traffic. The first principle of Wardrop is Wardrop's User Equilibrium (UE) which means an optimal state with same travel cost in the same origin-destination. Based on UE, it was replaced by the question of distributing the allocated traffic flow depends on vehicle type. To this end, it is necessary to apply the traffic assignment based on the route, not the link unit, so that each vehicle can be distributed to the route. This distribution is also an optimization problem, which is a Linear Programming (LP) problem with equality constraint and inequality constraint with the fuel consumption per vehicle derived for each route as a factor. This problem can be resolved through the process of replacing the constraints with the Lagrange multiplier, and the simple conditions for optimization are met. In conclusion, the goal of this study is to allocate a path-based dynamic traffic assignment (DTA) so that it can be applied in real-time with minimal computation and to distribute them by vehicle type. First, under the current road conditions, each vehicle moves toward the intersection. The intersection at the end of the road that is currently running by time unit was organized by Origin-Destination (O-D). In DTA studies, intricate and detailed model like the cell transmission model (CTM) is used for modeling. The traffic flow is calculated as a fluid, which needs high calculating costs and many complex constraints to optimization. Therefore, link time-series was suggested to be modeled for each link and applied as a kind of historical information. This approach can be regarded as Discretized-DTA based on link time-series. It is possible to apply the time axis to the traffic network with a small computing cost and to allocate O-D traffic that changes with time. This optimization problem can be resolved by the Gradient Projection algorithm, which was widely used in path-based traffic allocation. Different delay equations were applied for the intersections by traffic lights for the modeling of the time delay. The actual transportation network flow was predicted as much as possible by the Discretized-DTA algorithm. The allocated traffic was divided by the route, and the fuel consumption per vehicle was derived for each route. In the Sioux Falls Network, the most commonly used example of a traffic allocation simulation, the total energy cost was reduced by about 2% when applying the vehicle distribution used in this study after static traffic assignment. This performance is the result of no time loss between the vehicles, as it is in a UE state. And if traffic simulation case is limited to O-D allocated on multiple paths, it is an effect of more than 3%. This improvement could be replaced by a reduction in fuel cost of about 20 million won for 360,600 vehicles daily. For evaluation of the performance as a navigating system, four navigating systems, as a comparison group, are modeled with algorithms that recommend the optimal route in real-time. The system proposed in this study was able to improve 20% in total traffic time and 15% in the energy aspect compared to the comparison group. It was also applied to Gangdong-gu, Seoul, to simulate a somewhat congested transportation network. At this time, the performance improvement was reduced by 10% in traffic time and 5% in the energy aspect. In the case of the navigating system, indeed, the effect of energy optimization for distributing by vehicle type is not substantial because allocation for each vehicle causes rarely distributed path. However, this improvement can be a significant impact if the effects are accumulated in the transportation network. In this study, energy optimization in the transportation network was achieved based on fuel consumption tendency by vehicle type, and the navigation system was developed for this. Nowadays, with the development of various communication and control technologies, the navigation system based on them can contribute to reducing the cost of transportation, both personally and socially.사람들의 이동에 편의성을 제공하는 자동차는 100년 넘는 긴 시간 동안 연구되어왔다. 최근에는 단일 자동차의 성능에 대한 연구에 더불어 다른 자동차와의 상호 작용에 대한 연구가 많이 이루어지고 있다. 예로 차량 간 (vehicle-to-vehicle: V2V) 통신, 차량 인프라 간(vehicle-to-infrastructure: V2I) 통신, 지능형 운전자 보조 시스템(advanced driver assistance system: ADAS) 등의 기술에 대한 연구를 들 수 있다. 이 같은 변화는 연구 대상의 범위도 단일 차량에서 차량 fleet, 그리고 micro-traffic부터 macro-traffic까지 넓어지게 하고 있다. 주 실험 대상인 자동차의 경우에도 동력전달계의 전기화에 따라 내연기관자동차, 하이브리드자동차, 전기자동차, 연료전지자동차등으로 분류되며, 자율주행과 통신 가능 여부에 따라 또 다른 분류로 나뉠 수 있다. 본 연구는 오랫동안 환경에 큰 영향을 끼치는 자동차의 연료소모량을 교통망 차원으로 넓히는 것에 착안하였다. 물론 이러한 연구는 십년 넘게 이루어져왔지만, 최근 다양해진 동력전달계에 따른 최적화 연구는 찾기 힘들었다. 특히 동력전달계에 따라 도로 별 연료소모 경향이 달라지는 것에 착안하여, 도로 별 에너지적 우위를 반영한 시스템을 구축하기로 하였다. 수 십년 동안 차량들의 교통 상황을 정리하여 각 차량들의 루트를 정하는 연구는, 도로 건설 등의 도로 계획을 위한 통행 배정에 주로 적용되어왔다. 따라서 이용자들의 선택을 예측하고, 시간 단위 또는 일 단위의 거시적인 관점에서의 연구가 주 내용이었다. 하지만 근 시일 내에 일정 단위의 교통망에서는 차량들의 루트를 컨트롤할 수 있을 것이라 예상되기에 이러한 가정을 기반으로 교통망 내의 에너지를 최적화 하는 연구를 진행하였다. 연료소모량에 대한 연구는 많이 진행되었지만, 다양한 파워트레인으로 구성된 다수의 차량들에 대한 연구는 찾아보기 힘들다. 그 대표적인 이유는 연료소모량자체가 예측 및 계산하기 힘들고, 차량마다 그 편차가 크기 때문이다. 본 연구에서는 차량 비출력(Vehicle specific power: VSP)를 이용하여 각 변수들의 평균치로 예측한 후에, 이를 이용하여 차종 별 동력전달계에 맞는 연료소모량을 계산하는 방법을 활용하였다. 연료소모량에 대한 데이터는 미국의 Argonne national laboratory에서 공급하는 전방향 시뮬레이터인 Autonomie에서 가져왔다. 시뮬레이션 된 연료소모량과 VSP와의 관계를 변수로 하여 뉴턴법(Newtons method)로 편차를 최소화하도록 최적화하였다. 하지만 교통망 내에서 에너지 최적화 후, 차종에 따라 통행 시간이 달라져서 상대적 우위에 따른 시간 낭비가 생기면 운전자의 공정성 측면에서 문제가 된다. 따라서 각 도로의 통행시간을 기준으로 Wardrop의 첫번째 원칙을 적용한 최적 통행 배정을 수행하였다. 그리고 이를 기준으로 배정된 통행의 차량 흐름을 차종 별로 분배하는 문제로 치환하였다. 이를 위해서는 통행 배정을 링크 단위가 아니라 경로를 기반으로 적용하여야 각 차종을 경로에 분배할 수 있다. 이러한 분배 또한 최적화 문제로, 이는 각 경로에 대해 도출된 차량당 연료소모량을 계수로 하고, 등식 제한 조건과 부등식 제한 조건을 가지는 선형계획법(Linear Programming: LP)문제이다. 이는 제한조건을 라그랑주 상수(Lagrange Multiplier)로 치환하는 과정을 통해, 해결할 수 있으며 조건이 단순하기 때문에, 최적화를 위한 조건을 만족시킨다. 또한 본 연구에서는 실시간으로 루트를 정해주는 일종의 네비게이션을 목표로 하였기 때문에, Wardrop의 이용자 평형(User Equilibrium: UE)상태를 시간에 따라 변하는 동적 상태로 적용해야 했다. 결론적으로, 경로 기반의 동적 통행 배정(Dynamic Traffic Assignment: DTA)을 연산을 최소화하여 실시간으로 적용할 수 있도록 배정을 하고, 차종 별로 분배를 하는 것이 본 연구의 목표이다. 먼저 현재 차량 상황에서 각 도로는 교차로를 향해서 이동하기 때문에, 시간 단위 별로 현재 달리고 있는 도로의 끝의 교차로를 기점으로 하고 원래 가고자 하는 목적지를 종점으로 가지는 기 종점을 구성하였다. 동적 통행 배정의 연구에서는 세포 전이 모델(Cell Transmission Model: CTM) 등을 이용하여, 교통 흐름을 유체처럼 계산하여 시간 소모가 많다. 따라서 도로에 진입하는 각 통행 흐름을 도로의 시계열에 저장하는 이산화 된 동적 통행 배정(Discretized-DTA)방법을 고안하였다. 이는 적은 계산 비용으로 심플하게 시간 축을 교통망에 부여하였고, 이를 통해 시간에 따라 변화하는 기 종점을 통행을 배정할 수 있게 되었다. 이 최적화 문제는 경로 기반 통행배정에서 많이 사용된 경사 투영법(Gradient Projection) 알고리즘을 적용하였다. 각 도로의 교통 흐름에 따른 시간 지체도 신호등이 있는 교차로의 단속류와 신호등이 없는 도로의 연속류에 따라 다른 지체 식을 적용하여 실제 현실의 교통망의 흐름을 최대한 예측하였다. 이렇게 배정된 통행량을 경로 별로 나누어, 각 경로에 대한 차량당 연료소모량을 도출하였다. 통행 배정 시뮬레이션에서 가장 많이 사용하는 예제인 Sioux Falls 네트워크에서는 정적 통행 배정 이후 본 연구에 사용된 차종 분배를 적용할 경우에, 전체 에너지 코스트가 약 2% 감소하였다. 이는 Wardrop의 이용자 평형 상태이기 때문에, 차량들 간의 어떤 시간 손해도 없는 결과이다. 그리고 만약 통행량이 복수의 경로로 배정된 O-D에 한정할 경우에, 약 3%가 넘는 효과라고 할 수 있다. 이는 하루 기준으로, 360,600대의 차량에 대해 2000만원 정도의 연료비 감소로 치환할 수 있었다. 네비게이팅 시스템의 경우에는, 현재 도로의 상태를 4가지 정도로 나누어서 실시간 최적 경로로 추천하는 경우를 비교군으로 정하였다. 비교군에 비해 본 연구에서 제안하는 시스템은 전체 통행시간과 에너지적 측면을 개선할 수 있었다. 또한 이를 서울시 강동구에 적용하여 어느 정도 혼잡한 교통망을 모사하였다. 이 때 시뮬레이션 결과와 기존의 최소 시간 알고리즘을 적용하는 상용 네비게이션 시스템 결과를 비교하였다. 결과적으로 약 8000대의 차량이 주행하는 시나리오 1의 교통망을 기준으로 전체 통행시간은 66%, 에너지 소모 비용은 34%를 줄일 수 있었다. 이는 일정 혼잡도까지는 효과가 커졌지만, 어느 이상에서는 효과가 감소하기도 하였다. 물론 이산화 된 동적 교통 분배를 기반으로하기 때문에, 일정 시간 단위의 차량들을 편대로 묶어 통행을 배정하고, 그 결과 경로가 분산되는 경우 자체가 적기 때문에 분산된 경로에 차종 별로 분배하는 에너지 최적화의 효과는 크지 않은 것이 사실이다. 하지만 이 또한 교통망 내에서 그 효과를 누적하면 영향이 크다고 할 수 있다. 본 연구에서는 결과적으로 차종 별 연료 소모 경향에 근거하여 교통망 내 통행 시간 감소에 더불어 에너지 최적화를 이루었고, 이를 위한 네비게이팅 시스템을 개발했다. 각종 통신과 제어 기술이 발전한 요즘, 그에 기반한 네비게이팅 시스템은 개인적, 사회적으로 교통에서 발생하는 비용을 줄이는 데 기여할 수 있다.Chapter 1. Introduction 1 1.1. Background 1 1.2. Research Scope and Contents 7 Chapter 2. Theory and Literature Review 11 2.1. Traffic Assignment Problem 11 2.1.1. Wardrops Principle 11 2.1.2. Dynamic Traffic Assignment (DTA) 16 2.1.3. Volume-Delay Function (VDF) 18 2.2. Vehicle Fuel Consumption 20 2.2.1. Tendency Based on Driving Cycle 21 2.2.2. Tendency Based on Powertrain 22 2.3. Vehicle Specific Power (VSP) 24 2.4. Route Guidance System 26 2.4.1. Optimal Routing System Based on Fuel Economy 27 Chapter 3. Target Model Development 29 3.1. Vehicle Model Development 29 3.2. Fuel Consumption Trend Depends on Vehicle Model 32 3.3. Introduction of Vehicle Specific Power 35 3.4. Calibration of VSP Parameters 36 3.5. Regression of VSP Variables 38 3.5.1.. VSP Variables from General Vehicles 39 3.5.2. Regression of VSP Variables by Travel Time 40 Chapter 4. Traffic Assignment based on Energy Consumption 46 4.1. Model for Static Traffic Assignment 46 4.1.1. Sioux Falls Network 46 4.2. Gradient Projection (GP) Algorithm 48 4.3. Distribution of Vehicles to Energy Optimization 51 4.3.1. Problem Formulation for Vehicle Distribution 51 4.3.2. Linear Programming 53 4.4. Simulation Result in Test Network 54 Chapter 5. Navigating System using Discretized Dynamic Traffic Assignment 57 5.1. Modeling of Discretized Dynamic Traffic Assignment 57 5.1.1. Discretized-DTA with Vehicle Fleets 57 5.1.2. Discretized-DTA with Link Time-Series 60 5.1.3. Target Network 62 5.2. Navigating System 65 5.2.1. Structure of the Navigating System 65 5.2.2. Algorithm of the Navigating System 65 5.2.3. Assumption of the Navigating System 69 5.3. Result of Navigating System 70 5.3.1. Results of the Travel Time Prediction 70 5.3.2. Results in Scenario 1 71 5.3.3. Results in Scenario 2 79 5.3.4. Results in Scenario 3 81 Chapter 6. Conclusion and Future Works 85 6.1. Conclusion 85 6.2. Future Work 87 Bibliography 88 Abstract in Korean 100Docto

Shrinking VOD Traffic via Rényi-Entropic Optimal Transport

In response to the exponential surge in Internet Video on Demand (VOD) traffic, numerous research endeavors have concentrated on optimizing and enhancing infrastructure efficiency. In contrast, this paper explores whether users’ demand patterns can be shaped to reduce the pressure on infrastructure. Our main idea is to design a mechanism that alters the distribution of user requests to another distribution which is much more cache-efficient, but still remains ‘close enough’ (in the sense of cost) to fulfil each individual user’s preference. To quantify the cache footprint of VOD traffic, we propose a novel application of Rényi entropy as its proxy, capturing the ‘richness’ (the number of distinct videos or cache size) and the ‘evenness’ (the relative popularity of video accesses) of the on-demand video distribution. We then demonstrate how to decrease this metric by formulating a problem drawing on the mathematical theory of optimal transport (OT). Additionally, we establish a key equivalence theorem: minimizing Rényi entropy corresponds to maximizing soft cache hit ratio (SCHR) — a variant of cache hit ratio allowing similarity-based video substitutions. Evaluation on a real-world, city-scale video viewing dataset reveals a remarkable 83% reduction in cache size (associated with VOD caching traffic). Crucially, in alignment with the above-mentioned equivalence theorem, our approach yields a significant uplift to SCHR, achieving close to 100%

Impact of Different Fecal Processing Methods on Assessments of Bacterial Diversity in the Human Intestine.

The intestinal microbiota are integral to understanding the relationships between nutrition and health. Therefore, fecal sampling and processing protocols for metagenomic surveys should be sufficiently robust, accurate, and reliable to identify the microorganisms present. We investigated the use of different fecal preparation methods on the bacterial community structures identified in human stools. Complete stools were collected from six healthy individuals and processed according to the following methods: (i) randomly sampled fresh stool, (ii) fresh stool homogenized in a blender for 2 min, (iii) randomly sampled frozen stool, and (iv) frozen stool homogenized in a blender for 2 min, or (v) homogenized in a pneumatic mixer for either 10, 20, or 30 min. High-throughput DNA sequencing of the 16S rRNA V4 regions of bacterial community DNA extracted from the stools showed that the fecal microbiota remained distinct between individuals, independent of processing method. Moreover, the different stool preparation approaches did not alter intra-individual bacterial diversity. Distinctions were found at the level of individual taxa, however. Stools that were frozen and then homogenized tended to have higher proportions of Faecalibacterium, Streptococcus, and Bifidobacterium and decreased quantities of Oscillospira, Bacteroides, and Parabacteroides compared to stools that were collected in small quantities and not mixed prior to DNA extraction. These findings indicate that certain taxa are at particular risk for under or over sampling due to protocol differences. Importantly, homogenization by any method significantly reduced the intra-individual variation in bacteria detected per stool. Our results confirm the robustness of fecal homogenization for microbial analyses and underscore the value of collecting and mixing large stool sample quantities in human nutrition intervention studies

Conditional Gradient Methods

The purpose of this survey is to serve both as a gentle introduction and a coherent overview of state-of-the-art Frank--Wolfe algorithms, also called conditional gradient algorithms, for function minimization. These algorithms are especially useful in convex optimization when linear optimization is cheaper than projections. The selection of the material has been guided by the principle of highlighting crucial ideas as well as presenting new approaches that we believe might become important in the future, with ample citations even of old works imperative in the development of newer methods. Yet, our selection is sometimes biased, and need not reflect consensus of the research community, and we have certainly missed recent important contributions. After all the research area of Frank--Wolfe is very active, making it a moving target. We apologize sincerely in advance for any such distortions and we fully acknowledge: We stand on the shoulder of giants.Comment: 238 pages with many figures. The FrankWolfe.jl Julia package (https://github.com/ZIB-IOL/FrankWolfe.jl) providces state-of-the-art implementations of many Frank--Wolfe method

The relationship between biological and psychosocial risk factors and resting‐state functional connectivity in 2‐monthold Bangladeshi infants: A feasibility and pilot study

Childhood poverty has been associated with structural and functional alterations in the developing brain. However, poverty does not alter brain development directly, but acts through associated biological or psychosocial risk factors (e.g. malnutrition, family conflict). Yet few studies have investigated risk factors in the context of infant neurodevelopment, and none have done so in low‐resource settings such as Bangladesh, where children are exposed to multiple, severe biological and psychosocial hazards. In this feasibility and pilot study, usable resting‐state fMRI data were acquired in infants from extremely poor (n = 16) and (relatively) more affluent (n = 16) families in Dhaka, Bangladesh. Whole‐brain intrinsic functional connectivity (iFC) was estimated using bilateral seeds in the amygdala, where iFC has shown susceptibility to early life stress, and in sensory areas, which have exhibited less susceptibility to early life hazards. Biological and psychosocial risk factors were examined for associations with iFC. Three resting‐state networks were identified in within‐group brain maps: medial temporal/striatal, visual, and auditory networks. Infants from extremely poor families compared with those from more affluent families exhibited greater (i.e. less negative) iFC in precuneus for amygdala seeds; however, no group differences in iFC were observed for sensory area seeds. Height‐for‐age, a proxy for malnutrition/infection, was not associated with amygdala/precuneus iFC, whereas prenatal family conflict was positively correlated. Findings suggest that it is feasible to conduct infant fMRI studies in low‐resource settings. Challenges and practical steps for successful implementations are discussed

'Beautiful beasts' and brave warriors: the longevity of a Maasai stereotype

This chapter (in the fourth edition of a widely respected anthropological textbook) traces the production by imperial explorers and others of a stereotypical image of the Maasai people of Kenya, the continuities to the present day, and the uses to which this stereotype has been put

A Survey on Cache Management Mechanisms for Real-Time Embedded Systems

© ACM, 2015. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Computing Surveys, {48, 2, (November 2015)} http://doi.acm.org/10.1145/2830555Multicore processors are being extensively used by real-time systems, mainly because of their demand for increased computing power. However, multicore processors have shared resources that affect the predictability of real-time systems, which is the key to correctly estimate the worst-case execution time of tasks. One of the main factors for unpredictability in a multicore processor is the cache memory hierarchy. Recently, many research works have proposed different techniques to deal with caches in multicore processors in the context of real-time systems. Nevertheless, a review and categorization of these techniques is still an open topic and would be very useful for the real-time community. In this article, we present a survey of cache management techniques for real-time embedded systems, from the first studies of the field in 1990 up to the latest research published in 2014. We categorize the main research works and provide a detailed comparison in terms of similarities and differences. We also identify key challenges and discuss future research directions.King Saud University NSER