A reinforcement learning-based adaptive control model for future street planning an algorithm and a case study

With the emerging technologies in Intelligent Transportation System (ITS), the adaptive operation of road space is likely to be realised within decades. An intelligent street can learn and improve its decision-making on the right-of-way (ROW) for road users, liberating more active pedestrian space while maintaining traffic safety and efficiency. However, there is a lack of effective controlling techniques for these adaptive street infrastructures. To fill this gap in existing studies, we formulate this control problem as a Markov Game and develop a solution based on the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The proposed model can dynamically assign ROW for sidewalks, autonomous vehicles (AVs) driving lanes and on-street parking areas in real-time. Integrated with the SUMO traffic simulator, this model was evaluated using the road network of the South Kensington District against three cases of divergent traffic conditions: pedestrian flow rates, AVs traffic flow rates and parking demands. Results reveal that our model can achieve an average reduction of 3.87% and 6.26% in street space assigned for on-street parking and vehicular operations. Combined with space gained by limiting the number of driving lanes, the average proportion of sidewalks to total widths of streets can significantly increase by 10.13%

A reinforcement learning-based adaptive control model for future street planning an algorithm and a case study

With the emerging technologies in Intelligent Transportation System (ITS), the adaptive operation of road space is likely to be realised within decades. An intelligent street can learn and improve its decision-making on the right-of-way (ROW) for road users, liberating more active pedestrian space while maintaining traffic safety and efficiency. However, there is a lack of effective controlling techniques for these adaptive street infrastructures. To fill this gap in existing studies, we formulate this control problem as a Markov Game and develop a solution based on the multi-agent Deep Deterministic Policy Gradient (MADDPG) algorithm. The proposed model can dynamically assign ROW for sidewalks, autonomous vehicles (AVs) driving lanes and on-street parking areas in real-time. Integrated with the SUMO traffic simulator, this model was evaluated using the road network of the South Kensington District against three cases of divergent traffic conditions: pedestrian flow rates, AVs traffic flow rates and parking demands. Results reveal that our model can achieve an average reduction of 3.87% and 6.26% in street space assigned for on-street parking and vehicular operations. Combined with space gained by limiting the number of driving lanes, the average proportion of sidewalks to total widths of streets can significantly increase by 10.13%

Approximate optimum curbside utilisation for pick-up and drop-off (PUDO) and parking demands using reinforcement learning

With the uptake of automated transport, especially Pick-Up and Drop-Off (PUDO) operations of Shared Autonomous Vehicles (SAVs), the valet parking of passenger vehicles and delivery vans are envisaged to saturate our future streets. These emerging behaviours would join conventional on-street parking activities in an intensive competition for scarce curb resources. Existing curbside management approaches principally focus on those long-term parking demands, neglecting those short-term PUDO or docking events. Feasible solutions that coordinate diverse parking requests given limited curb space are still absent. We propose a Reinforcement Learning (RL) method to dynamically dispatch parking areas to accommodate a hybrid stream of parking behaviours. A partially-learning Deep Deterministic Policy Gradient (DDPG) algorithm is trained to approximate optimum dispatching strategies. Modelling results reveal satisfying convergence guarantees and robust learning patterns. Namely, the proposed model successfully discriminates parking demands of distinctive sorts and prioritises PUDOs and docking requests. Results also identify that when the demand-supply ratio situates at 2:1 to 4:1, the service rate approximates an optimal (83\%), and curbside occupancy surges to 80%. This work provides a novel intelligent dispatching model for diverse and fine-grained parking demands. Furthermore, it sheds light on deploying distinctive administrative strategies to the curbside in different contexts

Intelligent management of on-street parking provision for the autonomous vehicles era

The increasing degree of connectivity between vehicles and infrastructure, and the impending deployment of autonomous vehicles (AV) in urban streets, presents unique opportunities and challenges regarding the on-street parking provision for AVs. This study develops a novel simulation-optimisation approach for intelligent curbside management, based on a metaheuristic technique. The hybrid method balances curb lanes for driving or parking, aiming to minimise the average traffic delay. The model is tested using an idealised grid layout with a range of flow rates and parking policies. Results demonstrate delay decreased by 9%-27% from the benchmark case. Additionally, the traffic delay distribution shows the trade-offs between expanding road capacity and minimising traffic demand through curb management, indicating the interplay between curb parking and traffic management in the AV era

Adaptive road configurations for improved autonomous vehicle-pedestrian interactions using reinforcement learning

The deployment of Autonomous Vehicles (AVs) poses considerable challenges and unique opportunities for the design and management of future urban road infrastructure. In light of this disruptive transformation, the Right-Of-Way (ROW) composition of road space has the potential to be renewed. Design approaches and intelligent control models have been proposed to address this problem, but we lack an operational framework that can dynamically generate ROW plans for AVs and pedestrians in response to real-time demand. Based on microscopic traffic simulation, this study explores Reinforcement Learning (RL) methods for evolving ROW compositions. We implement a centralised paradigm and a distributive learning paradigm to separately perform the dynamic control on several road network configurations. Experimental results indicate that the algorithms have the potential to improve traffic flow efficiency and allocate more space for pedestrians. Furthermore, the distributive learning algorithm outperforms its centralised counterpart regarding computational cost (49.55%), benchmark rewards (25.35%), best cumulative rewards (24.58%), optimal actions (13.49%) and rate of convergence. This novel road management technique could potentially contribute to the flow-adaptive and active mobility-friendly streets in the AVs era

Screening the risk of obstructive sleep apnea by utilizing supervised learning techniques based on anthropometric features and snoring events

OBJECTIVES: Obstructive sleep apnea (OSA) is typically diagnosed by polysomnography (PSG). However, PSG is time-consuming and has some clinical limitations. This study thus aimed to establish machine learning models to screen for the risk of having moderate-to-severe and severe OSA based on easily acquired features. METHODS: We collected PSG data on 3529 patients from Taiwan and further derived the number of snoring events. Their baseline characteristics and anthropometric measures were obtained, and correlations among the collected variables were investigated. Next, six common supervised machine learning techniques were utilized, including random forest (RF), extreme gradient boosting (XGBoost), k-nearest neighbor (kNN), support vector machine (SVM), logistic regression (LR), and naïve Bayes (NB). First, data were independently separated into a training and validation dataset (80%) and a test dataset (20%). The approach with the highest accuracy in the training and validation phase was employed to classify the test dataset. Next, feature importance was investigated by calculating the Shapley value of every factor, which represented the impact on OSA risk screening. RESULTS: The RF produced the highest accuracy (of >70%) in the training and validation phase in screening for both OSA severities. Hence, we employed the RF to classify the test dataset, and results showed a 79.32% accuracy for moderate-to-severe OSA and 74.37% accuracy for severe OSA. Snoring events and the visceral fat level were the most and second most essential features of screening for OSA risk. CONCLUSIONS: The established model can be considered for screening for the risk of having moderate-to-severe or severe OSA

Machine learning approaches for predicting sleep arousal response based on heart rate variability, oxygen saturation, and body profiles.

OBJECTIVE: Obstructive sleep apnea is a global health concern, and several tools have been developed to screen its severity. However, most tools focus on respiratory events instead of sleep arousal, which can also affect sleep efficiency. This study employed easy-to-measure parameters-namely heart rate variability, oxygen saturation, and body profiles-to predict arousal occurrence. METHODS: Body profiles and polysomnography recordings were collected from 659 patients. Continuous heart rate variability and oximetry measurements were performed and then labeled based on the presence of sleep arousal. The dataset, comprising five body profiles, mean heart rate, six heart rate variability, and five oximetry variables, was then split into 80% training/validation and 20% testing datasets. Eight machine learning approaches were employed. The model with the highest accuracy, area under the receiver operating characteristic curve, and area under the precision recall curve values in the training/validation dataset was applied to the testing dataset and to determine feature importance. RESULTS: InceptionTime, which exhibited superior performance in predicting sleep arousal in the training dataset, was used to classify the testing dataset and explore feature importance. In the testing dataset, InceptionTime achieved an accuracy of 76.21%, an area under the receiver operating characteristic curve of 84.33%, and an area under the precision recall curve of 86.28%. The standard deviations of time intervals between successive normal heartbeats and the square roots of the means of the squares of successive differences between normal heartbeats were predominant predictors of arousal occurrence. CONCLUSIONS: The established models can be considered for screening sleep arousal occurrence or integrated in wearable devices for home-based sleep examination

Associations between risk of Alzheimer's disease and obstructive sleep apnea, intermittent hypoxia, and arousal responses: A pilot study.

OBJECTIVES: Obstructive sleep apnea (OSA) may increase the risk of Alzheimer's disease (AD). However, potential associations among sleep-disordered breathing, hypoxia, and OSA-induced arousal responses should be investigated. This study determined differences in sleep parameters and investigated the relationship between such parameters and the risk of AD. METHODS: Patients with suspected OSA were recruited and underwent in-lab polysomnography (PSG). Subsequently, blood samples were collected from participants. Patients' plasma levels of total tau (T-Tau) and amyloid beta-peptide 42 (Aβ42) were measured using an ultrasensitive immunomagnetic reduction assay. Next, the participants were categorized into low- and high-risk groups on the basis of the computed product (Aβ42 × T-Tau, the cutoff for AD risk). PSG parameters were analyzed and compared. RESULTS: We included 36 patients in this study, of whom 18 and 18 were assigned to the low- and high-risk groups, respectively. The average apnea-hypopnea index (AHI), apnea, hypopnea index [during rapid eye movement (REM) and non-REM (NREM) sleep], and oxygen desaturation index (≥3%, ODI-3%) values of the high-risk group were significantly higher than those of the low-risk group. Similarly, the mean arousal index and respiratory arousal index (R-ArI) of the high-risk group were significantly higher than those of the low-risk group. Sleep-disordered breathing indices, oxygen desaturation, and arousal responses were significantly associated with an increased risk of AD. Positive associations were observed among the AHI, ODI-3%, R-ArI, and computed product. CONCLUSIONS: Recurrent sleep-disordered breathing, intermittent hypoxia, and arousal responses, including those occurring during the NREM stage, were associated with AD risk. However, a longitudinal study should be conducted to investigate the causal relationships among these factors

Associations of the distance-saturation product and low-attenuation area percentage in pulmonary computed tomography with acute exacerbation in patients with chronic obstructive pulmonary disease

Background: Chronic obstructive pulmonary disease (COPD) has high global health concerns, and previous research proposed various indicators to predict mortality, such as the distance-saturation product (DSP), derived from the 6-min walk test (6MWT), and the low-attenuation area percentage (LAA%) in pulmonary computed tomographic images. However, the feasibility of using these indicators to evaluate the stability of COPD still remains to be investigated. Associations of the DSP and LAA% with other COPD-related clinical parameters are also unknown. This study, thus, aimed to explore these associations. Methods: This retrospective study enrolled 111 patients with COPD from northern Taiwan. Individuals’ data we collected included results of a pulmonary function test (PFT), 6MWT, life quality survey [i.e., the modified Medical Research Council (mMRC) scale and COPD assessment test (CAT)], history of acute exacerbation of COPD (AECOPD), and LAA%. Next, the DSP was derived by the distance walked and the lowest oxygen saturation recorded during the 6MWT. In addition, the DSP and clinical phenotype grouping based on clinically significant outcomes by previous study approaches were employed for further investigation (i.e., DSP of 290 m%, LAA% of 20%, and AECOPD frequency of ≥1). Mean comparisons and linear and logistic regression models were utilized to explore associations among the assessed variables. Results: The low-DSP group (<290 m%) had significantly higher values for the mMRC, CAT, AECOPD frequency, and LAA% at different lung volume scales (total, right, and left), whereas it had lower values of the PFT and 6MWT parameters compared to the high-DSP group. Significant associations (with high odds ratios) were observed of the mMRC, CAT, AECOPD frequency, and PFT with low- and high-DSP groupings. Next, the risk of having AECOPD was associated with the mMRC, CAT, DSP, and LAA% (for the total, right, and left lungs). Conclusion: A lower value of the DSP was related to a greater worsening of symptoms, more-frequent exacerbations, poorer pulmonary function, and more-severe emphysema (higher LAA%). These readily determined parameters, including the DSP and LAA%, can serve as indicators for assessing the COPD clinical course and may can serve as a guide to corresponding treatments

Glycemic responses to strenuous training in male professional cyclists with type 1 diabetes: a prospective observational study

Introduction This prospective observational study sought to establish the glycemic, physiological and dietary demands of strenuous exercise training as part of a 9-day performance camp in a professional cycling team with type 1 diabetes (T1D).Research design and methods Sixteen male professional cyclists with T1D on multiple daily injections (age: 27±4 years; duration of T1D: 11±5 years; body mass index: 22±2 kg/m2; glycated hemoglobin: 7%±1% (50±6 mmol/mol); maximum rate of oxygen consumption: 73±4 mL/kg/min) performed road cycle sessions (50%–90% of the anaerobic threshold, duration 1–6 hours) over 9 consecutive days. Glycemic (Dexcom G6), nutrition and physiological data were collected throughout. Glycemic data were stratified into predefined glycemic ranges and mapped alongside exercise physiology and nutritional parameters, as well as split into daytime and night-time phases for comparative analysis. Data were assessed by means of analysis of variance and paired t-tests. A p value of ≤0.05 (two-tailed) was statistically significant.Results Higher levels of antecedent hypoglycemia in the nocturnal hours were associated with greater time spent in next-day hypoglycemia overall (p=0.003) and during exercise (p=0.019). Occurrence of nocturnal hypoglycemia was associated with over three times the risk of next-day hypoglycemia (p<0.001) and a twofold risk of low glucose during cycling (p<0.001). Moreover, there was trend for a greater amount of time spent in mild hypoglycemia during the night compared with daytime hours (p=0.080).Conclusion The higher prevalence of nocturnal hypoglycemia was associated with an increased risk of next-day hypoglycemia, which extended to cycle training sessions. These data highlight the potential need for additional prebed carbohydrates and/or insulin dose reduction strategies around exercise training in professional cyclists with T1D.Trial registration number DRKS00019923