Multi-agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC)

The population is steadily increasing worldwide resulting in intractable traffic congestion in dense urban areas. Adaptive Traffic Signal Control (ATSC) has shown strong potential to effectively alleviate urban traffic congestion by adjusting the signal timing plans in real-time in response to traffic fluctuations to achieve the desired objectives (e.g., minimizing delay). Efficient and robust ATSC can be designed using a multi-agent reinforcement learning (MARL) approach in which each controller (agent) is responsible for the control of traffic lights around a single traffic junction. Applying MARL approaches to ATSC problem is associated with a few challenges as agents typically react to changes in the environment at the individual level but the overall behaviour of all agents may not be optimal. This dissertation presents the development and evaluation of a novel system of Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC). The MARLIN-ATSC control system is developed to provide a self-learning ATSC using a synergetic combination of reinforcement learning approaches and game theory. MARLIN-ATSC operates in two modes: (1) independent mode, i.e. each intersection controller operates independently of other agents; and (2) integrated mode, where each controller coordinates the signal control actions with the neighbouring intersections. The system was tested on three networks (i.e., small, medium, large-scale) to ensure seamless transferability of the system design and results. The large-scale application was conducted on a computerized testbed network of 60 intersections in the lower downtown core of the City of Toronto for the morning rush hour handling 25,000 trips. The results show unprecedented reduction in the average intersection delay ranging from 27% in mode 1 to 39% in mode 2 at the network level; and travel time savings of 15% in mode 1 and 26% in mode 2, along the busiest routes in downtown Toronto. The thesis shows how mathematical modelling of the traffic control problem as a stochastic control problem, combined with the utilisation of artificial intelligence techniques such as reinforcement learning in a game-theory setup, can provide highly useful and economically inexpensive solutions to real-life problems such as urban traffic congestion.Ph

Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLIN-ATSC)

Traffic congestion in Greater Toronto Area costs Canada $6 billion /year and is expected to grow up to$ 15 billion /year in the next few decades. Adaptive Traffic Signal Control(ATSC) is a promising technique to alleviate traffic congestion. For medium-large transportation networks, coordinated ATSC is becoming a challenging problem because the number of system states and actions grows exponentially as the number of networked intersections grows. Efficient and robust controllers can be designed using a multi-agent reinforcement learning (MARL) approach in which each controller (agent) is responsible for the control of traffic lights around a single traffic junction. This paper presents a novel, decentralized and coordinated adaptive real-time traffic signal control system using Multi-Agent Reinforcement Learning for Integrated Network of Adaptive Traffic Signal Controllers (MARLINATSC) that aims to minimize the total vehicle delay in the traffic network. The system is tested using microscopic traffic simulation software (PARAMICS) on a network of 5 signalized intersections in Downtown Toronto. The performance of MARLIN-ATSC is compared against two approaches: the conventional pretimed signal control (B1) and independent RL-based control agents (B2), i.e. with no coordination. The results show that network-wide average delay savings range from 32% to 63% relative to B1 and from 7% to 12% relative to B2 under different demand levels and arrival profiles

SELF-LEARNING ACYCLIC ADAPTIVE TRAFFIC SIGNAL CONTROL

Population is steadily increasing worldwide. Consequently the demand for mobility is increasing, traffic congestion is deteriorating, and undesirable changes in the environment are becoming major concerns. Infrastructure improvement have been has been th

The effect of Alnus incana (L.) Moench extracts in ameliorating iron overload-induced hepatotoxicity in male albino rats

Abstract Iron overload causes multiorgan dysfunction and serious damage. Alnus incana from the family Betulaceae, widely distributed in North America, is used for treating diseases. In this study, we investigated the iron chelating, antioxidant, anti-inflammatory, and antiapoptotic activities of the total and butanol extract from Alnus incana in iron-overloaded rats and identified the bioactive components in both extracts using liquid chromatography-mass spectrometry. We induced iron overload in the rats via six intramuscular injections of 12.5 mg iron dextran/100 g body weight for 30 days. The rats were then administered 60 mg ferrous sulfate /kg body weight once daily using a gastric tube. The total and butanol extracts were given orally, and the reference drug (deferoxamine) was administered subcutaneously for another month. After two months, we evaluated the biochemical, histopathological, histochemical, and immunohistochemical parameters. Iron overload significantly increased the serum iron level, liver biomarker activities, hepatic iron content, malondialdehyde, tumor necrosis factor-alpha, and caspase-3 levels. It also substantially (P < 0.05) reduced serum albumin, total protein, and total bilirubin content, and hepatic reduced glutathione levels. It caused severe histopathological alterations compared to the control rats, which were markedly (P < 0.05) ameliorated after treatment. The total extract exhibited significantly higher anti-inflammatory and antiapoptotic activities but lower antioxidant and iron-chelating activities than the butanol extract. Several polyphenolic compounds, including flavonoids and phenolic acids, were detected by ultraperformance liquid chromatography-electrospray ionization-quadrupole time-of-flight mass spectrometry (UPLC-ESI-QTOF-MS) analysis. Our findings suggest that both extracts might alleviate iron overload-induced hepatoxicity and other pathological conditions characterized by hepatic iron overload, including thalassemia and sickle-cell anemia

Synthesis and In Vitro Antiproliferative Activity of 11-Substituted Neocryptolepines with a Branched ω-Aminoalkylamino Chain

Neocryptolepine, which is a kind of tetracyclic indoloquinoline alkaloid, exhibits the inhibition of topoisomerase II and shows antiproliferative activity. The present study describes the synthesis and antiproliferative evaluation of several neocryptolepine analogues carrying a branched, functionalized dibasic side chain at C11. These 2-substituted 5-methyl-indolo[2,3-b]quinoline derivatives were prepared by nucleophilic aromatic substitution (SNAr) of 11-chloroneocryptolepines with appropriate 1,2- and 1,3-diamines. Some of the 11-(ω-aminoalkylamino) derivatives were further transformed into 11-ureido and thioureido analogues. Many of the prepared neocryptolepine derivatives showed submicromolar antiproliferative activity against the human leukemia MV4-11 cell line. Among them, 11-(3-amino-2-hydroxy)propylamino derivatives 2h and 2k were the most cytotoxic with a mean IC50 value of 0.042 μM and 0.057 μM against the MV4-11 cell line, 0.197 μM and 0.1988 μM against the A549 cell line, and 0.138 μM and 0.117 μM against the BALB/3T3 cell line, respectively