Unsupervised Dynamics Prediction with Object-Centric Kinematics

Human perception involves discerning complex multi-object scenes into time-static object appearance (ie, size, shape, color) and time-varying object motion (ie, location, velocity, acceleration). This innate ability to unconsciously understand the environment is the motivation behind the success of dynamics modeling. Object-centric representations have emerged as a promising tool for dynamics prediction, yet they primarily focus on the objects' appearance, often overlooking other crucial attributes. In this paper, we propose Object-Centric Kinematics (OCK), a framework for dynamics prediction leveraging object-centric representations. Our model utilizes a novel component named object kinematics, which comprises low-level structured states of objects' position, velocity, and acceleration. The object kinematics are obtained via either implicit or explicit approaches, enabling comprehensive spatiotemporal object reasoning, and integrated through various transformer mechanisms, facilitating effective object-centric dynamics modeling. Our model demonstrates superior performance when handling objects and backgrounds in complex scenes characterized by a wide range of object attributes and dynamic movements. Moreover, our model demonstrates generalization capabilities across diverse synthetic environments, highlighting its potential for broad applicability in vision-related tasks.Comment: 15 pages, 6 figures, 4 table

Algorithmic Framework for Improving Heuristics in Stochastic, Stage-Wise Optimization Problems

Algorithmic Framework for Improving Heuristics in Stochastic, Stage-Wise Optimization Problems Jaein Choi 172 Pages Directed by Dr. Jay H. Lee and Dr. Matthew J. Realff The goal of this thesis is the development of a computationally tractable solution method for stochastic, stage-wise optimization problems. In order to achieve the goal, we have developed a novel algorithmic framework based on Dynamic Programming (DP) for improving heuristics. The propose method represents a systematic way to take a family of solutions and patch them together as an improved solution. However, patching is accomplished in state space, rather than in solution space. Since the proposed approach utilizes simulation with heuristics to circumvent the curse of dimensionality of the DP, it is named as Dynamic Programming in Heuristically Restricted State Space. The proposed algorithmic framework is applied to stochastic Resource Constrained Project Scheduling problems, a real-world optimization problem with a high dimensional state space and significant uncertainty equivalent to billions of scenarios. The real-time decision making policy obtained by the proposed approach outperforms the best heuristic applied in simulation stage to form the policy. The proposed approach is extended with the idea of Q-Learning technique, which enables us to build empirical state transition rules through simulation, for stochastic optimization problems with complicated state transition rules. Furthermore, the proposed framework is applied to a stochastic supply chain management problem, which has high dimensional action space as well as high dimensional state space, with a novel concept of implicit sub-action space that efficiently restricts action space for each state in the restricted state space. The resulting real-time policy responds to the time varying demand for products by stitching together decisions made by the heuristics and improves overall performance of the supply chain. The proposed approach can be applied to any problem formulated as a stochastic DP, provided that there are reasonable heuristics available for simulation.Ph.D.Committee Chair: Lee, Jay H.; Committee Co-Chair: Realff, Matthew R.; Committee Member: Ahmed, Shabbir; Committee Member: Ayhan, Hayriye; Committee Member: Bommarius, Andreas S

Generalized Tsallis Entropy Reinforcement Learning and Its Application to Soft Mobile Robots

In this paper, we present a new class of Markov decision processes (MDPs), called Tsallis MDPs, with Tsallis entropy maximization, which generalizes existing maximum entropy reinforcement learning (RL). A Tsallis MDP provides a unified framework for the original RL problem and RL with various types of entropy, including the well-known standard Shannon-Gibbs (SG) entropy, using an additional real-valued parameter, called an entropic index. By controlling the entropic index, we can generate various types of entropy, including the SG entropy, and a different entropy results in a different class of the optimal policy in Tsallis MDPs. We also provide a full mathematical analysis of Tsallis MDPs. Our theoretical result enables us to use any positive entropic index in RL. To handle complex and large-scale problems such as learning a controller for soft mobile robot, we also propose a Tsallis actor-critic (TAC). For a different type of RL problems, we find that a different value of the entropic index is desirable and empirically show that TAC with a proper entropic index outperforms the state-of-the-art actor-critic methods. Furthermore, to alleviate the effort for finding the proper entropic index, we propose a linear scheduling method where an entropic index linearly increases as the number of interactions increases. In simulations, the linear scheduling shows the fast convergence speed and a similar performance to TAC with the optimal entropic index, which is a useful property for real robot applications. We also apply TAC with the linear scheduling to learn a feedback controller of a soft mobile robot and shows the best performance compared to other existing actor critic methods in terms of convergence speed and the sum of rewards. Consequently, we empirically show that the proposed method efficiently learns a controller of soft mobile robots

Modeling G2019S-LRRK2 Sporadic Parkinson's Disease in 3D Midbrain Organoids

Summary: Recent advances in generating three-dimensional (3D) organoid systems from stem cells offer new possibilities for disease modeling and drug screening because organoids can recapitulate aspects of in vivo architecture and physiology. In this study, we generate isogenic 3D midbrain organoids with or without a Parkinson's disease-associated LRRK2 G2019S mutation to study the pathogenic mechanisms associated with LRRK2 mutation. We demonstrate that these organoids can recapitulate the 3D pathological hallmarks observed in patients with LRRK2-associated sporadic Parkinson's disease. Importantly, analysis of the protein-protein interaction network in mutant organoids revealed that TXNIP, a thiol-oxidoreductase, is functionally important in the development of LRRK2-associated Parkinson's disease in a 3D environment. These results provide proof of principle for the utility of 3D organoid-based modeling of sporadic Parkinson's disease in advancing therapeutic discovery. : H. Kim and colleagues generated 3D midbrain organoids containing a G2019S mutation in LRRK2 and used this system for sporadic Parkinson’s disease (PD) modeling in vitro. Their results demonstrate that these 3D midbrain organoids can recapitulate the pathological features of LRRK2-associated sporadic PD. These results demonstrate that the 3D midbrain organoids are invaluable for recapitulating PD phenotypes and understanding the molecular underpinnings of these phenotypes. Keywords: Parkinson's disease, iPSC, organoids, disease modeling, midbrai

A missing component of Arctic warming: black carbon from gas flaring

Gas flaring during oil extraction over the Arctic region is the primary source of warming-inducing aerosols (e.g. black carbon (BC)) with a strong potential to affect regional climate change. Despite continual BC emissions near the Arctic Ocean via gas flaring, the climatic impact of BC related to gas flaring remains uncertain. Here, we present simulations of potential gas flaring using an earth system model with comprehensive aerosol physics to show that increases in BC from gas flaring can potentially explain a significant fraction of Arctic warming. BC emissions from gas flaring over high latitudes contribute to locally confined warming over the source region, especially during the Arctic spring through BC-induced local albedo reduction. This local warming invokes remote and temporally lagging sea-ice melting feedback processes over the Arctic Ocean during winter. Our findings imply that a regional change in anthropogenic aerosol forcing is capable of changing Arctic temperatures in regions far from the aerosol source via time-lagged, sea-ice-related Arctic physical processes. We suggest that both energy consumption and production processes can increase Arctic warming

Impacts of local vs. trans-boundary emissions from different sectors on PM2.5 exposure in South Korea during the KORUS-AQ campaign

High concentrations of PM2.5 have become a serious environmental issue in South Korea, which ranked 1st or 2nd among OECD countries in terms of population exposure to PM2.5. Quantitative understanding of PM2.5 source attribution is thus crucial for developing efficient air quality mitigation strategies. Here we use a suite of extensive observations of PM2.5 and its precursors concentrations during the international KORea-US cooperative Air Quality field study in Korea (KORUS-AQ) in May???June 2016 to investigate source contributions to PM2.5 in South Korea under various meteorological conditions. For the quantitative analysis, we updated a 3-D chemical transport model, GEOS-Chem, and its adjoint with the latest regional emission inventory and other recent findings. The updated model is evaluated by comparing against observed daily PM2.5 and its component concentrations from six ground sites (Bangnyung, Bulkwang, Olympic park, Gwangju, Ulsan, and Jeju). Overall, simulated concentrations of daily PM2.5 and its components are in a good agreement with observations over the peninsula. We conduct an adjoint sensitivity analysis for simulated surface level PM2.5 concentrations at five ground sites (except for Bangnyung because of its small population) under four different meteorological conditions: dynamic weather, stagnant, extreme pollution, and blocking periods. Source contributions by regions vary greatly depending on synoptic meteorological conditions. Chinese contribution accounts for almost 68% of PM2.5 in surface air in South Korea during the extreme pollution period of the campaign, whereas an enhanced contribution from domestic sources (57%) occurs for the blocking period. Results from our sensitivity analysis suggest that the reduction of domestic anthropogenic NH3 emissions could be most effective in reducing population exposure to PM2.5 in South Korea (effectiveness???=???14%) followed by anthropogenic SO2 emissions from Shandong region (effectiveness???=???11%), domestic anthropogenic NOx emissions (effectiveness???=???10%), anthropogenic NH3 emissions from Shandong region (effectiveness???=???8%), anthropogenic NOx emissions from Shandong region (effectiveness???=???7%), domestic anthropogenic OC emissions (effectiveness???=???7%), and domestic anthropogenic BC emissions (effectiveness???=???5%)

Global COVID-19 lockdown highlights humans as both threats and custodians of the environment

The global lockdown to mitigate COVID-19 pandemic health risks has altered human interactions with nature. Here, we report immediate impacts of changes in human activities on wildlife and environmental threats during the early lockdown months of 2020, based on 877 qualitative reports and 332 quantitative assessments from 89 different studies. Hundreds of reports of unusual species observations from around the world suggest that animals quickly responded to the reductions in human presence. However, negative effects of lockdown on conservation also emerged, as confinement resulted in some park officials being unable to perform conservation, restoration and enforcement tasks, resulting in local increases in illegal activities such as hunting. Overall, there is a complex mixture of positive and negative effects of the pandemic lockdown on nature, all of which have the potential to lead to cascading responses which in turn impact wildlife and nature conservation. While the net effect of the lockdown will need to be assessed over years as data becomes available and persistent effects emerge, immediate responses were detected across the world. Thus initial qualitative and quantitative data arising from this serendipitous global quasi-experimental perturbation highlights the dual role that humans play in threatening and protecting species and ecosystems. Pathways to favorably tilt this delicate balance include reducing impacts and increasing conservation effectiveness