Adversarial Imitation Learning from Incomplete Demonstrations

Imitation learning targets deriving a mapping from states to actions, a.k.a. policy, from expert demonstrations. Existing methods for imitation learning typically require any actions in the demonstrations to be fully available, which is hard to ensure in real applications. Though algorithms for learning with unobservable actions have been proposed, they focus solely on state information and overlook the fact that the action sequence could still be partially available and provide useful information for policy deriving. In this paper, we propose a novel algorithm called Action-Guided Adversarial Imitation Learning (AGAIL) that learns a policy from demonstrations with incomplete action sequences, i.e., incomplete demonstrations. The core idea of AGAIL is to separate demonstrations into state and action trajectories, and train a policy with state trajectories while using actions as auxiliary information to guide the training whenever applicable. Built upon the Generative Adversarial Imitation Learning, AGAIL has three components: a generator, a discriminator, and a guide. The generator learns a policy with rewards provided by the discriminator, which tries to distinguish state distributions between demonstrations and samples generated by the policy. The guide provides additional rewards to the generator when demonstrated actions for specific states are available. We compare AGAIL to other methods on benchmark tasks and show that AGAIL consistently delivers comparable performance to the state-of-the-art methods even when the action sequence in demonstrations is only partially available.Comment: Accepted to International Joint Conference on Artificial Intelligence (IJCAI-19

Are There Alternatives to Greenbelts? Evidence from a New Land-use Transport Interaction Model for Greater Beijing

Urban greenbelts are considered a key instrument for shaping sustainable urban growth and protecting the environment in a large number of cities in the world. In most cities, there is a widely shared belief that urban greenbelts are beneficial to the natural environment. By contrast, there is little understanding of the underlying economic impacts of greenbelts and other green space configurations in fast growing cities. The unprecedented rate and scale of urbanisation in the emerging economies has brought the role of greenbelts into an even sharper focus. In cities within these fast growing economies, the urban population is expected to double in the coming decades, which means that greenbelts are under great pressures to adapt to the large forthcoming growth. Few existing urban models are capable of addressing the dynamic nature of the urban transformations and predicting the impacts of urban greenbelts in the developing world. This prompts us to develop a new modelling method that is capable of assessing the impacts of different configurations, scales and locations of green spaces. We then use it to examine alternative futures to the greenbelt through a case study of Greater Beijing. The method we developed is a new variant in the land use-transport interaction (LUTI) model family. This model is capable of addressing the non-equilibrium nature of urban land use and transport development and the equilibrium nature of the day-to-day adaptations made by businesses and citizens. This LUTI model aims to answer the following questions: what are the short-term and long-term economic impacts of a greenbelt on a fast growing city? Which alternative green space configuration performs better in terms of economic well-being and travel costs? Where and how much should the greenbelt land be progressively reshaped or released as the city grows? The new LUTI model is calibrated and validated using data collected for 1990, 2000 and 2010 for Greater Beijing, The model is first tested retrospectively through revisiting the past greenbelt policies in Beijing from 1990 to 2010. Then the impacts of different future green space configurations from 2010 to 2030 are predicted and assessed through quantifying economic costs/benefits and travel costs for socio-economic groups. The model results suggest that under rapid transformative urban change, the configuration, scale and location of a greenbelt have a significant impact on a city’s economic efficiency. Such impacts will transcend the greenbelt boundary, and even the boundary of Beijing Municipality, onto the entire city region. A narrow greenbelt launched in the early age of urban expansion could lead to spatial mismatch of residents and jobs. A wide and strictly controlled ring-shaped greenbelt is not the highest performing intervention either, in terms of economic well-being. The green-wedges configuration is a remedial policy that balances the economic efficiency and environmental benefits. Intensive development around metro/rail stations in the designated greenbelt could reduce spatial costs and promote sustainable travel modes. This implies that a careful siting of new development within existing designations of the greenbelt can be beneficial in terms of economic well-being and sustainable transport.CSC Cambridge International Scholarshi

Are There Alternatives to Greenbelts? Evidence from a New Land-use Transport Interaction Model for Greater Beijing

Urban greenbelts are considered a key instrument for shaping sustainable urban growth and protecting the environment in a large number of cities in the world. In most cities, there is a widely shared belief that urban greenbelts are beneficial to the natural environment. By contrast, there is little understanding of the underlying economic impacts of greenbelts and other green space configurations in fast growing cities. The unprecedented rate and scale of urbanisation in the emerging economies has brought the role of greenbelts into an even sharper focus. In cities within these fast growing economies, the urban population is expected to double in the coming decades, which means that greenbelts are under great pressures to adapt to the large forthcoming growth. Few existing urban models are capable of addressing the dynamic nature of the urban transformations and predicting the impacts of urban greenbelts in the developing world. This prompts us to develop a new modelling method that is capable of assessing the impacts of different configurations, scales and locations of green spaces. We then use it to examine alternative futures to the greenbelt through a case study of Greater Beijing. The method we developed is a new variant in the land use-transport interaction (LUTI) model family. This model is capable of addressing the non-equilibrium nature of urban land use and transport development and the equilibrium nature of the day-to-day adaptations made by businesses and citizens. This LUTI model aims to answer the following questions: what are the short-term and long-term economic impacts of a greenbelt on a fast growing city? Which alternative green space configuration performs better in terms of economic well-being and travel costs? Where and how much should the greenbelt land be progressively reshaped or released as the city grows? The new LUTI model is calibrated and validated using data collected for 1990, 2000 and 2010 for Greater Beijing, The model is first tested retrospectively through revisiting the past greenbelt policies in Beijing from 1990 to 2010. Then the impacts of different future green space configurations from 2010 to 2030 are predicted and assessed through quantifying economic costs/benefits and travel costs for socio-economic groups. The model results suggest that under rapid transformative urban change, the configuration, scale and location of a greenbelt have a significant impact on a city’s economic efficiency. Such impacts will transcend the greenbelt boundary, and even the boundary of Beijing Municipality, onto the entire city region. A narrow greenbelt launched in the early age of urban expansion could lead to spatial mismatch of residents and jobs. A wide and strictly controlled ring-shaped greenbelt is not the highest performing intervention either, in terms of economic well-being. The green-wedges configuration is a remedial policy that balances the economic efficiency and environmental benefits. Intensive development around metro/rail stations in the designated greenbelt could reduce spatial costs and promote sustainable travel modes. This implies that a careful siting of new development within existing designations of the greenbelt can be beneficial in terms of economic well-being and sustainable transport.CSC Cambridge International Scholarshi

Investigating the Effects of Robot Engagement Communication on Learning from Demonstration

Robot Learning from Demonstration (RLfD) is a technique for robots to derive policies from instructors' examples. Although the reciprocal effects of student engagement on teacher behavior are widely recognized in the educational community, it is unclear whether the same phenomenon holds true for RLfD. To fill this gap, we first design three types of robot engagement behavior (attention, imitation, and a hybrid of the two) based on the learning literature. We then conduct, in a simulation environment, a within-subject user study to investigate the impact of different robot engagement cues on humans compared to a "without-engagement" condition. Results suggest that engagement communication significantly changes the human's estimation of the robots' capability and significantly raises their expectation towards the learning outcomes, even though we do not run actual learning algorithms in the experiments. Moreover, imitation behavior affects humans more than attention does in all metrics, while their combination has the most profound influences on humans. We also find that communicating engagement via imitation or the combined behavior significantly improve humans' perception towards the quality of demonstrations, even if all demonstrations are of the same quality.Comment: Under revie

A Unified Contrastive Transfer Framework with Propagation Structure for Boosting Low-Resource Rumor Detection

The truth is significantly hampered by massive rumors that spread along with breaking news or popular topics. Since there is sufficient corpus gathered from the same domain for model training, existing rumor detection algorithms show promising performance on yesterday's news. However, due to a lack of training data and prior expert knowledge, they are poor at spotting rumors concerning unforeseen events, especially those propagated in different languages (i.e., low-resource regimes). In this paper, we propose a unified contrastive transfer framework to detect rumors by adapting the features learned from well-resourced rumor data to that of the low-resourced. More specifically, we first represent rumor circulated on social media as an undirected topology, and then train a Multi-scale Graph Convolutional Network via a unified contrastive paradigm. Our model explicitly breaks the barriers of the domain and/or language issues, via language alignment and a novel domain-adaptive contrastive learning mechanism. To enhance the representation learning from a small set of target events, we reveal that rumor-indicative signal is closely correlated with the uniformity of the distribution of these events. We design a target-wise contrastive training mechanism with three data augmentation strategies, capable of unifying the representations by distinguishing target events. Extensive experiments conducted on four low-resource datasets collected from real-world microblog platforms demonstrate that our framework achieves much better performance than state-of-the-art methods and exhibits a superior capacity for detecting rumors at early stages.Comment: A significant extension of the first contrastive approach for low-resource rumor detection (arXiv:2204.08143

Optimization of a static headspace GC-MS method and its application in metabolic fingerprinting of the leaf volatiles of 42 citrus cultivars

Citrus leaves, which are a rich source of plant volatiles, have the beneficial attributes of rapid growth, large biomass, and availability throughout the year. Establishing the leaf volatile profiles of different citrus genotypes would make a valuable contribution to citrus species identification and chemotaxonomic studies. In this study, we developed an efficient and convenient static headspace (HS) sampling technique combined with gas chromatography-mass spectrometry (GC-MS) analysis and optimized the extraction conditions (a 15-min incubation at 100 ˚C without the addition of salt). Using a large set of 42 citrus cultivars, we validated the applicability of the optimized HS-GC-MS system in determining leaf volatile profiles. A total of 83 volatile metabolites, including monoterpene hydrocarbons, alcohols, sesquiterpene hydrocarbons, aldehydes, monoterpenoids, esters, and ketones were identified and quantified. Multivariate statistical analysis and hierarchical clustering revealed that mandarin (Citrus reticulata Blanco) and orange (Citrus sinensis L. Osbeck) groups exhibited notably differential volatile profiles, and that the mandarin group cultivars were characterized by the complex volatile profiles, thereby indicating the complex nature and diversity of these mandarin cultivars. We also identified those volatile compounds deemed to be the most useful in discriminating amongst citrus cultivars. This method developed in this study provides a rapid, simple, and reliable approach for the extraction and identification of citrus leaf volatile organic compound, and based on this methodology, we propose a leaf volatile profile-based classification model for citrus