Natural Language Instructions for Intuitive Human Interaction with Robotic Assistants in Field Construction Work

The introduction of robots is widely considered to have significant potential of alleviating the issues of worker shortage and stagnant productivity that afflict the construction industry. However, it is challenging to use fully automated robots in complex and unstructured construction sites. Human-Robot Collaboration (HRC) has shown promise of combining human workers' flexibility and robot assistants' physical abilities to jointly address the uncertainties inherent in construction work. When introducing HRC in construction, it is critical to recognize the importance of teamwork and supervision in field construction and establish a natural and intuitive communication system for the human workers and robotic assistants. Natural language-based interaction can enable intuitive and familiar communication with robots for human workers who are non-experts in robot programming. However, limited research has been conducted on this topic in construction. This paper proposes a framework to allow human workers to interact with construction robots based on natural language instructions. The proposed method consists of three stages: Natural Language Understanding (NLU), Information Mapping (IM), and Robot Control (RC). Natural language instructions are input to a language model to predict a tag for each word in the NLU module. The IM module uses the result of the NLU module and building component information to generate the final instructional output essential for a robot to acknowledge and perform the construction task. A case study for drywall installation is conducted to evaluate the proposed approach. The obtained results highlight the potential of using natural language-based interaction to replicate the communication that occurs between human workers within the context of human-robot teams

Enabling Real Time Simulation Of Architecture, Engineering, Construction, And Facility Management (Aec/Fm) Systems: A Review Of Formalism, Model Architecture, And Data Representation

Within the past few years, simulation and modeling have been adopted by the Architecture, Engineering, Construction, and Facility Management (AEC/FM) industry because it provides unique opportunities to transform traditional human-centered decision-making in design, planning, execution, and maintenance practices into simulation-centered decision-making. This allows a decision-maker to understand system behavior, and predict future events without having to test scenarios and interfere with the current operations of an existing system. However, a major challenge in transitioning from human-centered decisionmaking to simulation-centered decision-making is to establish methods and guidelines that facilitate seamless integration of field data into project planning, monitoring and control, and operation. This integration of field data into a simulation model allows the dynamics and evolving nature of events that may occur in a real system to be integrated into the simulation for effective decision-making. The goal of this paper is to conduct a vigorous review and investigate the underlying challenges associated with aspects such as modular design, data interfaces and translation, and model development and validation. In particular, this paper: 1) presents Discrete Event Simulation Specification (DEVS) as a simulation formalism that facilitates decision-making across a single operation (e.g. earthmoving); 2) reviews High Level Architecture (HLA) as a distributed simulation platform that allows for the coupling of several standalone simulation models and data sources to improve decision-making across complex systems; and 3) describes the hierarchy of multimodal data sources for simulation and different formalisms (i.e. model representations) that facilitate storage and querying of real time data to support simulation and distributed simulation models. The materials presented in this paper are the latest findings of an ongoing collaborative project being conducted by the Simulation Taskforce of the Visualization, Information Modeling, and Simulation (VIMS) Committee of the American Society of Civil Engineers (ASCE), and aim to lay the foundation for future research and implementation efforts in AEC/FM domain-specific real-time simulation systems. In the long term, work in this area will enable researchers to address some of the most fundamental problems in AEC/FM simulation modeling

Distributed Simulation Platforms and Data Passing Tools for Natural Hazards Engineering: Reviews, Limitations, and Recommendations

Abstract There has been a strong need for simulation environments that are capable of modeling deep interdependencies between complex systems encountered during natural hazards, such as the interactions and coupled effects between civil infrastructure systems response, human behavior, and social policies, for improved community resilience. Coupling such complex components with an integrated simulation requires continuous data exchange between different simulators simulating separate models during the entire simulation process. This can be implemented by means of distributed simulation platforms or data passing tools. In order to provide a systematic reference for simulation tool choice and facilitating the development of compatible distributed simulators for deep interdependent study in the context of natural hazards, this article focuses on generic tools suitable for integration of simulators from different fields but not the platforms that are mainly used in some specific fields. With this aim, the article provides a comprehensive review of the most commonly used generic distributed simulation platforms (Distributed Interactive Simulation (DIS), High Level Architecture (HLA), Test and Training Enabling Architecture (TENA), and Distributed Data Services (DDS)) and data passing tools (Robot Operation System (ROS) and Lightweight Communication and Marshalling (LCM)) and compares their advantages and disadvantages. Three specific limitations in existing platforms are identified from the perspective of natural hazard simulation. For mitigating the identified limitations, two platform design recommendations are provided, namely message exchange wrappers and hybrid communication, to help improve data passing capabilities in existing solutions and provide some guidance for the design of a new domain-specific distributed simulation framework.http://deepblue.lib.umich.edu/bitstream/2027.42/173943/1/13753_2021_Article_361.pd