Human behavior understanding for worker-centered intelligent manufacturing

“In a worker-centered intelligent manufacturing system, sensing and understanding of the worker’s behavior are the primary tasks, which are essential for automatic performance evaluation & optimization, intelligent training & assistance, and human-robot collaboration. In this study, a worker-centered training & assistant system is proposed for intelligent manufacturing, which is featured with self-awareness and active-guidance. To understand the hand behavior, a method is proposed for complex hand gesture recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. To sense and understand the worker in a more comprehensive way, a multi-modal approach is proposed for worker activity recognition using Inertial Measurement Unit (IMU) signals obtained from a Myo armband and videos from a visual camera. To automatically learn the importance of different sensors, a novel attention-based approach is proposed to human activity recognition using multiple IMU sensors worn at different body locations. To deploy the developed algorithms to the factory floor, a real-time assembly operation recognition system is proposed with fog computing and transfer learning. The proposed worker-centered training & assistant system has been validated and demonstrated the feasibility and great potential for applying to the manufacturing industry for frontline workers. Our developed approaches have been evaluated: 1) the multi-view approach outperforms the state-of-the-arts on two public benchmark datasets, 2) the multi-modal approach achieves an accuracy of 97% on a worker activity dataset including 6 activities and achieves the best performance on a public dataset, 3) the attention-based method outperforms the state-of-the-art methods on five publicly available datasets, and 4) the developed transfer learning model achieves a real-time recognition accuracy of 95% on a dataset including 10 worker operations”--Abstract, page iv

Space Applications of Automation, Robotics and Machine Intelligence Systems (ARAMIS). Volume 4: Supplement, Appendix 4.3: Candidate ARAMIS Capabilities

Potential applications of automation, robotics, and machine intelligence systems (ARAMIS) to space activities, and to their related ground support functions, in the years 1985-2000, so that NASA may make informed decisions on which aspects of ARAMIS to develop. The study first identifies the specific tasks which will be required by future space projects. It then defines ARAMIS options which are candidates for those space project tasks, and evaluates the relative merits of these options. Finally, the study identifies promising applications of ARAMIS, and recommends specific areas for further research. The ARAMIS options defined and researched by the study group span the range from fully human to fully machine, including a number of intermediate options (e.g., humans assisted by computers, and various levels of teleoperation). By including this spectrum, the study searches for the optimum mix of humans and machines for space project tasks

Sidekick agents for sequential planning problems

Thesis (Ph. D.)--Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 127-131).Effective Al sidekicks must solve the interlinked problems of understanding what their human collaborator's intentions are and planning actions to support them. This thesis explores a range of approximate but tractable approaches to planning for AI sidekicks based on decision-theoretic methods that reason about how the sidekick's actions will effect their beliefs about unobservable states of the world, including their collaborator's intentions. In doing so we extend an existing body of work on decision-theoretic models of assistance to support information gathering and communication actions. We also apply Monte Carlo tree search methods for partially observable domains to the problem and introduce an ensemble-based parallelization strategy. These planning techniques are demonstrated across a range of video game domains.by Owen Macindoe.Ph.D

Improving field management by machine vision - a review

Growing population of people around the world and thus increasing demand to food products as well as high tendency for declining the cost of operations and environmental preserving cares intensify inclination toward the application of variable rate systems for agricultural treatments, in which machine vision as a powerful appliance has been paid vast attention by agricultural researchers and farmers as this technology consumers. Various applications have introduced for machine vision in different fields of agricultural and food industry till now that confirms the high potential of this approach for inspection of different parameters affecting productivity. Computer vision has been utilized for quantification of factors affecting crop growth in field; such as, weed, irrigation, soil quality, plant nutrients and fertilizers in several cases. This paper presents some of these successful applications in addition to representing an introduction to machine vision

Lecture Video Transformation through An Intelligent Analysis and Post-processing System

Lecture videos are good sources for people to learn new things. Students commonly use online videos to explore various domains. However, some recorded videos are posted on online platforms without being post-processed due to technology and resource limitations. In this work, we focus on the research of developing an intelligent system to automatically extract essential information, including the main instructor and screen, in a lecture video in several scenarios by using modern deep learning techniques. This thesis aims to combine the extracted essential information to render the videos and generate a new layout with a smaller file size than the original one. Another benefit of using this approach is that the users may save video post-processing time and costs. State-of-the-art object detection models, an algorithm to correct screen display, tracking the instructor, and other deep learning techniques were adopted in the system to detect both the main instructor and the screen in given videos without much of the computational burden. There are four main contributions: 1. We built an intelligent video analysis and post-processing system to extract and reframe detected objects from lecture videos. 2. We proposed a post-processing algorithm to localize the frontal human torso position in processing a sequence of frames in the videos. 3. We proposed a novel deep learning approach to distinguish the main instructor from other instructors or audiences in several complex situations. 4. We proposed an algorithm to extract the four edge points of a screen at the pixel level and correct the screen display in various scenarios

Development and Validation of a Basic Ground Skills Assessment for Equine-Assisted Services

The equine-assisted services program at Utah State University produced and validated an assessment process used to test equines for suitability for equine-assisted services. Equine-assisted services incorporate the interaction of humans who face mental, physical, emotional, and/or social challenges and equines for therapeutic purposes. Recreational, physical, mental, social, and/or emotional goals are met through various equine-assisted services such as therapies, equine-assisted learning, and horsemanship. Due to the potential for human injury while interacting with equines, it is important to ensure equines are assessed for safe behavior prior to participation in equine-assisted services. This is why our group developed and tested the Basic Ground Skills Assessment. It was found to be valid and moderately reliable. Use of this assessment in the industry may increase the safety of humans by ensuring only suitable equines are engaged in equine-assisted services. Additionally, a survey was distributed to Professional Association of Therapeutic Horsemanship International centers to better understand the equine evaluation procedures currently in practice. The survey’s results indicated that centers may benefit from a validated assessment process, as we have developed, as there was not a common assessment process currently in place

A STUDY OF THE USE OF MIXED REALITY FOR CAPTURING HUMAN OBSERVATION AND INFERENCES IN PRODUCTION ENVIRONMENTS

Augmented and mixed reality is already considered as needful technology of the modern production systems. It is primarily employed to virtualize proper digital content, mainly related to 3D objects, into the human visual field allowing people to visualize and understand complex spatial shapes, their mutual relations, and positioning. Yet, the huge potential of the technology is waiting to be revealed in its usage for collecting and recording human observations and inferences about the context of the production environment. Its bi-directional interface makes it the most direct and the most efficient knowledge capturing means to date. The paper presents the challenges and benefits that come from the usage of a conceptual interface of an mixed reality application that is designed to collect data, semantics and knowledge about the production context directly from the man-in-process. As a production environment for the development, implementation, and testing of mixed reality applications for this purpose, various processes for the assembly and maintenance of medium-voltage equipment were used

Major requirements for building Smart Homes in Smart Cities based on Internet of Things technologies

The recent boom in the Internet of Things (IoT) will turn Smart Cities and Smart Homes (SH) from hype to reality. SH is the major building block for Smart Cities and have long been a dream for decades, hobbyists in the late 1970s made Home Automation (HA) possible when personal computers started invading home spaces. While SH can share most of the IoT technologies, there are unique characteristics that make SH special. From the result of a recent research survey on SH and IoT technologies, this paper defines the major requirements for building SH. Seven unique requirement recommendations are defined and classified according to the specific quality of the SH building blocks

手術者の手の動きの動画像解析に基づく縫合手術構成段階の認識と早期認識に関する研究

早大学位記番号:新7574早稲田大

Semi-structured decision processes : a conceptual framework for understanding human-automation systems

Thesis (Ph.D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 1999.Includes bibliographical references (p. 191-199).The purpose of this work is to improve understanding of existing and proposed decision systems, ideally to improve the design of future systems. A "decision system" is defined as a collection of information-processing components-often involving humans and automation (e.g., computers)-that interact towards a common set of objectives. Since a key issue in the design of decision systems is the division of work between humans and machines (a task known as "function allocation"), this thesis is primarily intended to help designers incorporate automation more appropriately within these systems. This thesis does not provide a design methodology, but introduces a way to qualitatively analyze potential designs early in the system design process. A novel analytical framework is presented, based on the concept of "semi-Structured" decision processes. It is believed that many decisions involve both well-defined "Structured" parts (e.g., formal procedures, traditional algorithms) and ill-defined "Unstructured" parts (e.g., intuition, judgment, neural networks) that interact in a known manner. While Structured processes are often desired because they fully prescribe how a future decision (during "operation") will be made, they are limited by what is explicitly understood prior to operation. A system designer who incorporates Unstructured processes into a decision system understands which parts are not understood sufficiently, and relinquishes control by deferring decision-making from design to operation. Among other things, this design choice tends to add flexibility and robustness. The value of the semi-Structured framework is that it forces people to consider system design concepts as operational decision processes in which both well-defined and ill-defined components are made explicit. This may provide more insight into decision systems, and improve understanding of the implications of design choices. The first part of this thesis defines the semi-Structured process and introduces a diagrammatic notation for decision process models. In the second part, the semi-Structured framework is used to understand and explain highly evolved decision system designs (these are assumed to be representative of "good" designs) whose components include feedback controllers, alerts, decision aids, and displays. Lastly, the semi-Structured framework is applied to a decision system design for a mobile robot.by William N. Kaliardos.Ph.D