Learning and Reacting with Inaccurate Prediction: Applications to Autonomous Excavation

Motivated by autonomous excavation, this work investigates solutions to a class of problem where disturbance prediction is critical to overcoming poor performance of a feedback controller, but where the disturbance prediction is intrinsically inaccurate. Poor feedback controller performance is related to a fundamental control problem: there is only a limited amount of disturbance rejection that feedback compensation can provide. It is known, however, that predictive action can improve the disturbance rejection of a control system beyond the limitations of feedback. While prediction is desirable, the problem in excavation is that disturbance predictions are prone to error due to the variability and complexity of soil-tool interaction forces. This work proposes the use of iterative learning control to map the repetitive components of excavation forces into feedforward commands. Although feedforward action shows useful to improve excavation performance, the non-repetitive nature of soil-tool interaction forces is a source of inaccurate predictions. To explicitly address the use of imperfect predictive compensation, a disturbance observer is used to estimate the prediction error. To quantify inaccuracy in prediction, a feedforward model of excavation disturbances is interpreted as a communication channel that transmits corrupted disturbance previews, for which metrics based on the sensitivity function exist. During field trials the proposed method demonstrated the ability to iteratively achieve a desired dig geometry, independent of the initial feasibility of the excavation passes in relation to actuator saturation. Predictive commands adapted to different soil conditions and passes were repeated autonomously until a pre-specified finish quality of the trench was achieved. Evidence of improvement in disturbance rejection is presented as a comparison of sensitivity functions of systems with and without the use of predictive disturbance compensation

A stochastic method for representation, modelling and fusion of excavated material in mining

The ability to safely and economically extract raw materials such as iron ore from a greater number of remote, isolated and possibly dangerous locations will become more pressing over the coming decades as easily accessible deposits become depleted. An autonomous mining system has the potential to make the mining process more efficient, predictable and safe under these changing conditions. One of the key parts of the mining process is the estimation and tracking of bulk material through the mining production chain. Current state-of-the-art tracking and estimation systems use a deterministic representation for bulk material. This is problematic for wide-scale automation of mine processes as there is no measurement of the uncertainty in the estimates provided. A probabilistic representation is critical for autonomous systems to correctly interpret and fuse the available data in order to make the most informed decision given the available information without human intervention. This thesis investigates whether bulk material properties can be represented probabilistically through a mining production chain to provide statistically consistent estimates of the material at each stage of the production chain. Experiments and methods within this thesis focus on the load-haul-dump cycle. The development of a representation of bulk material using lumped masses is presented. A method for tracking and estimation of these lumped masses within the mining production chain using an 'Augmented State Kalman Filter' (ASKF) is developed. The method ensures that the fusion of new information at different stages will provide statistically consistent estimates of the lumped mass. There is a particular focus on the feasibility and practicality of implementing a solution on a production mine site given the current sensing technology available and how it can be adapted for use within the developed estimation system (with particular focus on remote sensing and volume estimation)

Evaluation of the effects of user profile and interface characteristics on performance during robotic teleoperation.

Programa de P?s-Gradua??o em Instrumenta??o, Controle e Automa??o de Processos de Minera??o. Departamento de Engenharia de Controle e Automa??o, Escola de Minas, Universidade Federal de Ouro Preto.In the mining industry, operator safety has always been a crucial point, and therefore demands attention. In addition, given the high investments made in the enterprise, the search for a consistent and e cient production process is also relevant. To increase productivity and safety, teleoperation techniques can be used to allow the operator to be removed from the risk areas and operate the equipment from a safe distance. The work presented in this master's thesis was developed at the Instituto Tecnol?gico Vale (ITV) as a part of a broader project that aims the development of a conceptual framework that enables the analysis and validation of the application of teleoperation techniques in mining equipment. These equipment could be either used for exploration, engineering or extraction. More speci cally, this framework will permit the analysis of relevant information for teleoperation. This work represents the initial step taken, which comprises the execution of proof-of-concept tests in laboratory using a robotic system. Di erent teleoperation interfaces schemes consisting of video and haptic devices are analyzed under di erent aspects and the user point of view is taken into account. The goal is to analyze the e ects of the force feedback and of di erent visual feedback during teleoperation and to evaluate which prede ned users characteristics can contribute to better performance during teleoperation. Experimental tests were performed using a commercial mobile robotic platform Seekur Jr and teleoperation interfaces consisting of the haptic devices Novint Falcon and Geomagic Touch. This platform can be programmed in such a way that it operates emulating a mining equipment. For practical application-oriented tests, di erent operations from a mining process can be used as application of study (e.g. explosive charging or excavators operations). For the purposes of this work, interfaces for the control of the robotic platform locomotion were analyzed. In order to evaluate the in uence of interface characteristics and users pro le on performance during teleoperation two experiments were conducted. The experiments consisted of guiding the robotic platform in a scenario containing obstacles, using the di erent interfaces. In this work, delays in communication during teleoperation were not addressed. Results show that the use of force feedback e ectively contributes for better performances in term of task execution time, number of collisions with obstacles and task completion in teleoperation activities. Moreover, additional visual information, such as the environment map, can also increase perception of obstacles.Na ind?stria da minera??o, a seguran?a do operador sempre foi um ponto crucial, demandando, portanto, cuidadosa aten??o. Al?m disso, dados os altos investimentos realizados no empreendimento, a busca por um processo de produ??o consistente e e ciente ? tamb?m relevante. Com o objetivo de aumentar a produtividade e a seguran?a nas opera??es dentro da minera??o, t?cnicas de teleopera??o podem ser utilizadas para permitir que o operador seja removido das ?reas de risco e opere o equipamento ? dist?ncia. Em ess?ncia, a teleopera??o significa que ? poss?vel haver uma separa??o f?sica entre a unidade de comando (sistema local) e a unidade de execu??o (sistema remoto), ainda mantendo a rela??o entre os dois. Isto ?, apesar da separa??o f?sica, o sistema local ? ainda capaz de comandar o sistema remoto. Idealmente, para que tal comando seja feito de maneira mais e caz poss?vel, deseja-se prover ao operador uma experi?ncia com o ambiente remoto de forma natural e realista, de tal forma que ele tenha a sensa??o de estar interagindo diretamente com o ambiente remoto, por?m sem as perturba??es t?picas desses ambientes (ru?do, vibra??es, calor, etc.) e com a possibilidade de amplia??o dessa percep??o. Com a utiliza??o da teleopera??o, ? poss?vel, al?m de reduzir os riscos a que o operador est? exposto em suas atividades, melhorar seu desempenho na opera??o destes equipamentos durante a realiza??o das tarefas. O trabalho apresentado nesta disserta??o de mestrado foi desenvolvido no Instituto Tecnol?gico Vale (ITV) como parte de um projeto mais amplo que visa o desenvolvimento de um arcabou?o que permita a an?lise e valida??o da aplica??o de t?cnicas de teleopera??o a equipamentos de minera??o. Estes equipamentos podem ser usados tanto para tarefas de explora??o, quanto de engenharia ou extra??o. Mais especificamente, este arcabou?o permitir? a an?lise de qual informa??o ? relevante e auxilia o operador durante a opera??o do equipamento. No ?mbito deste projeto, o trabalho descrito nesta disserta??o de mestrado representa o primeiro passo dado, que compreende a execu??o de testes de conceito em laborat?rio usando um sistema rob?tico. Diferentes interfaces de teleopera??o compostas por dispositivos h?pticos e de v?deo s?o analisadas diante de diferentes aspectos e o ponto de vista do usu?rio ? levado em considera??o. O objetivo ? analisar os efeitos da realimenta??o de for?a e da realimenta??o visual durante a realiza??o de tarefas de maneira teleoperada e avaliar quais caracter?sticas predefinidas dos operadores podem contribuir para um melhor desempenho durante a teleopera??o de um equipamento. Para realizar os testes, a plataforma rob?tica m?vel comercial Seekur Jr e interfaces de teleopera??o compostas por dispositivos h?pticos Novint Falcon e Geomagic Touch s?o utilizados. Esta plataforma rob?tica pode ser programada de tal forma a operar emulando um equipamento de minera??o. Para a realiza??o de testes voltados a aplica??es pr?ticas, diferentes opera??es de um processo de minera??o podem ser utilizadas como aplica??o de estudo (carregamento de explosivos, opera??o de uma escavadeira, por exemplo). Para os fins de estudo deste trabalho, foram analisadas as interfaces para o controle da locomo??o da plataforma rob?tica. Para avaliar a influ?ncia das caracter?sticas da interface e do perfil dos usu?rios no desempenho durante a teleopera??o dois experimentos foram conduzidos. Os experimentos consistiram em comandar a plataforma rob?tica em um cen?rio contendo obst?culos utilizando as diferentes interfaces desenvolvidas. Neste trabalho, a presen?a de atrasos na comunica??o durante a teleopera??o n?o foi considerada. Os resultados mostram que o uso da realimenta??o de for?a efetivamente contribui para melhor desempenho durante a execu??o de tarefas de forma teleoperada, em termos do tempo de execu??o da tarefa, do n?mero de obst?culos colididos e do n?vel de conclus?o da tarefa. Al?m disso, informa??es visuais adicionais tais como um mapa do ambiente aumentam a percep??o de obst?culos

Evaluation of gaming environments for mixed reality interfaces and human supervisory control in telerobotics

Telerobotics refers to a branch of technology that deals with controlling a robot from a distance. It is commonly used to access difficult environments, reduce operating costs, and to improve comfort and safety. However, difficulties have emerged in telerobotics development. Effective telerobotics requires maximising operator performance and previous research has identified issues which reduce operator performance, such as operator attention being divided across the numerous custom built interfaces and continuous operator involvement in a high workload situation potentially causing exhaustion and subsequent operator error. This thesis evaluates mixed reality and human supervisory control concepts in a gaming engine environment for telerobotics. This concept is proposed in order to improve the effectiveness of current technology in telerobotic interfaces. Four experiments are reported in this thesis which covers virtual gaming environments, mixed reality interfaces, and human supervisory control and aims to advance telerobotics technology. This thesis argues that gaming environments are useful for building telerobotic interfaces and examines the properties required for telerobotics. A useful feature provided by gaming environments is that of overlying video on virtual objects to support mixed reality interfaces. Experiments in this thesis show that mixed reality interfaces provide useful information without distracting the operator from the task. This thesis introduces two response models based on the planning process of human supervisory control: Adaptation and Queue response models. The experimental results show superior user performance under these two response models compared to direct/manual control. In the final experiment a large number of novice users, with a diversity of backgrounds, used a robot arm to push blocks into a hole by using these two response models. Further analyses on evaluating the user performance on the interfaces with two response models were found to be well fitted by a Weibull distribution. Operators preferred the interface with the Queue response model over the interface with the Adaptation response model, and human supervisory control over direct/manual control. It is expected that the increased sophistication of control commands in a production system will usually be greater than those that were tested in this thesis, where limited time was available for automation development. Where that is the case the increases in human productivity using human supervisory control found in this experiment can be expected to be greater. The research conducted here has shown that mixed reality in gaming environments, when combined with human supervisory control, offers a good route for overcoming limitations in current telerobotics technology. Practical applications would benefit by the application of these methods, making it possible for the operator to have the necessary information available in a convenient and non-distracting form, considerably improving productivity

Application of neuroergonomics in the industrial design of mining equipment.

Neuroergonomics is an interdisciplinary field merging neuroscience and ergonomics to optimize performance. In order to design an optimal user interface, we must understand the cognitive processing involved. Traditional methodology incorporates self-assessment from the user. This dissertation examines the use of neurophysiological techniques in quantifying the cognitive processing involved in allocating cognitive resources. Attentional resources, cognitive processing, memory and visual scanning are examined to test the ecological validity of theoretical laboratory settings and how they translate to real life settings. By incorporating a non-invasive measurement technique, such as the quantitative electroencephalogram (QEEG), we are able to examine connectivity patterns in the brain during operation and discern whether or not a user has obtained expert status. Understanding the activation patterns during each phase of design will allow us to gauge whether our design has balanced the cognitive requirements of the user.Doctor of Philosophy (PhD) in Natural Resources Engineerin