Reasoning About Multi-Attribute Preferences (extended abstract)

Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Exploring Heuristic Action Selection in Agent Programming (extended abstract)

Rational agents programmed in agent programming languages derive their choice of action from their beliefs and goals. One of the main benefits of such programming languages is that they facilitate a highlevel and conceptually elegant specification of agent behaviour. Qualitative concepts alone, however, are not sufficient to specify that this behaviour is also nearly optimal, a quality typically also associated with rational agents. Optimality in this context refers to the costs and rewards associated with action execution. In this paper we extend the agent programming language GOAL with primitives that allow the specification of near-optimal behaviour and illustrate the use of these constructs by extending a GOAL Blocks World agent with various strategies to optimize its behaviour.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Crisis Decision Making Through a Shared Integrative Negotiation Mental Model

Decision making during crises takes place in (multi-agency) teams, in a bureaucratic political context. As a result, the common notion that during crises decision making should be done in line with a Command & Control structure is invalid. This paper shows that the best way for crisis decision making teams in a bureaucratic political context is to follow an integrative negotiation approach as the shared mental model of decision making. This conclusion is based on an analysis of crisis decision making by teams in a bureaucratic political context. First of all this explains why in a bureaucratic political context the Command & Control adage does not hold. Secondly, this paper motivates why crisis decision making in such context can be seen as a negotiation process. Further analysis of the given context shows that an assertive and cooperative approach suits crisis decision making best.Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

A Framework for Qualitative Multi-Criteria Preferences (extended abstract)

Intelligent SystemsElectrical Engineering, Mathematics and Computer Scienc

Viewpoint optimization for aiding grasp synthesis algorithms using reinforcement learning

Grasp synthesis for unknown objects is a challenging problem as the algorithms are expected to cope with missing object shape information. This missing information is a function of the vision sensor viewpoint. The majority of the grasp synthesis algorithms in literature synthesize a grasp by using one single image of the target object and making assumptions on the missing shape information. On the contrary, this paper proposes the use of robot's depth sensor actively: we propose an active vision methodology that optimizes the viewpoint of the sensor for increasing the quality of the synthesized grasp over time. By this way, we aim to relax the assumptions on the sensor's viewpoint and boost the success rates of the grasp synthesis algorithms. A reinforcement learning technique is employed to obtain a viewpoint optimization policy, and a training process and automated training data generation procedure are presented. The methodology is applied to a simple force-moment balance-based grasp synthesis algorithm, and a thousand simulations with five objects are conducted with random initial poses in which the grasp synthesis algorithm was not able to obtain a good grasp with the initial viewpoint. In 94% of these cases, the policy achieved to find a successful grasp.Accepted Author ManuscriptRobot DynamicsBiomechatronics & Human-Machine Contro

Active vision via extremum seeking for robots in unstructured environments: Applications in object recognition and manipulation

In this paper, a novel active vision strategy is proposed for optimizing the viewpoint of a robot's vision sensor for a given success criterion. The strategy is based on extremum seeking control (ESC), which introduces two main advantages: 1) Our approach is model free: It does not require an explicit objective function or any other task model to calculate the gradient direction for viewpoint optimization. This brings new possibilities for the use of active vision in unstructured environments, since a priori knowledge of the surroundings and the target objects is not required. 2) ESC conducts continuous optimization backed up with mechanisms to escape from local maxima. This enables an efficient execution of an active vision task. We demonstrate our approach with two applications in the object recognition and manipulation fields, where the model-free approach brings various benefits: for object recognition, our framework removes the dependence on offline training data for viewpoint optimization, and provides robustness of the system to occlusions and changing lighting conditions. In object manipulation, the model-free approach allows us to increase the success rate of a grasp synthesis algorithm without the need of an object model; the algorithm only uses continuous measurements of the objective value, i.e., the grasp quality. Our experiments show that continuous viewpoint optimization can efficiently increase the data quality for the underlying algorithm, while maintaining the robustness.Accepted Author ManuscriptRobot DynamicsBiomechatronics & Human-Machine Contro

Accelerating reinforcement learning on a robot by using subgoals in a hierarchical framework

Reinforcement learning is a way to learn control tasks by trial and error. Even for simple motor control tasks, however, this can take a long time. We can speed up learning by using prior knowledge, but this is not always available, especially for an autonomous agent. One way to add limited prior knowledge is to use subgoals, defining points that the controller should aim for on the way to reaching the real goal. In this study, we use the MAXQ hierarchical framework to specify subgoals. This decreased the learning time by a factor two on a robot leg step-up task and we show that tests on a real robot give similar results. The worse end performance that is a result of the reduced solution space can be partially canceled out by hierarchical greedy execution. To our knowledge, this is the first time the MAXQ framework is applied to a real robot.Biomechanical EngineeringMechanical, Maritime and Materials Engineerin

The Effects of Large Disturbances on On-Line Reinforcement Learning for aWalking Robot

Reinforcement Learning is a promising paradigm for adding learning capabilities to humanoid robots. One of the difficulties of the real world is the presence of disturbances. In Reinforcement Learning, disturbances are typically dealt with stochastically. However, large and infrequent disturbances do not fit well in this framework; essentially, they are outliers and not part of the underlying (stochastic) Markov Decision Process. Therefore, they can negatively influence learning. The main reasons for such disturbances for a humanoid robot are sudden changes in the dynamics (such as a sudden push), sensor noise and sampling time irregularities. We investigate the effects of these types of outliers on the on-line learning process of a simple walking robot simulation. We propose to exclude the outliers from the learning process with the aim to improve convergence and the final solution. While infrequent sensor and timing outliers had a negligible influence, infrequent pushes heavily disrupted the learning process. By excluding the outliers from the learning process, performance was again restored.Biomechanical EngineeringMechanical, Maritime and Materials Engineerin

Topographical scanning and reproduction of near-planar surfaces of paintings

Paintings are near-planar objects with material characteristics that vary widely. The fact that paint has a material presence is often overlooked, mostly because we often encounter these artworks in the form of two-dimensional reproductions. Capturing paintings in the third dimension is not only important for study, restoration and conservation, but it also inspires 3D printing methods1, particularly through the high demands it makes on reproducing color, gloss and texture. “A hybrid solution between fringe projection and stereo imaging is proposed as 3D imaging method, with a setup involving two cameras and a projector. Fringe projection is aided by sparse stereo matching to serve as image encoder. These encoded images processed by the stereo cameras solve the correspondence problem in stereo matching, leading to a dense and accurate topographical map, while simultaneously capturing the composition of the painting in full color”1. The topographical map and color data are used to make hardcopy 3D reproductions, using a specially developed printing system. Several paintings by Dutch masters Rembrandt and Van Gogh have been scanned and reproduced using this technique. These 3D printed reproductions have been evaluated by experts, both individually and in a side-by-side comparison with the original.Design EngineeringIndustrial Design Engineerin

Augmented Reality for Art, Design and Cultural Heritage: System Design and Evaluation

This paper describes the design of an optical see-through head-mounted display (HMD) system for Augmented Reality (AR). Our goals were to make virtual objects “perfectly” indistinguishable from real objects, wherever the user roams, and to find out to which extent imperfections are hindering applications in art and design. For AR, fast and accurate measuring of head motions is crucial. We made a head-pose tracker for the HMD that uses error-state Kalman filters to fuse data from an inertia tracker with data from a camera that tracks visual markers. This makes on-line head-pose based rendering of dynamic virtual content possible. We measured our system, and found that with an A4-sized marker viewed from > 20? at 5m distance with an SXGA camera (FOV 108?), the RMS error in the tracker angle was < 0.5? when moving the head slowly. Our Kalman filters suppressed the pose error due to camera delay, which is proportional to the angular and linear velocities, and the dynamic misalignment was comparable to the static misalignment. Applications of artists and designers lead to observations on the profitable use of our AR system. Their exhibitions at world-class museums showed that AR is a powerful tool for disclosing cultural heritage.Biomechanical EngineeringMechanical, Maritime and Materials Engineerin