Agente de visão semântica para robótica

Mestrado em Engenharia de Computadores e TelemáticaVisão semântica é uma importante linha de investigação na área de visão por computador. A palavra-chave “semântica” implica a extracção de características não apenas visuais (cor, forma, textura), mas também qualquer tipo de informação de “alto-nível”. Em particular, a visão semântica procura compreender ou interpretar imagens de cenas em termos dos objectos presentes e eventualmente das relações entre eles. Uma das principais áreas de aplicação actual é a robótica. Sendo o mundo que nos rodeia extremamente visual, a interacção entre um utilizador humano não especializado e um robô requer que o robô seja capaz de detectar, reconhecer e compreender qualquer tipo de referências visuais fornecidas no âmbito da comunicação entre o utilizador e o robô. Para que tal seja possível, é necessária uma fase de aprendizagem, através da qual várias categorias de objectos são aprendidas pelo robô. Depois deste processo, o robô será capaz de reconhecer novas instâncias das categorias anteriormente aprendidas. Foi desenvolvido um novo agente de visão semântica que recorre a serviços de pesquisa de imagens na Web para aprender um conjunto de categorias gerais a partir apenas dos seus respectivos nomes. O trabalho teve como ponto de partida o agente UA@SRVC, anteriormente desenvolvido na Universidade de Aveiro para participação no Semantic Robot Vision Challenge. O trabalho começou pelo desenvolvimento de uma nova técnica de segmentação de objectos baseada nas suas arestas e na diversidade de cor. De seguida, a técnica de pesquisa semântica e selecção de imagens de treino do agente UA@SRVC foi revista e reimplementada utilizando, entre outros componentes, o novo módulo de segmentação. Por fim foram desenvolvidos novos classificadores para o reconhecimento de objectos. Apreendemos que, mesmo com pouca informação prévia sobre um objecto, é possível segmentá-lo correctamente utilizando para isso uma heurística simples que combina a diversidade da cor e a distância entre segmentos. Recorrendo a uma técnica de agrupamento conceptual, é possível criar um sistema de votos que permite efectuar uma boa selecção de instâncias para o treino de categorias. Conclui-se também que diferentes classificadores são mais eficientes quando a fase de aprendizagem é supervisionada ou automatizada.Semantic vision is an important line of research in computer vision. The keyword “semantic” means the extraction of features, not only visual (color, shape, texture), but also any “higher level” information. In particular, semantic vision seeks to understand or interpret images of scenes in terms of present objects and possible relations between them. One of the main areas of current application is robotics. As the world around us is extremely visual, interaction between a non specialized human user and a robot requires the robot to be able to detect, recognize and understand any kind of visual cues provided in the communication between user and robot. To make this possible, a learning phase is needed, in which various categories of objects are learned by the robot. After this process, the robot will be able to recognize new instances of the categories previously learned. We developed a new semantic vision agent that uses image search web services to learn a set of general categories based only on their respective names. The work had as starting point the agent UA@SRVC, previously developed at the University of Aveiro for participation in the Semantic Robot Vision Challenge. This work began by developing a new technique for segmentation of objects based on their edges and diversity of color. Then, the technique of semantic search and selection of images from the agent UA@SRVC was revised and reimplemented using, among other components, the new object extracting module. Finally new classifiers were developed for the recognition of objects. We learned that, even with little prior information about an object, it is possible to segment it correctly using a simple heuristic that combines colour disparity and distance between segments. Drawing on a conceptual clustering technique, we can create a voting system that allows a good selection of instances for training the categories. We also conclude that various classifiers are most effective when the learning phase is supervised or automated

Open-ended category learning for language acquisition

Motivated by the need to support language-based communication between robots and their human users, as well as grounded symbolic reasoning, this paper presents a learning architecture that can be used by robotic agents for long-term and open-ended category acquisition. To be more adaptive and to improve learning performance as well as memory usage, this learning architecture includes a metacognitive processing component. Multiple object representations and multiple classifiers and classifier combinations are used. At the object level, the main similarity measure is based on a multi-resolution matching algorithm. Categories are represented as sets of known instances. In this instance-based approach, storing and forgetting rules optimise memory usage. Classifier combinations are based on majority voting and the Dempster-Shafer evidence theory. All learning computations are carried out during the normal execution of the agent, which allows continuous monitoring of the performance of the different classifiers. The measured classification successes of the individual classifiers support an attentional selection mechanism, through which classifier combinations are dynamically reconfigured and a specific classifier is chosen to predict the category of a new unseen object. A simple physical agent, incorporating these learning capabilities, is used to test the approach. A long-term experiment was carried out having in mind the open-ended nature of category learning. With the help of a human mediator, the agent incrementally learned 68 categories of real-world objects visually perceivable through an inexpensive camera. Various aspects of the approach are evaluated through systematic experiments.</p

Embodied Language Acquisition: A Proof Of Concept

For robots to interact with humans at the language level, it becomes fundamental that robots and humans share a common language. In this paper, a social language grounding paradigm is adopted to teach a robotic arm basic vocabulary about objects in its environment. A human user, acting as an instructor, teaches the names of the objects present in their shared field of view. The robotic agent grounds these words by associating them to visual category descriptions. A component-based object representation is presented. An instance based approach is used for category representation. An instance is described by its components and geometric relations between them. Each component is a color blob or an aggregation of neighboring color blobs. The categorization strategy is based on graph matching. The learning/grounding capacity of the robot is assessed over a series of semi-automated experiments and the results are reported. © 2009 Springer Berlin Heidelberg.5816 LNAI263274Brady, M., Artificial Intelligence and Robotics (1985) Artificial Intelligence, 26 (1), pp. 79-121Burgard, W., Fox, D., Hähnel, D., Lakemeyer, G., Schulz, D., Steiner, W., Thrun, S., Cremers, A.B., Real Robots for the Real World - The RHINO Museum Tour-Guide Project (1998) Proc. of the AAAI 1998 Spring Symposium on Integrating Robotics Research, Taking the Next Leap, , Stanford, CACunningham, C., Weber, R., Proctor, J.M., Fowler, C., Murphy, M., Investigating Graphs in Textual Case-Based Reasoning (2004) ECBR 2004, pp. 573-586Diestel, R., (2000) Graph Theory, , Springer, HeidelbergHarnad, S., The symbol grounding problem (1990) Physica D, 42, pp. 335-346Kennedy, W.G., Trafton, J.G., Long-Term Symbolic Learning (2007) Cognitive Systems Research, 8, pp. 237-247Kirby, S., Hurford, J., The Emergence of Linguistic Structure: An overview of the Iterated Learning Model (2002) Simulating the Evolution of Language, pp. 121-148. , Cangelosi, A., Parisi D. (eds.) .Springer, HeidelbergKozima, H., Nakagawa, C., Social robots for children: Practice in communication-care (2006) 9th IEEE International Workshop on Advanced Motion ControlLevinson, S.E., Squire, K., Lin, R.S., McClain, M., Automatic language acquisition by an autonomous robot (2005) Proceedings of the AAAI Spring Symposium on Developmental Robotics, pp. 21-23. , MarchLove, N., Cognition and the language myth (2004) Language Sciences, 26, pp. 525-544Seabra Lopes, L., Connell, J.H., Semisentient robots: Routes to integrated intelligence (2001) IEEE Intelligent Systems, 16 (5), pp. 10-14Seabra Lopes, L., Chauhan, A., How many Words can my Robot learn? An Approach and Experiments with One-Class Learning (2007) Interaction Studies, 8 (1), pp. 53-81Seabra Lopes, L., Chauhan, A., Silva, J., Towards long-term visual learning of object categories in human-robot interaction (2007) New Trends in Artificial Intelligence, APPIA, pp. 623-634. , Maia Neves, J.C., Santos, M.F., Machado, J.M. (eds.)Seabra Lopes, L., Chauhan, A., Open-ended category learning for language acquisition (2008) Connection Science, 8 (4)Steels, L., Language games for autonomous robots (2001) IEEE Intelligent Systems, 16 (5), pp. 16-22Steels, L., Kaplan, F., AIBO's first words: The social learning of language and meaning (2002) Evolution of Communication, 4 (1), pp. 3-32Steels, L., Evolving Grounded Communication for Robots (2003) Trends in Cognitive Science, 7 (7), pp. 308-312Thomaz, A.L., Breazeal, C., Robot Learning via Socially Guided Exploration (2007) Proc. of ICDL 2006, , Imperial College, LondonYu, C., The emergence of links between lexical acquisition and object categorization: A computational study (2005) Connection Science, 17 (3), pp. 381-39