Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

In this paper, we explore the potential of using a large language model (LLM) to understand the driving environment in a human-like manner and analyze its ability to reason, interpret, and memorize when facing complex scenarios. We argue that traditional optimization-based and modular autonomous driving (AD) systems face inherent performance limitations when dealing with long-tail corner cases. To address this problem, we propose that an ideal AD system should drive like a human, accumulating experience through continuous driving and using common sense to solve problems. To achieve this goal, we identify three key abilities necessary for an AD system: reasoning, interpretation, and memorization. We demonstrate the feasibility of employing an LLM in driving scenarios by building a closed-loop system to showcase its comprehension and environment-interaction abilities. Our extensive experiments show that the LLM exhibits the impressive ability to reason and solve long-tailed cases, providing valuable insights for the development of human-like autonomous driving. The related code are available at https://github.com/PJLab-ADG/DriveLikeAHuman

Systematic AI Approach for AGI: Addressing Alignment, Energy, and AGI Grand Challenges

AI faces a trifecta of grand challenges the Energy Wall, the Alignment Problem and the Leap from Narrow AI to AGI. Contemporary AI solutions consume unsustainable amounts of energy during model training and daily operations.Making things worse, the amount of computation required to train each new AI model has been doubling every 2 months since 2020, directly translating to increases in energy consumption.The leap from AI to AGI requires multiple functional subsystems operating in a balanced manner, which requires a system architecture. However, the current approach to artificial intelligence lacks system design; even though system characteristics play a key role in the human brain from the way it processes information to how it makes decisions. Similarly, current alignment and AI ethics approaches largely ignore system design, yet studies show that the brains system architecture plays a critical role in healthy moral decisions.In this paper, we argue that system design is critically important in overcoming all three grand challenges. We posit that system design is the missing piece in overcoming the grand challenges.We present a Systematic AI Approach for AGI that utilizes system design principles for AGI, while providing ways to overcome the energy wall and the alignment challenges.Comment: International Journal on Semantic Computing (2024) Categories: Artificial Intelligence; AI; Artificial General Intelligence; AGI; System Design; System Architectur

SciTech News- 68(4)-2014

Columns and Reports From the Editor 5 SciTech News Call for Articles 5 Conference Report, Diane K. Foster International Student Travel Award Recipient 8 Conference Report, S. Kirk Cabeen Travel Stipend Award Recipient 9 Conference Report, Bonnie Hilditch International Librarian Award Recipient 11 Conference Report, IEEE Continuing Education Award Recipient 19 Division News Science-Technology Division 6 Chemistry Division 14 Engineering Division 17 Call for Nominations & Applications Bonnie Hilditch International Librarian Award 13 IEEE Continuing Education Stipend 20 Engineering Librarian of the Year Award 21 SPIE Digital Library Student Travel Stipend 22 Reviews Sci-Tech Book News Reviews 23 Advertisements Annual Reviews 3 IEEE

Data ethics : building trust : how digital technologies can serve humanity

Data is the magic word of the 21st century. As oil in the 20th century and electricity in the 19th century: For citizens, data means support in daily life in almost all activities, from watch to laptop, from kitchen to car, from mobile phone to politics. For business and politics, data means power, dominance, winning the race. Data can be used for good and bad, for services and hacking, for medicine and arms race. How can we build trust in this complex and ambiguous data world? How can digital technologies serve humanity? The 45 articles in this book represent a broad range of ethical reflections and recommendations in eight sections: a) Values, Trust and Law, b) AI, Robots and Humans, c) Health and Neuroscience, d) Religions for Digital Justice, e) Farming, Business, Finance, f) Security, War, Peace, g) Data Governance, Geopolitics, h) Media, Education, Communication. The authors and institutions come from all continents. The book serves as reading material for teachers, students, policy makers, politicians, business, hospitals, NGOs and religious organisations alike. It is an invitation for dialogue, debate and building trust! The book is a continuation of the volume “Cyber Ethics 4.0” published in 2018 by the same editors

Reconstruction and recognition of confusable models using three-dimensional perception

Perception is one of the key topics in robotics research. It is about the processing of external sensor data and its interpretation. The necessity of fully autonomous robots makes it crucial to help them to perform tasks more reliably, flexibly, and efficiently. As these platforms obtain more refined manipulation capabilities, they also require expressive and comprehensive environment models: for manipulation and affordance purposes, their models have to involve each one of the objects present in the world, coincidentally with their location, pose, shape and other aspects. The aim of this dissertation is to provide a solution to several of these challenges that arise when meeting the object grasping problem, with the aim of improving the autonomy of the mobile manipulator robot MANFRED-2. By the analysis and interpretation of 3D perception, this thesis covers in the first place the localization of supporting planes in the scenario. As the environment will contain many other things apart from the planar surface, the problem within cluttered scenarios has been solved by means of Differential Evolution, which is a particlebased evolutionary algorithm that evolves in time to the solution that yields the cost function lowest value. Since the final purpose of this thesis is to provide with valuable information for grasping applications, a complete model reconstructor has been developed. The proposed method holdsmany features such as robustness against abrupt rotations, multi-dimensional optimization, feature extensibility, compatible with other scan matching techniques, management of uncertain information and an initialization process to reduce convergence timings. It has been designed using a evolutionarybased scan matching optimizer that takes into account surface features of the object, global form and also texture and color information. The last tackled challenge regards the recognition problem. In order to procure with worthy information about the environment to the robot, a meta classifier that discerns efficiently the observed objects has been implemented. It is capable of distinguishing between confusable objects, such as mugs or dishes with similar shapes but different size or color. The contributions presented in this thesis have been fully implemented and empirically evaluated in the platform. A continuous grasping pipeline covering from perception to grasp planning including visual object recognition for confusable objects has been developed. For that purpose, an indoor environment with several objects on a table is presented in the nearby of the robot. Items are recognized from a database and, if one is chosen, the robot will calculate how to grasp it taking into account the kinematic restrictions associated to the anthropomorphic hand and the 3D model for this particular object. -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------La percepción es uno de los temas más relevantes en el mundo de la investigaci ón en robótica. Su objetivo es procesar e interpretar los datos recibidos por un sensor externo. La gran necesidad de desarrollar robots autónomos hace imprescindible proporcionar soluciones que les permita realizar tareas más precisas, flexibles y eficientes. Dado que estas plataformas cada día adquieren mejores capacidades para manipular objetos, también necesitarán modelos expresivos y comprensivos: para realizar tareas de manipulación y prensión, sus modelos han de tener en cuenta cada uno de los objetos presentes en su entorno, junto con su localizaci ón, orientación, forma y otros aspectos. El objeto de la presente tesis doctoral es proponer soluciones a varios de los retos que surgen al enfrentarse al problema del agarre, con el propósito final de aumentar la capacidad de autonomía del robot manipulador MANFRED-2. Mediante el análisis e interpretación de la percepción tridimensional, esta tesis cubre en primer lugar la localización de planos de soporte en sus alrededores. Dado que el entorno contendrá muchos otros elementos aparte de la superficie de apoyo buscada, el problema en entornos abarrotados ha sido solucionado mediante Evolución Diferencial, que es un algoritmo evolutivo basado en partículas que evoluciona temporalmente a la solución que contempla el menor resultado en la función de coste. Puesto que el propósito final de este trabajo de investigación es proveer de información valiosa a las aplicaciones de prensión, se ha desarrollado un reconstructor de modelos completos. El método propuesto posee diferentes características como robustez a giros abruptos, optimización multidimensional, extensión a otras características, compatibilidad con otras técnicas de reconstrucción, manejo de incertidumbres y un proceso de inicialización para reducir el tiempo de convergencia. Ha sido diseñado usando un registro optimizado mediante técnicas evolutivas que tienen en cuenta las particularidades de la superficie del objeto, su forma global y la información relativa a la textura. El último problema abordado está relacionado con el reconocimiento de objetos. Con la intención de abastecer al robot con la mayor información posible sobre el entorno, se ha implementado un meta clasificador que diferencia de manera eficaz los objetos observados. Ha sido capacitado para distinguir objetos confundibles como tazas o platos con formas similares pero con diferentes colores o tamaños. Las contribuciones presentes en esta tesis han sido completamente implementadas y probadas de manera empírica en la plataforma. Se ha desarrollado un sistema que cubre el problema de agarre desde la percepción al cálculo de la trayectoria incluyendo el sistema de reconocimiento de objetos confundibles. Para ello, se ha presentado una mesa con objetos en un entorno cerrado cercano al robot. Los elementos son comparados con una base de datos y si se desea agarrar uno de ellos, el robot estimará cómo cogerlo teniendo en cuenta las restricciones cinemáticas asociadas a una mano antropomórfica y el modelo tridimensional generado del objeto en cuestión

Grounding the Interaction : Knowledge Management for Interactive Robots

Avec le développement de la robotique cognitive, le besoin d’outils avancés pour représenter, manipuler, raisonner sur les connaissances acquises par un robot a clairement été mis en avant. Mais stocker et manipuler des connaissances requiert tout d’abord d’éclaircir ce que l’on nomme connaissance pour un robot, et comment celle-ci peut-elle être représentée de manière intelligible pour une machine. \ud \ud Ce travail s’efforce dans un premier temps d’identifier de manière systématique les besoins en terme de représentation de connaissance des applications robotiques modernes, dans le contexte spécifique de la robotique de service et des interactions homme-robot. Nous proposons une typologie originale des caractéristiques souhaitables des systèmes de représentation des connaissances, appuyée sur un état de l’art détaillé des outils existants dans notre communauté. \ud \ud Dans un second temps, nous présentons en profondeur ORO, une instanciation particulière d’un système de représentation et manipulation des connaissances, conçu et implémenté durant la préparation de cette thèse. Nous détaillons le fonctionnement interne du système, ainsi que son intégration dans plusieurs architectures robotiques complètes. Un éclairage particulier est donné sur la modélisation de la prise de perspective dans le contexte de l’interaction, et de son interprétation en terme de théorie de l’esprit. \ud \ud La troisième partie de l’étude porte sur une application importante des systèmes de représentation des connaissances dans ce contexte de l’interaction homme-robot : le traitement du dialogue situé. Notre approche et les algorithmes qui amènent à l’ancrage interactif de la communication verbale non contrainte sont présentés, suivis de plusieurs expériences menées au Laboratoire d’Analyse et d’Architecture des Systèmes au CNRS à Toulouse, et au groupe Intelligent Autonomous System de l’université technique de Munich. Nous concluons cette thèse sur un certain nombre de considérations sur la viabilité et l’importance d’une gestion explicite des connaissances des agents, ainsi que par une réflexion sur les éléments encore manquant pour réaliser le programme d’une robotique “de niveau humain”.-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------With the rise of the so-called cognitive robotics, the need of advanced tools to store, manipulate, reason about the knowledge acquired by the robot has been made clear. But storing and manipulating knowledge requires first to understand what the knowledge itself means to the robot and how to represent it in a machine-processable way. \ud \ud This work strives first at providing a systematic study of the knowledge requirements of modern robotic applications in the context of service robotics and human-robot interaction. What are the expressiveness requirement for a robot? what are its needs in term of reasoning techniques? what are the requirement on the robot's knowledge processing structure induced by other cognitive functions like perception or decision making? We propose a novel typology of desirable features for knowledge representation systems supported by an extensive review of existing tools in our community. \ud \ud In a second part, the thesis presents in depth a particular instantiation of a knowledge representation and manipulation system called ORO, that has been designed and implemented during the preparation of the thesis. We elaborate on the inner working of this system, as well as its integration into several complete robot control stacks. A particular focus is given to the modelling of agent-dependent symbolic perspectives and their relations to theories of mind. \ud \ud The third part of the study is focused on the presentation of one important application of knowledge representation systems in the human-robot interaction context: situated dialogue. Our approach and associated algorithms leading to the interactive grounding of unconstrained verbal communication are presented, followed by several experiments that have taken place both at the Laboratoire d'Analyse et d'Architecture des Systèmes at CNRS, Toulouse and at the Intelligent Autonomous System group at Munich Technical University. \ud \ud The thesis concludes on considerations regarding the viability and importance of an explicit management of the agent's knowledge, along with a reflection on the missing bricks in our research community on the way towards "human level robots". \ud \u

Multidisciplinary perspectives on Artificial Intelligence and the law

This open access book presents an interdisciplinary, multi-authored, edited collection of chapters on Artificial Intelligence (‘AI’) and the Law. AI technology has come to play a central role in the modern data economy. Through a combination of increased computing power, the growing availability of data and the advancement of algorithms, AI has now become an umbrella term for some of the most transformational technological breakthroughs of this age. The importance of AI stems from both the opportunities that it offers and the challenges that it entails. While AI applications hold the promise of economic growth and efficiency gains, they also create significant risks and uncertainty. The potential and perils of AI have thus come to dominate modern discussions of technology and ethics – and although AI was initially allowed to largely develop without guidelines or rules, few would deny that the law is set to play a fundamental role in shaping the future of AI. As the debate over AI is far from over, the need for rigorous analysis has never been greater. This book thus brings together contributors from different fields and backgrounds to explore how the law might provide answers to some of the most pressing questions raised by AI. An outcome of the Católica Research Centre for the Future of Law and its interdisciplinary working group on Law and Artificial Intelligence, it includes contributions by leading scholars in the fields of technology, ethics and the law.info:eu-repo/semantics/publishedVersio

cii Student Papers - 2022

In this collection of papers, we, the Research Group Critical Information Infrastructures (cii) from the Karlsruhe Institute of Technology, present eight selected student research articles contributing to the design, development, and evaluation of critical information infrastructures. During our courses, students mostly work in groups and deal with problems and issues related to sociotechnical challenges in the realm of (critical) information systems. Student papers came from five different cii courses, namely Emerging Trends in Internet Technologies, Emerging Trends in Digital Health, Digital Health, Critical Information Infrastructures, and Selected Issues on Critical Information Infrastructures: Collaborative Development of Innovative Teaching Concepts in summer term of 2021 and the winter term of 2021/2022