Reinforcement Learning for Agents with Many Sensors and Actuators Acting in Categorizable Environments

In this paper, we confront the problem of applying reinforcement learning to agents that perceive the environment through many sensors and that can perform parallel actions using many actuators as is the case in complex autonomous robots. We argue that reinforcement learning can only be successfully applied to this case if strong assumptions are made on the characteristics of the environment in which the learning is performed, so that the relevant sensor readings and motor commands can be readily identified. The introduction of such assumptions leads to strongly-biased learning systems that can eventually lose the generality of traditional reinforcement-learning algorithms. In this line, we observe that, in realistic situations, the reward received by the robot depends only on a reduced subset of all the executed actions and that only a reduced subset of the sensor inputs (possibly different in each situation and for each action) are relevant to predict the reward. We formalize this property in the so called 'categorizability assumption' and we present an algorithm that takes advantage of the categorizability of the environment, allowing a decrease in the learning time with respect to existing reinforcement-learning algorithms. Results of the application of the algorithm to a couple of simulated realistic-robotic problems (landmark-based navigation and the six-legged robot gait generation) are reported to validate our approach and to compare it to existing flat and generalization-based reinforcement-learning approaches

Digital filter implementation over FPGA platform with LINUX OS

AbstractThe embedded processors on FPGA's are a good tool to specific propose works. In this work we present how the FPGA is used to apply a Sobel filter to a set of images, also the step needed to set-up the entire system is described. An embedded processor, with a Linux distribution implemented is used to run a special compilation of C filter program, the filter is compared with the results obtained with a PC running the same filter, in the embedded system all the process runs in the FPGA and the exit file can be accessed by ftp or http server embedded into the Linux system

Prognostics health management: perspectives in engineering systems reliability prognostics

The Prognostic Health Management (PHM) has been asserting itself as the most promising methodology to enhance the effective reliability and availability of a product or system during its life-cycle conditions by detecting current and approaching failures, thus, providing mitigation of the system risks with reduced logistics and support costs. However, PHM is at an early stage of development, it also expresses some concerns about possible shortcomings of its methods, tools, metrics and standardization. These factors have been severely restricting the applicability of PHM and its adoption by the industry. This paper presents a comprehensive literature review about the PHM main general weaknesses. Exploring the research opportunities present in some recent publications, are discussed and outlined the general guide-lines for finding the answer to these issues.(undefined

From back to front: A functional model for the cerebellar modulation in the establishment of conditioned preferences for cocaine‐related cues

This is the pre-peer reviewed version of the following article: From back to front: A functional model for the cerebellar modulation in the establishment of conditioned preferences for cocaine‐related cues, which has been published in final form at https://doi.org/10.1111/adb.12834. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.It is now increasingly clear that the cerebellum may modulate brain functions altered in drug addiction. We previously demonstrated that cocaine‐induced conditioned preference increased activity at the dorsal posterior cerebellar vermis. Unexpectedly, a neurotoxic lesion at this region increased the probability of cocaine‐induced conditioned preference acquisition. The present research aimed at providing an explanatory model for such as facilitative effect of the cerebellar lesion. First, we addressed a tracing study in which we found a direct projection from the lateral (dentate) nucleus to the ventral tegmental area (VTA) that also receives Purkinje axons from lobule VIII in the vermis. This pathway might control the activity and plasticity of the cortico‐striatal circuitry. Then we evaluated cFos expression in different regions of the medial prefrontal cortex and striatum after a lesion in lobule VIII before conditioning. Additionally, perineuronal net (PNN) expression was assessed to explore whether the cerebellar lesion might affect synaptic stabilization mechanisms in the medial prefrontal cortex (mPFC). Damage in this region of the vermis induced general disinhibition of the mPFC and striatal subdivisions that receive dopaminergic projections, mainly from the VTA. Moreover, cerebellar impairment induced an upregulation of PNN expression in the mPFC. The major finding of this research was to provide an explanatory model for the function of the posterior cerebellar vermis on drug‐related memory. In this model, damage of the posterior vermis would release striatum‐cortical networks from the inhibitory tonic control exerted by the cerebellar cortex over VTA, thereby promoting drug effects

Podcasting: en la frontera entre comunicación y educación

Este trabajo ahonda en la potencialidad de los medios digitales para educar a las personas en sus múltiples dimensiones. Parte de la cultura participativa en el seno de la sociedad de la información, más ubicua y transparente tras el advenimiento de la Web 2.0. La sociedad digital abre nuevos espacios de comunicación, mediante recursos como el podcasting. Se trata de un medio que ofrece audio bajo demanda a través de internet, pero también, un canal por el que las personas comparten contenidos y reflexiones. Al margen de sus funciones para el ocio, el podcasting entraña una virtualidad educativa que puede aprovecharse en el ámbito escolar, así como en la autoformación de los oyentes. En definitiva: un medio en la frontera entre comunicación y educación

Predicting the onset and persistence of episodes of depression in primary health care. The predictD-Spain study: Methodology

Background: The effects of putative risk factors on the onset and/or persistence of depression remain unclear. We aim to develop comprehensive models to predict the onset and persistence of episodes of depression in primary care. Here we explain the general methodology of the predictD-Spain study and evaluate the reliability of the questionnaires used. Methods: This is a prospective cohort study. A systematic random sample of general practice attendees aged 18 to 75 has been recruited in seven Spanish provinces. Depression is being measured with the CIDI at baseline, and at 6, 12, 24 and 36 months. A set of individual, environmental, genetic, professional and organizational risk factors are to be assessed at each follow-up point. In a separate reliability study, a proportional random sample of 401 participants completed the test-retest (251 researcher-administered and 150 self-administered) between October 2005 and February 2006. We have also checked 118,398 items for data entry from a random sample of 480 patients stratified by province. Results: All items and questionnaires had good test-retest reliability for both methods of administration, except for the use of recreational drugs over the previous six months. Cronbach's alphas were good and their factorial analyses coherent for the three scales evaluated (social support from family and friends, dissatisfaction with paid work, and dissatisfaction with unpaid work). There were 191 (0.16%) data entry errors. Conclusion: The items and questionnaires were reliable and data quality control was excellent. When we eventually obtain our risk index for the onset and persistence of depression, we will be able to determine the individual risk of each patient evaluated in primary health car

Usos del podcast para fines educativos. Mapeo sistemático de la literatura en WoS y Scopus (2014-2019)

Introducción:la investigación disponible sobre el aprovechamiento educativo del podcast de audio es escasa. Se revisó la literatura publicada (2014-2019) clasificando usos, contextos y categorías del podcast de audio con fines educativos, e identificando autores y revistas de referencia en el ámbito.Metodología:se aplicó el método de Mapeo Sistemático de la Literaturaa una muestra de artículos indexados de acceso abierto en las bases de datos Web of Science y Scopus. El filtrado de la muestra se llevó a cabo de acuerdo con unos criterios de inclusión y exclusión.Resultados y conclusiones: se hallaron 81artículos que destacan los principales usos educativos del podcast y coinciden en su utilidad para apoyar el aprendizaje en instituciones y entornos educativos formales, no formales e informales. Este trabajo brinda a investigadores, educadores e instituciones una línea base actualizada para seguir explorando las virtudes educativas del podcast.Introduction:There is little research available on the educational use of the audio podcast. The published literature (2014-2019) was reviewed, classifying uses, contexts,and categories of the audio podcast for educational purposes, and identifying authors and reference journals in the field. Methodology:The Systematic Mapping of Literature method was applied to a sample of open access articlesindexedin the Web of Science and Scopus databases. The filtering of the sample was carried out according to inclusion and exclusion criteria.Results and conclusions:81articles were found that highlight the main educational uses of the podcast and coincide in its usefulness to support learning in formal, non-formal,and informal educational institutions and settings. This work provides researchers, educators, and institutions with an updated baseline to further explore the educational strengths of the podcast

Feature Selection Using Genetic Algorithms for the Generation of a Recognition and Classification of Children Activities Model Using Environmental Sound

In the area of recognition and classification of children activities, numerous works have been proposed that make use of different data sources. In most of them, sensors embedded in children’s garments are used. In this work, the use of environmental sound data is proposed to generate a recognition and classification of children activities model through automatic learning techniques, optimized for application on mobile devices. Initially, the use of a genetic algorithm for a feature selection is presented, reducing the original size of the dataset used, an important aspect when working with the limited resources of a mobile device. For the evaluation of this process, five different classification methods are applied, k-nearest neighbor (k-NN), nearest centroid (NC), artificial neural networks (ANNs), random forest (RF), and recursive partitioning trees (Rpart). Finally, a comparison of the models obtained, based on the accuracy, is performed, in order to identify the classification method that presents the best performance in the development of a model that allows the identification of children activity based on audio signals. According to the results, the best performance is presented by the five-feature model developed through RF, obtaining an accuracy of 0.92, which allows to conclude that it is possible to automatically classify children activity based on a reduced set of features with significant accuracy.In the area of recognition and classification of children activities, numerous works have been proposed that make use of different data sources. In most of them, sensors embedded in children’s garments are used. In this work, the use of environmental sound data is proposed to generate a recognition and classification of children activities model through automatic learning techniques, optimized for application on mobile devices. Initially, the use of a genetic algorithm for a feature selection is presented, reducing the original size of the dataset used, an important aspect when working with the limited resources of a mobile device. For the evaluation of this process, five different classification methods are applied, k-nearest neighbor (k-NN), nearest centroid (NC), artificial neural networks (ANNs), random forest (RF), and recursive partitioning trees (Rpart). Finally, a comparison of the models obtained, based on the accuracy, is performed, in order to identify the classification method that presents the best performance in the development of a model that allows the identification of children activity based on audio signals. According to the results, the best performance is presented by the five-feature model developed through RF, obtaining an accuracy of 0.92, which allows to conclude that it is possible to automatically classify children activity based on a reduced set of features with significant accuracy