Efficient Embedded Decoding of Neural Network Language Models in a Machine Translation System

[EN] Neural Network Language Models (NNLMs) are a successful approach to Natural Language Processing tasks, such as Machine Translation. We introduce in this work a Statistical Machine Translation (SMT) system which fully integrates NNLMs in the decoding stage, breaking the traditional approach based on n-best list rescoring. The neural net models (both language models (LMs) and translation models) are fully coupled in the decoding stage, allowing to more strongly influence the translation quality. Computational issues were solved by using a novel idea based on memorization and smoothing of the softmax constants to avoid their computation, which introduces a trade-off between LM quality and computational cost. These ideas were studied in a machine translation task with different combinations of neural networks used both as translation models and as target LMs, comparing phrase-based and N-gram-based systems, showing that the integrated approach seems more promising for N-gram-based systems, even with nonfull-quality NNLMs.This work was partially supported by the Spanish MINECO and FEDER found under project TIN2017-85854-C4-2-R.Zamora Martínez, FJ.; Castro-Bleda, MJ. (2018). Efficient Embedded Decoding of Neural Network Language Models in a Machine Translation System. International Journal of Neural Systems. 28(9). https://doi.org/10.1142/S0129065718500077S28

Epothilone-d rescues cognition and attenuates alzheimer’s disease-like pathology in APP/PS1 mice

AIMS: Cognitive decline in Alzheimer's disease (AD) patients has been linked to synaptic damage and neuronal loss. Hyperphosphorylation of tau protein destabilizes microtubules leading to the accumulation of autophagy/vesicular material and the generation of dystrophic neurites, thus contributing to axonal/synaptic dysfunction. In this study, we analyzed the effect of a microtubule-stabilizing compound in the progression of the disease in the hippocampus of APP751SL/PS1M146L transgenic model. METHODS: APP/PS1 mice (3 month-old) were treated with a weekly intraperitoneal injection of 2 mg/kg epothilone-D (Epo-D) for 3 months. Vehicle-injected animals were used as controls. Mice were tested on the Morris water maze, Y-maze and object-recognition tasks for memory performance. Abeta, AT8, ubiquitin and synaptic markers levels were analyzed by Western-blots. Hippocampal plaque, synaptic and dystrophic loadings were quantified by image analysis after immunohistochemical stainings. RESULTS: Epo-D treated mice exhibited a significant improvement in the memory tests compared to controls. The rescue of cognitive deficits was associated to a significant reduction in the AD-like hippocampal pathology. Levels of Abeta, APP and ubiquitin were significantly reduced in treated animals. This was paralleled by a decrease in the amyloid burden, and more importantly, in the plaque-associated axonal dystrophy pathology. Finally, synaptic levels were significantly restored in treated animals compared to controls. CONCLUSION: Epo-D treatment promotes synaptic and spatial memory recovery, reduces the accumulation of extracellular Abeta and the associated neuritic pathology in the hippocampus of APP/PS1 model. Therefore, microtubule stabilizing drugs could be considered therapeutical candidates to slow down AD progression. Supported by FIS-PI12/01431 and PI15/00796 (AG),FIS-PI12/01439 and PI15/00957(JV)Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

CINEMA PRODUCTION IN ANDALUCIA: REGIONAL AUDIO-VISUAL ECOSYSTEM

Los objetivos principales de esta investigación son analizar el estado de la producción cinematográfica en Andalucía y evaluar su particularidad o su adecuación a las dinámicas de la industria nacional. La producción de cine en España se caracteriza por la fuerte polarización de las empresas productoras, la concentración del éxito en pocos títulos, la importancia de la implicación de las televisiones y la necesaria complicidad de las filiales nacionales de las majors de Hollywood en la distribución. Intentaremos comprobar si este modelo se reproduce a escala autonómica en Andalucía. Aplicaremos una metodología basada en el estudio de caso andaluz, a partir de la revisión bibliográfica de los estudios e informes dedicados a la producción cinematográfica en Andalucía y en España.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tec

SYSTEMIC ADMINISTRATION OF EPOTHYLONE-D RECUES MEMORY AND AMELIORATES ALZHEIMER’S DISEASE-LIKE PATHOLOGY IN APP/PS1 MICE

Aims Cognitive and memory decline in Alzheimer's disease (AD) patients is highly related to synaptic dysfunction and neuronal loss. Tau hyperphosphorylation destabilizes microtubules leading to axonal transport failure, accumulation of autophagy/vesicular material and the generation of dystrophic neurites, thus contributing to axonal/synaptic dysfunction. In this study, we analyzed the effect of a microtubule-stabilizing drug in the progression of the disease in an APP751SL/PS1M146L transgenic model. Method APP/PS1 mice (3 month-old) were weekly treated with 2 mg/kg intraperitoneal injections of Epothilone-D (Epo-D) for 3 months. Vehicle-injected animals were used as controls. For memory performance, animals were tested on the object-recognition tasks, Y-maze and Morris water maze. Levels of Abeta, ubiquitin, AT8 and synaptic markers were analyzed by Western-blot. Hippocampal plaque burden, dystrophic and synaptic loadings were quantified after immunostaining by image analysis. Results Epo-D treated mice showed a significant improvement in the performance of hippocampus-associated cognitive tests compared to controls. This memory recovery correlated with a significant reduction in the AD-like hippocampal pathology. Abeta, APP and ubiquitin levels were significantly reduced in treated animals, and a decrease in both the plaque loading and the axonal pathology was also found. Finally, synaptic levels were significantly preserved in treated animals in comparison with controls. Conclusion Epo-D treatment promotes synaptic and cognitive improvement, reduces the accumulation of extracellular Abeta and the associated neuritic pathology in the hippocampus of APP/PS1 model. Therefore, microtubule stabilizing drugs could be considered therapeutical candidates to slow down AD progression.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. Supported by FIS-PI15/00796 (AG), FIS-PI15/00957(JV) and co-financed by FEDER funds from European Union

A Hybrid Piece-Wise Slowdown Model for Concurrent Kernel Execution on GPU

Current execution of kernels on GPUs allows improving the use of hardware resources and reducing the execution time of co-executed kernels. In addition, efficient kernel-oriented scheduling policies pursuing criteria based on fairness or Quality of Service can be implemented. However, achieved co-executing performance strongly depends on how GPU resources are partitioned between kernels. Thus, precise slowdown models that predict accurate co-execution performance must be used to fulfill scheduling policy requirements. Most recent slowdown models work with Spatial Multitask (SMT) partitioning, where Stream Multiprocessors (SMs) are distributed among tasks. In this work, we show that Simultaneous Multikernel (SMK) partitioning, where kernels share the SMs, obtains better performance. However, kernel interference in SMK occurs not only in global memory, as in the SMT case, but also within the SM, leading to high prediction errors. Here, we propose a modification of a previous state-of-the-art slowdown model to reduce median prediction error from 27.92% to 9.50%. Moreover, this new slowdown model is used to implement a scheduling policy that improves fairness by 1.41x on average compared to even partitioning, whereas previous models reach only 1.21x on average.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech P18-FR-3130 UMA20-FEDERJA-059 PID2019-105396RB-I0

The NoisyOffice Database: A Corpus To Train Supervised Machine Learning Filters For Image Processing

[EN] This paper presents the `NoisyOffice¿ database. It consists of images of printed text documents with noise mainly caused by uncleanliness from a generic office, such as coffee stains and footprints on documents or folded and wrinkled sheets with degraded printed text. This corpus is intended to train and evaluate supervised learning methods for cleaning, binarization and enhancement of noisy images of grayscale text documents. As an example, several experiments of image enhancement and binarization are presented by using deep learning techniques. Also, double-resolution images are also provided for testing super-resolution methods. The corpus is freely available at UCI Machine Learning Repository. Finally, a challenge organized by Kaggle Inc. to denoise images, using the database, is described in order to show its suitability for benchmarking of image processing systems.This research was undertaken as part of the project TIN2017-85854-C4-2-R, jointly funded by the Spanish MINECO and FEDER founds.Castro-Bleda, MJ.; España Boquera, S.; Pastor Pellicer, J.; Zamora Martínez, FJ. (2020). The NoisyOffice Database: A Corpus To Train Supervised Machine Learning Filters For Image Processing. The Computer Journal. 63(11):1658-1667. https://doi.org/10.1093/comjnl/bxz098S165816676311Bozinovic, R. M., & Srihari, S. N. (1989). Off-line cursive script word recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(1), 68-83. doi:10.1109/34.23114Plamondon, R., & Srihari, S. N. (2000). Online and off-line handwriting recognition: a comprehensive survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 63-84. doi:10.1109/34.824821Vinciarelli, A. (2002). A survey on off-line Cursive Word Recognition. Pattern Recognition, 35(7), 1433-1446. doi:10.1016/s0031-3203(01)00129-7Impedovo, S. (2014). More than twenty years of advancements on Frontiers in handwriting recognition. Pattern Recognition, 47(3), 916-928. doi:10.1016/j.patcog.2013.05.027Baird, H. S. (2007). The State of the Art of Document Image Degradation Modelling. Advances in Pattern Recognition, 261-279. doi:10.1007/978-1-84628-726-8_12Egmont-Petersen, M., de Ridder, D., & Handels, H. (2002). Image processing with neural networks—a review. Pattern Recognition, 35(10), 2279-2301. doi:10.1016/s0031-3203(01)00178-9Marinai, S., Gori, M., & Soda, G. (2005). Artificial neural networks for document analysis and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(1), 23-35. doi:10.1109/tpami.2005.4Rehman, A., & Saba, T. (2012). Neural networks for document image preprocessing: state of the art. Artificial Intelligence Review, 42(2), 253-273. doi:10.1007/s10462-012-9337-zLazzara, G., & Géraud, T. (2013). Efficient multiscale Sauvola’s binarization. International Journal on Document Analysis and Recognition (IJDAR), 17(2), 105-123. doi:10.1007/s10032-013-0209-0Fischer, A., Indermühle, E., Bunke, H., Viehhauser, G., & Stolz, M. (2010). Ground truth creation for handwriting recognition in historical documents. Proceedings of the 8th IAPR International Workshop on Document Analysis Systems - DAS ’10. doi:10.1145/1815330.1815331Belhedi, A., & Marcotegui, B. (2016). Adaptive scene‐text binarisation on images captured by smartphones. IET Image Processing, 10(7), 515-523. doi:10.1049/iet-ipr.2015.0695Kieu, V. C., Visani, M., Journet, N., Mullot, R., & Domenger, J. P. (2013). An efficient parametrization of character degradation model for semi-synthetic image generation. Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing - HIP ’13. doi:10.1145/2501115.2501127Fischer, A., Visani, M., Kieu, V. C., & Suen, C. Y. (2013). Generation of learning samples for historical handwriting recognition using image degradation. Proceedings of the 2nd International Workshop on Historical Document Imaging and Processing - HIP ’13. doi:10.1145/2501115.2501123Journet, N., Visani, M., Mansencal, B., Van-Cuong, K., & Billy, A. (2017). DocCreator: A New Software for Creating Synthetic Ground-Truthed Document Images. Journal of Imaging, 3(4), 62. doi:10.3390/jimaging3040062Walker, D., Lund, W., & Ringger, E. (2012). A synthetic document image dataset for developing and evaluating historical document processing methods. Document Recognition and Retrieval XIX. doi:10.1117/12.912203Dong, C., Loy, C. C., He, K., & Tang, X. (2016). Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2), 295-307. doi:10.1109/tpami.2015.2439281Suzuki, K., Horiba, I., & Sugie, N. (2003). Neural edge enhancer for supervised edge enhancement from noisy images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12), 1582-1596. doi:10.1109/tpami.2003.1251151Hidalgo, J. L., España, S., Castro, M. J., & Pérez, J. A. (2005). Enhancement and Cleaning of Handwritten Data by Using Neural Networks. Lecture Notes in Computer Science, 376-383. doi:10.1007/11492429_46Pastor-Pellicer, J., España-Boquera, S., Zamora-Martínez, F., Afzal, M. Z., & Castro-Bleda, M. J. (2015). Insights on the Use of Convolutional Neural Networks for Document Image Binarization. Lecture Notes in Computer Science, 115-126. doi:10.1007/978-3-319-19222-2_10España-Boquera, S., Zamora-Martínez, F., Castro-Bleda, M. J., & Gorbe-Moya, J. (s. f.). Efficient BP Algorithms for General Feedforward Neural Networks. Lecture Notes in Computer Science, 327-336. doi:10.1007/978-3-540-73053-8_33Zamora-Martínez, F., España-Boquera, S., & Castro-Bleda, M. J. (s. f.). Behaviour-Based Clustering of Neural Networks Applied to Document Enhancement. Lecture Notes in Computer Science, 144-151. doi:10.1007/978-3-540-73007-1_18Graves, A., Fernández, S., & Schmidhuber, J. (2007). Multi-dimensional Recurrent Neural Networks. Artificial Neural Networks – ICANN 2007, 549-558. doi:10.1007/978-3-540-74690-4_56Sauvola, J., & Pietikäinen, M. (2000). Adaptive document image binarization. Pattern Recognition, 33(2), 225-236. doi:10.1016/s0031-3203(99)00055-2Pastor-Pellicer, J., Castro-Bleda, M. J., & Adelantado-Torres, J. L. (2015). esCam: A Mobile Application to Capture and Enhance Text Images. Lecture Notes in Computer Science, 601-604. doi:10.1007/978-3-319-19222-2_5

Surface-Enhanced Raman spectroscopy of thiobenzoic acid on metal nanoparticles

Molecules adsorbed on some metal surfaces such as silver, copper and gold, can exhibit enormous Surface-enhanced Raman Scattering (SERS). The SERS effect has historically been associated with substrate roughness on two characteristic length scales [1,2]. Surface roughness on the 10 to 100 nm length scale supports the electromagnetic resonances which are the dominant mechanism of enhancement. A second mechanism often thought to require atomic scale roughness, is referred to as the chemical enhancement mechanism. This second mechanism involves the creation of new electronic excited states which result from adsorbate–substrate chemical interactions. These two mechanisms operate simultaneously making it difficult to isolate the role and magnitude of each one. In this work the SERS spectra of thiobenzoic acid (TBA) adsorbed on several silver colloids are recorded by using different excitation wavelengths. Taking advantage of the fact that SERS spectroscopy is both, surface selective and highly sensitive, we have attempted to determine the molecular structure of the surface complex once TBA is adsorbed on the metal. The analysis of the vibrational wavenumbers of the Raman and SERS spectra suggests that this molecule shows unidentate coordination to the silver surface through the sulphur atom. In order to confirm this conclusion DFT calculations have been carried out for different TBA-silver complexes concluding that the unidentate coordination is the most likely interaction of TBA on the metal surface. Wavelength-scanned SERS excitation spectroscopy involves the measurement of SERS signal by using several excitation wavelengths and it was recognized as a useful tool for checking the mechanisms responsible for the SERS enhancement [3]. We have studied the effect of the wavelength within the SERS spectra of TBA on silver colloid prepared by different methods and we have analyzed the intensity of the 8a vibrational mode of TBA recorded at about 1590 cm-1. The intensity of this mode is noticeably higher in the spectrum recorded with the 514.5 nm line (Fig. 1). This result is attributed to the presence of a resonant Raman effect associated to a photoinduced charge-transfer process when using the most energetic excitation line. Finally, it is important to mention the high affinity of TBA for the silver surfaces what allows for a detection limit estimated to be 0.01 microM.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech

Raman, SERS and DFT study of chemically-adsorbed thiobenzoic acid on silver nanoparticles

Thiocarboxylic acids are organosulphur compounds with general formula RC(O)SH. They are related to carboxylic acids by the replacement of one oxygen by sulphur. Two tautomers are possible, written as RC(S)OH and RC(O)SH. The second one is the majority species in solid state and solution of thiobenzoic acid (TBA) at room temperature [1], but derivatives from both tautomers are known so that the SERS spectrum can be originated by either one. Taking advantage of the fact that SERS spectroscopy is both surface selective and highly sensitive we have attempted to determine the molecular structure of TBA once it is adsorbed on the metal surface. To accomplish this SERS spectra of TBA have been recorded on different silver colloids. A combination of layer-by-layer method with spin-coating deposition of silver nanoparticles have been used to prepare SERS active substrates on which the homogeneity of the SERS signal of TBA has been analyzed. Fig. 1 (I) shows the Raman spectra of TBA in the neat liquid (a), 1 M aqueous solution at pH 14 (b), SERS spectrum of a 5x10-4 M silver colloid prepared by reduction of silver nitrate with sodium borohydride at pH 7 (c) and SERS spectrum of a 5x10-4 M silver colloid prepared by reduction of silver nitrate with hydroxylamine hydrochloride (d). The assignment of the Raman spectra has been based on the present work as well as on previous studies [2-4]. The two bands recorded at 1662 and 2572 cm-1 in Fig.1a, are assigned to (C=O) and (SH) modes, what confirms that the Raman spectrum in the neat liquid is due to the thiolic specie of TBA, RC(O)SH. In the Raman spectrum of the solution (Fig. 1b) a significant redshift of (C=O) mode of 52 cm-1 and the absence of the (SH) band are detected in agreement with the behavior observed in the SERS spectrum (Fig. 1c). The latter confirms that the thiol tautomer of thiobenzoate anion, RC(O)S-, is adsorbed on silver nanoparticles. Other important SERS enhancements have been registered for the following vibrational modes: 8a;ring, (C=O) and (CS) recorded at 1592, 1554 and 928 cm-1 in Fig. 1c and 1d, respectively. The last two modes, (C=O) and (CS), undergo wavenumber shifts of +40 and -40 cm-1 respectively, which are closely related with the coordination of thiobenzoate anion to the metal surface [5]. The analysis of the vibrational wavenumber of these modes suggests that this molecule shows unidentate coordination through the sulphur atom to the metal surface. In order to confirm this fact DFT calculations have been carried out for different silver complexes: I) bridging bidentate ligand, (II) chelating ligand and (III and IV) unidentate ligand (Fig. 2). Theoretical wavenumber of representative bands of these compounds have been compared to the experimental one concluding that the behavior of the unidentate ligand (III) is the most probably coordination type of TBA on the metal surface. In order to confirm these conclusions the SERS spectra of TBA on silver colloid prepared by reduction of silver nitrate with hydroxylamine hydrochloride at different concentration of analyte have been recorded as well (Fig. 1 (II)). It is well known that the SERS enhancement factor depends strongly on different factors and in particular on the adsorption properties of the probe and the analyte concentration on the surface coverage. In this sense TBA has shown a very good detection level for this particular silver colloid it being a highly SERS active molecule. The detection limit is estimated to be 0.01 molar. Finally, Fig. 3 shows a representative Raman mapping of TBA adsorbed on a silver substrate prepared by spin-coating. Generally speaking, the image represents a fairly homogeneous distribution of the SERS intensity highlighting some points where the intensity is stronger as is expected in areas with heterogeneous coverage. The reproducibility of this type of substrate is under study focusing their application as reproducible and ultrasensitive sensing assemblies by using TBA as the target molecule due its good SERS sensitivity.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tec