Reinforcement Learning and Physics

Machine learning techniques provide a remarkable tool for advancing scientific research, and this area has significantly grown in the past few years. In particular, reinforcement learning, an approach that maximizes a (long-term) reward by means of the actions taken by an agent in a given environment, can allow one for optimizing scientific discovery in a variety of fields such as physics, chemistry, and biology. Morover, physical systems, in particular quantum systems, may allow one for more efficient reinforcement learning protocols. In this review, we describe recent results in the field of reinforcement learning and physics. We include standard reinforcement learning techniques in the computer science community for enhancing physics research, as well as the more recent and emerging area of quantum reinforcement learning, inside quantum machine learning, for improving reinforcement learning computations.Ministerio de Ciencia e Innovación PGC2018- 095113-B-I00, PID2019-104002GB-C21 and PID2019-104002GB-C2

Quantum Machine Learning: A tutorial

This tutorial provides an overview of Quantum Machine Learning (QML), a relatively novel discipline that brings together concepts from Machine Learning (ML), Quantum Computing (QC) and Quantum Information (QI). The great development experienced by QC, partly due to the involvement of giant technological companies as well as the popularity and success of ML have been responsible of making QML one of the main streams for researchers working on fuzzy borders between Physics, Mathematics and Computer Science. A possible, although arguably coarse, classification of QML methods may be based on those approaches that make use of ML in a quantum experimentation environment and those others that take advantage of QC and QI to find out alternative and enhanced solutions to problems driven by data, oftentimes offering a considerable speedup and improved performances as a result of tackling problems from a complete different standpoint. Several examples will be provided to illustrate both classes of methods.Ministerio de Ciencia, Innovación y Universidades GC2018-095113-B-I00,PID2019-104002GB-C21, and PID2019-104002GB-C22 (MCIU/AEI/FEDER, UE

Supervised Quantum Learning without Measurements

We propose a quantum machine learning algorithm for efficiently solving a class of problems encoded in quantum controlled unitary operations. The central physical mechanism of the protocol is the iteration of a quantum time-delayed equation that introduces feedback in the dynamics and eliminates the necessity of intermediate measurements. The performance of the quantum algorithm is analyzed by comparing the results obtained in numerical simulations with the outcome of classical machine learning methods for the same problem. The use of time-delayed equations enhances the toolbox of the field of quantum machine learning, which may enable unprecedented applications in quantum technologies

Active Learning in Physics: From 101, to Progress, and Perspective

Active Learning (AL) is a family of machine learning (ML) algorithms that predates the current era of artificial intelligence. Unlike traditional approaches that require labeled samples for training, AL iteratively selects unlabeled samples to be annotated by an expert. This protocol aims to prioritize the most informative samples, leading to improved model performance compared to training with all labeled samples. In recent years, AL has gained increasing attention, particularly in the field of physics. This paper presents a comprehensive and accessible introduction to the theory of AL reviewing the latest advancements across various domains. Additionally, we explore the potential integration of AL with quantum ML, envisioning a synergistic fusion of these two fields rather than viewing AL as a mere extension of classical ML into the quantum realm.Comment: 15 page

Actual treatment of attention deficit hyperactivity disorder (ADHD)

El tratamiento del trastorno por défi cit de atención e hiperactividad (TDAH) incluye intervenciones farmacológicas, psicosociales y educativas, y en él se aconseja un diseño personalizado teniendo en cuenta las características del paciente, el tipo de trastorno y la comorbilidad que lo acompaña. Los fármacos de primera línea son el psicoestimulante metilfenidato (MTF) y atomoxetina (ATX), un simpaticomimético de acción central no estimulante. Ambos reducen las manifestaciones clínicas de inquietud, inatención e impulsividad, mejorando la calidad de las relaciones sociales y el rendimiento académico. Metilfenidato bloquea el transportador presináptico de dopamina (DA) y noradrenalina (NA), aumentando la concentración de estos neurotransmisores en el espacio presináptico neuronal. Se presenta en formas de liberación inmediata (LI) (Rubifen® y Medicebran® en preparados de acción prolongada con tecnología OROS® [osmotic controlled-release oral delivery system], Concerta® y Metilfenidato Sandoz®) y en pellets (Medikinet®), que permiten seleccionar adecuadamente la dosis y la pauta posológica. Las formas de LI pueden inducir efecto rebote al provocar un pico plasmático elevado que decae en poco tiempo. Atomoxetina (Strattera®) es un inhibidor muy selectivo y potente del transportador presináptico de NA; aumenta los niveles de NA y DA en la corteza prefrontal, pero no en las regiones corticales relacionadas con el desarrollo de tics o riesgo de abusos de sustancias. Puede ser la alternativa a MTF cuando éste pierde efi cacia o está contraindicado. La efectividad de ambos fármacos debe considerarse a partir de las 2-4 semanas. Sus reacciones adversas son numerosas y con frecuencia causan malestar, lo que difi culta la adherencia. Por ello es necesario el seguimiento de estos pacientes, y el farmacéutico puede ejercer un papel destacado para mejorar el cumplimiento y los efectos de la farmacoterapiaTreatment of attention defi cit hyperactivity disorder (ADHD) includes pharmacological, psychosocial and educational interventions. A custom designed treatment taking into account patient characteristics, type of disorder and comorbidity must be advisable. First election drugs are the psychostimulant methylphenidate (MTF) and the sympathomimetic not stimulant atomoxetine (ATX). These drugs reduce the clinical manifestations of restlessness, inattention and impulsivity, improving the quality of social relationships and academic performance. MTF blocks the presynaptic dopamine (DA) and norepinephrine (NA) transporters increasing the concentration of these neurotransmitters in the presynaptic neuron. Both of them are available in the pharmaceutical forms of immediate release (IR) (Rubifen ® and Medicebran®, prolonged acting preparations with OROS® [osmotic controlled-release oral delivery system] technology, Con certa® and Metilfenidato Sandoz®) and pellets (Medikinet®), allowing a proper selection of dosage pattern. IR pharmaceutical forms can induce rebounding effect by causing high plasma peak that decays quickly. ATX is a highly selective and a potent inhibitor of presynaptic NA transporter, increasing levels of NA and DA in the prefrontal cortex, but not in cortical regions related to the development of tics or risk of substance abuse. It can be an alternative to MTF when this loses effectiveness or is contraindicated. The effectiveness of both drugs must be considered after 2 to 4 weeks of treatment. Their side effects are numerous and often cause discomfort making diffi cult adherence. Therefore it is necessary to monitor these patients playing pharmacist a leading role in improving the performance and the effects of pharmacotherap

Risk Assessment of Hip Fracture Based on Machine Learning

[EN] Identifying patients with high risk of hip fracture is a great challenge in osteoporosis clinical assessment. Bone Mineral Density (BMD) measured by Dual-Energy X-Ray Absorptiometry (DXA) is the current gold standard in osteoporosis clinical assessment. However, its classification accuracy is only around 65%. In order to improve this accuracy, this paper proposes the use of Machine Learning (ML) models trained with data from a biomechanical model that simulates a sideways-fall. Machine Learning (ML) models are models able to learn and to make predictions from data. During a training process, ML models learn a function that maps inputs and outputs without previous knowledge of the problem. The main advantage of ML models is that once the mapping function is constructed, they can make predictions for complex biomechanical behaviours in real time. However, despite the increasing popularity of Machine Learning (ML) models and their wide application to many fields of medicine, their use as hip fracture predictors is still limited. This paper proposes the use of ML models to assess and predict hip fracture risk. Clinical, geometric, and biomechanical variables from the finite element simulation of a side fall are used as independent variables to train the models. Among the different tested models, Random Forest stands out, showing its capability to outperform BMD-DXA, achieving an accuracy over 87%, with specificity over 92% and sensitivity over 83%.This study was partially funded by the FPI grant (FPI-SP20170111) from the Universitat Politecnica de Valencia obtained by Eduardo Villamor.Galassi, A.; Martín-Guerrero, JD.; Villamor, E.; Monserrat Aranda, C.; Rupérez Moreno, MJ. (2020). Risk Assessment of Hip Fracture Based on Machine Learning. Applied bionics and biomechanics (Online). 2020:1-13. https://doi.org/10.1155/2020/8880786S1132020World Health OrganizationAssessment of fracture risk and its application to screening for postmenopausal osteoporosis. Report of a WHO Study Group1994http://www.who.int/iris/handle/10665/39142, http://apps.who.int//iris/handle/10665/39142Cooper, C., Campion, G., & Melton, L. J. (1992). Hip fractures in the elderly: A world-wide projection. Osteoporosis International, 2(6), 285-289. doi:10.1007/bf01623184El Maghraoui, A., & Roux, C. (2008). DXA scanning in clinical practice. QJM, 101(8), 605-617. doi:10.1093/qjmed/hcn022Testi, D., Viceconti, M., Cappello, A., & Gnudi, S. (2002). Prediction of Hip Fracture Can Be Significantly Improved by a Single Biomedical Indicator. Annals of Biomedical Engineering, 30(6), 801-807. doi:10.1114/1.1495866Nguyen, N. D., Frost, S. A., Center, J. R., Eisman, J. A., & Nguyen, T. V. (2008). Development of prognostic nomograms for individualizing 5-year and 10-year fracture risks. Osteoporosis International, 19(10), 1431-1444. doi:10.1007/s00198-008-0588-0Bolland, M. J., Siu, A. T., Mason, B. H., Horne, A. M., Ames, R. W., Grey, A. B., … Reid, I. R. (2011). Evaluation of the FRAX and Garvan fracture risk calculators in older women. Journal of Bone and Mineral Research, 26(2), 420-427. doi:10.1002/jbmr.215Fountoulis, G., Kerenidi, T., Kokkinis, C., Georgoulias, P., Thriskos, P., Gourgoulianis, K., … Vlychou, M. (2016). Assessment of Bone Mineral Density in Male Patients with Chronic Obstructive Pulmonary Disease by DXA and Quantitative Computed Tomography. International Journal of Endocrinology, 2016, 1-6. doi:10.1155/2016/6169721Pellicer-Valero, O. J., Rupérez, M. J., Martínez-Sanchis, S., & Martín-Guerrero, J. D. (2020). Real-time biomechanical modeling of the liver using Machine Learning models trained on Finite Element Method simulations. Expert Systems with Applications, 143, 113083. doi:10.1016/j.eswa.2019.113083Martínez-Martínez, F., Rupérez-Moreno, M. J., Martínez-Sober, M., Solves-Llorens, J. A., Lorente, D., Serrano-López, A. J., … Martín-Guerrero, J. D. (2017). A finite element-based machine learning approach for modeling the mechanical behavior of the breast tissues under compression in real-time. Computers in Biology and Medicine, 90, 116-124. doi:10.1016/j.compbiomed.2017.09.019Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2), 94-98. doi:10.7861/futurehosp.6-2-94Kruse, C., Eiken, P., & Vestergaard, P. (2016). Clinical fracture risk evaluated by hierarchical agglomerative clustering. Osteoporosis International, 28(3), 819-832. doi:10.1007/s00198-016-3828-8Ho-Le, T. P., Center, J. R., Eisman, J. A., Nguyen, T. V., & Nguyen, H. T. (2017). Prediction of hip fracture in post-menopausal women using artificial neural network approach. 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). doi:10.1109/embc.2017.8037784Dall’Ara, E., Eastell, R., Viceconti, M., Pahr, D., & Yang, L. (2016). Experimental validation of DXA-based finite element models for prediction of femoral strength. Journal of the Mechanical Behavior of Biomedical Materials, 63, 17-25. doi:10.1016/j.jmbbm.2016.06.004Enns-Bray, W. S., Bahaloo, H., Fleps, I., Pauchard, Y., Taghizadeh, E., Sigurdsson, S., … Helgason, B. (2019). Biofidelic finite element models for accurately classifying hip fracture in a retrospective clinical study of elderly women from the AGES Reykjavik cohort. Bone, 120, 25-37. doi:10.1016/j.bone.2018.09.014Testi, D., Viceconti, M., Baruffaldi, F., & Cappello, A. (1999). Risk of fracture in elderly patients: a new predictive index based on bone mineral density and finite element analysis. Computer Methods and Programs in Biomedicine, 60(1), 23-33. doi:10.1016/s0169-2607(99)00007-3Yang, L., Palermo, L., Black, D. M., & Eastell, R. (2014). Prediction of Incident Hip Fracture with the Estimated Femoral Strength by Finite Element Analysis of DXA Scans in the Study of Osteoporotic Fractures. Journal of Bone and Mineral Research, 29(12), 2594-2600. doi:10.1002/jbmr.2291Luo, Y., Ahmed, S., & Leslie, W. D. (2018). Automation of a DXA-based finite element tool for clinical assessment of hip fracture risk. Computer Methods and Programs in Biomedicine, 155, 75-83. doi:10.1016/j.cmpb.2017.11.020Terzini, M., Aldieri, A., Rinaudo, L., Osella, G., Audenino, A. L., & Bignardi, C. (2019). Improving the Hip Fracture Risk Prediction Through 2D Finite Element Models From DXA Images: Validation Against 3D Models. Frontiers in Bioengineering and Biotechnology, 7. doi:10.3389/fbioe.2019.00220Nishiyama, K. K., Ito, M., Harada, A., & Boyd, S. K. (2013). Classification of women with and without hip fracture based on quantitative computed tomography and finite element analysis. Osteoporosis International, 25(2), 619-626. doi:10.1007/s00198-013-2459-6Jiang, P., Missoum, S., & Chen, Z. (2015). Fusion of clinical and stochastic finite element data for hip fracture risk prediction. Journal of Biomechanics, 48(15), 4043-4052. doi:10.1016/j.jbiomech.2015.09.044Ferizi, U., Besser, H., Hysi, P., Jacobs, J., Rajapakse, C. S., Chen, C., … Chang, G. (2018). Artificial Intelligence Applied to Osteoporosis: A Performance Comparison of Machine Learning Algorithms in Predicting Fragility Fractures From MRI Data. Journal of Magnetic Resonance Imaging, 49(4), 1029-1038. doi:10.1002/jmri.26280Villamor, E., Monserrat, C., Del Río, L., Romero-Martín, J. A., & Rupérez, M. J. (2020). Prediction of osteoporotic hip fracture in postmenopausal women through patient-specific FE analyses and machine learning. Computer Methods and Programs in Biomedicine, 193, 105484. doi:10.1016/j.cmpb.2020.105484Rossman, T., Kushvaha, V., & Dragomir-Daescu, D. (2015). QCT/FEA predictions of femoral stiffness are strongly affected by boundary condition modeling. Computer Methods in Biomechanics and Biomedical Engineering, 19(2), 208-216. doi:10.1080/10255842.2015.1006209Si, H. (2015). TetGen, a Delaunay-Based Quality Tetrahedral Mesh Generator. ACM Transactions on Mathematical Software, 41(2), 1-36. doi:10.1145/2629697Morgan, E. F., & Keaveny, T. M. (2001). Dependence of yield strain of human trabecular bone on anatomic site. Journal of Biomechanics, 34(5), 569-577. doi:10.1016/s0021-9290(01)00011-2Morgan, E. F., Bayraktar, H. H., & Keaveny, T. M. (2003). Trabecular bone modulus–density relationships depend on anatomic site. Journal of Biomechanics, 36(7), 897-904. doi:10.1016/s0021-9290(03)00071-xBayraktar, H. H., Morgan, E. F., Niebur, G. L., Morris, G. E., Wong, E. K., & Keaveny, T. M. (2004). Comparison of the elastic and yield properties of human femoral trabecular and cortical bone tissue. Journal of Biomechanics, 37(1), 27-35. doi:10.1016/s0021-9290(03)00257-4Wirtz, D. C., Schiffers, N., Pandorf, T., Radermacher, K., Weichert, D., & Forst, R. (2000). Critical evaluation of known bone material properties to realize anisotropic FE-simulation of the proximal femur. Journal of Biomechanics, 33(10), 1325-1330. doi:10.1016/s0021-9290(00)00069-5Eckstein, F., Wunderer, C., Boehm, H., Kuhn, V., Priemel, M., Link, T. M., & Lochmüller, E.-M. (2003). Reproducibility and Side Differences of Mechanical Tests for Determining the Structural Strength of the Proximal Femur. Journal of Bone and Mineral Research, 19(3), 379-385. doi:10.1359/jbmr.0301247Orwoll, E. S., Marshall, L. M., Nielson, C. M., Cummings, S. R., Lapidus, J., … Cauley, J. A. (2009). Finite Element Analysis of the Proximal Femur and Hip Fracture Risk in Older Men. Journal of Bone and Mineral Research, 24(3), 475-483. doi:10.1359/jbmr.081201Maas, S. A., Ellis, B. J., Ateshian, G. A., & Weiss, J. A. (2012). FEBio: Finite Elements for Biomechanics. Journal of Biomechanical Engineering, 134(1). doi:10.1115/1.4005694Choi, W. J., Cripton, P. A., & Robinovitch, S. N. (2014). Effects of hip abductor muscle forces and knee boundary conditions on femoral neck stresses during simulated falls. Osteoporosis International, 26(1), 291-301. doi:10.1007/s00198-014-2812-4Van den Kroonenberg, A. J., Hayes, W. C., & McMahon, T. A. (1995). Dynamic Models for Sideways Falls From Standing Height. Journal of Biomechanical Engineering, 117(3), 309-318. doi:10.1115/1.2794186Robinovitch, S. N., McMahon, T. A., & Hayes, W. C. (1995). Force attenuation in trochanteric soft tissues during impact from a fall. Journal of Orthopaedic Research, 13(6), 956-962. doi:10.1002/jor.1100130621Dufour, A. B., Roberts, B., Broe, K. E., Kiel, D. P., Bouxsein, M. L., & Hannan, M. T. (2011). The factor-of-risk biomechanical approach predicts hip fracture in men and women: the Framingham Study. Osteoporosis International, 23(2), 513-520. doi:10.1007/s00198-011-1569-2BowyerK. W.ChawlaN. V.HallL. O.KegelmeyerW. P.SMOTE: synthetic minority over-sampling techniqueCoRRhttps://arxiv.org/abs/1106.181

Towards Prediction of Financial Crashes with a D-Wave Quantum Computer

Prediction of financial crashes in a complex financial network is known to be an NP-hard problem, i.e., a problem which cannot be solved efficiently with a classical computer. We experimentally explore a novel approach to this problem by using a D-Wave quantum computer to obtain financial equilibrium more efficiently. To be specific, the equilibrium condition of a nonlinear financial model is embedded into a higher-order unconstrained binary optimization (HUBO) problem, which is then transformed to a spin-$1/2$ Hamiltonian with at most two-qubit interactions. The problem is thus equivalent to finding the ground state of an interacting spin Hamiltonian, which can be approximated with a quantum annealer. Our experiment paves the way to study quantitative macroeconomics, enlarging the number of problems that can be handled by current quantum computers