Fey, Rob H.B.

Janssen, Tom M.E.

Kessels, B.M.

van de Wouw, Nathan

English

To decrease the mismatch between a model and a physical system, physically interpretable model parameter values of nonlinear systems can be updated in real-time by using the Inverse Mapping Parameter Updating (IMPU) method. In this method, an Inverse Mapping Model (IMM), constituted by an Artificial Neural Network (ANN), is trained, offline, using simulated data that consists of features of output responses (ANN inputs) and corresponding parameter values (ANN outputs). In an online phase, the trained ANN can then be used to infer parameter values with high computational efficiency. The (non-trivial) choice of ANN-hyperparameters, e.g., ANN structure and training settings, may significantly influence the accuracy of the trained ANN. Therefore, this work discusses multiple ANN-hyperparameter tuning techniques to increase the accuracy of the IMPU method, of which the Bayesian search technique is the most promising considering accuracy and efficiency as it learns from previously evaluated hyperparameter values

Pure OAI Repository

 Neural network hyperparameter tuning for online modelparameter updating using inverse mapping modelsCitation for published version (APA):Kessels, B. M., Janssen, T. M. E., Fey, R. H. B., & van de Wouw, N. (2023). Neural network hyperparametertuning for online model parameter updating using inverse mapping models. Abstract from Nonlinear DynamicsConference, Rome, Italy.Document status and date:Published: 01/01/2023Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)Please check the document version of this publication:• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publicationGeneral rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal.If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverneTake down policyIf you believe that this document breaches copyright please contact us at:openaccess@tue.nlproviding details and we will investigate your claim.Download date: 04. Oct. 2023Neural network hyperparameter tuning for online model parameter updating usinginverse mapping modelsBas M. Kessels, Tom M.E. Janssen, Rob H.B. Fey, and Nathan van de WouwDynamics and Control, Department of Mechanical Engineering, Eindhoven University of Technology, The NetherlandsAbstract. To decrease the mismatch between a model and a physical system, physically interpretable model parame-ter values of nonlinear systems can be updated in real-time by using the Inverse Mapping Parameter Updating (IMPU)method. In this method, an Inverse Mapping Model (IMM), constituted by an Artificial Neural Network (ANN), is trained,offline, using simulated data that consists of features of output responses (ANN inputs) and corresponding parametervalues (ANN outputs). In an online phase, the trained ANN can then be used to infer parameter values with high com-putational efficiency. The (non-trivial) choice of ANN-hyperparameters, e.g., ANN structure and training settings, maysignificantly influence the accuracy of the trained ANN. Therefore, this work discusses multiple ANN-hyperparametertuning techniques to increase the accuracy of the IMPU method, of which the Bayesian search technique is the mostpromising considering accuracy and efficiency as it learns from previously evaluated hyperparameter values.IntroductionIn a digital twin context, model updating is required to ensure that a model accurately represents a physicalsystems throughout its operational life. In this research, an IMM is employed that enables (near) real-timeinference of interpretable parameter values of nonlinear dynamics systems on the basis of a set of featuresrepresenting measured output response data. This is achieved by training the IMM, which is constituted by anANN, offline, using simulated training data (pairs of output response functions and corresponding parametervalues) should be chosen appropriately. Afterwards, in an online phase, the trained ANN is used to inferparameter values from measured features with little computational cost. For a detailed explanation of this IMPUmethod, see [1]. To ensure that the ANN accurately estimates parameter values, careful design of the ANN isessential, i.e., its activation functions, its number of layers and neurons, and its training settings (learning rate,number of epochs, and batch size). These hyperparameters should be tuned to minimize the validation loss,i.e., the mean squared error of the inferred parameter values (normalized between 0 and 1), as evaluated on avalidation set. In this paper, different tuning strategies are compared: grid, random, and Bayesian searches [2].Results and discussionANN hyperparameter tuning is applied to a closed-loop nonlinear multibody dynamics system consisting offour connected rigid bodies with, in total, 10 Degrees of Freedom (DoFs) of which 3 DoFs are excited and‘measured’. Using the ‘measured’ output responses, we tune the above-mentioned ANN-hyperparameters suchthat the values of eight parameters (among which stiffness constants and damping factors) are inferred withhigh accuracy. In Figure 1, the validation losses obtained using different ANN-hyperparameter searches, aswell as one manually tuned ANN (using engineering insight and experience), are ranked in ascending order.Note that, for all search types, the hyperparameter values lie within the same bounded space and the sametraining and validation data is used. Performing these grid, random, and Bayesian searches takes roughly 78,15, and 20 hours, respectively and especially the grid search is thus a time consuming (offline) task. In contrast,online inferring of one set of parameter values using the trained ANN only takes approximately 6 ms. FromFigure 1, we observe that the Bayesian search finds relatively many ANNs that outperform the manually tunedANN, while requiring much less computation time than the grid search. The mean relative (non-normalized)parameter value error (averaged over the eight parameters) for a test data set as obtained using the IMPU methodequals 0.26% for the manually tuned ANN and only 0.14% for the best performing ANN of the Bayesian search.0 200 400 600 800 1000 1200 1400 1600 180010-410-21000 10 20 30 40 5024681010-5Figure 1: Validation losses as obtained by grid, random, and Bayesian searches. The validation loss of the manually tuned ANN isindicated by the dashed line. The right figure presents a detail of the 50 best performing ANNs per search. The total number of trainedANNs and the relative percentage of ANNs that outperform the manually tuned ANN per search are indicated by n and m, respectively.References[1] Kessels, B.M., Fey, R.H.B, van de Wouw, N. (2022) Real-time parameter updating for nonlinear digital twins using inversemapping models and transient-based features. Nonlinear Dynamics (submitted, preprint available at researchsquare.com).[2] Zulfiqar, M., Gamage, K.A.A., Kamran, M., Rasheed, M. B. (2022) Sensors 22(12):4446.

Neural network hyperparameter tuning for online model parameter updating using inverse mapping models

https://pure.tue.nl/ws/files/303128626/BMKessels_ExtAbstr_NODYCON2023_NeuralNetworkHyperparameterTuningForOnlineModelParameterUpdatingUsingInverseMappingModels.pdf

Neural network hyperparameter tuning for online model parameter updating using inverse mapping models

Abstract

Similar works

Full text

Available Versions

Pure OAI Repository