Is One Epoch All You Need For Multi-Fidelity Hyperparameter Optimization?

Hyperparameter optimization (HPO) is crucial for fine-tuning machine learning models but can be computationally expensive. To reduce costs, Multi-fidelity HPO (MF-HPO) leverages intermediate accuracy levels in the learning process and discards low-performing models early on. We compared various representative MF-HPO methods against a simple baseline on classical benchmark data. The baseline involved discarding all models except the Top-K after training for only one epoch, followed by further training to select the best model. Surprisingly, this baseline achieved similar results to its counterparts, while requiring an order of magnitude less computation. Upon analyzing the learning curves of the benchmark data, we observed a few dominant learning curves, which explained the success of our baseline. This suggests that researchers should (1) always use the suggested baseline in benchmarks and (2) broaden the diversity of MF-HPO benchmarks to include more complex cases.Comment: 5 pages, with extended appendice

Multi-objective constrained Bayesian optimization for structural design

The planning and design of buildings and civil engineering concrete structures constitutes a complex problem subject to constraints, for instance, limit state constraints from design codes, evaluated by expensive computations such as finite element (FE) simulations. Traditionally, the focus has been on minimizing costs exclusively, while the current trend calls for good trade-offs of multiple criteria such as sustainability, buildability, and performance, which can typically be computed cheaply from the design parameters. Multi-objective methods can provide more relevant design strategies to find such trade-offs. However, the potential of multi-objective optimization methods remains unexploited in structural concrete design practice, as the expensiveness of structural design problems severely limits the scope of applicable algorithms. Bayesian optimization has emerged as an efficient approach to optimizing expensive functions, but it has not been, to the best of our knowledge, applied to constrained multi-objective optimization of structural concrete design problems. In this work, we develop a Bayesian optimization framework explicitly exploiting the features inherent to structural design problems, that is, expensive constraints and cheap objectives. The framework is evaluated on a generic case of structural design of a reinforced concrete (RC) beam, taking into account sustainability, buildability, and performance objectives, and is benchmarked against the well-known Non-dominated Sorting Genetic Algorithm II (NSGA-II) and a random search procedure. The results show that the Bayesian algorithm performs considerably better in terms of rate-of-improvement, final solution quality, and variance across repeated runs, which suggests it is well-suited for multi-objective constrained optimization problems in structural design

Schemes Based on Federated Learning for Decentralized Training in Machine Learning Models

Standard Machine Learning approaches require large amounts of data usually centralized in data centers. In these approaches, there is only one device responsible for the training of the whole process. New collaborative approaches allow the training of common models from different decentralized devices, each one holding local data samples. An example is Federated Learning. In recent years, along with the blooming of Machine Learning based applications and services, ensuring data privacy and security have become a critical obligation. In this work, three training procedures based on Federated Learning were tested: FedAvg, FedADA, and LoADABoost comparing their performance versus a traditional centralized training method. Using public information from written reviews about movies, a neural network algorithm was implemented. The objective of the model was to predict whether a review is positive or negative. Utilizing the F1 Score as a performance metric, the hypothesis was to validate whether the Federated Learning training methods are similar to traditional centralized training methodologies. After the implementation of the same neural network with different training methodologies, no major differences or changes in performance were noted, concluding that Federated Learning is indeed a similar and viable training methodology