On discretisation drift and smoothness regularisation in neural network training

The deep learning recipe of casting real-world problems as mathematical optimisation and tackling the optimisation by training deep neural networks using gradient-based optimisation has undoubtedly proven to be a fruitful one. The understanding behind why deep learning works, however, has lagged behind its practical significance. We aim to make steps towards an improved understanding of deep learning with a focus on optimisation and model regularisation. We start by investigating gradient descent (GD), a discrete-time algorithm at the basis of most popular deep learning optimisation algorithms. Understanding the dynamics of GD has been hindered by the presence of discretisation drift, the numerical integration error between GD and its often studied continuous-time counterpart, the negative gradient flow (NGF). To add to the toolkit available to study GD, we derive novel continuous-time flows that account for discretisation drift. Unlike the NGF, these new flows can be used to describe learning rate specific behaviours of GD, such as training instabilities observed in supervised learning and two-player games. We then translate insights from continuous time into mitigation strategies for unstable GD dynamics, by constructing novel learning rate schedules and regularisers that do not require additional hyperparameters. Like optimisation, smoothness regularisation is another pillar of deep learning's success with wide use in supervised learning and generative modelling. Despite their individual significance, the interactions between smoothness regularisation and optimisation have yet to be explored. We find that smoothness regularisation affects optimisation across multiple deep learning domains, and that incorporating smoothness regularisation in reinforcement learning leads to a performance boost that can be recovered using adaptions to optimisation methods.Comment: PhD thesis. arXiv admin note: text overlap with arXiv:2302.0195

On discretisation drift and smoothness regularisation in neural network training

The deep learning recipe of casting real-world problems as mathematical optimisation and tackling the optimisation by training deep neural networks using gradient-based optimisation has undoubtedly proven to be a fruitful one. The understanding behind why deep learning works, however, has lagged behind its practical significance. We aim to make steps towards an improved understanding of deep learning with a focus on optimisation and model regularisation. We start by investigating gradient descent (GD), a discrete-time algorithm at the basis of most popular deep learning optimisation algorithms. Understanding the dynamics of GD has been hindered by the presence of discretisation drift, the numerical integration error between GD and its often studied continuous-time counterpart, the negative gradient flow (NGF). To add to the toolkit available to study GD, we derive novel continuous-time flows that account for discretisation drift. Unlike the NGF, these new flows can be used to describe learning rate specific behaviours of GD, such as training instabilities observed in supervised learning and two-player games. We then translate insights from continuous time into mitigation strategies for unstable GD dynamics, by constructing novel learning rate schedules and regularisers that do not require additional hyperparameters. Like optimisation, smoothness regularisation is another pillar of deep learning's success with wide use in supervised learning and generative modelling. Despite their individual significance, the interactions between smoothness regularisation and optimisation have yet to be explored. We find that smoothness regularisation affects optimisation across multiple deep learning domains, and that incorporating smoothness regularisation in reinforcement learning leads to a performance boost that can be recovered using adaptions to optimisation methods

Reverse logistics and space allocation for recovery management in new urban settlements

This paper presents the authors’ vision about the planning foundation for the new residential areas from the large cities outskirts, in a sustainable development framework. One considers the great generation potential of the high and very high income population in case of the used products with remained reuse value, or new and undesired products, available in the residential places. We propose a space allocation model with a hexagonal hierarchical structure for the centralized return centers in a reverse logistics. The space allocation model for the recovery centers implementation takes into consideration: the recovery habits, environmental care and sustainable development education, “moral” compensations, centralized recovery centers facilities, walking willingness of the average inhabitant of the considered area, decision makings involvement at the local Public Authority level, and local community. One reveals the importance of the data collecting stage for the potential and availability of the exhausted products (having reuse value) in a specific area with high and very high income populationreverse logistics; centralized return centres; recovery potential; space allocation

Individualizarea marketingului sportiv in cadrul marketingului general

Ca orice domeniu nou aparut in tara noastra, si marketingul sportiv se cere a fi definit pentru a putea percepe, la adevarata sa valoare, importanta pe care o are in dezvoltarea economica (prin veniturile pe care le genereaza) si sociala (prin implicarea populatiei in activitatile sportive). Conceptualizarea acestui domeniu trebuie pornita de la cadrul general in care a aparut si s-a dezvoltat marketingul sportiv. Tinand cont de acest lucru, lucrare de fata isi propune sa dezbata diferitele puncte de vedere intalnite in teorie referitor la definitia si clasificarea marketingului sportiv.marketing sportiv, marketing pentru sport, marketing prin sport, sports marketing, outdoor marketing, marketing through sport

Decisions of hypermarkets location in dense urban area – effects on streets network congestion in the Bucharest case

The paper represents some partial results of the research carried out by the Transportation, Traffic and Logistics Department - University POLITEHNICA of Bucharest, funded by the Romanian Ministry of Research and Education through the National University Research Council. In this paper we provide: a brief description of the interrelation between the life style changes of Romanian people during the last decades and the car traffic congestion in large cities; the streets network modelling of a radial-circular urban structure (the characteristic of a historically developed city as Bucharest city is), in case of car traffic congestion; the assessment model of the additional car traffic congestion for certain locations with large attractivity. Having an important effect on the entire lifestyle of urban people, the decision of a hypermarket location might be a complex one, taking into consideration the new leisure and shopping tendencies but also the additional car traffic congestion caused by the chosen location

COMPARATIVE ANALYSIS OF OBJECT CLASSIFICATION ALGORITHMS: TRADITIONAL IMAGE PROCESSING VERSUS ARTIFICIAL INTELLIGENCE – BASED APPROACH

In the current era of advanced digital technologies, form recognition is integrated into numerous applications, from computer vision to industrial automation. This paper focuses on a comparative analysis of two distinct form recognition algorithms, namely harnessing the power of artificial intelligence (AI) and image processing techniques. The research is motivated by the need to address the trade-off between speed and complexity in form recognition, with a center on real-world applicability. Traditional image processing-based form recognition approaches often require complex coding, substantial domain expertise, and significant computational resources. This complexity can hinder rapid adaptation to changing requirements and the addition of new forms. The aim is to explore whether AI-powered algorithms can offer a more efficient and versatile alternative, reducing the barriers to entry for form recognition tasks. The primary goal of the paper is to compare the performance of AI-based form recognition with image processing-based methods in terms of speed and accuracy. The second goal is to assess the ease of adapting AI-based algorithms to new forms without extensive code changes. Two form recognition algorithms were designed and implemented, one based on artificial intelligence and a second relying on image processing. The AI-powered algorithm uses neural network architecture trained on a predefined dataset of forms. The image processing algorithm employs edge detection and contour analysis techniques