Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Tasks

Large Language Models (LLMs) evaluation is a patchy and inconsistent landscape, and it is becoming clear that the quality of automatic evaluation metrics is not keeping up with the pace of development of generative models. We aim to improve the understanding of current models' performance by providing a preliminary and hybrid evaluation on a range of open and closed-source generative LLMs on three NLP benchmarks: text summarisation, text simplification and grammatical error correction (GEC), using both automatic and human evaluation. We also explore the potential of the recently released GPT-4 to act as an evaluator. We find that ChatGPT consistently outperforms many other popular models according to human reviewers on the majority of metrics, while scoring much more poorly when using classic automatic evaluation metrics. We also find that human reviewers rate the gold reference as much worse than the best models' outputs, indicating the poor quality of many popular benchmarks. Finally, we find that GPT-4 is capable of ranking models' outputs in a way which aligns reasonably closely to human judgement despite task-specific variations, with a lower alignment in the GEC task.Comment: Accepted at EMNLP 202

A Product-Form Model for the Performance Evaluation of a Bandwidth Allocation Strategy in WSNs

Wireless Sensor Networks (WSNs) are important examples of Collective Adaptive System, which consist of a set of motes that are spatially distributed in an indoor or outdoor space. Each mote monitors its surrounding conditions, such as humidity, intensity of light, temperature, and vibrations, but also collects complex information, such as images or small videos, and cooperates with the whole set of motes forming the WSN to allow the routing process. The traffic in the WSN consists of packets that contain the data harvested by the motes and can be classified according to the type of information that they carry. One pivotal problem in WSNs is the bandwidth allocation among the motes. The problem is known to be challenging due to the reduced computational capacity of the motes, their energy consumption constraints, and the fully decentralised network architecture. In this article, we study a novel algorithm to allocate the WSN bandwidth among the motes by taking into account the type of traffic they aim to send. Under the assumption of a mesh network and Poisson distributed harvested packets, we propose an analytical model for its performance evaluation that allows a designer to study the optimal configuration parameters. Although the Markov chain underlying the model is not reversible, we show it to be.-reversible under a certain renaming of states. By an extensive set of simulations, we show that the analytical model accurately approximates the performance of networks that do not satisfy the assumptions. The algorithm is studied with respect to the achieved throughput and fairness. We show that it provides a good approximation of the max-min fairness requirements

Sviluppo di un criterio energetico con effetto della tensione media per l'analisi sperimentale della vita a fatica di provini in acciaio inossidabile

Verifica del criterio energetico con analisi dell'effetto della tensiome media per la previsione sperimentale della vita a fatica di componenti realizzati in acciaio inossidabile (AISI 304L trafilato a freddo) proposto dal prof. Meneghetti. Segue la comparazione con il metodo classico proposto da woehle

Nutrition in the first 1000 days of life. A cross-sectional study on the perception and knowledge of both current and prospective mothers on Local Health Unit 2 - Veneto Region

openINTRODUZIONE: Il fattore nutrizionale durante i primi 1000 giorni di vita ricopre un ruolo di notevole rilevanza, sia perché la rapida crescita fisiologica e lo sviluppo funzionale del neonato richiedono un elevato fabbisogno nutritivo, sia perché una scorretta alimentazione materna e infantile, può interferire con le diverse fasi del processo di sviluppo del bambino, portando a conseguenze a breve e lungo termine per la sua salute e per quella della madre; e nella misura in cui riescono ad innescare mutazioni ereditarie, di quella delle future generazioni. Alla luce di questo, la conoscenza dei fattori di rischio e di quelli protettivi, devono rappresentare una priorità per tutte le mamme e future mamme. A tal proposito il presente studio ha lo scopo di indagare e valutare il grado di conoscenza del campione in merito all’importanza di assumere una corretta alimentazione durante questo periodo. MATERIALI E METODI: Lo studio ha previsto la somministrazione di un questionario autoredatto alle mamme che hanno partorito da 0-24 mesi e alle donne dal primo al terzo trimestre di gravidanza, presso gli ambulatori vaccinali pediatrici del Servizio Igiene e Sanità Pubblica e i Consultori Familiari dell’Azienda Sanitaria Ulss 2 – Marca Trevigiana. Si è analizzato il profilo socio-demografico del campione, le sue conoscenze generali sui primi 1000 giorni di vita, le sue conoscenze specifiche sulle varie fasi alimentari che compongono tale periodo e le fonti di accesso alle informazioni. RISULTATI E DISCUSSIONE: Sono state raccolte 118 risposte, di cui l’81% rappresentano le già mamme e il 19% le donne in gravidanza. L’indagine ha fatto emergere che la maggioranza del campione ha una buona percezione del livello di conoscenza generale sui primi 1000 giorni di vita, ma, nonostante ciò, la percentuale di donne che dichiarano di saperne nulla o poco è comunque elevata. Il 93% delle rispondenti sostiene che l’alimentazione della donna in gravidanza è un importante fattore in grado di influenzare la salute del bambino, però, alcune loro abitudini e affermazioni rilevate fanno intendere che non siano correttamente informate. D’altra parte, le fasi alimentari dell’allattamento e dell’alimentazione complementare sono caratterizzate in maggior misura da livelli di conoscenza elevati. Come principali fonti di informazioni sono indicati i Consultori Familiari, i ginecologi privati, i medici di base e/o pediatri, internet e libri e/o riviste, ma la maggioranza dichiara comunque di aver sentito parlare poco dei primi 1000 giorno di vita. CONCLUSIONI: Benché sono diverse le istituzioni coinvolte nella promozione di progetti a sostegno dell’importanza dei primi 1000 giorni di vita, a fronte dei risultati ottenuti dallo studio, si crede che non basti il solo coinvolgimento di più figure professionali, istituzioni, enti o associazioni per sensibilizzare l’importanza di questo periodo, bensì è opportuno che l’intera società raggiunga una consapevolezza tale da permettere a tutti i cittadini e in particolare ai genitori di diventare loro stessi promotori di scelte e stili di vita sani

Drammatico romanzesco. Vittorini tra autore e personaggio

This study focuses on Elio Vittorini’s work. It is based on a field of research defined as that of the relationship between theatre and novel. In order to explore this phenomenon in its complexity on the direction of the epicization of drama as much as the dramaturgisation of novel, we referred to the modal theory and to the oscillations between the two poles of writing labeled mimetic and diegetic by a long tradition which dates back to Plato. In this process of contamination of forms, a paradigmatic role has been noticed in the narrative work of Elio Vittorini, first of all in the predominant presence of the dialogical dimension and, secondly, in a exploration of form sensitive and attentive to the most innovative international models wich was conducted outside a rigid gender framework. In addition, the autograph documents testify to the existence of a dramatic production that, although ephemeral and inconsistent, feeds in the depth the novels’ mechanism of composition. In particular, we analyzed this approach in the composition of Les Hommes et les Autres, inspired by the unfinished drama Atto primo which, since it does not go beyond the draft stage, reveals the theatrical models of experimentation and helps in the interpretation of the novel. Finally, after assessing the mimetic dimension of the work, we determined its success on stage. The alleged theatricality of the novel has been identified above all in relation to contemporary theatrical experiences that are based precisely on the tension between mimetic and diegetic modes.Cette étude porte sur la production littéraire d’Elio Vittorini. Elle s’appuie sur un domaine d’analyse défini comme celui du rapport entre théâtre et roman. Afin de mettre en valeur toutes les manifestations de ce complexe phénomène de proximité et de superposition partielle, dans le sens de l’épicisation du drame autant que de la dramaturgisation du roman, nous avons fait référence à la théorie modale et aux oscillations entre les deux pôles d’écriture auxquels une longue tradition, remontant à Platon, a assigné l’étiquette de mimétique et diégétique. Nous avons cru entrevoir un rôle paradigmatique dans ce processus de contamination des formes dans l’œuvre narrative d’Elio Vittorini, d’abord dans la présence prépondérante de la dimension dialogique et dans une recherche formelle sensible et attentive aux modèles internationaux les plus innovants, menée en dehors d'un cadre rigide de genre. De plus, les documents autographes témoignent l’existence d’une production dramatique que, bien qu’éphémère et inconsistante, alimente en profondeur la production narrative. En particulier, nous avons analysé cette démarche dans la composition de Les Hommes et les Autres, inspirée par le drame inachevé Atto primo qui, puisqu’il ne dépasse pas le stade d’ébauche, dévoile les modèles théâtraux de l’expérimentation et aide à l'interprétation du roman. Enfin, après avoir évalué la dimension mimétique de l’ œuvre, on a déterminé son succès scénique. La prétendue théâtralité du roman a été identifiée surtout par rapport aux expériences théâtrales contemporaines qui reposent précisément sur la tension entre mimétique et diégétique

Drammatico romanzesco. Vittorini tra autore e personaggio

Cette étude porte sur la production littéraire d’Elio Vittorini. Elle s’appuie sur un domaine d’analyse défini comme celui du rapport entre théâtre et roman. Afin de mettre en valeur toutes les manifestations de ce complexe phénomène de proximité et de superposition partielle, dans le sens de l’épicisation du drame autant que de la dramaturgisation du roman, nous avons fait référence à la théorie modale et aux oscillations entre les deux pôles d’écriture auxquels une longue tradition, remontant à Platon, a assigné l’étiquette de mimétique et diégétique. Nous avons cru entrevoir un rôle paradigmatique dans ce processus de contamination des formes dans l’œuvre narrative d’Elio Vittorini, d’abord dans la présence prépondérante de la dimension dialogique et dans une recherche formelle sensible et attentive aux modèles internationaux les plus innovants, menée en dehors d'un cadre rigide de genre. De plus, les documents autographes témoignent l’existence d’une production dramatique que, bien qu’éphémère et inconsistante, alimente en profondeur la production narrative. En particulier, nous avons analysé cette démarche dans la composition de Les Hommes et les Autres, inspirée par le drame inachevé Atto primo qui, puisqu’il ne dépasse pas le stade d’ébauche, dévoile les modèles théâtraux de l’expérimentation et aide à l'interprétation du roman. Enfin, après avoir évalué la dimension mimétique de l’ œuvre, on a déterminé son succès scénique. La prétendue théâtralité du roman a été identifiée surtout par rapport aux expériences théâtrales contemporaines qui reposent précisément sur la tension entre mimétique et diégétique.This study focuses on Elio Vittorini’s work. It is based on a field of research defined as that of the relationship between theatre and novel. In order to explore this phenomenon in its complexity on the direction of the epicization of drama as much as the dramaturgisation of novel, we referred to the modal theory and to the oscillations between the two poles of writing labeled mimetic and diegetic by a long tradition which dates back to Plato. In this process of contamination of forms, a paradigmatic role has been noticed in the narrative work of Elio Vittorini, first of all in the predominant presence of the dialogical dimension and, secondly, in a exploration of form sensitive and attentive to the most innovative international models wich was conducted outside a rigid gender framework. In addition, the autograph documents testify to the existence of a dramatic production that, although ephemeral and inconsistent, feeds in the depth the novels’ mechanism of composition. In particular, we analyzed this approach in the composition of Les Hommes et les Autres, inspired by the unfinished drama Atto primo which, since it does not go beyond the draft stage, reveals the theatrical models of experimentation and helps in the interpretation of the novel. Finally, after assessing the mimetic dimension of the work, we determined its success on stage. The alleged theatricality of the novel has been identified above all in relation to contemporary theatrical experiences that are based precisely on the tension between mimetic and diegetic modes