Overfreezing Meets Overparameterization: A Double Descent Perspective on Transfer Learning of Deep Neural Networks

We study the generalization behavior of transfer learning of deep neural networks (DNNs). We adopt the overparameterization perspective -- featuring interpolation of the training data (i.e., approximately zero train error) and the double descent phenomenon -- to explain the delicate effect of the transfer learning setting on generalization performance. We study how the generalization behavior of transfer learning is affected by the dataset size in the source and target tasks, the number of transferred layers that are kept frozen in the target DNN training, and the similarity between the source and target tasks. We show that the test error evolution during the target DNN training has a more significant double descent effect when the target training dataset is sufficiently large with some label noise. In addition, a larger source training dataset can delay the arrival to interpolation and double descent peak in the target DNN training. Moreover, we demonstrate that the number of frozen layers can determine whether the transfer learning is effectively underparameterized or overparameterized and, in turn, this may affect the relative success or failure of learning. Specifically, we show that too many frozen layers may make a transfer from a less related source task better or on par with a transfer from a more related source task; we call this case overfreezing. We establish our results using image classification experiments with the residual network (ResNet) and vision transformer (ViT) architectures

Cisplatin, 5-fluorouracil and interferon alpha 2b for recurrent or metastatic head and neck cancer.

On the basis of preclinical data suggesting the possibility of maximising the efficacy of 5-fluorouracil and cisplatin by interferon, a pilot clinical trial was initiated in recurrent and/or metastatic head and neck cancer. Thirty-four patients were treated with cisplatin at 100 mg m-2, followed by 5-fluorouracil at 1,000 mg m-2 by continuous infusion for 5 days. Interferon alpha 2b was administered at the dose of 3 million U i.m. daily for 7 days, beginning the day before chemotherapy. Courses were repeated every 3 weeks. Two patients achieved a complete remission, six a partial response, 14 had stable disease and 12 progressed on therapy, for an overall response rate of 23% (95% confidence interval 10-36%). Median survival time was 5 months. Toxicity was severe. Stomatitis, diarrhoea and myelosuppression were the most common side-effects. Because of the poor response rate and the presence of severe toxicity, in our opinion further clinical trials in head and neck cancer should be attempted only after a better definition in preclinical studies of interactions among 5-fluorouracil, cisplatin and interferon

The Mw 6.3, 2009 L’Aquila earthquake: source, path and site effects from spectral analysis of strong motion data

The strong motion data of 2009 April 6 L’Aquila (Central Italy) earthquake (Mw = 6.3) and of 12 aftershocks (4.1 ≤ Mw ≤ 5.6) recorded by 56 stations of the Italian strong motion network are spectrally analysed to estimate the source parameters, the seismic attenuation, and the site amplification effects. The obtained source spectra for S wave have stress drop values ranging from 2.4 to 16.8 MPa, being the stress drop of the main shock equal to 9.2 MPa. The spectral curves describing the attenuation with distance show the presence of shoulders and bumps, mainly around 50 and 150 km, as consequence of significant reflected and refracted arrivals from crustal interfaces. The attenuation in the first 50 km is well described by a quality factor equal to Q( f ) = 59 f 0.56 obtained by fixing the geometrical spreading exponent to 1. Finally, the horizontal-to-vertical spectral ratio provides unreliable estimates of local site effects for those stations showing large amplifications over the vertical component of motion

CARPET: a web-based package for the analysis of ChIP-chip and expression tiling data

Summary: CARPET (Collection of Automated Routine Programs for Easy Tiling) is a set of Perl, Python and R scripts, integrated on the Galaxy2 web-based platform, for the analysis of ChIP-chip and expression tiling data, both for standard and custom chip designs. CARPET allows rapid experimental data entry, simple quality control, normalization, easy identification and annotation of enriched ChIP-chip regions, detection of the absolute or relative transcriptional status of genes assessed by expression tiling experiments and, more importantly, it allows the integration of ChIP-chip and expression data. Results can be visualized instantly in a genomic context within the UCSC genome browser as graph-based custom tracks through Galaxy2. All generated and uploaded data can be stored within sessions and are easily shared with other users. Availability: http://bio.ifom-ieo-campus.it/galaxy Contacts: [email protected] lucilla.luzi@if om-ieo-campus.i

Comparison between empirical predictive equations calibrated at global and national scale and the Italian strong-motion data

In Italy in the last years many ground motion prediction equations (hereinafter GMPEs) were calibrated both at national and regional scale using weak and strong motion data recorded in the last 30 years by several networks. Moreover many of the Italian strongest earthquakes were included in global datasets in order to calibrate GMPEs suitable to predict ground-motion at very large scale. In the last decade the Sabetta and Pugliese (1996) relationships represented a reference for the ground motion predictions in Italy. At present all Italian strong-motion data, recorded from 1972 by RAN (Italian Accelerometric Network), and more recently by other regional networks (e.g. RAIS, Strong motion network of Northern Italy), are collected in ITACA (ITalian ACcelerometric Archive). Considering Italian strong-motion data with Mw  4.0 and distance (Joyner-Boore or epicentral) up to 100 km, new GMPEs were developed by Bindi et al. (2009), aimed at replacing the older Italian relationships. The occurrence of the recent 23rd December 2008, Mw 5.4, Parma (Northern Italy) earthquake and the 6th April 2009, Mw 6.3, L’Aquila earthquake, allowed to upgrade the ITACA data set and gave us the possibility to validate the predictive capability of many GMPEs, developed using Italian, European and global data sets. The results are presented in terms of quality of performance (fit between recorded and predicted values) using the maximum likelihood approach as explained in Spudich et al. (1999). Considering the strong-motion data recorded during the L’Aquila sequence the considered GMPEs, in average, overestimate the observed data, showing a dependence of the residuals with distance in particular at higher frequencies. An improvement of fit is obtained comparing all Italian strong-motion data included in ITACA with the European GMPEs calibrated by Akkar and Bommer (2007 a,b) and the global models calibrated by Cauzzi and Faccioli (2008). In contrast, Italian data seem to attenuate faster than the NGA models calibrated by Boore and Atkinson (2008), in particular at higher frequencies

Damage distribution and seismological model of the November 24, 2004, Salo' (Northern Italy) earthquake

The West side of lake of Garda, in Northern Italy, was struck by a ML=5.2 earthquake on November 24, 2004. The felt area is rather large (from Venice to Milan) and the damaged area consists of 66 municipalities, with a number of homeless of about 2200 and estimated direct damages of 215 millions of euros. Most of the damaged structures are old masonry buildings and churches, while there were almost no damage to reinforced concrete structures. The observed distribution of macroseismic intensity shows a strong azimuthal dependence, with high intensity level in a 10x10 km2 area located SW to the epicentre and rather large dispersion of values (ranging from V to VII-VIII) in the first 10 km epicentral distance. Taking into account the vulnerability level of the damaged structures and the features of the geological formations, we tried to explain the observed damage distribution in terms of finite fault properties of the source, despite the moderate magnitude of the event. Thus we hypothesised a fault geometry from seismotectonic considerations and we simulated the event by a high frequency simulation technique (Deterministic Stochastic Method, DSM). The synthetic ground motion parameters were converted into intensity values by empirical relationships and local geological conditions were considered to explain some discrepancies between simulated and observed intensities. It was possible to adequately reproduce both the observed distribution of macroseismic intensity and the ground motion recorded by an accelerometric station located at about 13 km epicentral distance

Ground Motion Prediction Equations Derived from the Italian Strong Motion Database

We present a set of ground motion prediction equations (GMPEs) derived for the geometrical mean of the horizontal components and the vertical, considering the latest release of the strong motion database for Italy. The regressions are performed over the magnitude range 4–6.9 and considering distances up to 200 km. The equations are derived for peak ground acceleration (PGA), peak ground velocity (PGV) and 5%-damped spectral acceleration at periods between 0.04 and 2 s. The total standard deviation (sigma) varies between 0.34 and 0.38 log10 unit, confirming the large variability of ground shaking parameters when regional data sets containing small to moderate magnitude events (M < 6) are used. The between-stations variability provides the largest values for periods shorter than 0.2 s while, for longer periods, the between-events and between-stations distributions of error provide similar contribution to the total variabilit

What can we learn from the January 2012 Northern Italy earthquakes?

This note focuses on the ground motion recorded during the recent moderate earthquakes occurred in the central part of Northern Italy (panel 1), a region characterized by low seismicity. For this area the Italian seismic hazard map (Stucchi et al., 2011) assigns a maximum horizontal acceleration (rock site) up to 0.2 g (10% probability of exceedance in 50 yrs). In the last 4 years, the region was struck by 9 earthquakes in the magnitude range 4≤Mw≤5.0, with the three largest located in the Northern Apennines (Mw 4.9 and 5.0 Parma events, December 2008 and January 2012) and in the Po plain (Mw 4.9 Reggio Emila event of January 2012). We analyze the strongmotion data (distance < 300 km) from these events recorded by stations belonging to the INGV (RAIS, http://rais.mi.ingv.it; RSNC http://iside.rm.ingv.it) and DPC (RAN, www.protezionecivile.it; http://itaca.mi.ingv.it). The 2008 and 2012 Parma events, both characterized by reverse focal mechanisms (http://cnt.rm.ingv.it/), have depths of 27 and 60 km respectively. The deep event produced a maximum peak ground acceleration (PGA) of 97 cm/s2 at Novellara (NVL, EC8 C class) station (70 km from the epicenter). The 25th January 2012 event (depth of 34 km) produced a maximum PGA of 114 cm/s2 at Sorbolo (SRP) station (7 km from the epicenter). Preliminary analyses show: 1) a peculiar ground-motion attenuation of the deep Parma event with respect to the shallow one. In panel 2, the PGAs for the two Parma events are plotted as a function of hypocentral distance and compared to the global ground motion prediction equation (GMPE) calibrated by Cauzzi and Faccioli (2008) using events with depth < 30 km. The different distance-decay of PGA for the deep event (blue for A class of EC8 and red for B and C classes, CEN 2003) is evident, in particular for distance up to 100 km. On the other hand, the PGAs of the 2008 Parma crustal event (grey) are well explained by this GMPE. In panel 3, the PGAs for the deep 2012 event, grouped for EC8 classes, are compared to the national GMPE calibrated by Bindi et al. (2011) using crustal events and epicentral distance. Also in this case, the GMPE underestimates the PGAs up to 200 km. Although most of the class C sites (red) show the largest PGAs, the underestimation cannot be completely ascribed to site effects. The large PGAs from the Parma deep event, with respect to the shallow one, could be explained in terms of source effects (e.g. large stress drop values enhancing the high-frequency radiation). In addition, as explained by Castro et al. (2008), the different attenuation in the lower and upper crust could explain the large PGAs recorded for the 2012 deep event. 2) seismic amplification at Po Plain sites: In panel 4, the PGAs of the January 25th, Mw 4.9, Reggio Emilia event are plotted as a function of the epicentral distance, together with the Bindi et al. (2011) GMPE. In general, the largest amplitudes occur at the Po plain sites (red), suggesting possible peculiar site response. An overall increase of the PGAs is observed around 100km, in agreement with the results of Bragato et al. (2011) that studied the regional influence of Moho S-wave reflections in the area. An example of site response is shown in panel 5, considering TREG (class C) and ZEN8 (class A) stations (panel 5a), located at 88 km from the Reggio Emila epicentre. The rotational standard spectral ratio (panel 5b) for 10 s of S wave shows polarized amplifications around 2 Hz, detected also at others Po plain sites (not reported), as well as amplification on the vertical component. The points discussed above should to be interpreted as a warning for future applications dealing with ground motion estimation in the aftermath of an earthquake in this area (e.g. ShakeMap calculation): currently used GMPEs, based on different events and sites characteristics could lead to significant bias in the final results

Qui INGV

L’INGV, a partire dal 2006, ha iniziato una fase di potenziamento del monitoraggio accelerometrico, installando nelle aree centrali della pianura padana 22 sensori strong-motion (Rete Accelerometrica Italia Settentrionale, RAIS, http://rais.mi.ingv.it/). Dal 2008, sensori accelerometrici sono stati via via installati in 105 siti a Rete Sismica Nazionale (RSN), gestita dal Centro Nazionale Terremoti (CNT). Nel complesso le 127 stazioni accelerometriche presenti sul territorio nazionale costituiscono a tutti gli effetti la rete accelerometrica nazionale INGV. I dati acquisiti da tutte le stazioni accelerometriche sono attualmente distribuiti in tempo reale tramite il portale EIDA (European Integrated Data Archive; http://eida.rm.ingv.it/) e sono principalmente utilizzati per il calcolo delle Shakemaps a scala nazionale. Attualmente, l’INGV sta realizzando un portale web per la distribuzione dei dati accelerometrici registrati dalle stazioni INGV, composto da 2 moduli distinti: il primo, denominato ISMD, ha lo scopo di archiviaziare e distribuire in tempo quasi reale (poche ore dopo l’evento) le forme d’onda accelerometriche in formato non corretto ed i relativi metadati ottenuti a seguito di una procedura di processamento automatico; il secondo, denominato DYNA, è una banca dati relazionale, contenente le forme d’onda di accelerazione, velocità e spostamento e gli spettri di risposta di accelerazione, ottenuti attraverso il processamento manuale dei segnali non corretti, oltre ai relativi metadati associati agli eventi sismici ed alle stazioni di registrazione Il prototipo del portale dei dati accelerometrici INGV (Figura 1) è stato pubblicato lo scorso maggio, a seguito della sequenza sismica Emiliana