Assessing Operational Accuracy of CNN-based Image Classifiers using an Oracle Surrogate

Context Assessing the accuracy in operation of a Machine Learning (ML) system for image classification on arbitrary (unlabeled) inputs is hard. This is due to the oracle problem, which impacts the ability of automatically judging the output of the classification, thus hindering the accuracy of the assessment when unlabeled previously unseen inputs are submitted to the system. Objective We propose the Image Classification Oracle Surrogate (ICOS), a technique to automatically evaluate the accuracy in operation of image classifiers based on Convolutional Neural Networks (CNNs). Method To establish whether the classification of an arbitrary image is correct or not, ICOS leverages three knowledge sources: operational input data, training data, and the ML algorithm. Knowledge is expressed through likely invariants - properties which should not be violated by correct classifications. ICOS infers and filters invariants to improve the correct detection of misclassifications, reducing the number of false positives. We evaluate ICOS experimentally on twelve CNNs – using the popular MNIST, CIFAR10, CIFAR100, and ImageNet datasets. We compare it to two alternative strategies, namely cross-referencing and self-checking. Results Experimental results show that ICOS exhibits performance comparable to the other strategies in terms of accuracy, showing higher stability over a variety of CNNs and datasets with different complexity and size. Conclusions ICOS likely invariants are shown to be effective in automatically detecting misclassifications by CNNs used in image classification tasks when the expected output is unknown; ICOS ultimately yields faithful assessments of their accuracy in operation. Knowledge about input data can also be manually incorporated into ICOS, to increase robustness against unexpected phenomena in operation, like label shift

Monitoring tools for DevOps and microservices: A systematic grey literature review

Microservice-based systems are usually developed according to agile practices like DevOps, which enables rapid and frequent releases to promptly react and adapt to changes. Monitoring is a key enabler for these systems, as they allow to continuously get feedback from the field and support timely and tailored decisions for a quality-driven evolution. In the realm of monitoring tools available for microservices in the DevOps-driven development practice, each with different features, assumptions, and performance, selecting a suitable tool is an as much difficult as impactful task. This article presents the results of a systematic study of the grey literature we performed to identify, classify and analyze the available monitoring tools for DevOps and microservices. We selected and examined a list of 71 monitoring tools, drawing a map of their characteristics, limitations, assumptions, and open challenges, meant to be useful to both researchers and practitioners working in this area. Results are publicly available and replicable

Triplet vs doublet lenalidomide-containing regimens for the treatment of elderly patients with newly diagnosed multiple myeloma

Lenalidomide-dexamethasone improved outcome in newly diagnosed elderly multiple myeloma patients. We randomly assigned 662 patients who were age \u202165 years or transplantation-ineligible to receive induction with melphalan-prednisone-lenalidomide (MPR) or cyclophosphamide-prednisone-lenalidomide (CPR) or lenalidomide plus lowdose dexamethasone (Rd). The primary end point was progression-free survival (PFS) in triplet (MPR and CPR) vs doublet (Rd) lenalidomide-containing regimens. After a median follow-up of 39 months, the medianPFSwas22 months for the triplet combinations and 21 months for the doublet (P 5 .284). The median overall survival (OS) was not reached in either arms, and the 4-year OS was 67% for the triplet and 58% for the doublet arms (P 5 .709). By considering the 3 treatment arms separately, no difference in outcome was detected among MPR, CPR, and Rd. The most common grade \u20213 toxicity was neutropenia: 64% in MPR, 29% in CPR, and 25% in Rd patients (P < .0001). Grade \u20213 nonhematologic toxicities were similar among arms and were mainly infections (6.5% to 11%), constitutional (3.5% to 9.5%), and cardiac (4.5% to 6%), with no difference among the arms. In conclusion, in the overall population, the alkylator-containing tripletsMPRandCPRwere not superior to the alkylator-free doublet Rd, which was associated with lower toxicit

EARLY METABOLIC RESPONSE IN FOLLICULAR LYMPHOMA: A SUBSET ANALYSIS OF THE FOLL12 TRIAL BY THE FONDAZIONE ITALIANA LINFOMI (FIL)

n/

Brentuximab vedotin consolidation after autologous stem cell transplantation for Hodgkin lymphoma: A Fondazione Italiana Linfomi real-life experience

The standard management for relapsed or refractory classical Hodgkin lymphoma (cHL) is salvage therapy followed by autologous stem cell transplantation (ASCT). This strategy allows almost 50% of patients to be cured. Post-ASCT maintenance treatment with brentuximab vedotin (BV) confers improved progression-free survival (PFS) to cHL patients at high risk of relapse. We investigated the outcome of 105 cHL patients receiving post-ASCT BV maintenance in the real-life setting of 23 Italian hematology centers. This population included naïve patients and those previously exposed to BV. Median follow-up was 20 months. Patients presented a median of two lines of treatment pre-ASCT, with 51% receiving BV. Twenty-nine percent of patients had at least two high-risk factors (refractory disease, complete response [CR] less than 12 months, extranodal disease at relapse), while 16% presented none. At PET-CT, a Deauville score (DS) of 1–3 was reported in 75% and 78% of pre- and post-ASCT evaluations, respectively. Grade 3–4 adverse events (AEs), mainly peripheral neuropathy, were observed in 16% of patients. Three-year PFS and overall survival (OS) were 62% and 86%, respectively. According to BV exposure, 3-year PFS and OS were 54% and 71%, respectively, for naïve and 77% and 96%, respectively, for previously exposed patients. Refractory disease (hazard ratio [HR] 4.46; p&nbsp;=&nbsp;0.003) and post-ASCT DS 4–5 (HR 3.14; p&nbsp;=&nbsp;0.005) were the only two factors significantly associated with PFS reduction in multivariable analysis. Post-ASCT BV maintenance is an effective, safe treatment option for cHL naïve patients and those previously exposed to BV

CPX-351 treatment in secondary acute myeloblastic leukemia is effective and improves the feasibility of allogeneic stem cell transplantation: results of the Italian compassionate use program

Secondary acute myeloid leukemia (sAML) poorly responds to conventional treatments and allogeneic stem cell transplantation (HSCT). We evaluated toxicity and efficacy of CPX-351 in 71 elderly patients (median age 66 years) with sAML enrolled in the Italian Named (Compassionate) Use Program. Sixty days treatment-related mortality was 7% (5/71). The response rate at the end of treatment was: CR/CRi in 50/71 patients (70.4%), PR in 6/71 (8.5%), and NR in 10/71 (19.7%). After a median follow-up of 11 months relapse was observed in 10/50 patients (20%) and 12 months cumulative incidence of relapse (CIR) was 23.6%. Median duration of response was not reached. In competing risk analysis, CIR was reduced when HSCT was performed in first CR (12 months CIR of 5% and 37.4%, respectively, for patients receiving (=20) or not (=30) HSCT, p = 0.012). Twelve-months OS was 68.6% (median not reached). In landmark analysis, HSCT in CR1 was the only significant predictor of longer survival (12 months OS of 100 and 70.5%, for patients undergoing or not HSCT in CR1, respectively, p = 0.011). In conclusion, we extend to a real-life setting, the notion that CPX is an effective regimen for high risk AML patients and may improve the results of HSCT