Identifying the Machine Learning Family from Black-Box Models

[EN] We address the novel question of determining which kind of machine learning model is behind the predictions when we interact with a black-box model. This may allow us to identify families of techniques whose models exhibit similar vulnerabilities and strengths. In our method, we first consider how an adversary can systematically query a given black-box model (oracle) to label an artificially-generated dataset. This labelled dataset is then used for training different surrogate models (each one trying to imitate the oracle¿s behaviour). The method has two different approaches. First, we assume that the family of the surrogate model that achieves the maximum Kappa metric against the oracle labels corresponds to the family of the oracle model. The other approach, based on machine learning, consists in learning a meta-model that is able to predict the model family of a new black-box model. We compare these two approaches experimentally, giving us insight about how explanatory and predictable our concept of family is.This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0287, the EU (FEDER), and the Spanish MINECO under grant TIN 2015-69175-C4-1-R, the Generalitat Valenciana PROMETEOII/2015/013. F. Martinez-Plumed was also supported by INCIBE under grant INCIBEI-2015-27345 (Ayudas para la excelencia de los equipos de investigacion avanzada en ciberseguridad). J. H-Orallo also received a Salvador de Madariaga grant (PRX17/00467) from the Spanish MECD for a research stay at the CFI, Cambridge, and a BEST grant (BEST/2017/045) from the GVA for another research stay at the CFI.Fabra-Boluda, R.; Ferri Ramírez, C.; Hernández-Orallo, J.; Martínez-Plumed, F.; Ramírez Quintana, MJ. (2018). Identifying the Machine Learning Family from Black-Box Models. Lecture Notes in Computer Science. 11160:55-65. https://doi.org/10.1007/978-3-030-00374-6_6S556511160Angluin, D.: Queries and concept learning. Mach. Learn. 2(4), 319–342 (1988)Benedek, G.M., Itai, A.: Learnability with respect to fixed distributions. Theor. Comput. Sci. 86(2), 377–389 (1991)Biggio, B., et al.: Security Evaluation of support vector machines in adversarial environments. In: Ma, Y., Guo, G. (eds.) Support Vector Machines Applications, pp. 105–153. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-02300-7_4Blanco-Vega, R., Hernández-Orallo, J., Ramírez-Quintana, M.J.: Analysing the trade-off between comprehensibility and accuracy in mimetic models. In: Suzuki, E., Arikawa, S. (eds.) DS 2004. LNCS (LNAI), vol. 3245, pp. 338–346. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30214-8_29Dalvi, N., Domingos, P., Sanghai, S., Verma, D., et al.: Adversarial classification. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 99–108. ACM (2004)Dheeru, D., Karra Taniskidou, E.: UCI machine learning repository (2017). http://archive.ics.uci.edu/mlDomingos, P.: Knowledge discovery via multiple models. Intell. Data Anal. 2(3), 187–202 (1998)Duin, R.P.W., Loog, M., Pȩkalska, E., Tax, D.M.J.: Feature-based dissimilarity space classification. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 46–55. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17711-8_5Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems. J. Mach. Learn. Res. 15(1), 3133–3181 (2014)Ferri, C., Hernández-Orallo, J., Modroiu, R.: An experimental comparison of performance measures for classification. Pattern Recognit. Lett. 30(1), 27–38 (2009)Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9(1), 69–82 (2008)Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.: Adversarial machine learning. In: Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, pp. 43–58 (2011)Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)Landis, J.R., Koch, G.G.: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics 33, 363–374 (1977)Lowd, D., Meek, C.: Adversarial learning. In: Proceedings of the 11th ACM SIGKDD International Conference on Knowledge Discovery in Data mining, pp. 641–647. ACM (2005)Martınez-Plumed, F., Prudêncio, R.B., Martınez-Usó, A., Hernández-Orallo, J.: Making sense of item response theory in machine learning. In: Proceedings of 22nd European Conference on Artificial Intelligence (ECAI). Frontiers in Artificial Intelligence and Applications, vol. 285, pp. 1140–1148 (2016)Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016)Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 372–387. IEEE (2016)Papernot, N., McDaniel, P., Wu, X., Jha, S., Swami, A.: Distillation as a defense to adversarial perturbations against deep neural networks. In: 2016 IEEE Symposium on Security and Privacy (SP), pp. 582–597. IEEE (2016)Sesmero, M.P., Ledezma, A.I., Sanchis, A.: Generating ensembles of heterogeneous classifiers using stacked generalization. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5(1), 21–34 (2015)Smith, M.R., Martinez, T., Giraud-Carrier, C.: An instance level analysis of data complexity. Mach. Learn. 95(2), 225–256 (2014)Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction APIs. In: USENIX Security Symposium, pp. 601–618 (2016)Valiant, L.G.: A theory of the learnable. Commun. ACM 27(11), 1134–1142 (1984)Wallace, C.S., Boulton, D.M.: An information measure for classification. Comput. J. 11(2), 185–194 (1968)Wolpert, D.H.: Stacked generalization. Neural Netw. 5(2), 241–259 (1992

How Do Hospital Pharmacists Approach Substitution of Nanomedicines? Insights from a Qualitative Pilot Study and a Quantitative Market Research Analysis in Five European Countries

We conducted research to assess hospital pharmacists’ familiarity with/interpretation of data requirements for the different regulatory approval frameworks and the impact of this on their approach to substitution in the formulary. The online questionnaire included a small molecule (acetylsalicylic acid—follow-ons approved via the generic pathway), two biologic drugs (insulin glargine and etanercept—follow-ons approved via the biosimilar pathway), a non-biologic complex drug (NBCD; glatiramer acetate—follow-ons approved via the hybrid pathway) and a nanomedicine, ferric carboxymaltose (no follow-ons approved as yet). The study was conducted in two phases: an initial qualitative pilot study with 30 participants, followed by a quantitative stage involving 201 pharmacists from five European countries. Most expected negligible safety/efficacy differences between reference and follow-on products. Head-to-head clinical data showing therapeutic equivalence as a prerequisite for reference product/follow-on substitution was perceived to be needed most for biologics (47%), followed by NBCDs (44%)/nanomedicines (39%) and small molecules (23%). Overall, 28% did not know the data requirements for follow-on approval via the hybrid pathway; 16% were familiar with this pathway, compared with 50% and 55% for the generic and biosimilar pathways, respectively. Overall, 19% of respondents thought the European Medicines Agency (EMA) was responsible for defining the substitutability of follow-ons. Education is required to increase hospital pharmacist’s knowledge of regulatory approval frameworks and their relevance to substitution practices

The Urological Association of Asia clinical guideline for urinary stone disease

The Urological Association of Asia, consisting of 25 member associations and one affiliated member since its foundation in 1990, has planned to develop Asian guidelines for all urological fields. The field of stone diseases is the third of its guideline projects. Because of the different climates, and social, economic and ethnic environments, the clinical practice for urinary stone diseases widely varies among the Asian countries. The committee members of the Urological Association of Asia on the clinical guidelines for urinary stone disease carried out a surveillance study to better understand the diversity of the treatment strategy among different regions and subsequent systematic literature review through PubMed and MEDLINE database between 1966 and 2017. Levels of evidence and grades of recommendation for each management were decided according to the relevant strategy. Each clinical question and answer were thoroughly reviewed and discussed by all committee members and their colleagues, with suggestions from expert representatives of the American Urological Association and European Association of Urology. However, we focused on the pragmatic care of patients and our own evidence throughout Asia, which included recent surgical trends, such as miniaturized percutaneous nephrolithotomy and endoscopic combined intrarenal surgery. This guideline covers all fields of stone diseases, from etiology to recurrence prevention. Here, we present a short summary of the first version of the guideline – consisting 43 clinical questions – and overview its key practical issues

Anti-tumour necrosis factor discontinuation in inflammatory bowel disease patients in remission: study protocol of a prospective, multicentre, randomized clinical trial

Background: Patients with inflammatory bowel disease who achieve remission with anti-tumour necrosis factor (anti-TNF) drugs may have treatment withdrawn due to safety concerns and cost considerations, but there is a lack of prospective, controlled data investigating this strategy. The primary study aim is to compare the rates of clinical remission at 1?year in patients who discontinue anti-TNF treatment versus those who continue treatment. Methods: This is an ongoing, prospective, double-blind, multicentre, randomized, placebo-controlled study in patients with Crohn?s disease or ulcerative colitis who have achieved clinical remission for ?6?months with an anti-TNF treatment and an immunosuppressant. Patients are being randomized 1:1 to discontinue anti-TNF therapy or continue therapy. Randomization stratifies patients by the type of inflammatory bowel disease and drug (infliximab versus adalimumab) at study inclusion. The primary endpoint of the study is sustained clinical remission at 1?year. Other endpoints include endoscopic and radiological activity, patient-reported outcomes (quality of life, work productivity), safety and predictive factors for relapse. The required sample size is 194 patients. In addition to the main analysis (discontinuation versus continuation), subanalyses will include stratification by type of inflammatory bowel disease, phenotype and previous treatment. Biological samples will be obtained to identify factors predictive of relapse after treatment withdrawal. Results: Enrolment began in 2016, and the study is expected to end in 2020. Conclusions: This study will contribute prospective, controlled data on outcomes and predictors of relapse in patients with inflammatory bowel disease after withdrawal of anti-TNF agents following achievement of clinical remission. Clinical trial reference number: EudraCT 2015-001410-1

Construction of Critical Periods for Water Resources Management and Their Application in the FEW Nexus

Amidst the growing population, urbanization, globalization, and economic growth, along with the impacts of climate change, decision-makers, stakeholders, and researchers need tools for better assessment and communication of the highly interconnected food–energy–water (FEW) nexus. This study aimed to identify critical periods for water resources management for robust decision-making for water resources management at the nexus. Using a 4610 ha agricultural watershed as a pilot site, historical data (2006–2012), scientific literature values, and SWAT model simulations were utilized to map out critical periods throughout the growing season of corn and soybeans. The results indicate that soil water deficits are primarily seen in June and July, with average deficits and surpluses ranging from −134.7 to +145.3 mm during the study period. Corresponding water quality impacts include average monthly surface nitrate-N, subsurface nitrate-N, and soluble phosphorus losses of up to 0.026, 0.26, and 0.0013 kg/ha, respectively, over the growing season. Estimated fuel requirements for the agricultural practices ranged from 24.7 to 170.3 L/ha, while estimated carbon emissions ranged from 0.3 to 2.7 kg CO2/L. A composite look at all the FEW nexus elements showed that critical periods for water management in the study watershed occurred in the early and late season—primarily related to water quality—and mid-season, related to water quantity. This suggests the need to adapt agricultural and other management practices across the growing season in line with the respective water management needs. The FEW nexus assessment methodologies developed in this study provide a framework in which spatial, temporal, and literature data can be implemented for improved water resources management in other areas

Construction of Critical Periods for Water Resources Management and Their Application in the FEW Nexus

Amidst the growing population, urbanization, globalization, and economic growth, along with the impacts of climate change, decision-makers, stakeholders, and researchers need tools for better assessment and communication of the highly interconnected food–energy–water (FEW) nexus. This study aimed to identify critical periods for water resources management for robust decision-making for water resources management at the nexus. Using a 4610 ha agricultural watershed as a pilot site, historical data (2006–2012), scientific literature values, and SWAT model simulations were utilized to map out critical periods throughout the growing season of corn and soybeans. The results indicate that soil water deficits are primarily seen in June and July, with average deficits and surpluses ranging from −134.7 to +145.3 mm during the study period. Corresponding water quality impacts include average monthly surface nitrate-N, subsurface nitrate-N, and soluble phosphorus losses of up to 0.026, 0.26, and 0.0013 kg/ha, respectively, over the growing season. Estimated fuel requirements for the agricultural practices ranged from 24.7 to 170.3 L/ha, while estimated carbon emissions ranged from 0.3 to 2.7 kg CO2/L. A composite look at all the FEW nexus elements showed that critical periods for water management in the study watershed occurred in the early and late season—primarily related to water quality—and mid-season, related to water quantity. This suggests the need to adapt agricultural and other management practices across the growing season in line with the respective water management needs. The FEW nexus assessment methodologies developed in this study provide a framework in which spatial, temporal, and literature data can be implemented for improved water resources management in other areas