Learning from microarray interlaboratory studies: measures of precision for gene expression

<p>Abstract</p> <p>Background</p> <p>The ability to demonstrate the reproducibility of gene expression microarray results is a critical consideration for the use of microarray technology in clinical applications. While studies have asserted that microarray data can be "highly reproducible" under given conditions, there is little ability to quantitatively compare amongst the various metrics and terminology used to characterize and express measurement performance. Use of standardized conceptual tools can greatly facilitate communication among the user, developer, and regulator stakeholders of the microarray community. While shaped by less highly multiplexed systems, measurement science (metrology) is devoted to establishing a coherent and internationally recognized vocabulary and quantitative practice for the characterization of measurement processes.</p> <p>Results</p> <p>The two independent aspects of the metrological concept of "accuracy" are "trueness" (closeness of a measurement to an accepted reference value) and "precision" (the closeness of measurement results to each other). A carefully designed collaborative study enables estimation of a variety of gene expression measurement precision metrics: repeatability, several flavors of intermediate precision, and reproducibility. The three 2004 Expression Analysis Pilot Proficiency Test collaborative studies, each with 13 to 16 participants, provide triplicate microarray measurements on each of two reference RNA pools. Using and modestly extending the consensus ISO 5725 documentary standard, we evaluate the metrological precision figures of merit for individual microarray signal measurement, building from calculations appropriate to single measurement processes, such as technical replicate expression values for individual probes on a microarray, to the estimation and display of precision functions representing all of the probes in a given platform.</p> <p>Conclusion</p> <p>With only modest extensions, the established metrological framework can be fruitfully used to characterize the measurement performance of microarray and other highly multiplexed systems. Precision functions, summarizing routine precision metrics estimated from appropriately repeated measurements of one or more reference materials as functions of signal level, are demonstrated and merit further development for characterizing measurement platforms, monitoring changes in measurement system performance, and comparing performance among laboratories or analysts.</p

Exploring the use of internal and externalcontrols for assessing microarray technical performance

<p>Abstract</p> <p>Background</p> <p>The maturing of gene expression microarray technology and interest in the use of microarray-based applications for clinical and diagnostic applications calls for quantitative measures of quality. This manuscript presents a retrospective study characterizing several approaches to assess technical performance of microarray data measured on the Affymetrix GeneChip platform, including whole-array metrics and information from a standard mixture of external spike-in and endogenous internal controls. Spike-in controls were found to carry the same information about technical performance as whole-array metrics and endogenous "housekeeping" genes. These results support the use of spike-in controls as general tools for performance assessment across time, experimenters and array batches, suggesting that they have potential for comparison of microarray data generated across species using different technologies.</p> <p>Results</p> <p>A layered PCA modeling methodology that uses data from a number of classes of controls (spike-in hybridization, spike-in polyA+, internal RNA degradation, endogenous or "housekeeping genes") was used for the assessment of microarray data quality. The controls provide information on multiple stages of the experimental protocol (e.g., hybridization, RNA amplification). External spike-in, hybridization and RNA labeling controls provide information related to both assay and hybridization performance whereas internal endogenous controls provide quality information on the biological sample. We find that the variance of the data generated from the external and internal controls carries critical information about technical performance; the PCA dissection of this variance is consistent with whole-array quality assessment based on a number of quality assurance/quality control (QA/QC) metrics.</p> <p>Conclusions</p> <p>These results provide support for the use of both external and internal RNA control data to assess the technical quality of microarray experiments. The observed consistency amongst the information carried by internal and external controls and whole-array quality measures offers promise for rationally-designed control standards for routine performance monitoring of multiplexed measurement platforms.</p

NIST Interlaboratory Study on Glycosylation Analysis of Monoclonal Antibodies: Comparison of Results from Diverse Analytical Methods

Glycosylation is a topic of intense current interest in the development of biopharmaceuticals because it is related to drug safety and efficacy. This work describes results of an interlaboratory study on the glycosylation of the Primary Sample (PS) of NISTmAb, a monoclonal antibody reference material. Seventy-six laboratories from industry, university, research, government, and hospital sectors in Europe, North America, Asia, and Australia submit- Avenue, Silver Spring, Maryland 20993; 22Glycoscience Research Laboratory, Genos, Borongajska cesta 83h, 10 000 Zagreb, Croatia; 23Faculty of Pharmacy and Biochemistry, University of Zagreb, A. Kovacˇ ic´ a 1, 10 000 Zagreb, Croatia; 24Department of Chemistry, Georgia State University, 100 Piedmont Avenue, Atlanta, Georgia 30303; 25glyXera GmbH, Brenneckestrasse 20 * ZENIT / 39120 Magdeburg, Germany; 26Health Products and Foods Branch, Health Canada, AL 2201E, 251 Sir Frederick Banting Driveway, Ottawa, Ontario, K1A 0K9 Canada; 27Graduate School of Advanced Sciences of Matter, Hiroshima University, 1-3-1 Kagamiyama Higashi-Hiroshima 739–8530 Japan; 28ImmunoGen, 830 Winter Street, Waltham, Massachusetts 02451; 29Department of Medical Physiology, Jagiellonian University Medical College, ul. Michalowskiego 12, 31–126 Krakow, Poland; 30Department of Pathology, Johns Hopkins University, 400 N. Broadway Street Baltimore, Maryland 21287; 31Mass Spec Core Facility, KBI Biopharma, 1101 Hamlin Road Durham, North Carolina 27704; 32Division of Mass Spectrometry, Korea Basic Science Institute, 162 YeonGuDanji-Ro, Ochang-eup, Cheongwon-gu, Cheongju Chungbuk, 363–883 Korea (South); 33Advanced Therapy Products Research Division, Korea National Institute of Food and Drug Safety, 187 Osongsaengmyeong 2-ro Osong-eup, Heungdeok-gu, Cheongju-si, Chungcheongbuk-do, 363–700, Korea (South); 34Center for Proteomics and Metabolomics, Leiden University Medical Center, P.O. Box 9600, 2300 RC Leiden, The Netherlands; 35Ludger Limited, Culham Science Centre, Abingdon, Oxfordshire, OX14 3EB, United Kingdom; 36Biomolecular Discovery and Design Research Centre and ARC Centre of Excellence for Nanoscale BioPhotonics (CNBP), Macquarie University, North Ryde, Australia; 37Proteomics, Central European Institute for Technology, Masaryk University, Kamenice 5, A26, 625 00 BRNO, Czech Republic; 38Max Planck Institute for Dynamics of Complex Technical Systems, Sandtorstrasse 1, 39106 Magdeburg, Germany; 39Department of Biomolecular Sciences, Max Planck Institute of Colloids and Interfaces, 14424 Potsdam, Germany; 40AstraZeneca, Granta Park, Cambridgeshire, CB21 6GH United Kingdom; 41Merck, 2015 Galloping Hill Rd, Kenilworth, New Jersey 07033; 42Analytical R&D, MilliporeSigma, 2909 Laclede Ave. St. Louis, Missouri 63103; 43MS Bioworks, LLC, 3950 Varsity Drive Ann Arbor, Michigan 48108; 44MSD, Molenstraat 110, 5342 CC Oss, The Netherlands; 45Exploratory Research Center on Life and Living Systems (ExCELLS), National Institutes of Natural Sciences, 5–1 Higashiyama, Myodaiji, Okazaki 444–8787 Japan; 46Graduate School of Pharmaceutical Sciences, Nagoya City University, 3–1 Tanabe-dori, Mizuhoku, Nagoya 467–8603 Japan; 47Medical & Biological Laboratories Co., Ltd, 2-22-8 Chikusa, Chikusa-ku, Nagoya 464–0858 Japan; 48National Institute for Biological Standards and Control, Blanche Lane, South Mimms, Potters Bar, Hertfordshire EN6 3QG United Kingdom; 49Division of Biological Chemistry & Biologicals, National Institute of Health Sciences, 1-18-1 Kamiyoga, Setagaya-ku, Tokyo 158–8501 Japan; 50New England Biolabs, Inc., 240 County Road, Ipswich, Massachusetts 01938; 51New York University, 100 Washington Square East New York City, New York 10003; 52Target Discovery Institute, Nuffield Department of Medicine, University of Oxford, Roosevelt Drive, Oxford, OX3 7FZ, United Kingdom; 53GlycoScience Group, The National Institute for Bioprocessing Research and Training, Fosters Avenue, Mount Merrion, Blackrock, Co. Dublin, Ireland; 54Department of Chemistry, North Carolina State University, 2620 Yarborough Drive Raleigh, North Carolina 27695; 55Pantheon, 201 College Road East Princeton, New Jersey 08540; 56Pfizer Inc., 1 Burtt Road Andover, Massachusetts 01810; 57Proteodynamics, ZI La Varenne 20–22 rue Henri et Gilberte Goudier 63200 RIOM, France; 58ProZyme, Inc., 3832 Bay Center Place Hayward, California 94545; 59Koichi Tanaka Mass Spectrometry Research Laboratory, Shimadzu Corporation, 1 Nishinokyo Kuwabara-cho Nakagyo-ku, Kyoto, 604 8511 Japan; 60Children’s GMP LLC, St. Jude Children’s Research Hospital, 262 Danny Thomas Place Memphis, Tennessee 38105; 61Sumitomo Bakelite Co., Ltd., 1–5 Muromati 1-Chome, Nishiku, Kobe, 651–2241 Japan; 62Synthon Biopharmaceuticals, Microweg 22 P.O. Box 7071, 6503 GN Nijmegen, The Netherlands; 63Takeda Pharmaceuticals International Co., 40 Landsdowne Street Cambridge, Massachusetts 02139; 64Department of Chemistry and Biochemistry, Texas Tech University, 2500 Broadway, Lubbock, Texas 79409; 65Thermo Fisher Scientific, 1214 Oakmead Parkway Sunnyvale, California 94085; 66United States Pharmacopeia India Pvt. Ltd. IKP Knowledge Park, Genome Valley, Shamirpet, Turkapally Village, Medchal District, Hyderabad 500 101 Telangana, India; 67Alberta Glycomics Centre, University of Alberta, Edmonton, Alberta T6G 2G2 Canada; 68Department of Chemistry, University of Alberta, Edmonton, Alberta T6G 2G2 Canada; 69Department of Chemistry, University of California, One Shields Ave, Davis, California 95616; 70Horva´ th Csaba Memorial Laboratory for Bioseparation Sciences, Research Center for Molecular Medicine, Doctoral School of Molecular Medicine, Faculty of Medicine, University of Debrecen, Debrecen, Egyetem ter 1, Hungary; 71Translational Glycomics Research Group, Research Institute of Biomolecular and Chemical Engineering, University of Pannonia, Veszprem, Egyetem ut 10, Hungary; 72Delaware Biotechnology Institute, University of Delaware, 15 Innovation Way Newark, Delaware 19711; 73Proteomics Core Facility, University of Gothenburg, Medicinaregatan 1G SE 41390 Gothenburg, Sweden; 74Department of Medical Biochemistry and Cell Biology, University of Gothenburg, Institute of Biomedicine, Sahlgrenska Academy, Medicinaregatan 9A, Box 440, 405 30, Gothenburg, Sweden; 75Department of Clinical Chemistry and Transfusion Medicine, Sahlgrenska Academy at the University of Gothenburg, Bruna Straket 16, 41345 Gothenburg, Sweden; 76Department of Chemistry, University of Hamburg, Martin Luther King Pl. 6 20146 Hamburg, Germany; 77Department of Chemistry, University of Manitoba, 144 Dysart Road, Winnipeg, Manitoba, Canada R3T 2N2; 78Laboratory of Mass Spectrometry of Interactions and Systems, University of Strasbourg, UMR Unistra-CNRS 7140, France; 79Natural and Medical Sciences Institute, University of Tu¨ bingen, Markwiesenstrae 55, 72770 Reutlingen, Germany; 80Bijvoet Center for Biomolecular Research and Utrecht Institute for Pharmaceutical Sciences, Utrecht University, Padualaan 8, 3584 CH Utrecht, The Netherlands; 81Division of Bioanalytical Chemistry, Amsterdam Institute for Molecules, Medicines and Systems, Vrije Universiteit Amsterdam, de Boelelaan 1085, 1081 HV Amsterdam, The Netherlands; 82Department of Chemistry, Waters Corporation, 34 Maple Street Milford, Massachusetts 01757; 83Zoetis, 333 Portage St. Kalamazoo, Michigan 49007 Author’s Choice—Final version open access under the terms of the Creative Commons CC-BY license. Received July 24, 2019, and in revised form, August 26, 2019 Published, MCP Papers in Press, October 7, 2019, DOI 10.1074/mcp.RA119.001677 ER: NISTmAb Glycosylation Interlaboratory Study 12 Molecular & Cellular Proteomics 19.1 Downloaded from https://www.mcponline.org by guest on January 20, 2020 ted a total of 103 reports on glycan distributions. The principal objective of this study was to report and compare results for the full range of analytical methods presently used in the glycosylation analysis of mAbs. Therefore, participation was unrestricted, with laboratories choosing their own measurement techniques. Protein glycosylation was determined in various ways, including at the level of intact mAb, protein fragments, glycopeptides, or released glycans, using a wide variety of methods for derivatization, separation, identification, and quantification. Consequently, the diversity of results was enormous, with the number of glycan compositions identified by each laboratory ranging from 4 to 48. In total, one hundred sixteen glycan compositions were reported, of which 57 compositions could be assigned consensus abundance values. These consensus medians provide communityderived values for NISTmAb PS. Agreement with the consensus medians did not depend on the specific method or laboratory type. The study provides a view of the current state-of-the-art for biologic glycosylation measurement and suggests a clear need for harmonization of glycosylation analysis methods. Molecular & Cellular Proteomics 19: 11–30, 2020. DOI: 10.1074/mcp.RA119.001677.L

in the Analysis of Foods and Dietary Supplements: A Practical Guide

This publication is available free of charge from

Evaluating Droplet Digital Polymerase Chain Reaction for the Quantification of Human Genomic DNA: Lifting the Traceability Fog

Digital polymerase chain reaction (dPCR) end point platforms directly estimate the number of DNA target copies per reaction partition, λ, where the partitions are fixed-location chambers (cdPCR) or aqueous droplets floating in oil (ddPCR). For use in the certification of target concentration in primary calibrant certified reference materials (CRMs), both λ and the partition volume, <i>V</i>, must be metrologically traceable to some accessible reference system, ideally, the International System of Units (SI). The fixed spatial distribution of cdPCR chambers enables real-time monitoring of PCR amplification. Analysis of the resulting reaction curves enables validation of the critical dPCR assumptions that are essential for establishing the SI traceability of λ. We know of no direct method for validating these assumptions for ddPCR platforms. The manufacturers of the cdPCR and ddPCR systems available to us do not provide traceable partition volume specifications. Our colleagues at the National Institute of Standards and Technology (NIST) have developed a reliable method for determining ddPCR droplet volume and have demonstrated that different ddPCR reagents yield droplets of somewhat different size. Thus, neither dPCR platform by itself provides metrologically traceable estimates of target concentration. We show here that evaluating split samples with both cdPCR and ddPCR platforms can transfer the λ traceability characteristics of a cdPCR assay to its ddPCR analogue, establishing fully traceable ddPCR estimates of CRM target concentration

Evaluating Droplet Digital Polymerase Chain Reaction for the Quantification of Human Genomic DNA: Lifting the Traceability Fog

Digital polymerase chain reaction (dPCR) end point platforms directly estimate the number of DNA target copies per reaction partition, λ, where the partitions are fixed-location chambers (cdPCR) or aqueous droplets floating in oil (ddPCR). For use in the certification of target concentration in primary calibrant certified reference materials (CRMs), both λ and the partition volume, <i>V</i>, must be metrologically traceable to some accessible reference system, ideally, the International System of Units (SI). The fixed spatial distribution of cdPCR chambers enables real-time monitoring of PCR amplification. Analysis of the resulting reaction curves enables validation of the critical dPCR assumptions that are essential for establishing the SI traceability of λ. We know of no direct method for validating these assumptions for ddPCR platforms. The manufacturers of the cdPCR and ddPCR systems available to us do not provide traceable partition volume specifications. Our colleagues at the National Institute of Standards and Technology (NIST) have developed a reliable method for determining ddPCR droplet volume and have demonstrated that different ddPCR reagents yield droplets of somewhat different size. Thus, neither dPCR platform by itself provides metrologically traceable estimates of target concentration. We show here that evaluating split samples with both cdPCR and ddPCR platforms can transfer the λ traceability characteristics of a cdPCR assay to its ddPCR analogue, establishing fully traceable ddPCR estimates of CRM target concentration