A Commentary on

Commentary: Reducing Viability Bias in Analysis of Gut Microbiota in Preterm Infants at Risk of NEC and Sepsis



by Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212. doi: 10.3389/fcimb.2018.0021

Berrington, JE

Cummings, SP

Embleton, ND

Lanyon, C

Schwalbe, EC

Smith, DL

van der Gast, CJ

Young, GR

Young, Gregory

Smith, Darren

Embleton, Nicholas D.

Berrington, Janet Elizabeth

Schwalbe, Ed

Cummings, Stephen Paul

van der Gast, Christopher J.

Lanyon, Clare

Northumbria Research Link

Northumbria Research Link
Citation:  Young,  Gregory,  Smith,  Darren,  Embleton,  Nicholas  D.,  Berrington,  Janet  Elizabeth, 
Schwalbe, Ed, Cummings, Stephen Paul,  van der Gast, Christopher J. and Lanyon, Clare (2018) 
Response: Commentary: Reducing Viability Bias in Analysis of Gut Microbiota in Preterm Infants at  
Risk of NEC and Sepsis. Frontiers in Cellular and Infection Microbiology, 8. p. 374. ISSN 2235-2988 
Published by: Frontiers
URL: https://doi.org/10.3389/fcimb.2018.00374 <https://doi.org/10.3389/fcimb.2018.00374>
This version was downloaded from Northumbria Research Link: http://nrl.northumbria.ac.uk/42545/
Northumbria University has developed Northumbria Research Link (NRL) to enable users to access 
the University’s research output. Copyright © and moral rights for items on NRL are retained by the 
individual author(s) and/or other copyright owners.  Single copies of full items can be reproduced, 
displayed or performed, and given to third parties in any format or medium for personal research or 
study, educational, or not-for-profit purposes without prior permission or charge, provided the authors, 
title and full bibliographic details are given, as well as a hyperlink and/or URL to the original metadata 
page. The content must not be changed in any way. Full items must not be sold commercially in any  
format or medium without formal permission of the copyright holder.  The full policy is available online: 
http://nrl.northumbria.ac.uk/pol  i cies.html  
This  document  may differ  from the  final,  published version of  the research  and has been made 
available online in accordance with publisher policies. To read and/or cite from the published version 
of the research, please visit the publisher’s website (a subscription may be required.)
                        

GENERAL COMMENTARY
published: 24 October 2018
doi: 10.3389/fcimb.2018.00374
Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 1 October 2018 | Volume 8 | Article 374
Edited by:
Pascale Alard,
University of Louisville, United States
Reviewed by:
Bjoern O. Schroeder,
University of Gothenburg, Sweden
Natalia Shulzhenko,
Oregon State University,
United States
*Correspondence:
Gregory R. Young
gregory.young@northumbria.ac.uk
Clare Lanyon
clare.lanyon@northumbria.ac.uk
Specialty section:
This article was submitted to
Microbiome in Health and Disease,
a section of the journal
Frontiers in Cellular and Infection
Microbiology
Received: 28 June 2018
Accepted: 05 October 2018
Published: 24 October 2018
Citation:
Young GR, Smith DL, Embleton ND,
Berrington JE, Schwalbe EC,
Cummings SP, van der Gast CJ and
Lanyon C (2018) Response:
Commentary: Reducing Viability Bias
in Analysis of Gut Microbiota in
Preterm Infants at Risk of NEC and
Sepsis.
Front. Cell. Infect. Microbiol. 8:374.
doi: 10.3389/fcimb.2018.00374
Response: Commentary: Reducing
Viability Bias in Analysis of Gut
Microbiota in Preterm Infants at Risk
of NEC and Sepsis
Gregory R. Young 1*, Darren L. Smith 1, Nicholas D. Embleton 2,
Janet Elizabeth Berrington 2, Edward C. Schwalbe 1, Stephen Paul Cummings 3,
Christopher J. van der Gast 4 and Clare Lanyon 1*
1 Faculty of Health and Life Sciences, University of Northumbria, Newcastle upon Tyne, United Kingdom, 2Newcastle
Neonatal Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom, 3 School of
Science and Engineering, Teesside University, Middlesbrough, United Kingdom, 4 School of Healthcare Science, Manchester
Metropolitan University, Manchester, United Kingdom
Keywords: PMA, viability analyses, microbiota, methods—analytic, bias, clinical samples
A Commentary on
Commentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Risk
of NEC and Sepsis
by Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212.
doi: 10.3389/fcimb.2018.00212
We would like to thank Agustí and Codony (2018), for their interest in our recent manuscript
(Young et al., 2017), and valuable comments. We agree that non-viable cell exclusion is an
important consideration, worth making when conducting analyses of microbial communities via
targeted DNA sequencing and amplification approaches. The technique is especially pertinent in
environments where large volumes of non-viable bacteria are expected, such as in preterm infant
stool, where multiple clinical interventions manifest deliberate bacterial killing.
Whilst we agree with the general principles with regards to sample collection and handling
described by Agustí and Codony (2018) we maintain that these are not always possible in real-life,
clinical patient samples. For example, as patient care is the primary concern in this cohort, sample
collection is convenience based rather than experimentally dictated. This requires samples to be
spontaneously collected and stored prior to processing. We concede this increases the likelihood
of loss of anaerobic bacterial viability. We would, however point out that the study by Brusa et al.
(1989), highlighted in the commentary defines viability purely as culturability. Studies (Contreras
et al., 2011; Nocker et al., 2011) have reported loss of culturability occurs at lower stress levels
than loss of membrane integrity. Thus, suggesting greater cellular stress may be required to deplete
DNA signals from non-viable cells, as determined during PMA-based viability assays. Moreover,
we highlight in our manuscript that PMA is a conservative parameter for loss of viability. We
also propose that, in the specific cohort investigated, conservative non-viable determination is
preferable to over-exaggerated determination or a complete absence of it. This is especially true
when non-treated samples can be analyzed in parallel.
DNA retention on microtube surfaces and subsequent inclusion in the viable communities may
well be an issue in PMA-based viability determination. In our study however, samples were not
heat-killed at any point. This is in contrast to the study by Agustí et al. (2017), in which DNA
binding to microtube walls may have occurred prior to PMA-treatment. Furthermore, the methods
Young et al. Response: Reducing vPCR Bias
described by ourselves outline transfer between sterile glass pots
during initial sample collection and several microtubes and well
plates from PMA-addition to incubation and photo-activation
and, finally DNA isolation. We propose that these several steps
are sufficient to reduce the impact of DNA retained onmicrotube
surfaces on the assigned live DNA fraction.
In addition to the valuable comments made by Agustí
and Codony (2018), where the target sequence is <300
bp we recommend combination of a nested-PCR approach,
comprising initial long fragment pre-amplification, followed
by subsequent target amplicon sequencing/quantification, first
described by Luo et al. (2010). This increases the likelihood of
encountering intercalated PMA during the DNA amplification
process.
The use of PMA as a viability-dye is extremely useful in
microbial community analysis. As a greater volume of literature
becomes available regarding it’s use, the technique will no doubt
refine. From our experience with the technique, we propose
several important considerations to be made when planning any
study in which PMA-based non-viable cell exclusion is employed:
• Is the sample likely to contain substantial volumes of DNA
originating from non-viable bacteria?
• Are the collection and storage methods likely to impact
community viability?
• What is the composition of the sample? (Homogenisation
may be required)
• What is the optimal concentration of PMA required to ensure
non-viable cell DNA quenching?
• Is there potential for extra-cellular DNA carry-over in the
methods? (specifically fromDNA retained on plastics surfaces)
• What is my light-source for photoactivation (464 nm is
excitation maxima of PMA therefore substantial emission at
this wavelength is required)
• Can I employ a nested-PCR approach tomaximize intercalated
PMA impact?
• Can I afford to sequence the non-treated sample also?
(Particularly useful in clinical environments to ID bacterial
communities that may have been viable recently)
• Can I afford to supplement this technique with another
to confirm presence/viability? (e.g., FISH, flow cytometry,
selective culture)
We would further direct anyone interested to a comprehensive
review of the technique by Fittipaldi et al. (2012).
AUTHOR CONTRIBUTIONS
GY wrote the response. All authors proofread and approved this
commentary response.
REFERENCES
Agustí, G., and Codony, F. (2018). Commentary: Reducing viability bias in analysis
of gut microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.
Microbiol. 8:212. doi: 10.3389/fcimb.2018.00212
Agustí, G., Fittipaldi, M., and Codony, F. (2017). False-positive viability
PCR results: an association with microtubes. Curr. Microbiol. 74, 377–380.
doi: 10.1007/s00284-016-1189-3
Brusa, T., Canzi, E., Pacini, N., Zanchi, R., and Ferrari, A. (1989). Oxygen tolerance
of anaerobic bacteria isolated from human feces. Curr. Microbiol. 19, 39–43.
Contreras, P. J., Urrutia, H., Sossa, K., and Nocker, A. (2011). Effect of
PCR amplicon length on suppressing signals from membrane-compromised
cells by propidium monoazide treatment. J. Microbiol. Methods 87, 89–95.
doi: 10.1016/j.mimet.2011.07.016
Fittipaldi, M., Nocker, A., and Codony, F. (2012). Progress in understanding
preferential detection of live cells using viability dyes in combination
with DNA amplification. J. Microbiol. Methods 91, 276–289.
doi: 10.1016/j.mimet.2012.08.007
Luo, J. F., Lin, W. T., and Guo, Y. (2010). Method to detect only
viable cells in microbial ecology. Appl. Microbiol. Biotechnol. 86, 377–384.
doi: 10.1007/s00253-009-2373-1
Nocker, A., Caspers, M., Esveld-Amanatidou, A., van der Vossen, J.,
Schuren, F., Montijn, R., et al. (2011). Multiparameter viability assay
for stress profiling applied to the food pathogen Listeria monocytogenes
F2365. Appl. Environ. Microbiol. 77, 6433–6440. doi: 10.1128/AEM.001
42-11
Young, G. R., Smith, D. L., Embleton, N. D., Berrington, J. E., Schwalbe, E.
C., Cummings, S. P., et al. (2017). Reducing Viability bias in analysis of gut
microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.
Microbiol. 7:237. doi: 10.3389/fcimb.2017.00237
Conflict of Interest Statement: The authors declare that the research was
conducted in the absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.
Copyright © 2018 Young, Smith, Embleton, Berrington, Schwalbe, Cummings, van
der Gast and Lanyon. This is an open-access article distributed under the terms
of the Creative Commons Attribution License (CC BY). The use, distribution or
reproduction in other forums is permitted, provided the original author(s) and the
copyright owner(s) are credited and that the original publication in this journal
is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.
Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 2 October 2018 | Volume 8 | Article 374


Response: Commentary: Reducing Viability Bias in Analysis of Gut Microbiota in Preterm Infants at Risk of NEC and Sepsis

Young, Gregory R.

Smith, Darren L.

Schwalbe, Edward C.

Van Der Gast, Christopher J.

Teesside University Research

GENERAL COMMENTARY
published: 24 October 2018
doi: 10.3389/fcimb.2018.00374
Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 1 October 2018 | Volume 8 | Article 374
Edited by:
Pascale Alard,
University of Louisville, United States
Reviewed by:
Bjoern O. Schroeder,
University of Gothenburg, Sweden
Natalia Shulzhenko,
Oregon State University,
United States
*Correspondence:
Gregory R. Young
gregory.young@northumbria.ac.uk
Clare Lanyon
clare.lanyon@northumbria.ac.uk
Specialty section:
This article was submitted to
Microbiome in Health and Disease,
a section of the journal
Frontiers in Cellular and Infection
Microbiology
Received: 28 June 2018
Accepted: 05 October 2018
Published: 24 October 2018
Citation:
Young GR, Smith DL, Embleton ND,
Berrington JE, Schwalbe EC,
Cummings SP, van der Gast CJ and
Lanyon C (2018) Response:
Commentary: Reducing Viability Bias
in Analysis of Gut Microbiota in
Preterm Infants at Risk of NEC and
Sepsis.
Front. Cell. Infect. Microbiol. 8:374.
doi: 10.3389/fcimb.2018.00374
Response: Commentary: Reducing
Viability Bias in Analysis of Gut
Microbiota in Preterm Infants at Risk
of NEC and Sepsis
Gregory R. Young 1*, Darren L. Smith 1, Nicholas D. Embleton 2,
Janet Elizabeth Berrington 2, Edward C. Schwalbe 1, Stephen Paul Cummings 3,
Christopher J. van der Gast 4 and Clare Lanyon 1*
1 Faculty of Health and Life Sciences, University of Northumbria, Newcastle upon Tyne, United Kingdom, 2Newcastle
Neonatal Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom, 3 School of
Science and Engineering, Teesside University, Middlesbrough, United Kingdom, 4 School of Healthcare Science, Manchester
Metropolitan University, Manchester, United Kingdom
Keywords: PMA, viability analyses, microbiota, methods—analytic, bias, clinical samples
A Commentary on
Commentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Risk
of NEC and Sepsis
by Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212.
doi: 10.3389/fcimb.2018.00212
We would like to thank Agustí and Codony (2018), for their interest in our recent manuscript
(Young et al., 2017), and valuable comments. We agree that non-viable cell exclusion is an
important consideration, worth making when conducting analyses of microbial communities via
targeted DNA sequencing and amplification approaches. The technique is especially pertinent in
environments where large volumes of non-viable bacteria are expected, such as in preterm infant
stool, where multiple clinical interventions manifest deliberate bacterial killing.
Whilst we agree with the general principles with regards to sample collection and handling
described by Agustí and Codony (2018) we maintain that these are not always possible in real-life,
clinical patient samples. For example, as patient care is the primary concern in this cohort, sample
collection is convenience based rather than experimentally dictated. This requires samples to be
spontaneously collected and stored prior to processing. We concede this increases the likelihood
of loss of anaerobic bacterial viability. We would, however point out that the study by Brusa et al.
(1989), highlighted in the commentary defines viability purely as culturability. Studies (Contreras
et al., 2011; Nocker et al., 2011) have reported loss of culturability occurs at lower stress levels
than loss of membrane integrity. Thus, suggesting greater cellular stress may be required to deplete
DNA signals from non-viable cells, as determined during PMA-based viability assays. Moreover,
we highlight in our manuscript that PMA is a conservative parameter for loss of viability. We
also propose that, in the specific cohort investigated, conservative non-viable determination is
preferable to over-exaggerated determination or a complete absence of it. This is especially true
when non-treated samples can be analyzed in parallel.
DNA retention on microtube surfaces and subsequent inclusion in the viable communities may
well be an issue in PMA-based viability determination. In our study however, samples were not
heat-killed at any point. This is in contrast to the study by Agustí et al. (2017), in which DNA
binding to microtube walls may have occurred prior to PMA-treatment. Furthermore, the methods
Young et al. Response: Reducing vPCR Bias
described by ourselves outline transfer between sterile glass pots
during initial sample collection and several microtubes and well
plates from PMA-addition to incubation and photo-activation
and, finally DNA isolation. We propose that these several steps
are sufficient to reduce the impact of DNA retained onmicrotube
surfaces on the assigned live DNA fraction.
In addition to the valuable comments made by Agustí
and Codony (2018), where the target sequence is <300
bp we recommend combination of a nested-PCR approach,
comprising initial long fragment pre-amplification, followed
by subsequent target amplicon sequencing/quantification, first
described by Luo et al. (2010). This increases the likelihood of
encountering intercalated PMA during the DNA amplification
process.
The use of PMA as a viability-dye is extremely useful in
microbial community analysis. As a greater volume of literature
becomes available regarding it’s use, the technique will no doubt
refine. From our experience with the technique, we propose
several important considerations to be made when planning any
study in which PMA-based non-viable cell exclusion is employed:
• Is the sample likely to contain substantial volumes of DNA
originating from non-viable bacteria?
• Are the collection and storage methods likely to impact
community viability?
• What is the composition of the sample? (Homogenisation
may be required)
• What is the optimal concentration of PMA required to ensure
non-viable cell DNA quenching?
• Is there potential for extra-cellular DNA carry-over in the
methods? (specifically fromDNA retained on plastics surfaces)
• What is my light-source for photoactivation (464 nm is
excitation maxima of PMA therefore substantial emission at
this wavelength is required)
• Can I employ a nested-PCR approach tomaximize intercalated
PMA impact?
• Can I afford to sequence the non-treated sample also?
(Particularly useful in clinical environments to ID bacterial
communities that may have been viable recently)
• Can I afford to supplement this technique with another
to confirm presence/viability? (e.g., FISH, flow cytometry,
selective culture)
We would further direct anyone interested to a comprehensive
review of the technique by Fittipaldi et al. (2012).
AUTHOR CONTRIBUTIONS
GY wrote the response. All authors proofread and approved this
commentary response.
REFERENCES
Agustí, G., and Codony, F. (2018). Commentary: Reducing viability bias in analysis
of gut microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.
Microbiol. 8:212. doi: 10.3389/fcimb.2018.00212
Agustí, G., Fittipaldi, M., and Codony, F. (2017). False-positive viability
PCR results: an association with microtubes. Curr. Microbiol. 74, 377–380.
doi: 10.1007/s00284-016-1189-3
Brusa, T., Canzi, E., Pacini, N., Zanchi, R., and Ferrari, A. (1989). Oxygen tolerance
of anaerobic bacteria isolated from human feces. Curr. Microbiol. 19, 39–43.
Contreras, P. J., Urrutia, H., Sossa, K., and Nocker, A. (2011). Effect of
PCR amplicon length on suppressing signals from membrane-compromised
cells by propidium monoazide treatment. J. Microbiol. Methods 87, 89–95.
doi: 10.1016/j.mimet.2011.07.016
Fittipaldi, M., Nocker, A., and Codony, F. (2012). Progress in understanding
preferential detection of live cells using viability dyes in combination
with DNA amplification. J. Microbiol. Methods 91, 276–289.
doi: 10.1016/j.mimet.2012.08.007
Luo, J. F., Lin, W. T., and Guo, Y. (2010). Method to detect only
viable cells in microbial ecology. Appl. Microbiol. Biotechnol. 86, 377–384.
doi: 10.1007/s00253-009-2373-1
Nocker, A., Caspers, M., Esveld-Amanatidou, A., van der Vossen, J.,
Schuren, F., Montijn, R., et al. (2011). Multiparameter viability assay
for stress profiling applied to the food pathogen Listeria monocytogenes
F2365. Appl. Environ. Microbiol. 77, 6433–6440. doi: 10.1128/AEM.001
42-11
Young, G. R., Smith, D. L., Embleton, N. D., Berrington, J. E., Schwalbe, E.
C., Cummings, S. P., et al. (2017). Reducing Viability bias in analysis of gut
microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.
Microbiol. 7:237. doi: 10.3389/fcimb.2017.00237
Conflict of Interest Statement: The authors declare that the research was
conducted in the absence of any commercial or financial relationships that could
be construed as a potential conflict of interest.
Copyright © 2018 Young, Smith, Embleton, Berrington, Schwalbe, Cummings, van
der Gast and Lanyon. This is an open-access article distributed under the terms
of the Creative Commons Attribution License (CC BY). The use, distribution or
reproduction in other forums is permitted, provided the original author(s) and the
copyright owner(s) are credited and that the original publication in this journal
is cited, in accordance with accepted academic practice. No use, distribution or
reproduction is permitted which does not comply with these terms.
Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 2 October 2018 | Volume 8 | Article 374


Teeside University's Research Repository

GENERAL COMMENTARYpublished: 24 October 2018doi: 10.3389/fcimb.2018.00374Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 1 October 2018 | Volume 8 | Article 374Edited by:Pascale Alard,University of Louisville, United StatesReviewed by:Bjoern O. Schroeder,University of Gothenburg, SwedenNatalia Shulzhenko,Oregon State University,United States*Correspondence:Gregory R. Younggregory.young@northumbria.ac.ukClare Lanyonclare.lanyon@northumbria.ac.ukSpecialty section:This article was submitted toMicrobiome in Health and Disease,a section of the journalFrontiers in Cellular and InfectionMicrobiologyReceived: 28 June 2018Accepted: 05 October 2018Published: 24 October 2018Citation:Young GR, Smith DL, Embleton ND,Berrington JE, Schwalbe EC,Cummings SP, van der Gast CJ andLanyon C (2018) Response:Commentary: Reducing Viability Biasin Analysis of Gut Microbiota inPreterm Infants at Risk of NEC andSepsis.Front. Cell. Infect. Microbiol. 8:374.doi: 10.3389/fcimb.2018.00374Response: Commentary: ReducingViability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and SepsisGregory R. Young 1*, Darren L. Smith 1, Nicholas D. Embleton 2,Janet Elizabeth Berrington 2, Edward C. Schwalbe 1, Stephen Paul Cummings 3,Christopher J. van der Gast 4 and Clare Lanyon 1*1 Faculty of Health and Life Sciences, University of Northumbria, Newcastle upon Tyne, United Kingdom, 2NewcastleNeonatal Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom, 3 School ofScience and Engineering, Teesside University, Middlesbrough, United Kingdom, 4 School of Healthcare Science, ManchesterMetropolitan University, Manchester, United KingdomKeywords: PMA, viability analyses, microbiota, methods—analytic, bias, clinical samplesA Commentary onCommentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and Sepsisby Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212.doi: 10.3389/fcimb.2018.00212We would like to thank Agustí and Codony (2018), for their interest in our recent manuscript(Young et al., 2017), and valuable comments. We agree that non-viable cell exclusion is animportant consideration, worth making when conducting analyses of microbial communities viatargeted DNA sequencing and amplification approaches. The technique is especially pertinent inenvironments where large volumes of non-viable bacteria are expected, such as in preterm infantstool, where multiple clinical interventions manifest deliberate bacterial killing.Whilst we agree with the general principles with regards to sample collection and handlingdescribed by Agustí and Codony (2018) we maintain that these are not always possible in real-life,clinical patient samples. For example, as patient care is the primary concern in this cohort, samplecollection is convenience based rather than experimentally dictated. This requires samples to bespontaneously collected and stored prior to processing. We concede this increases the likelihoodof loss of anaerobic bacterial viability. We would, however point out that the study by Brusa et al.(1989), highlighted in the commentary defines viability purely as culturability. Studies (Contreraset al., 2011; Nocker et al., 2011) have reported loss of culturability occurs at lower stress levelsthan loss of membrane integrity. Thus, suggesting greater cellular stress may be required to depleteDNA signals from non-viable cells, as determined during PMA-based viability assays. Moreover,we highlight in our manuscript that PMA is a conservative parameter for loss of viability. Wealso propose that, in the specific cohort investigated, conservative non-viable determination ispreferable to over-exaggerated determination or a complete absence of it. This is especially truewhen non-treated samples can be analyzed in parallel.DNA retention on microtube surfaces and subsequent inclusion in the viable communities maywell be an issue in PMA-based viability determination. In our study however, samples were notheat-killed at any point. This is in contrast to the study by Agustí et al. (2017), in which DNAbinding to microtube walls may have occurred prior to PMA-treatment. Furthermore, the methodsYoung et al. Response: Reducing vPCR Biasdescribed by ourselves outline transfer between sterile glass potsduring initial sample collection and several microtubes and wellplates from PMA-addition to incubation and photo-activationand, finally DNA isolation. We propose that these several stepsare sufficient to reduce the impact of DNA retained onmicrotubesurfaces on the assigned live DNA fraction.In addition to the valuable comments made by Agustíand Codony (2018), where the target sequence is <300bp we recommend combination of a nested-PCR approach,comprising initial long fragment pre-amplification, followedby subsequent target amplicon sequencing/quantification, firstdescribed by Luo et al. (2010). This increases the likelihood ofencountering intercalated PMA during the DNA amplificationprocess.The use of PMA as a viability-dye is extremely useful inmicrobial community analysis. As a greater volume of literaturebecomes available regarding it’s use, the technique will no doubtrefine. From our experience with the technique, we proposeseveral important considerations to be made when planning anystudy in which PMA-based non-viable cell exclusion is employed:• Is the sample likely to contain substantial volumes of DNAoriginating from non-viable bacteria?• Are the collection and storage methods likely to impactcommunity viability?• What is the composition of the sample? (Homogenisationmay be required)• What is the optimal concentration of PMA required to ensurenon-viable cell DNA quenching?• Is there potential for extra-cellular DNA carry-over in themethods? (specifically fromDNA retained on plastics surfaces)• What is my light-source for photoactivation (464 nm isexcitation maxima of PMA therefore substantial emission atthis wavelength is required)• Can I employ a nested-PCR approach tomaximize intercalatedPMA impact?• Can I afford to sequence the non-treated sample also?(Particularly useful in clinical environments to ID bacterialcommunities that may have been viable recently)• Can I afford to supplement this technique with anotherto confirm presence/viability? (e.g., FISH, flow cytometry,selective culture)We would further direct anyone interested to a comprehensivereview of the technique by Fittipaldi et al. (2012).AUTHOR CONTRIBUTIONSGY wrote the response. All authors proofread and approved thiscommentary response.REFERENCESAgustí, G., and Codony, F. (2018). Commentary: Reducing viability bias in analysisof gut microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 8:212. doi: 10.3389/fcimb.2018.00212Agustí, G., Fittipaldi, M., and Codony, F. (2017). False-positive viabilityPCR results: an association with microtubes. Curr. Microbiol. 74, 377–380.doi: 10.1007/s00284-016-1189-3Brusa, T., Canzi, E., Pacini, N., Zanchi, R., and Ferrari, A. (1989). Oxygen toleranceof anaerobic bacteria isolated from human feces. Curr. Microbiol. 19, 39–43.Contreras, P. J., Urrutia, H., Sossa, K., and Nocker, A. (2011). Effect ofPCR amplicon length on suppressing signals from membrane-compromisedcells by propidium monoazide treatment. J. Microbiol. Methods 87, 89–95.doi: 10.1016/j.mimet.2011.07.016Fittipaldi, M., Nocker, A., and Codony, F. (2012). Progress in understandingpreferential detection of live cells using viability dyes in combinationwith DNA amplification. J. Microbiol. Methods 91, 276–289.doi: 10.1016/j.mimet.2012.08.007Luo, J. F., Lin, W. T., and Guo, Y. (2010). Method to detect onlyviable cells in microbial ecology. Appl. Microbiol. Biotechnol. 86, 377–384.doi: 10.1007/s00253-009-2373-1Nocker, A., Caspers, M., Esveld-Amanatidou, A., van der Vossen, J.,Schuren, F., Montijn, R., et al. (2011). Multiparameter viability assayfor stress profiling applied to the food pathogen Listeria monocytogenesF2365. Appl. Environ. Microbiol. 77, 6433–6440. doi: 10.1128/AEM.00142-11Young, G. R., Smith, D. L., Embleton, N. D., Berrington, J. E., Schwalbe, E.C., Cummings, S. P., et al. (2017). Reducing Viability bias in analysis of gutmicrobiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 7:237. doi: 10.3389/fcimb.2017.00237Conflict of Interest Statement: The authors declare that the research wasconducted in the absence of any commercial or financial relationships that couldbe construed as a potential conflict of interest.Copyright © 2018 Young, Smith, Embleton, Berrington, Schwalbe, Cummings, vander Gast and Lanyon. This is an open-access article distributed under the termsof the Creative Commons Attribution License (CC BY). The use, distribution orreproduction in other forums is permitted, provided the original author(s) and thecopyright owner(s) are credited and that the original publication in this journalis cited, in accordance with accepted academic practice. No use, distribution orreproduction is permitted which does not comply with these terms.Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 2 October 2018 | Volume 8 | Article 374

English

E-space: Manchester Metropolitan University's Research Repository

Young, GR and Smith, DL and Embleton, ND and Berrington, JE andSchwalbe, EC and Cummings, SP and van der Gast, CJ and Lanyon, C(2018) Response: Commentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Risk of NEC and Sepsis. Frontiers in Cellularand Infection Microbiology, 8. p. 374. ISSN 2235-2988Downloaded from: https://e-space.mmu.ac.uk/627997/Version: Published VersionPublisher: Frontiers MediaDOI: https://doi.org/10.3389/fcimb.2018.00374Usage rights: Creative Commons: Attribution 4.0Please cite the published versionhttps://e-space.mmu.ac.ukGENERAL COMMENTARYpublished: 24 October 2018doi: 10.3389/fcimb.2018.00374Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 1 October 2018 | Volume 8 | Article 374Edited by:Pascale Alard,University of Louisville, United StatesReviewed by:Bjoern O. Schroeder,University of Gothenburg, SwedenNatalia Shulzhenko,Oregon State University,United States*Correspondence:Gregory R. Younggregory.young@northumbria.ac.ukClare Lanyonclare.lanyon@northumbria.ac.ukSpecialty section:This article was submitted toMicrobiome in Health and Disease,a section of the journalFrontiers in Cellular and InfectionMicrobiologyReceived: 28 June 2018Accepted: 05 October 2018Published: 24 October 2018Citation:Young GR, Smith DL, Embleton ND,Berrington JE, Schwalbe EC,Cummings SP, van der Gast CJ andLanyon C (2018) Response:Commentary: Reducing Viability Biasin Analysis of Gut Microbiota inPreterm Infants at Risk of NEC andSepsis.Front. Cell. Infect. Microbiol. 8:374.doi: 10.3389/fcimb.2018.00374Response: Commentary: ReducingViability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and SepsisGregory R. Young 1*, Darren L. Smith 1, Nicholas D. Embleton 2,Janet Elizabeth Berrington 2, Edward C. Schwalbe 1, Stephen Paul Cummings 3,Christopher J. van der Gast 4 and Clare Lanyon 1*1 Faculty of Health and Life Sciences, University of Northumbria, Newcastle upon Tyne, United Kingdom, 2NewcastleNeonatal Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom, 3 School ofScience and Engineering, Teesside University, Middlesbrough, United Kingdom, 4 School of Healthcare Science, ManchesterMetropolitan University, Manchester, United KingdomKeywords: PMA, viability analyses, microbiota, methods—analytic, bias, clinical samplesA Commentary onCommentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and Sepsisby Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212.doi: 10.3389/fcimb.2018.00212We would like to thank Agustí and Codony (2018), for their interest in our recent manuscript(Young et al., 2017), and valuable comments. We agree that non-viable cell exclusion is animportant consideration, worth making when conducting analyses of microbial communities viatargeted DNA sequencing and amplification approaches. The technique is especially pertinent inenvironments where large volumes of non-viable bacteria are expected, such as in preterm infantstool, where multiple clinical interventions manifest deliberate bacterial killing.Whilst we agree with the general principles with regards to sample collection and handlingdescribed by Agustí and Codony (2018) we maintain that these are not always possible in real-life,clinical patient samples. For example, as patient care is the primary concern in this cohort, samplecollection is convenience based rather than experimentally dictated. This requires samples to bespontaneously collected and stored prior to processing. We concede this increases the likelihoodof loss of anaerobic bacterial viability. We would, however point out that the study by Brusa et al.(1989), highlighted in the commentary defines viability purely as culturability. Studies (Contreraset al., 2011; Nocker et al., 2011) have reported loss of culturability occurs at lower stress levelsthan loss of membrane integrity. Thus, suggesting greater cellular stress may be required to depleteDNA signals from non-viable cells, as determined during PMA-based viability assays. Moreover,we highlight in our manuscript that PMA is a conservative parameter for loss of viability. Wealso propose that, in the specific cohort investigated, conservative non-viable determination ispreferable to over-exaggerated determination or a complete absence of it. This is especially truewhen non-treated samples can be analyzed in parallel.DNA retention on microtube surfaces and subsequent inclusion in the viable communities maywell be an issue in PMA-based viability determination. In our study however, samples were notheat-killed at any point. This is in contrast to the study by Agustí et al. (2017), in which DNAbinding to microtube walls may have occurred prior to PMA-treatment. Furthermore, the methodsYoung et al. Response: Reducing vPCR Biasdescribed by ourselves outline transfer between sterile glass potsduring initial sample collection and several microtubes and wellplates from PMA-addition to incubation and photo-activationand, finally DNA isolation. We propose that these several stepsare sufficient to reduce the impact of DNA retained onmicrotubesurfaces on the assigned live DNA fraction.In addition to the valuable comments made by Agustíand Codony (2018), where the target sequence is <300bp we recommend combination of a nested-PCR approach,comprising initial long fragment pre-amplification, followedby subsequent target amplicon sequencing/quantification, firstdescribed by Luo et al. (2010). This increases the likelihood ofencountering intercalated PMA during the DNA amplificationprocess.The use of PMA as a viability-dye is extremely useful inmicrobial community analysis. As a greater volume of literaturebecomes available regarding it’s use, the technique will no doubtrefine. From our experience with the technique, we proposeseveral important considerations to be made when planning anystudy in which PMA-based non-viable cell exclusion is employed:• Is the sample likely to contain substantial volumes of DNAoriginating from non-viable bacteria?• Are the collection and storage methods likely to impactcommunity viability?• What is the composition of the sample? (Homogenisationmay be required)• What is the optimal concentration of PMA required to ensurenon-viable cell DNA quenching?• Is there potential for extra-cellular DNA carry-over in themethods? (specifically fromDNA retained on plastics surfaces)• What is my light-source for photoactivation (464 nm isexcitation maxima of PMA therefore substantial emission atthis wavelength is required)• Can I employ a nested-PCR approach tomaximize intercalatedPMA impact?• Can I afford to sequence the non-treated sample also?(Particularly useful in clinical environments to ID bacterialcommunities that may have been viable recently)• Can I afford to supplement this technique with anotherto confirm presence/viability? (e.g., FISH, flow cytometry,selective culture)We would further direct anyone interested to a comprehensivereview of the technique by Fittipaldi et al. (2012).AUTHOR CONTRIBUTIONSGY wrote the response. All authors proofread and approved thiscommentary response.REFERENCESAgustí, G., and Codony, F. (2018). Commentary: Reducing viability bias in analysisof gut microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 8:212. doi: 10.3389/fcimb.2018.00212Agustí, G., Fittipaldi, M., and Codony, F. (2017). False-positive viabilityPCR results: an association with microtubes. Curr. Microbiol. 74, 377–380.doi: 10.1007/s00284-016-1189-3Brusa, T., Canzi, E., Pacini, N., Zanchi, R., and Ferrari, A. (1989). Oxygen toleranceof anaerobic bacteria isolated from human feces. Curr. Microbiol. 19, 39–43.Contreras, P. J., Urrutia, H., Sossa, K., and Nocker, A. (2011). Effect ofPCR amplicon length on suppressing signals from membrane-compromisedcells by propidium monoazide treatment. J. Microbiol. Methods 87, 89–95.doi: 10.1016/j.mimet.2011.07.016Fittipaldi, M., Nocker, A., and Codony, F. (2012). Progress in understandingpreferential detection of live cells using viability dyes in combinationwith DNA amplification. J. Microbiol. Methods 91, 276–289.doi: 10.1016/j.mimet.2012.08.007Luo, J. F., Lin, W. T., and Guo, Y. (2010). Method to detect onlyviable cells in microbial ecology. Appl. Microbiol. Biotechnol. 86, 377–384.doi: 10.1007/s00253-009-2373-1Nocker, A., Caspers, M., Esveld-Amanatidou, A., van der Vossen, J.,Schuren, F., Montijn, R., et al. (2011). Multiparameter viability assayfor stress profiling applied to the food pathogen Listeria monocytogenesF2365. Appl. Environ. Microbiol. 77, 6433–6440. doi: 10.1128/AEM.00142-11Young, G. R., Smith, D. L., Embleton, N. D., Berrington, J. E., Schwalbe, E.C., Cummings, S. P., et al. (2017). Reducing Viability bias in analysis of gutmicrobiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 7:237. doi: 10.3389/fcimb.2017.00237Conflict of Interest Statement: The authors declare that the research wasconducted in the absence of any commercial or financial relationships that couldbe construed as a potential conflict of interest.Copyright © 2018 Young, Smith, Embleton, Berrington, Schwalbe, Cummings, vander Gast and Lanyon. This is an open-access article distributed under the termsof the Creative Commons Attribution License (CC BY). The use, distribution orreproduction in other forums is permitted, provided the original author(s) and thecopyright owner(s) are credited and that the original publication in this journalis cited, in accordance with accepted academic practice. No use, distribution orreproduction is permitted which does not comply with these terms.Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 2 October 2018 | Volume 8 | Article 374

Northumbria University Research Portal

Northumbria Research LinkCitation:  Young,  Gregory,  Smith,  Darren,  Embleton,  Nicholas  D.,  Berrington,  Janet  Elizabeth, Schwalbe, Ed, Cummings, Stephen Paul,  van der Gast, Christopher J. and Lanyon, Clare (2018) Response: Commentary: Reducing Viability Bias in Analysis of Gut Microbiota in Preterm Infants at  Risk of NEC and Sepsis. Frontiers in Cellular and Infection Microbiology, 8. p. 374. ISSN 2235-2988 Published by: FrontiersURL: https://doi.org/10.3389/fcimb.2018.00374 <https://doi.org/10.3389/fcimb.2018.00374>This version was downloaded from Northumbria Research Link: http://nrl.northumbria.ac.uk/42545/Northumbria University has developed Northumbria Research Link (NRL) to enable users to access the University’s research output. Copyright © and moral rights for items on NRL are retained by the individual author(s) and/or other copyright owners.  Single copies of full items can be reproduced, displayed or performed, and given to third parties in any format or medium for personal research or study, educational, or not-for-profit purposes without prior permission or charge, provided the authors, title and full bibliographic details are given, as well as a hyperlink and/or URL to the original metadata page. The content must not be changed in any way. Full items must not be sold commercially in any  format or medium without formal permission of the copyright holder.  The full policy is available online: http://nrl.northumbria.ac.uk/pol  i cies.html  This  document  may differ  from the  final,  published version of  the research  and has been made available online in accordance with publisher policies. To read and/or cite from the published version of the research, please visit the publisher’s website (a subscription may be required.)                        GENERAL COMMENTARYpublished: 24 October 2018doi: 10.3389/fcimb.2018.00374Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 1 October 2018 | Volume 8 | Article 374Edited by:Pascale Alard,University of Louisville, United StatesReviewed by:Bjoern O. Schroeder,University of Gothenburg, SwedenNatalia Shulzhenko,Oregon State University,United States*Correspondence:Gregory R. Younggregory.young@northumbria.ac.ukClare Lanyonclare.lanyon@northumbria.ac.ukSpecialty section:This article was submitted toMicrobiome in Health and Disease,a section of the journalFrontiers in Cellular and InfectionMicrobiologyReceived: 28 June 2018Accepted: 05 October 2018Published: 24 October 2018Citation:Young GR, Smith DL, Embleton ND,Berrington JE, Schwalbe EC,Cummings SP, van der Gast CJ andLanyon C (2018) Response:Commentary: Reducing Viability Biasin Analysis of Gut Microbiota inPreterm Infants at Risk of NEC andSepsis.Front. Cell. Infect. Microbiol. 8:374.doi: 10.3389/fcimb.2018.00374Response: Commentary: ReducingViability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and SepsisGregory R. Young 1*, Darren L. Smith 1, Nicholas D. Embleton 2,Janet Elizabeth Berrington 2, Edward C. Schwalbe 1, Stephen Paul Cummings 3,Christopher J. van der Gast 4 and Clare Lanyon 1*1 Faculty of Health and Life Sciences, University of Northumbria, Newcastle upon Tyne, United Kingdom, 2NewcastleNeonatal Service, Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, United Kingdom, 3 School ofScience and Engineering, Teesside University, Middlesbrough, United Kingdom, 4 School of Healthcare Science, ManchesterMetropolitan University, Manchester, United KingdomKeywords: PMA, viability analyses, microbiota, methods—analytic, bias, clinical samplesA Commentary onCommentary: Reducing Viability Bias in Analysis of GutMicrobiota in Preterm Infants at Riskof NEC and Sepsisby Agustí, G., and Codony, F. (2018). Front. Cell. Infect. Microbiol. 8:212.doi: 10.3389/fcimb.2018.00212We would like to thank Agustí and Codony (2018), for their interest in our recent manuscript(Young et al., 2017), and valuable comments. We agree that non-viable cell exclusion is animportant consideration, worth making when conducting analyses of microbial communities viatargeted DNA sequencing and amplification approaches. The technique is especially pertinent inenvironments where large volumes of non-viable bacteria are expected, such as in preterm infantstool, where multiple clinical interventions manifest deliberate bacterial killing.Whilst we agree with the general principles with regards to sample collection and handlingdescribed by Agustí and Codony (2018) we maintain that these are not always possible in real-life,clinical patient samples. For example, as patient care is the primary concern in this cohort, samplecollection is convenience based rather than experimentally dictated. This requires samples to bespontaneously collected and stored prior to processing. We concede this increases the likelihoodof loss of anaerobic bacterial viability. We would, however point out that the study by Brusa et al.(1989), highlighted in the commentary defines viability purely as culturability. Studies (Contreraset al., 2011; Nocker et al., 2011) have reported loss of culturability occurs at lower stress levelsthan loss of membrane integrity. Thus, suggesting greater cellular stress may be required to depleteDNA signals from non-viable cells, as determined during PMA-based viability assays. Moreover,we highlight in our manuscript that PMA is a conservative parameter for loss of viability. Wealso propose that, in the specific cohort investigated, conservative non-viable determination ispreferable to over-exaggerated determination or a complete absence of it. This is especially truewhen non-treated samples can be analyzed in parallel.DNA retention on microtube surfaces and subsequent inclusion in the viable communities maywell be an issue in PMA-based viability determination. In our study however, samples were notheat-killed at any point. This is in contrast to the study by Agustí et al. (2017), in which DNAbinding to microtube walls may have occurred prior to PMA-treatment. Furthermore, the methodsYoung et al. Response: Reducing vPCR Biasdescribed by ourselves outline transfer between sterile glass potsduring initial sample collection and several microtubes and wellplates from PMA-addition to incubation and photo-activationand, finally DNA isolation. We propose that these several stepsare sufficient to reduce the impact of DNA retained onmicrotubesurfaces on the assigned live DNA fraction.In addition to the valuable comments made by Agustíand Codony (2018), where the target sequence is <300bp we recommend combination of a nested-PCR approach,comprising initial long fragment pre-amplification, followedby subsequent target amplicon sequencing/quantification, firstdescribed by Luo et al. (2010). This increases the likelihood ofencountering intercalated PMA during the DNA amplificationprocess.The use of PMA as a viability-dye is extremely useful inmicrobial community analysis. As a greater volume of literaturebecomes available regarding it’s use, the technique will no doubtrefine. From our experience with the technique, we proposeseveral important considerations to be made when planning anystudy in which PMA-based non-viable cell exclusion is employed:• Is the sample likely to contain substantial volumes of DNAoriginating from non-viable bacteria?• Are the collection and storage methods likely to impactcommunity viability?• What is the composition of the sample? (Homogenisationmay be required)• What is the optimal concentration of PMA required to ensurenon-viable cell DNA quenching?• Is there potential for extra-cellular DNA carry-over in themethods? (specifically fromDNA retained on plastics surfaces)• What is my light-source for photoactivation (464 nm isexcitation maxima of PMA therefore substantial emission atthis wavelength is required)• Can I employ a nested-PCR approach tomaximize intercalatedPMA impact?• Can I afford to sequence the non-treated sample also?(Particularly useful in clinical environments to ID bacterialcommunities that may have been viable recently)• Can I afford to supplement this technique with anotherto confirm presence/viability? (e.g., FISH, flow cytometry,selective culture)We would further direct anyone interested to a comprehensivereview of the technique by Fittipaldi et al. (2012).AUTHOR CONTRIBUTIONSGY wrote the response. All authors proofread and approved thiscommentary response.REFERENCESAgustí, G., and Codony, F. (2018). Commentary: Reducing viability bias in analysisof gut microbiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 8:212. doi: 10.3389/fcimb.2018.00212Agustí, G., Fittipaldi, M., and Codony, F. (2017). False-positive viabilityPCR results: an association with microtubes. Curr. Microbiol. 74, 377–380.doi: 10.1007/s00284-016-1189-3Brusa, T., Canzi, E., Pacini, N., Zanchi, R., and Ferrari, A. (1989). Oxygen toleranceof anaerobic bacteria isolated from human feces. Curr. Microbiol. 19, 39–43.Contreras, P. J., Urrutia, H., Sossa, K., and Nocker, A. (2011). Effect ofPCR amplicon length on suppressing signals from membrane-compromisedcells by propidium monoazide treatment. J. Microbiol. Methods 87, 89–95.doi: 10.1016/j.mimet.2011.07.016Fittipaldi, M., Nocker, A., and Codony, F. (2012). Progress in understandingpreferential detection of live cells using viability dyes in combinationwith DNA amplification. J. Microbiol. Methods 91, 276–289.doi: 10.1016/j.mimet.2012.08.007Luo, J. F., Lin, W. T., and Guo, Y. (2010). Method to detect onlyviable cells in microbial ecology. Appl. Microbiol. Biotechnol. 86, 377–384.doi: 10.1007/s00253-009-2373-1Nocker, A., Caspers, M., Esveld-Amanatidou, A., van der Vossen, J.,Schuren, F., Montijn, R., et al. (2011). Multiparameter viability assayfor stress profiling applied to the food pathogen Listeria monocytogenesF2365. Appl. Environ. Microbiol. 77, 6433–6440. doi: 10.1128/AEM.00142-11Young, G. R., Smith, D. L., Embleton, N. D., Berrington, J. E., Schwalbe, E.C., Cummings, S. P., et al. (2017). Reducing Viability bias in analysis of gutmicrobiota in preterm infants at risk of NEC and sepsis. Front. Cell Infect.Microbiol. 7:237. doi: 10.3389/fcimb.2017.00237Conflict of Interest Statement: The authors declare that the research wasconducted in the absence of any commercial or financial relationships that couldbe construed as a potential conflict of interest.Copyright © 2018 Young, Smith, Embleton, Berrington, Schwalbe, Cummings, vander Gast and Lanyon. This is an open-access article distributed under the termsof the Creative Commons Attribution License (CC BY). The use, distribution orreproduction in other forums is permitted, provided the original author(s) and thecopyright owner(s) are credited and that the original publication in this journalis cited, in accordance with accepted academic practice. No use, distribution orreproduction is permitted which does not comply with these terms.Frontiers in Cellular and Infection Microbiology | www.frontiersin.org 2 October 2018 | Volume 8 | Article 374

Response: Commentary: Reducing Viability Bias in Analysis of Gut Microbiota in Preterm Infants at Risk of NEC and Sepsis

Abstract

Similar works

Full text

Available Versions

Northumbria Research Link

Teesside University Research

Teeside University's Research Repository

E-space: Manchester Metropolitan University's Research Repository

Northumbria University Research Portal

Teeside University's Research Repository

Northumbria Research Link

Frontiers - Publisher Connector

Directory of Open Access Journals