Objective: To translate and to perform the cross-cultural adaptation of the Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items (MG-QOL15). Method: the original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. the authors analyzed the difficulties and misunderstandings in the application of the questionnaire. Results: the questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores. Conclusion: the Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG)Univ Fed Minas Gerais, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Hosp Clin, Neurol Serv, Belo Horizonte, MG, BrazilUniv Virginia, Charlottesville, VA USAUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Prof Fac Med, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilWeb of Scienc

Araujo, Caroline Martins

Barbosa, Luiz Sergio Mageste

Burns, Ted M.

Gomez, Rodrigo Santiago

Lemos, Stela Maris Aguiar

Mourao, Aline Mansueto

Teixeira, Antonio Lúcio

Arquivos de Neuro-Psiquiatria

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG)Objective: To translate and to perform the cross-cultural adaptation of the Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items (MG-QOL15). Method: the original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. the authors analyzed the difficulties and misunderstandings in the application of the questionnaire. Results: the questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores. Conclusion: the Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG.Univ Fed Minas Gerais, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Hosp Clin, Neurol Serv, Belo Horizonte, MG, BrazilUniv Virginia, Charlottesville, VA USAUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Prof Fac Med, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilWeb of Scienc

Lemos, Stela Maris Aguiar [UNIFESP]

Teixeira, Antonio Lúcio [UNIFESP]

LAReferencia - Red Federada de Repositorios Institucionales de Publicaciones Científicas Latinoamericanas

Brazilian cross-cultural translation and adaptation of the Questionnaire of Life Quality Specific for Myasthenia Gravis-15 items

Objective: To translate and to perform the cross-cultural adaptation of the Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items (MG-QOL15). Method: the original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. the authors analyzed the difficulties and misunderstandings in the application of the questionnaire. Results: the questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores. Conclusion: the Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG.Univ Fed Minas Gerais, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Hosp Clin, Neurol Serv, Belo Horizonte, MG, BrazilUniv Virginia, Charlottesville, VA USAUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Prof Fac Med, Belo Horizonte, MG, BrazilUniv Fed Minas Gerais, Universidade Federal de São Paulo, Dept Fonoaudiol, Belo Horizonte, MG, BrazilWeb of ScienceCoordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)Fundação de Amparo à Pesquisa do Estado de Minas Gerais (FAPEMIG

Repositório Institucional UNIFESP

955DOI: 10.1590/0004-282X20130180ARTICLEBrazilian cross-cultural translation and adaptation of the “Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items”Tradução e adaptação transcultural do “Questionário de Qualidade de Vida Específico para Miastenia Gravis – 15 itens” para o BrasilAline Mansueto Mourão1, Caroline Martins Araújo1, Luiz Sérgio Mageste Barbosa2,  Rodrigo Santiago Gomez2, Ted M. Burns3, Stela Maris Aguiar Lemos4, Antonio Lucio Teixeira5Clínica de Doenças Neuromusculares, Serviço de Neurologia, Hospital das Clínicas, Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil.1Fonoaudióloga, Mestre em Neurociências pela Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil;2Médico Neurologista, Serviço de Neurologia do Hospital das Clínicas da Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil;3Médico Neurologista, University of Virginia, Charlottesville, Virginia, United States;4Fonoaudióloga, Doutora em Ciências pela Universidade Federal de São Paulo, Professora do Departamento de Fonoaudiologia da Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil;5Médico Neurologista e Psiquiatra, Professor da Faculdade de Medicina da Universidade Federal de Minas Gerais, Belo Horizonte MG, Brazil.Correspondence: Antonio Lucio Teixeira; Laboratório Interdisciplinar de Investigação Médica (LIIM) / sala 281 / Faculdade de Medicina,UFMG; Av Professor Alfredo Balena 190; 30130-100 Belo Horizonte MG - Brasil; E-mail: altexr@gmail.comSupport: This study was supported by CAPES, CNPq and Fapemig. AMM received a CAPES scholarship during her Master course.Conflict of interest: There is no conflict of interest to declare.Received 28 January 2013; Received in final form 10 July 2013; Accepted 17 July 2013.ABSTRACTObjective: To translate and to perform the cross-cultural adaptation of the “Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items” (MG-QOL15). Method: The original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. The authors analyzed the difficulties and misunderstandings in the application of the question-naire. Results: The questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores. Conclusion: The Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG. Keywords: myasthenia gravis, quality of life, questionnaire.RESuMoObjetivo: Realizar a tradução e a adaptação transcultural do “Questionário de Qualidade de Vida Específico para Miastenia Gravis – 15 itens” (MG-QOL15). Método: Inicialmente, foi realizada a tradução do instrumento original em inglês para o português. Esta versão foi revisada e retrotraduzida para o inglês. A versão retrotraduzida foi comparada à versão original em inglês, sendo corrigidas as divergências no texto em português. Em um segundo momento, dez pacientes com MG acompanhados na Clínica de Doenças Neuromusculares do Hospital das Clíni-cas da Universidade Federal de Minas Gerais responderam ao questionário. Na aplicação do questionário, os itens mal compreendidos foram analisados pelos autores. Resultados: Os itens 8, 13 e 15 foram considerados de difícil compreensão e modificados na versão final em por-tuguês. A maioria dos pacientes (70%) apresentou uma pontuação total superior a 25, sendo que os itens 3, 8 e 9 apresentaram os escores mais elevados. Conclusão: A versão brasileira do MG-QOL15 é um instrumento promissor para a avaliação de pacientes brasileiros com MG. Palavras-chave: miastenia gravis, qualidade de vida, questionário. Myasthenia Gravis (MG) is an autoimmune neuromus-cular junction disorder that results in fatigue and fluctuating weakness. The incidence of MG varies according to sex, age, and ethnic group1-6, with an annual incidence ranging from 1 to 15 cases per million7,8. The disease is more prevalent in young women when compared to men, with a gender ratio of 956 Arq Neuropsiquiatr 2013;71(12):955-958approximately 2:19. It is also reported to be common in elder-ly men1,2,5. Over the past decades, advances in treatment have changed the prognosis of MG, but spontaneous remission is not frequent and the impact on the quality of life of patients still significant10.The report of the quality of life by the patient may in­fluence therapeutic decisions and enable a better understan­ding of his/her demands, paving the way for an optimal clini-cal support11,12. The more specific the questionnaire, the more likely it includes only items relevant to the disease, avoiding unnecessary statements13.The aim of the current study is to perform the transla­tion and cross-cultural adaptation of an instrument desig-ned to specifically assess the quality of life of MG patients named “Questionnaire of Life Quality Specific for Myas­thenia Gravis - 15 items” (MG-QOL15)12. We expect this instrument may contribute to the evaluation of patients in both clinical and research settings, providing a reliable view on the physical, psychological and social well­being of Brazilian patients with MG.METhodInstrumentThe MG­QOL1512 originated from a list of 60 items co ve-ring specific features of health related­quality of life in MG. The main objective of MG­QOL15 is to provide an effec tive, fast and easy way to evaluate, and to interpret the quality of life of MG patients. Each item is scored from zero to four accor-ding to its frequency, scoring a maximum of 60. The higher the score, the worse it is perceived the quality of life by the patient.Translation, reverse translation and final versionTo translate the MG­QOL15 into Portuguese, two authors performed independently the initial translation of the instru-ment. Both versions were then reviewed by a third author to build a single revised version in Portuguese. This revised ver-sion was back­translated into English by a fourth author. The re-verse-translated version was compared to the original English version, and any discrepancy was corrected in the Portuguese text to keep the original meaning. This final Portuguese ver-sion was used in the testing phase described bellow.Testing phaseWe randomly recruited ten patients over 18 years­old who are followed at the Neuromuscular Diseases Clinic of the University Hospital, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.The following demographic and clinical data were col-lected: age, gender, marital status, educational level, early and current symptoms of MG, and therapeutic strategies. MG was categorized according to the Myasthenia Gravis Foundation of America Clinical Classification (MGFA)1. The MG Composite13 was used to assess the current severity of MG clinical signs and symptoms. In addition, the Medical Outcomes Study 36­Item Short­Form Health Survey (SF­36)14 and the Hospital Anxiety and Depression Scale (HADS)15 were applied. SF­36 is one of the most widely used instru-ments to assess quality of life in clinical and research settings, being composed of eight health related domains (physical functioning, role­physical, bodily pain, general health, vitali-ty, social functioning, role­emotional and mental health). The HADS comprises 14 items, seven to assess anxiety (HAD­A) and seven depression (HAD­D).MG-QOL15 was applied to 10 patients with MG. Patients commented on their difficulties in understanding each state-ment and the misunderstood items were pointed out and subsequently revised by the authors. The objective of this step was to verify whether each item was clearly understood and consistent with its original meaning.The study was approved by the Research Ethics Com­mittee of the Universidade Federal de Minas Gerais (ETIC no. 0476.0.203.000­11). All subjects gave their written informed consent to participate in the study.RESuLTSThe clinical and socio­demographic features of the pa-tients are depicted in Table 1. Nine patients (90%) were fe-male, aged 25 to 57 years­old, and 80% attended at least 8 Table 1. Clinical and demographic features of the patients with myasthenia gravis (n=10) participating in the current study.Data Patients (n) (%)GenderFemale/Male 9/1AgeAge 18 – 50 6 60Age > 50 4 40Length of disease (years) 30First symptomsOcular 5 50Bulbar 3 30Generalized 2 20MGFAIIIA 3 30IIIB 6 60Mean MG composite 13Mean number of crisis 5Mean HADS 6Thymechtomy 7 70TreatmentAzathioprine + Prednisone 6 60Prednisone 4 40MG: myasthenia gravis; MGFA: Myasthenia Gravis Foundation of America Clinical Classification; HADS: Hospital Anxiety and Depression Scale.957Aline Mansueto Mourão ET al. MG-QOL15years of school. Five patients (50%) had ocular symptoms as the initial complaint, and seven (70%) reported bulbar mani-festations as being the main problem.During testing phase, patients had difficulty in interpre­ting and understanding the items 8, 13 and 15. As an exam-ple, regarding the adaptations in statement 8, the expression ‘professional status’ (original version) was replaced by ‘my position at work’ (back­translation version) due to the diffi-culty of all patients to understand the word ‘status’. To ease comprehension, the word “cargo”(meaning job function) was introduced in parenthesis after the expression ‘my position at work’. After entering the word “cargo”, the comprehension became effective. For the analysis and discussion of all state-ments, meetings were held among the authors, reaching the final Portuguese version.Seven patients had a score above 25 in MG­QOL15, and items 3, 5, 7, 8 and 9 showed the highest scores (Table 2). A negative correlation was observed between MG-QOL15 and total SF­36 score (p<0.05; rho=60) due to a very significant correlation between MG-QOL15 and the general health do-main of SF­36 (p<0.001; rho=91). A significant positive corre-lation was also observed between MG­QOL15 and the HADS (p<0.05; rho=63).dISCuSSIon The MG­QOL15 was originally proposed and published in English, being derived from a list of 60 items covering features of health related-quality of life in MG16. The ori­ginal study showed that MG-QOL15 correlated with clini-cal measures of MG, indicating its construct validity. In the current study, we performed the translation and cross­cul-tural adaptation to Portuguese, and the initial validation of the MG-QOL1512.The Brazilian version of MG­QOL15 was well understood and accepted by the patients. To perform a concurrent va-lidation, MG­QOL15 was correlated with the SF­36. SF­36 has been already validated in Portuguese and is one of the most popular instruments to assess the quality of life rela ted to health14. There was a negative correlation between MG­QOL15 and the general health domain of SF­36. As expected, the higher the MG-QOL15 score (i.e. more symptoms and in-capacity attributed to the disease), the worse the perception of the general health status by the patient. This is in line with the results of the other two studies with the MG-QOL1512,17. Indeed the frequency and severity of MG symptoms are well-known factors that influence the patient’s perception of his/her quality of life1­5,12. Therefore, besides translating and adapting MG­QOL15, this study provides preliminary evi-dence for the validity of the instrument. Anxiety disorder and depression are frequent comor-bid conditions in MG, and they can also influence quality of life9,10. The QOLMG­15 correlated positively with the HADS, suggesting that the worse the perception of the quality of life, the higher the likelihood of being depressed or anxious, which is in agreement with the literature9-11. This indicates that it is important to look beyond the physical or somatic symptoms of the disease in order to provide an optimal clini-cal assistance to MG patients.Other disease related­factors may influence quality of life in MG such as the type of the predominant symptom, the number of myasthenic crisis and drug treatment3­5,12. The small number of patients enrolled in this report prevented the assessment of these associations, and further investiga-tion is warranted.Table 2. Scores of individual items from the “Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items”. Statements (original version) Min-max Mean  (SD)1. I’m frustrated by my myasthenia gravis. 0-4 1.8 (1.48)2. I have trouble using my eyes because of my myasthenia gravis. 1-4 2.4 (1.14)3. I have trouble eating because of my myasthenia gravis. 0-4 2.6 (1.67)4. I have limited my social activity because of my myasthenia gravis. 0-3 1.0 (1.41)5. My myasthenia gravis limits my ability to enjoy hobbies and ctivities. 0-4 2.2 (2.04)6. I have trouble meeting the needs of my family because of my myasthenia gravis. 0-0 1.6 (2.19)7. I have to make plans around my myasthenia gravis. 0-4 2.4 (2.19)8. My occupational skills and job status have been negatively affected by my myasthenia gravis. 0-4 3.2 (1.78)9. I have difficulty speaking due to my myasthenia gravis. 2-4 3.0 (1.0)10. I have trouble driving due to my myasthenia gravis. 0-0 0.0 (0.0)11. I am depressed about my myasthenia gravis 0-4 1.4 (1.83)12. I have trouble walking due to my myasthenia gravis 2-4 2.4 (0.89)13. I have trouble getting around public places because of my myasthenia gravis. 0-4 1.6 (2.19)14. I feel overwhelmed by my myasthenia gravis. 0-4 1.4 (1.94)15. I have trouble performing my personal grooming needs because of myasthenia gravis 0-4 1.2 (1.78)Total score: 11-46 28.4 (2.53)SD: standard deviation.958 Arq Neuropsiquiatr 2013;71(12):955-958It is worth mentioning that the authors of the original questionnaire did not recommend any cut off value for the categorization of the quality of life of patients with MG12. They only assumed that the higher the score the worse the quality of life, with a maximum score of 60. Despite limited sample size, the mean value of MG­QOL15 of our patients was similar to the value reported in the original study of the instrument12.In conclusion, this study presents a promising instrument, which was easily applied and interpreted by the physician, to assess the quality of life in MG. It also indicates that the seve-rity of motor symptoms and the presence of psychiatric symp-toms influence the quality of life of Brazilian patients with MG. Further studies involving lar ger samples are necessary to vali-date the MG­QOL15, and to provide more robust data on the factors influencing the quality of life in MG.References1. Jaretzki A, Barohn RJ, Ernstoff RM, et al. Myasthenia Gravis: Recommendations for Clinical Research Standards. Neurology 2000;55:16-23. 2. Alshekhlee A, Miles JD, Katirji B, Preston DC, Kaminski HJ. Incidence and mor tality rates of myasthenia gravis and myasthenic crisis in US hospitals. Neurology 2009; 72:1548-1554. 3. Phillips LH, Torner JC. Epidemiologic evidence for a changing natural histo ry of myasthenia gravis. Neurology 1996;47:1233-1238. 4. Jacobson DL, Gange SJ, Rose NR, Graham NM. Epidemiology and estimated population burden of selected autoimmune diseases in the United States. Clin Immunol Immunopathol 1997;84:223-243.5. Drachman DB. Myasthenia gravis. N Engl J Med 1994;330:1797-1810.6. Grob D, Brunner N, Namba T, Pagala M. Lifetime course of myasthenia gravis. Muscle Nerve 2008;37:141–149.7. Carr AS, Cardwell CR, McCarron PO, McConville J. A systematic review of population based epidemiological studies in Myasthenia Gravis. BMC Neurology 2010;10:1-9. 8. Mao ZF, Mo XA, Qin C, Lai YR, Hartman TC. Course and prognosis of myasthenia gravis: a systematic review. Eur J Neurol 2010;17:913-921. 9. Ybarra MI, Kummer A, Frota ERC, Oliveira JT, Gomez RS, Teixeira AL. Psychiatric disorders in myastenia gravis. Arq Neuropsiquiatr 2011;69:176-179. 10. Suzuki Y, Utsugisawa K, Suzuki S, et al. Factors associated with depressive state in patients with myasthenia gravis: a multicentre cross-sectional study. BMJ Open 2011; 1:1-6. 11. Morris J, Perez D, McNoe B. The use of quality of life data in clinical practice. Qual Life Res 1998;7:85-91. 12. Burns TM, Conaway M, Sanders DB, et al. Construct and concurrent validation of tha MG-QOL15 in the practice setting. Muscle Nerve 2010;41:219-226.13. Burns TM, Conaway M, Sanders DB, et al. The MG Composite. A valid and reliable outcome measure for myasthenia gravis. Neurology 2010;74:1434-1440.14. Ware JE, Sherbourne CD. The MOS 36 Item Short-form Health Survey (SF-36). I Conceptual framework and item selection. Medical Care 1992;30:473-483. 15. Zigmond AS, Snaith RP: The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983;67:361-370. 16. Mullins LL, Carpentier MY, Paul RH, Sanders DB, Muscle Study Group. Disease-specific measure of quality of life for myasthenia gravis. Muscle Nerve 2008;38:947–956.17. Masuda M, Utsugisawa K, Suzuki S, et al. The MG-QOL15 Japanese version: validation and associations with clinical factors. Rinsho Shinkeigaku 2012;52:1043-1046.

Objective To translate and to perform the cross-cultural adaptation of the “Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items” (MG-QOL15). Method The original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. The authors analyzed the difficulties and misunderstandings in the application of the questionnaire. Results The questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores. Conclusion The Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG.</jats:p

Aline Mansueto Mourao

Caroline Martins Araujo

Luiz Sergio Mageste Barbosa

Rodrigo Santiago Gomez

Ted M. Burns

Stela Maris Aguiar Lemos

Antonio Lucio Teixeira

Crossref

Brazilian cross-cultural translation and adaptation of the "Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items"

Objective To translate and to perform the cross-cultural adaptation of the “Questionnaire of Life Quality Specific for Myasthenia Gravis - 15 items” (MG-QOL15).   Method The original English version of the questionnaire was translated into Portuguese. This version was revised and translated back into English. Later, both English versions were compared and the divergences were corrected in the Portuguese text. At a second stage, ten patients with MG followed at the Neuromuscular Diseases Clinic from the University Hospital, Universidade Federal de Minas Gerais answered the questionnaire. The authors analyzed the difficulties and misunderstandings in the application of the questionnaire.   Results The questions 8, 13 and 15 were considered difficult to understand and were modified in the final Portuguese version. Most patients (70%) had a total score above 25, and the statements 3, 8 and 9 showed the highest scores.   Conclusion The Brazilian version of the questionnaire MG-QOL15 seems to be a promising tool for the assessment of Brazilian patients with MG

Directory of Open Access Journals

http://dx.doi.org/10.1590/0004-282X20130180

Brazilian cross-cultural translation and adaptation of the Questionnaire of Life Quality Specific for Myasthenia Gravis-15 items

Abstract

Similar works

Full text

Available Versions

LAReferencia - Red Federada de Repositorios Institucionales de Publicaciones Científicas Latinoamericanas

Repositório Institucional UNIFESP

Crossref

Directory of Open Access Journals