Reflections on reproducible research

Reproducibility in language documentation and description means that the analysis given in descriptive publication is presented in a way that allows the reader to access the data on which the claims are based, to verify the analysis for themself. Linguists, including Himmelmann, have long pointed to the centrality of documentation data to linguistic description. Over the twenty years since Himmelmann’s 1998 paper we have seen a growth in digital archiving, and the rise of the Open Access movement. Although there is good infrastructure in place to make reproducible research possible, few descriptive publications clearly link to underlying data, and very little documentation data is publicly accessible. We discuss some of the institutional roadblocks to reproducibility, including a lack of support for the development of published primary data. We also look at what work on language documentation and description can learn from the recent replication crisis in psychology.National Foreign Language Resource Cente

A survey of current reproducibility practices in linguistics publications

Poster: In order to move forward toward reproducible research in linguistics, we first need to know where we are now with regard to our practices for methodological clarity and data citation in publications. In this poster we share the results of a study of over 370 journal articles, dissertations, and grammars, which is taken as a sample of current practices in the field. The publications all come from a ten-year span. The journals were selected for broad coverage. Grammars included published grammars and dissertations written as grammars, with broad geographic coverage, both in terms of subject language and publisher or university.These publications are critiqued on the basis of transparency of data source, data collection methods, analysis, and storage. While we find examples of transparent reporting, most of the surveyed research does not include key metadata, methodological information, or citations that are resolvable to the data on which the analyses are based.In order to move forward toward reproducible research in linguistics, we first need to know where we are now with regard to our practices for methodological clarity and data citation in publications. In this poster we share the results of a study of over 370 journal articles, dissertations, and grammars, which is taken as a sample of current practices in the field. The publications all come from a ten-year span. The journals were selected for broad coverage. Grammars included published grammars and dissertations written as grammars, with broad geographic coverage, both in terms of subject language and publisher or university.These publications are critiqued on the basis of transparency of data source, data collection methods, analysis, and storage. While we find examples of transparent reporting, most of the surveyed research does not include key metadata, methodological information, or citations that are resolvable to the data on which the analyses are based.This material is based upon work supported by the National Science Foundation under grant SMA-1447886

Effect of tin fluoride content on the structure and properties of phosphate glass – polyamide 11 hybrids

This research investigates tin-fluoride phosphate glasses as liquid-phase fillers in polyamide 11. Optimizing the tin-fluoride content was shown to reduce the glass-transition temperature of the glass down to 106oC and enable both constituents of the composite to be fluid during processing. During extrusion, the phosphate glass was elongated into threads and ultimately broken up into droplets. The interaction parameter between the phosphate and polyamide was found to be - 0.012 due to molecular interactions and enhanced solubility of the ultrafine glass particles. All hybrids showed limited ductility due to the inability of the pinned polymer chains to be drawn into an expanding stable neck. However, there were major increases in both the Young's modulus and flexural modulus owing to the strengthening of molecular bonds by the phosphate-polyamide interactions

Data Citation in Linguistic Typology: Working Towards a Data Citation Standard in Linguistics

Poster presented at the 12th Conferenced of the Association for Linguistic Typology (ALT), Canberra, Australia, December 12-14 201

Putting practice into words: The state of data and methods transparency in grammatical descriptions

Language documentation and description are closely related practices, often performed as part of the same fieldwork project on an un(der)-studied language. Research trends in recent decades have seen a great volume of publishing in regards to the methods of language documentation, however, it is not clear that linguists' awareness of the importance of robust data-collection methods is translating into transparency about those methods or data citation in resultant publications. We analyze 50 dissertations and 50 grammars from a ten-year span (2003-2012) to assess the current state of the field. Publications are critiqued on the basis of transparency of data collection methods, analysis and storage, as well as citation of primary data. While we found examples of transparent reporting in these areas, much of the surveyed research does not include key information about methodology or data. We acknowledge that descriptive linguists often practice good methodology in data collection, but as a field we need to build a better culture with regard to making this clear in research writing. Thus we conclude with suggested benchmarks for the kind of information we believe is vital for creating a rich and useful research methodology in both long and short format descriptive research writing.National Foreign Language Resource Cente

Data Citation in Linguistics: Looking forward to new standards

Paper presented at the 20th International Congress of Linguists, Capetown, South Africa, July 2-6, 2018

Research Data Alliance et les données linguistiques

Presentation from the Journées FLOral-PFC: Dialectologie et Phonologie de Corpus conference, November 23-25 2017, Pari

Putting practice into words: Fieldwork methodology in grammatical descriptions

Language documentation and description are closely related tasks, often performed as part of the same fieldwork project on an un(der)-studied language. However, since Himmelmann (1998) we have been encouraged to consider that documentation and description are methodologically different, and that data collected with documentary methods can enable verification of descriptive claims based upon them. The last decade has seen a surge in the literature on good fieldwork methodology, including Gippert, Himmelmann & Mosel (2006), Crowley (2007), Bowern (2008), Chelliah & De Reuse (2011) and Thieberger (2012). The result is that linguists are more aware of good methodological practices for data collection than ever. These include attention to metadata about speaker demographics, setting, linguistic and discourse types; information about tools, equipment, and stimuli; a description of the fieldwork conditions including time spent among speakers; and a description of archiving practices and locatability of data. However, it is not clear that linguists' awareness of the importance of robust data-collection methods is translating into transparency about those methods in resultant publications. Clear methodological description is a hallmark of reproducible and reliable scientific research (Author 2014, Authors In Prep), but documentary and descriptive linguists rarely receive clear advice on how to discuss the methods they use. In this paper we present a survey of 50 published grammars, 50 grammar-based dissertations and 200+ journal articles with regard to how explicitly authors discuss their data collection methods, and what kinds of information they include. The publications surveyed were selected from a ten-year period beginning five years after Himmelmann 1998 encouraged the use of language documentation to provide verification for language description; journal articles come from nine journals selected for breadth of geography, linguistic subfield, and theoretical approach. We find that while there are some examples of strong methodologically-driven writing, the majority of authors do not include key documentary metadata or methodological information. The result is that it is often difficult or impossible to verify or reproduce descriptive linguistic claims, making descriptive linguistics one of the few social sciences to not require researchers to back up claims with an explicit statement of methodology. We acknowledge that descriptive linguists often practice good methodology in data collection, but need encouragement to make this clear in their writing. Thus we conclude with clear benchmarks for the kind of information we believe is vital for creating a rich and useful research methodology in both long and short format descriptive research writing. References Author. 2014. [Title omitted for anonymity]. In Amanda Harris, Nick Thieberger & Linda Barwick (eds.), Research, records, and responsibility: Ten years of the Pacific and Regional Archive for Digital Sources in Endangered Cultures. Sydney: University of Sydney Press. Authors. In prep. Citation and transparency in descriptive linguistics. Bowern, Claire. 2008. Linguistic fieldwork: a practical guide. Basingstoke [England] ; New York: Palgrave Macmillan. Chelliah, Shobhana L., and Willem J. De Reuse. 2011. Handbook of descriptive linguistic fieldwork. London: Springer. Crowley, Terry. 2007. Field linguistics: a beginner's guide. Edited by Nicholas Thieberger, Oxford linguistics. Oxford ; New York: Oxford University Press. Gippert, Jost, Nikolaus P. Himmelmann, & Ulrike Mosel. 2006. Essentials of language documentation. Berlin: Mouton de Gruyter. Himmelmann, Nikolaus P. 1998. "Documentary and descriptive linguistics." Linguistics no. 36:161–195. Thieberger, Nicholas. 2012. The Oxford handbook of linguistic fieldwork. Oxford: Oxford University Press

Network information and connected correlations

Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease in entropy for the joint distribution of $N$ variables relative to the maximum entropy allowed by all the observed $N-1$ variable distributions. We calculate the ``connected information'' terms for several examples, and show that it also enables the decomposition of the information that is carried by a population of elements about an outside source.Comment: 4 pages, 3 figure