After statistics reform

In the longer term null hypothesis significance testing (NHST) will disappear because p- values are not informative and not replicable. Should we continue to teach in the future the procedures of then abolished routines (i.e., NHST)? Three arguments are discussed for not teaching NHST in introductory courses in inferential statistics: NHST is too difficult for introductory courses; dichotomous thinking is a cognitive obstacle for interpretation; students can find relevant information in research reports without knowing NHST

Theorie toetsen in kwalitatief onderzoek

Inleiding Kwalitatieve onderzoekers betreuren het over het algemeen dat kwalitatief onderzoek doorgaans een exploratieve rol krijgt toebedeeld. In dit essay betoog ik dat zij zelf deels schuldig zijn aan deze roltoedeling, omdat zij geen overtuigende strategie ontwikkeld hebben om theorie te toetsen met kwalitatieve methoden. Ik wil laten zien dat en hoe de gangbare methodologische noties over het ontwikkelen van een generaliseerbare theorie in kwalitatief onderzoek tekortschieten. Ik zal ook proberen een alternatief daarvoor te formuleren

The Theory-Testing Questionnaire Business Survey Has Passed Its Expiry Date

Response rates in business surveys are low, which implies a huge risk of selection bias. Usually no attempt is made to assess the extent of selection bias and published survey results might, therefore, not be a correct reflection of actual population characteristics. In this paper, it is argued that response rates cannot be improved to a sufficient degree and that assessment of selection bias is difficult in practice. It is concluded that academic questionnaire surveys of businesses should be abandoned and that an alternative way of testing theory in populations must be found

Pecheux's contribution to discourse analysis

'Pecheux (1938-1983) war von den 1960er Jahren bis in die frühen 1980er Jahre einer der Hauptvertreter einer wichtigen und produktiven Phase der französischen Diskursanalyse. Zusammen mit anderen (auch bekannteren) Diskursforschern wie Michel Foucault war er durch die Epistemologie Bachelards und den Poststrukturalismus Althussers beeinflusst. Er teilte mit den Diskursforschern das Interesse an der Diskurstheorie. Aber sein wichtigster Beitrag zur Diskursanalyse besteht in der Entwicklung von Techniken für die empirische Diskursanalyse. Pecheux geht es dabei um den Bruch mit der 'Spontanideologie' der Inhaltsanalyse, wofür er ein formales und potenziell automatisierbares Vorgehen entworfen hat, das er 'Automatische Diskursanalyse' genannt hat. Diese Vorgehensweise sollte eine strukturalistische Diskursanalyse ermöglichen, mit dem die syntaktischen Elemente identifiziert wurden und die Beziehungen zwischen Auswahl und Substituierung der syntaktischen Elemente in einem Textkorpus beschrieben werden konnten. In der Auseinandersetzung mit den Kritiken an der Automatischen Diskursanalyse und in dem Versuch, deren Begrenzungen zu überwinden, hat sich Pecheux von der strukturalistischen Perspektive entfernt und eine reflexivere Theorie des Interdiskurses entwickelt. Damit sollten die ideologischen Konflikte und die veränderlichen Ungleichheiten zwischen Diskursen erklärt werden können. Der vorliegende Beitrag präsentiert die verschiedenen Phasen, die Pecheuxs Arbeiten durchlaufen haben, um ein empirisches Instrument für die Diskursanalyse zu entwickeln.' (Autorenreferat)'Michel Pecheux (1938-1983) was one of the main representatives of a critical and productive episode in French discourse analysis, from the late 1960s to the early 1980s. He shared with more famous contemporaries such as Michel Fouvcaulta background in Bachelardian epistemology and Althusserian 'post-structuralism' and an interest in theories of discourse, but his most important contribution to discourse analysis consisted in the development of tools for conducting empirical studies of discourses. In an attempt to break away from the 'spontaneous ideology' of content analysis, Pecheux developed a formal, potentially automatic instrument, which he called Automatic Discourse Analysis. This instrument could generate a structuralist description of a discourse by identifying and describing relations of selection and substitution of syntactic elements in a corpus of texts representing that discourse. When dealing with criticisms of this approach and attempting to overcome its limitations, Pecheux moved away from structuralism and developed a more reflective theory of 'interdiscourse' in which he tried to account for the ideological struggle and dynamic inequality between discourses. This article discusses the rationale of the different stages Pecheux went through to develop an empirical instrument of discourse analysis.' (author's abstract)

Replication

Replication is conducting a study in another case (or population) in order to assess whether a research finding from previous studies can be confirmed. The aim of replication is to assess the generalizability of a theoretical claim and the “research finding” that is (or is not) confirmed in the replication study is a claim that is derived from the previous studies. Sometimes the term replication is also used for conducting a study in the same case (or population) in order to assess whether the finding from the previous study can be reproduced. To avoid confusion, the latter method of assessing the reliability of a study might better be called duplication. Duplication will not be discussed here

Pattern matching

Pattern matching is comparing two patterns in order to determine whether they match (i.e., that they are the same) or do not match (i.e., that they differ). Pattern matching is the core procedure of theory-testing with cases. Testing consists of matching an “observed pattern” (a pattern of measured values) with an “expected pattern” (a hypothesis), and deciding whether these patterns match (resulting in a confirmation of the hypothesis) or do not match (resulting in a disconfirmation). Essential to pattern matching (as opposed to pattern recognition, which is a procedure by which theory is built) is that the expected pattern is precisely specified before the matching takes place

Theory-Testing With Cases

Theory-testing with cases is ascertaining whether the empirical evidence in a case or in a sample of cases either supports or does not support the theory. There are two methodologies for theory-testing with cases, (a) testing in a single case (‘theory-testing single case study’), and (b) testing in a sample of cases (‘theory-testing sample case study’). The functional form of the proposition that is tested determines which of these two methodologies should be used

Theory-Building With Cases

Theory-building with cases is (a) formulating new propositions that emerge from the empirical evidence in a sample of cases and (b) testing them in the same sample. The main difference with most other forms of generating new propositions (such as analyzing the theoretical literature, brainstorming, etc.) is its empirical character. The main difference with other forms of discovering new propositions in empirical evidence (such as in ‘exploratory’ research) is that only those theoretical formulations are accepted as a result of the theory-building study that are confirmed in a test in the sample from which the proposition was built. It is possible that a proposition about a relationship between two variables emerges from an exploratory single case study (e.g., when both variables have extreme values in that case), but it is not possible to test that new proposition in the same study because this would require a comparison in a sample of cases. The term theory-building study (as distinct from an exploratory study) is used here only for studies in which a proper test of the new proposition has been conducted

Not Willing, Not Able: Causes of Measurement Error in Business Surveys

National statistical institutes must collect accurate data from businesses in a timely and cost-effective way and without causing too much response burden. An adequate design of the information request is critical in achieving this goal. This paper describes the lessons we have learned about the design of business survey questionnaires from a thorough evaluation of the questionnaires of a typical business survey for official statistics, the Structural Business Survey. The paper presents a framework for understanding factors that contribute to missing and inaccurate data and draws a number of conclusions regarding how the design of business surveys can be improved to take these factors into account

The Productivity of the Three-Step Test-Interview (TSTI) Compared to an Expert Review of a Self-administered Questionnaire on Alcohol Consumption

The three-step test interview (TSTI) is a recently developed observation-based procedure for the identification of response problems in self-administered survey questionnaires. The TSTI was applied in field test interviews to a quantity-frequency-variability questionnaire on alcohol consumption. For an assessment of its productivity the results are compared to a previously performed expert review. Most response problems that were identified in the expert review were confirmed in the field test interviews. Additionally, the TSTI identified many unexpected problems, mostly stemming from unanticipated “deviant” drinking patterns and from local normative connotations attached to drinking alcohol. From these findings we conclude that the TSTI is a powerful test tool with a high ecological validity