Other-Settings Generalizability in IS Research

This paper presents a simple conceptualization of generalization, called other-settings generalization, that is valid for any IS researcher who claims that his or her results have applicability beyond the sample where data were collected. An other-settings generalization is the researcher’s act of arguing, based on the representativeness of the sample, that there is a reasonable expectation that a knowledge claim already believed to be true in one or more settings is also true in other clearly defined settings. Features associated with this conceptualization of generalization include (a) recognition that all human knowledge is bounded, (b) recognition that all knowledge claims—including generalizations—are subject to revision, (c) an ontological assumption that objective reality exists, (d) a scientific-realist definition of truth, and (e) identification of the following three essential characteristics of sound other-settings generalizations: (1) the researcher must clearly define the larger set of things to which the generalization applies; (2) the justification for making other-settings generalizations ultimately depends on the representativeness of the sample, not statistical inference; (3) representativeness is judged by comparing key characteristics of the proposition being generalized in the sample and target population. The paper concludes with the recommendation that future empirical IS research should include an explicit discussion of the other-settings generalizability of research findings

Lost in Translation: Piloting a Novel Framework to Assess the Challenges in Translating Scientific Uncertainty From Empirical Findings to WHO Policy Statements.

BACKGROUND:Calls for evidence-informed public health policy, with implicit promises of greater program effectiveness, have intensified recently. The methods to produce such policies are not self-evident, requiring a conciliation of values and norms between policy-makers and evidence producers. In particular, the translation of uncertainty from empirical research findings, particularly issues of statistical variability and generalizability, is a persistent challenge because of the incremental nature of research and the iterative cycle of advancing knowledge and implementation. This paper aims to assess how the concept of uncertainty is considered and acknowledged in World Health Organization (WHO) policy recommendations and guidelines. METHODS:We selected four WHO policy statements published between 2008-2013 regarding maternal and child nutrient supplementation, infant feeding, heat action plans, and malaria control to represent topics with a spectrum of available evidence bases. Each of these four statements was analyzed using a novel framework to assess the treatment of statistical variability and generalizability. RESULTS:WHO currently provides substantial guidance on addressing statistical variability through GRADE (Grading of Recommendations Assessment, Development, and Evaluation) ratings for precision and consistency in their guideline documents. Accordingly, our analysis showed that policy-informing questions were addressed by systematic reviews and representations of statistical variability (eg, with numeric confidence intervals). In contrast, the presentation of contextual or "background" evidence regarding etiology or disease burden showed little consideration for this variability. Moreover, generalizability or "indirectness" was uniformly neglected, with little explicit consideration of study settings or subgroups. CONCLUSION:In this paper, we found that non-uniform treatment of statistical variability and generalizability factors that may contribute to uncertainty regarding recommendations were neglected, including the state of evidence informing background questions (prevalence, mechanisms, or burden or distributions of health problems) and little assessment of generalizability, alternate interventions, and additional outcomes not captured by systematic review. These other factors often form a basis for providing policy recommendations, particularly in the absence of a strong evidence base for intervention effects. Consequently, they should also be subject to stringent and systematic evaluation criteria. We suggest that more effort is needed to systematically acknowledge (1) when evidence is missing, conflicting, or equivocal, (2) what normative considerations were also employed, and (3) how additional evidence may be accrued

Knowledge and information needs of informal caregivers in palliative care : a qualitative systematic review

Objectives: To review current understanding of the knowledge and information needs of informal caregivers in palliative settings. Data sources: Seven electronic databases were searched for the period January 1994–November 2006: Medline, CINAHL, PsychINFO, Embase, Ovid, Zetoc and Pubmed using a meta-search engine (Metalib®). Key journals and reference lists of selected papers were hand searched. Review methods: Included studies were peer-reviewed journal articles presenting original research. Given a variety of approaches to palliative care research, a validated systematic review methodology for assessing disparate evidence was used in order to assign scores to different aspects of each study (introduction and aims, method and data, sampling, data analysis, ethics and bias, findings/results, transferability/generalizability, implications and usefulness). Analysis was assisted by abstraction of key details of study into a table. Results: Thirty-four studies were included from eight different countries. The evidence was strongest in relation to pain management, where inadequacies in caregiver knowledge and the importance of education were emphasized. The significance of effective communication and information sharing between patient, caregiver and service provider was also emphasized. The evidence for other caregiver knowledge and information needs, for example in relation to welfare and social support was weaker. There was limited literature on non-cancer conditions and the care-giving information needs of black and minority ethnic populations. Overall, the evidence base was predominantly descriptive and dominated by small-scale studies, limiting generalizability. Conclusions: As palliative care shifts into patients’ homes, a more rigorously researched evidence base devoted to understanding caregivers knowledge and information needs is required. Research design needs to move beyond the current focus on dyads to incorporate the complex, three-way interactions between patients, service providers and caregivers in end-of-life care setting

A new approach for quantifying the effects of response elaboration training

Response Elaboration Training (RET; Kearns, 1985, 1986) is a content-driven treatment for individuals with aphasia. Several studies by Kearns and colleagues indicate RET’s unique “loose training” paradigm increases the amount of verbal information produced by individuals with non-fluent Broca’s aphasia in response to picture stimuli (Gaddie, Kearns, & Yedor, 1991; Kearns, 1985, 1986; Kearns & Scher, 1989; Kearns & Yedor, 1991; Nessler, 2009), and that in some cases, RET effects generalize to other speaking partners, stimuli, and settings (Bennett, Wambaugh, & Nesslar, 2005; Gaddie et al., 1991; Kearns & Yedor, 1991). In most RET research, training effects are quantified by having the participant describe the same pictures used in RET. This provision of pictorial support limits conclusions that can be drawn about the effectiveness of RET and its generalizability. The present study examined the effectiveness and generalizability of RET on speaking performance of individuals with non-fluent Broca’s aphasia with a sentence production task (SPT) that did not provide picture support

Organizational diversity: making the case for contextual interpretivism

Purpose The purpose of this paper is to make a case for contextual interpretivism in managing diversity in organizational settings, specifically in its bearing on internal communication, going against the dominating functionalistic stance of venerated and ubiquitous approaches. Design/methodology/approach Qualitative and quantitative methodologies were employed to explore the potential of contextual interpretivism within the mining and construction industries of South Africa, due to the fecund diversity context of its employee population. Findings This paper points to the enriched understanding that could result from following a contextual interpretivistic approach to internal communication for diversity management, and in so doing discusses the ways in which this could take hold in organizations through the application of germane theoretical assertions of revered internal organizational communication literature, specifically the excellence theory and communication satisfaction. Research limitations/implications The main limitation to this research is the restricted generalizability of its empirical research. Further research is required for the exploration of the central premise in other organizational contexts. Practical implications The paper provides insights into the ways in which organizations could approach its diversity management so as to speak to more than just the functional aspects thereof, and rather to the importance of nurturing an understanding of employees’ interpretation of the organization’s diversity endeavors. Originality/value The implications of applying a new approach to diversity management in organizational settings is discussed and argued, offering an empirical application thereof, which gives way to practical, data-driven recommendations for use in organizational settings

The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network

Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability

The Psychological Science Accelerator: Advancing Psychology Through a Distributed Collaborative Network

Source at https://doi.org/10.1177/2515245918797607.Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability

The Psychological Science Accelerator: Advancing Psychology through a Distributed Collaborative Network

Concerns have been growing about the veracity of psychological research. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions, or attempt to replicate prior research, in large, diverse samples. The PSA\u27s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time-limited), efficient (in terms of re-using structures and principles for different projects), decentralized, diverse (in terms of participants and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside of the network). The PSA and other approaches to crowdsourced psychological science will advance our understanding of mental processes and behaviors by enabling rigorous research and systematically examining its generalizability