Inter-rater reliability of treatment fidelity and therapeutic alliance measures for psychological therapies for anxiety in young people with autism spectrum disorders

Objectives: This article presents work undertaken to establish inter-rater reliability for a measure of treatment fidelity and a measure of therapeutic alliance for therapies for anxiety for young people with autism spectrum disorders. The discussion and decision-making processes behind achieving consensus of raters are rarely published. Margolin et al. (1998) have highlighted this issue and called for researchers to communicate the details of their observational and rating procedures. This article is a response to their call for greater transparency so that these methods are readily accessible for comparison with other studies. Methods: Participants were young people with autism spectrum disorders receiving treatment for anxiety, clinical staff treating these young people and the independent raters assessing the treatment sessions. We report: (i) the processes involved in establishing inter-rater reliability for two instruments, (ii) the results obtained with a sample of young people with autism spectrum disorders using these instruments. Results and conclusions: Results demonstrate that it was possible to attain satisfactory inter-rater reliability with each of these two instruments with a client group with autism spectrum disorders, even though the instruments were originally designed for typically-developing populations

Inter-rater reliability of treatment fidelity and therapeutic alliance measures for psychological therapies for anxiety in young people with autism spectrum disorders

Objectives: This article presents work undertaken to establish inter-rater reliability for a measure of treatment fidelity and a measure of therapeutic alliance for therapies for anxiety for young people with autism spectrum disorders. The discussion and decision-making processes behind achieving consensus of raters are rarely published. Margolin et al. (1998) have highlighted this issue and called for researchers to communicate the details of their observational and rating procedures. This article is a response to their call for greater transparency so that these methods are readily accessible for comparison with other studies. Methods: Participants were young people with autism spectrum disorders receiving treatment for anxiety, clinical staff treating these young people and the independent raters assessing the treatment sessions. We report: (i) the processes involved in establishing inter-rater reliability for two instruments, (ii) the results obtained with a sample of young people with autism spectrum disorders using these instruments. Results and conclusions: Results demonstrate that it was possible to attain satisfactory inter-rater reliability with each of these two instruments with a client group with autism spectrum disorders, even though the instruments were originally designed for typically-developing populations

Standardized or simple effect size: what should be reported?

It is regarded as best practice for psychologists to report effect size when disseminating quantitative research findings. Reporting of effect size in the psychological literature is patchy – though this may be changing – and when reported it is far from clear that appropriate effect size statistics are employed. This paper considers the practice of reporting point estimates of standardized effect size and explores factors such as reliability, range restriction and differences in design that distort standardized effect size unless suitable corrections are employed. For most purposes simple (unstandardized) effect size is more robust and versatile than standardized effect size. Guidelines for deciding what effect size metric to use and how to report it are outlined. Foremost among these are: i) a preference for simple effect size over standardized effect size, and ii) the use of confidence intervals to indicate a plausible range of values the effect might take. Deciding on the appropriate effect size statistic to report always requires careful thought and should be influenced by the goals of the researcher, the context of the research and the potential needs of readers

Techniques for the Fast Simulation of Models of Highly dependable Systems

With the ever-increasing complexity and requirements of highly dependable systems, their evaluation during design and operation is becoming more crucial. Realistic models of such systems are often not amenable to analysis using conventional analytic or numerical methods. Therefore, analysts and designers turn to simulation to evaluate these models. However, accurate estimation of dependability measures of these models requires that the simulation frequently observes system failures, which are rare events in highly dependable systems. This renders ordinary Simulation impractical for evaluating such systems. To overcome this problem, simulation techniques based on importance sampling have been developed, and are very effective in certain settings. When importance sampling works well, simulation run lengths can be reduced by several orders of magnitude when estimating transient as well as steady-state dependability measures. This paper reviews some of the importance-sampling techniques that have been developed in recent years to estimate dependability measures efficiently in Markov and nonMarkov models of highly dependable system

Quantitative research

This article describes the basic tenets of quantitative research. The concepts of dependent and independent variables are addressed and the concept of measurement and its associated issues, such as error, reliability and validity, are explored. Experiments and surveys – the principal research designs in quantitative research – are described and key features explained. The importance of the double-blind randomised controlled trial is emphasised, alongside the importance of longitudinal surveys, as opposed to cross-sectional surveys. Essential features of data storage are covered, with an emphasis on safe, anonymous storage. Finally, the article explores the analysis of quantitative data, considering what may be analysed and the main uses of statistics in analysis

Measures Matter: Scales for Adaptation, Cultural Distance, and Acculturation Orientation Revisited

Building upon existing measures, four new brief acculturation scales are presented, measuring sociocultural adaptation, psychological adaptation, perceived cultural distance, and acculturation orientation. Following good scale reliability in initial samples, the English scales were translated into nine different languages (Chinese, French, German, Italian, Japanese, Portuguese, Spanish, Thai, and Turkish). The translated scales were administered to a large sample of sojourners (N = 1,929), demonstrating good reliability and adequate structural equivalence across languages. In line with existing theory, sociocultural adaptation and psychological adaptation were positively correlated, and showed a negative association with perceived cultural distance. General measures of well-being were correlated with adaptation and distance, with better adaptation relating to higher well-being, and more distance relating to lower well-being. Acculturation orientation toward the home and host culture were measured separately and a weak negative correlation was found between the two, supporting their independence. Arguing against dichotomization, these subscales were analyzed as continuous variables. Regression analysis showed sojourners to be better adapted, if they were oriented more toward the host culture and less toward the home culture. These new scales are proposed as alternatives to existing measures

The Five Accomplishments: A Framework for Obtaining Customer Feedback in a Health Service Community Learning Disability Team

The usefulness of using the philosophy of normalisation and, in particular, O`Brien`s (1992) Five Accomplishments as a basis for evaluating client satisfaction with a community based health service learning disabilities team was examined. A survey of a group of people with mild learning disabilities took place using the Five Accomplishments as a framework for a semi-structured interview. Areas of client satisfaction and dissatisfaction are discussed in the context of the use of a value-based means of obtaining feedback from clients. Limitations of interviewing techniques and specifically those used in this study are reviewed

Meta-models for structural reliability and uncertainty quantification

A meta-model (or a surrogate model) is the modern name for what was traditionally called a response surface. It is intended to mimic the behaviour of a computational model M (e.g. a finite element model in mechanics) while being inexpensive to evaluate, in contrast to the original model which may take hours or even days of computer processing time. In this paper various types of meta-models that have been used in the last decade in the context of structural reliability are reviewed. More specifically classical polynomial response surfaces, polynomial chaos expansions and kriging are addressed. It is shown how the need for error estimates and adaptivity in their construction has brought this type of approaches to a high level of efficiency. A new technique that solves the problem of the potential biasedness in the estimation of a probability of failure through the use of meta-models is finally presented.Comment: Keynote lecture Fifth Asian-Pacific Symposium on Structural Reliability and its Applications (5th APSSRA) May 2012, Singapor