A Note On The Reliability Of Accounting Lobbying Studies

Studies that are based on content analyses of portions of the Financial Accounting Standards Board “Public Record” have appeared regularly in accounting and business literature since 1978.  Inter-rater reliability is a crucial determinant of the validity of content analyses, yet none of the studies based on content analysis of the “Public Record” report any measure of inter-rater reliability.  This study provides some evidence of the degree of inter-rater reliability of these studies.  Krippendorff’s coefficient of agreement, a measure of inter-rater reliability is derived for each of eight issues from four raters performing a content analysis of respondent letters in the Public Record volume, Exposure Draft:  Accounting for Certain Acquisitions of Banking or Thrift Institutions.”  In general, the coefficients indicated that extreme caution should be exercised in making inferences from studies based on content analyses of the Financial Accounting Standards Board “Public Record.