https://egrove.olemiss.edu/dl_proceedings/1089/thumbnail.jp

Kinney, William R.

English

eGrove (Univ. of Mississippi)

University of Mississippi eGrove Proceedings of the University of Kansas Symposium on Auditing Problems Deloitte Collection 1-1-1992 Discussant's response to "Using regression analysis to assist audit judgments in substantive testing" William R. Kinney Follow this and additional works at: https://egrove.olemiss.edu/dl_proceedings  Part of the Accounting Commons, and the Taxation Commons Recommended Citation Auditing Symposium XI: Proceedings of the 1992 Deloitte & Touche/University of Kansas Symposium on Auditing Problems, pp. 136-140; This Article is brought to you for free and open access by the Deloitte Collection at eGrove. It has been accepted for inclusion in Proceedings of the University of Kansas Symposium on Auditing Problems by an authorized administrator of eGrove. For more information, please contact egrove@olemiss.edu. Discussant's Response to "Using Regression Analysis to Assist Audit Judgments in Substantive Testing" William R. Kinney, Jr. University of  Texas at Austin I'm very pleased to have the opportunity to discuss the updated version of STAR. Many of  my papers have addressed problems in analytical review in auditing and particularly regression analysis as a tool. Hearing about STAR's revision was like hearing that an old friend  hadn't died after  all. Thus, it was with some enthusiasm that I accepted Raj's invitation to discuss the updated, interactive version of  STAR with its new bells and whistles. My comments are divided into four  basic areas and are generally favorable toward the software  and the approach. Rather than being overly technical, I will try to stimulate your thinking about STAR, provide some perspective, and assess where we might go from  here. First is a brief  history of  STAR and some STAR-related regression analysis research in auditing. Second is an analysis of what's good about STAR and what's new in the current version, and third will be some areas that need elaboration or additional thought. Finally, there is an overall evaluation of  STAR and its impact. History As many of  you know, STAR, dollar unit sampling, and the AICPA's audit risk model were developed by Ken Stringer of  the former  Deloitte Haskins & Sells. I began my research on regression in auditing after  a 1977 conversation with Jim Loebbecke. We were discussing his research on "combined attributes and variables" sampling which was related to Stringer's "cumulative monetary amount" version of  dollar unit sampling. Jim said that he had based his efforts on the presumption that Stringer was probably right, so Jim took what he knew about CMA and tried to derive what he didn't. I decided that I would try the same approach for  STAR. Using Stringer [1975], I set out to derive what must be in a regression pack-age that could satisfy  the requirements for  a substantive test. My primary prob-lem was determining what Stringer meant by the "most adverse distribution of error." Stringer [1975] gave no clues but said that STAR was designed to be effective  even under that most feared  of  circumstances. I finally  decided that that must mean that the procedure was based on the sum of  estimated misstate-ments, and therefore  it didn't matter how misstatements were distributed. My solution appeared in Kinney [1979]. At a conference  sponsored by DHS, I found  out that I had not guessed correctly about STAR but still had a useful result. Both STAR and Kinney [1979] use an upper precision limit (UPL) on error 136 calculation that is then compared to a monetary precision measure (monetary precision is the magnitude of  "intolerable" misstatement for  the assertion or account under audit). The decision rule is: "If  UPL (Error) > MP measure, then don't rely." There are two basic approaches to relating UPL to MP, as represented here today. They differ  on how the UPL is calculated and how MP is measured. The Price Waterhouse approach is based on Kinney [1979] and calculates an UPL on total error for  the year (in a time series model) using the standard error for the total. The resulting UPL for  the year is then compared to a materiality or tol-erable error measure for  the year. Specifically,  for  Price Waterhouse the calcu-lation is: U P L A P E ) = E + t (AP) x SE(E) ≥ MP where AP denotes the analytical procedure risk level, E is sum of  the estimated monthly misstatements for  the year as a whole, and SE(E) denotes the standard error for  total misstatement for  the year. The STAR approach of  Deloitte and Touche calculates the UPL by month and compares with a monthly MP measure (see Kinney [1979] and Stringer and Stewart [1985]). STAR makes the following  comparison for  all values of  n > 0: UPLAP (e) = e + t N√AP) x SE(e) > MP/n where e is the estimated error for  the month, and SE(e) is the standard error for the month. It can be shown that the STAR comparison has a unique minimum that occurs at generally small values of  n. Thus, the calculation need not be made for  all values of  n > 0. Research since 1975 has found  the following.  Regression analysis is reliable for  the data tested (it has been tested using simulated data and actual data with simulated misstatements). That is, the actual rate of  failure  to indicate material misstatements does not exceed the nominal level [e.g., see Kinney and Salamon, 1982; Knechel, 1988]. Also, the procedure is "fail  safe."  If  a precise model can-not be developed, then the SE is so large that the UPL will exceed the MP mea-sure and the auditor is warned that there is insufficient  evidence for  reliance. As to the success of  field  application, there is circumstantial evidence that STAR may be effective  in locating potential material misstatements. Kinney and McDaniel [1989] show that the rate of  correction of  errors discovered in quarterly statements of  Deloitte Haskins & Sells clients is about twice that for the population of  Big Eight firms  as a whole. While the result may be due to poor clients or to other factors,  these alternative explanations do not seem likely. In regard to auditing standards, STAR and other regression-based procedures are perhaps the only fully  operational and practical means of  complying with the provisions of  SAS No. 56 [AICPA, 1988] for  substantive evidence. Regression provides a basis for  forming  conditional expectations, and a means of  quantifying  precision and relating the result to materiality—two difficult requirements of  SAS No. 56. Ratio analysis and ARIMA models may partially satisfy  these conditions, but generally they suffer  from  excessive standard errors and, thus, are not effective  as audit evidence. Finally, STAR has an advantage under the SAS No. 31 [AICPA, 1980] approach of  assessing risk at the asser-tions level. In contrast to tests of  details which often  test only a single assertion, STAR may be effective  in detecting misstatements in more than one assertion and more than one account. 137 What's good about new STAR, and regression in auditing? In addition to the desirable features  discussed above, the new STAR approach is an improvement because of  the new bells and whistles that guide the auditor in developing an acceptable model of  an account. There is increased emphasis on understanding the client's business and many hints are provided to the auditors on how to better understand the business. First, the model-building exercise itself  requires understanding of  the basic covariation among and between financial  and physical elements. Second, the new diagnostic testing can confirm  or deny the auditor's preliminary understanding. For example, the audi-tor is directed to try to understand why an expected covariation is not observed. Furthermore, the diagnostic approach is extended through consideration of omitted variables. Specifically,  the program tests for  discontinuity (or changed parameters since the base period), autocorrelation, heteroscedasticity, and abnormal residuals. In each case, the auditor is given hints as to what the pattern that violates regression assumptions might mean in terms of  an improperly specified  model. For example, the auditor is directed to consider whether there are patterns over time such as a positive residual followed  by a negative resid-ual. This pattern may indicate a cutoff  problem. Also, it gives guidance about omitted causal or structural variables. Beyond auditing applications, there are several additional uses of  STAR and the auditor's skills in using STAR. A partial list includes: interim reviews of financial  information,  preparation and review of  forward-looking  information, and incorporation in a client's integrated internal control system [COSO 1991]. In fact,  the latter two can be combined in developing client forecasting  systems useful  in formulating  plans or budgets for  the future  and then providing early warning that things aren't working out as planned. The regression model could be used by the client to direct attention to implementation problems (including errors and fraud),  to revise the planning model estimates, or to revise the model itself  by including variables that had been omitted. Such a system should be helpful  in business operation as well as in demonstrating to others that controls are good [Kinney, Maher and Wright, 1990]. Finally, I pose a question for  professors  and practitioners alike (I don't expect an answer now, but I am curious about your thoughts). Given all of  the advantages of  STAR, why hasn't this product and approach been advertised? Regression analysis seems to offer  solutions to several problems of  auditors and offers  considerable benefits  to clients. Why hasn't D & T advertised it? Why don't public accounting firms  in general advertise their leading edge technolo-gies? Why isn't it useful  to advertise audit excellence to clients, financial  state-ment users, audit committees, and prospective employees? What needs elaboration? As to limitations of  STAR and the Stewart and Thornton [1992] paper, I have three general comments. The first  relates to the paper and how it could be made more useful  for  professors  who are interested in giving their students per-spective on practical application of  tools such as regression analysis (Scott and Wallace [1992] provide some insights in this regard). The second concerns the guidance in SAS No. 56 and its incorporation in STAR, and the third involves questions about STAR itself. As a teacher, I would appreciate answers to three questions about the appli-138 cation of  STAR in practice. First, what is the distribution of  the ratio of  the stan-dard error of  the regression to monetary precision? This ratio relates the preci-sion of  the estimate measured in dollars (SE) to allowable imprecision also measured in dollars (MP). I believe that the distribution of  SE/MP would be more useful  than correlation coefficients  since, in the former,  both the numera-tor and denominator are measured in dollars instead of  proportion of  variation explained. It would be especially useful  to see the distribution of  SE/MP across accounts, clients, and industries. Second, what is the mix of  internal, external, non-financial,  and indicator variables across accounts, clients and industries? This knowledge would allow professors  to assess the importance of  internal variables in designing analytical procedure research projects and to see how sophisticated the practice models are. Realistic classroom examples could then be developed. Third, what have STAR's costs been— training costs, implemen-tation costs, and the costs of  making the transition for  staff  auditors from Deloitte & Touche? As one interested in auditing standards, I am torn between the use of  regres-sion diagnostics to better understand the client's business vs. signalling possible misstatements. This same concern was expressed in the recent Expectations Gap Roundtable [Blocher and Loebbecke, 1992], and in a presentation at this confer-ence two years ago [Kinney and Haynes, 1991]. The approach taken in STAR is consistent with SAS No. 56, para. 21, which focuses  the auditor on explaining unexpected results in terms of  non-error causes. Basically, para. 21 says that if UPL exceeds MP, then the auditor should first  consider whether the model is wrong (auditor mistake), then ask management for  an explanation. If  both of these fail,  then the auditor is directed to consider accounting misstatement as the possible cause. Since behavioral research has shown that auditors may focus unduly on nonerror causes identified  by either of  the first  two foils,  they may underweight the probability of  error or fraud.  This problem is not unique to STAR, but STAR's focus  on understanding the client's business may increase the tendency. Turning now to STAR itself,  I note two issues that provide food  for  thought. First, stepwise regression includes the variable(s) that best fit  the data during the base period. Each period, by chance, certain variables will exhibit particularly good fits  in explaining the dependent variable even when there is a truly causal variable available. The best fitting  variables in the base period may not exhibit much explanatory power in the prediction (audit) period. Thus, STAR may to some degree select randomly irrelevant variables. An alternative is a theoretical basis for  the model in each application. The model (or perhaps an industry model) might be developed once and updated through appropriate consideration of  omitted variables. A theoretically-based model would require more skill in model building, but may be more precise in the long run. Second, a reading of  SAS No. 31 in conjunction with SAS No. 56 raises an evidence integration issue that has not been adequately addressed. The issue is how to combine evidence across assertions and across accounts. There are at least two levels of  analysis for  integrating results using regression as an analyti-cal procedure. Within an application, care must be taken to account for  the lack of  independence due to use of  the same regression equation to estimate multiple components of  an account balance [see Kinney, 1979]. Across applications, there is the problem of  how to combine results. Since STAR uses an investiga-tion rule that considers the "most adverse distribution of  error," it may provide 139 protection in signalling possible misstatement when a material amount of  mis-statements is spread over several accounts that are audited using STAR. Overall STAR was a very useful  tool in 1975. It has led to considerable research and to SAS No. 56. It holds much promise in the 1990s as a tool for  substantive test-ing in auditing as well as many other areas of  client services and direct use by the clients themselves. The new "bells and whistles" should add value through better understanding of  clients' businesses, and increased value as a substantive audit tool. Furthermore, its potential as an analytical tool seems even greater now than it did in 1975. I'm delighted that the technique has survived the merger, and competitive and cost pressures. I hope that other firms  will consider its use as an audit and business tool. References American Institute of  Certified  Public Accountants (AICPA), Statement  on Auditing  Standards No.  31: Evidential  Matter  (August 1980). American Institute of  Certified  Public Accountants (AICPA), Statement  on Auditing  Standards No.  56: Analytical  Procedures  (April 1988). Blocher, E. and J. K. Loebbecke, "Research in Analytical Procedures: Implications for  Establishing and Implementing Auditing Standards," Expectation  Gap Roundtable  (AICPA, forthcoming 1992). Committee of  Sponsoring Organizations of  the Treadway Commission, Internal  Control-Integrated Framework  (COSO, February 1992). Kinney, W.R., Jr., "Integrating Audit Tests: Regression Analysis and Partitioned Dollar Unit Sampling," Journal  of  Accounting Research (Autumn 1979), pp. 456-475. Kinney, W.R., Jr. and C. Haynes, "Analytical Procedure Results as Substantive Evidence," Proceedings  of  the 1990 Deloitte  Touche/University  of  Kansas  Symposium on Auditing Problems  (University of  Kansas, 1990), pp. 83-103. Kinney, W.R., Jr. and L. S. McDaniel, "Characteristics of  Firms Correcting Previously Reported Quarterly Earnings," Journal  of  Accounting and  Economics (February 1989), pp. 71-93. Kinney, W.R., Jr., M. W. Maher and D. A. Wright, "Assertions-based Standards for  Integrated Internal Control," Accounting Horizons  (December 1990), pp. 1-8. Kinney, W.R., Jr. and G. L. Salamon, "Regression Analysis in Auditing: A Comparison of Alternative Investigation Rules," Journal  of  Accounting Research (Autumn 1982), pp. 350-366. Knechel, W.R., "The Effectiveness  of  Statistical Analytical Review as a Substantive Auditing Procedure," Accounting Review (January 1988), pp. 74-95. Scott, D. and W. A. Wallace, "Practical Experiences with Regression Analysis," Proceedings  of  the 1992 Deloitte  Touche/University  of  Kansas  Symposium on Auditing  Problems,  University of Kansas (1992), pp. 141-169. Stewart, T. and A. Thornton, "Using Regression Analysis to Assist Audit Judgments in Substantive Testing," Proceedings  of  the 1992 Deloitte  Touche/University  of  Kansas  Symposium on Auditing  Problems,  University of  Kansas (1992), pp. 113-135. Stringer, K.W., "A Statistical Technique for  Analytical Review," Journal  of  Accounting Research Supplement  (1975), pp. 1-9. Stringer, K.W. and T. Stewart, Statistical  Techniques  for  Analytical  Review in Auditing,  New York: Ronald Press, John Wiley & Sons, (1986). 140 

Discussant\u27s response to  Using regression analysis to assist audit judgments in substantive testing

https://egrove.olemiss.edu/cgi/viewcontent.cgi?article=1089&amp;context=dl_proceedings

Discussant\u27s response to Using regression analysis to assist audit judgments in substantive testing

Abstract

Similar works

Full text

Available Versions

eGrove (Univ. of Mississippi)