Crompton, John L

Jeong, Ji Youn

English

ScholarWorks@UMass Amherst

University of Massachusetts AmherstScholarWorks@UMass AmherstTravel and Tourism Research Association:Advancing Tourism Research Globally 2015 ttra International ConferenceThe Potential Influence of Researchers’ “Hidden”Procedure Decisions on Estimates of VisitorSpending and Economic ImpactJi Youn JeongTexas A & M University - College StationJohn L. CromptonTexas A & M University - College StationFollow this and additional works at: https://scholarworks.umass.edu/ttraThis Event is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Travel and TourismResearch Association: Advancing Tourism Research Globally by an authorized administrator of ScholarWorks@UMass Amherst. For moreinformation, please contact scholarworks@library.umass.edu.Jeong, Ji Youn and Crompton, John L., "The Potential Influence of Researchers’ “Hidden” Procedure Decisions on Estimates of VisitorSpending and Economic Impact" (2015). Travel and Tourism Research Association: Advancing Tourism Research Globally. 30.https://scholarworks.umass.edu/ttra/ttra2015/Academic_Papers_Oral/30The Potential Influence of Researchers’ “Hidden” Procedure Decisions on Estimates of Visitor Spending and Economic Impact  Economic impact analysis is an inexact process and the output numbers should be regarded as a “best guess,” rather than as being inviolably accurate. Since there are multiple points in an analysis where underlying assumptions are made or at which alternative procedures can be adopted, there is a temptation to embrace inappropriate procedures and assumptions to generate high economic numbers that will support the sponsors’ position.  This paper identified, discussed and empirically tested the potential impacts of five practices on estimates of economic impact: aggregating per person per day expenditures by group weighting rather than by individual weighting; omitting a measure of the extent to which visiting a park was the exclusive trip purpose; retaining outlier values; aggregating different visitor segments; and using gross sales rather than output measures. The five practices discussed, for the most part are “hidden.” That is, they are internal process and procedural decisions made by researchers that most lay audiences are likely to consider esoteric, arcane and mundane, and to view with disinterest. They are frequently invisible, because they are rarely mentioned in reports. Nevertheless, they have potentially large hyperbolic impacts on visitors’ expenditure and economic impact estimates. Hence, awareness of the trade-offs inherent in selecting alternative procedures is critical for any meaningful evaluation of the legitimacy of the “best guess” outcome estimates.  Methods  Data were collected at nine state parks in Texas over a four and a half month period. Surveyors intercepted visitors at the park entrance gates and campgrounds. They were convenience samples. The leader of each group of visitors was asked to report the group’s expenditures in the local community which was defined as “within a 20 mile radius of the park.” The number of usable questionnaires obtained totaled 5,634, ranging from 382 at Daingerfield and 390 at Lake Corpus Christi, to 1,186 and 1,286 at Enchanted Rock and Garner, respectively.   Results Aggregating Per Person Per Day Expenditures by Group Weighting Rather Than by Individual Weighting  The results obtained from applying each of these weighting alternatives to the data for both day and overnight visitors in the nine parks showed that in all cases the group weighting procedure yielded higher dollar amounts than the individual weightings. The differences ranged from 4 % to 160 % with a median of 24%. The consistency of these results suggests systemic bias stemming from a disproportionate number of groups reporting a relatively high number of visitor days and relatively low per person per day expenditures. Empirical support for this explanation was provided by correlation analyses. When numbers of visitor days for both day and overnight visitors were correlated with per person per day expenditures in each category, the relationship was consistently negative indicating that as visitor days per group increased, per person per day expenditure declined. This suggests there are economies of scale both as the group size increases and as the length of stay increases.  Omitting a Measure of the Extent to which Visiting a Park was the Primary Trip Purpose  To capture both the relative pull influence of a park within the broader context of the area’s cumulative attractions, and the appropriate discount for those who qualified as casuals (that is, the park was not the main reason for them visiting the area), the survey instrument incorporated a 10 point “proportionality measure” on which respondents were asked to “circle the number that best represents the extent to which visiting the park was the primary purpose of your trip to this area.”   The data showed the parks tended to be the primary reason for overnight visitors coming to the area. At none of the nine parks did the proportionality measure reduce their expenditure estimates by more than 19 percent. However, among day visitors at all 9 sites, the parks were much less influential in decisions to visit the area. At Lake Ray Roberts, for example, when the proportionality measure was applied, the estimate of day visitor expenditures was dramatically reduced from $15.7 million to $8.9 million. It appears that many day visitors to Lake Ray Roberts, Dinosaur Valley, Enchanted Rock, Pedernales Falls, Tyler and Garner were either casuals or were persuaded to visit the area by the presence of multiple attractions, rather than only the park. These analyses illustrate the importance of incorporating a scale so attribution of expenditures accurately reflects the importance of a given attraction in decisions to visit an area.  Retaining Outlier Values   When estimates derived from relatively small samples are extrapolated to relatively large populations, sampling “accidents” can lead to substantial misrepresentation. For this reason, extreme values should be omitted. This was operationalized by removing the top and bottom 1% of expenditure estimates from the samples.  Among overnight visitors the inclusion of outlier values at 8 of the 9 parks led to an increase in expenditure estimates of from 2% to 12%, but at the other park the increase was 104%. Similarly, among day visitors the increase was relatively small at 7 of the parks ranging from 2% to 16%, but at the remaining 2 parks the increases were 376% and 496%.   These data confirm that omitting outliers should be a fundamental procedure in estimating average direct expenditures. Omitting 1% of the samples resulted in removal of the potential for substantially inflated estimates.   Aggregating Different Visitor Segments  Per person per day expenditures by overnight visitors were smaller than those of day visitors. In most cases the differences were substantial, because overnight groups remained in the parks for a longer period of time. This translates into more visitor days, which results in economies of scale.  Total visitor expenditures were calculated using both aggregated and disaggregated strategies. The results showed visitor expenditures at 8 of the parks were lower when the two segments were aggregated and at 6 parks the differences were substantial ranging from 20% to 72%.  Using Gross Sales Rather than Output Measures  There is frequent semantic and conceptual confusion between those two indices. The gross sales measure reports the effect of visitor spending on total economic activity within a host community. In contrast, the output measure includes all sales in the service sector, but for wholesale and retail sales it includes only gross margin not gross sales. The margin is defined as the selling price of an item, less the cost of goods sold (essentially production or acquisition cost).   When both of these economic impact measures were used at 9 sites, the percent by which impact on gross sales exceeded output ranged from 43% to 143% with a median of 101%. It has been pointed out that gross sales and output are esoteric measures with limited practical value, and that neither of them offers any useful insights for guiding elected officials in making tourism policy decisions (Crompton 2006). They are used because sponsors usually want to generate large numbers for advocacy purposes. Nevertheless, if they are to be used, it is important that stakeholders understand the distinction between them and the impact that distinction has on economic impact estimates.  Concluding Comments  Economic impact studies should be regarded as suggestive of the impacts of an attraction, rather than as being definitively accurate. Even when every effort is made by knowledgeable researchers to do them with integrity, it is inevitable they will have relatively large error margins.  In our view, the increasing skepticism with which economic impact studies are viewed can only be rebutted by avoidance of the mischievous practices described by Crompton (2006) and by embracing methodological transparency relating to the issues addressed in this paper. The five issues addressed in this paper have not previously been presented and empirically tested in the literature, despite their demonstrated impact on study results. Since they represent an under-investigated area of economic impact studies in tourism, it is our hope this paper will stimulate others to address these issues.   References Crompton, J. L. (2006). “Economic impact studies: Instruments for political shenanigans?” Journal of Travel Research, 45 (1):  67-82.  

The Potential Influence of Researchers’ “Hidden” Procedure Decisions on Estimates of Visitor Spending and Economic Impact

https://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1007&amp;context=ttra

The Potential Influence of Researchers’ “Hidden” Procedure Decisions on Estimates of Visitor Spending and Economic Impact

Abstract

Similar works

Full text

Available Versions

ScholarWorks@UMass Amherst