Turfgrass specialists and Extension educators responsible for developing educational materials in the Master Gardener Program sought stakeholder input for an innovative curriculum by using innovative data collection methods. County agents, program coordinators, and volunteers from 11 Cooperative Extension Service districts responded to a Web-based, Delphi study. Interactive, online data collection methods provided rapid feedback in the consensus-building process. Extension personnel can use this methodology to develop similar consensus-building activities for other programming issues. Stakeholder input can be achieved, with minimum time and expense, while curriculum developers minimize wasted time in programming development that clientele may not find useful

Chalmers, David R

Mayfield, Chyrel A

Wingenbach, Gary J

English

Clemson University: TigerPrints

The Journal of Extension Volume 43 Number 3 Article 19 6-1-2005 Assessing Stakeholder Needs: Delphi Meets the Internet Chyrel A. Mayfield Texas A&M University, c-mayfield@tamu.edu Gary J. Wingenbach Texas A&M University, g-wingenbach@tamu.edu David R. Chalmers Texas A&M University, dchalmers@tamu.edu This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License. Recommended Citation Mayfield, C. A., Wingenbach, G. J., & Chalmers, D. R. (2005). Assessing Stakeholder Needs: Delphi Meets the Internet. The Journal of Extension, 43(3), Article 19. https://tigerprints.clemson.edu/joe/vol43/iss3/19 This Ideas at Work is brought to you for free and open access by the Conferences at TigerPrints. It has been accepted for inclusion in The Journal of Extension by an authorized editor of TigerPrints. For more information, please contact kokeefe@clemson.edu. 	 JOEHOME JOURNAL GUIDELINES ABOUT	JOE CONTACT NATIONAL	JOB	BANKCurrent	Issues Back	IssuesJune	2005	//	Volume	43	//	Number	3	//	Ideas	at	Work	//	3IAW1Assessing	Stakeholder	Needs:	Delphi	Meets	the	InternetAbstractTurfgrass	specialists	and	Extension	educators	responsible	for	developing	educational	materialsin	the	Master	Gardener	Program	sought	stakeholder	input	for	an	innovative	curriculum	by	usinginnovative	data	collection	methods.	County	agents,	program	coordinators,	and	volunteers	from11	Cooperative	Extension	Service	districts	responded	to	a	Web-based,	Delphi	study.	Interactive,online	data	collection	methods	provided	rapid	feedback	in	the	consensus-building	process.Extension	personnel	can	use	this	methodology	to	develop	similar	consensus-building	activitiesfor	other	programming	issues.	Stakeholder	input	can	be	achieved,	with	minimum	time	andexpense,	while	curriculum	developers	minimize	wasted	time	in	programming	development	thatclientele	may	not	find	useful.	IntroductionCooperative	Extension	has	a	rich	history	of	developing	outreach	programs	that	have	a	direct	andrelevant	impact	on	stakeholders'	lives.	In	order	to	develop	relevant	programming,	clientele	areasked	for	input	during	the	development	stages	for	many	programs.	Often,	requesting	andincorporating	timely	and	relevant	input	to	program	curricula	can	be	a	time-consuming,	expensiveprocess.	Decreasing	state	and	federal	resources	are	forcing	Extension	personnel	to	seekalternative	methods	to	continue	their	rich	tradition	of	stakeholder	input	in	the	program	curriculadevelopment	processes.Conceptual	FrameworkAlternative	methods	for	collecting	stakeholder	input	to	Extension	program	curricula	provideExtension	personnel	with	timely,	relevant	feedback	during	the	curricula	development	process.	Oneinexpensive	alternative	to	holding	several	face-to-face	or	traditional	postal	mail	surveys	isachieved	through	the	Delphi	technique,	using	a	Web-based	medium.The	Delphi	technique	was	developed	by	the	Rand	Corporation	in	the	late	1950's	as	a	forecastingmethodology.	Unlike	the	nominal	group	process,	the	Delphi	does	not	require	face-to	faceparticipation.	It	is	a	"systematic	solicitation	and	collation	of	judgments	on	a	particular	topic	througha	set	of	carefully	designed	sequential	questionnaires	interspersed	with	summarized	informationChyrel	A.	MayfieldResearch	AssociateDepartment	of	Forest	Sciencec-mayfield@tamu.eduGary	J.	WingenbachAssociate	ProfessorDepartment	of	Agricultural	Educationg-wingenbach@tamu.eduDavid	R.	ChalmersExtension	Turfgrass	Specialist	and	Associate	ProfessorTexas	Cooperative	ExtensionDepartment	of	Soil	and	Crop	Sciencesdchalmers@tamu.eduTexas	A&M	UniversityCollege	Station,	Texasand	feedback	of	opinions	derived	from	earlier	responses"	(Debecq,	Van	de	Ven,	&	Gustafson,	1975,p.	10).	The	Delphi	technique	affords	researchers	an	opportunity	to	collect	large	amounts	of	inputover	a	wide	geographic	area.	Delphi	techniques	incorporate	expert	panel	members'	opinions,	valuejudgments,	and	agreement	in	the	consensus-building	process	(Somers,	Baker,	&	Isbell,	1984).Decisions	about	which	participants	to	invite	to	a	Delphi	should	be	considered	carefully.	Ludwig(1997)	recommended:Randomly	selecting	participants	is	NOT	acceptable.	Instead,	characteristics	andqualifications	of	desirable	respondents	should	be	identified	and	a	nomination	processused	to	select	participants.	Because	the	group	number	will	be	small	(12-15),	theresearcher	needs	to	locate	and	target	individuals	who	are	"expert,"	have	knowledge	andexperience	to	base	their	futuring	activities	upon,	and	are	self-motivated.	Delphi	shouldnot	be	used	with	groups	that	have	difficulty	in	reading	or	expressing	themselves	inwritten	communication.	(p.	2)Ladner,	Wingenbach,	and	Raven	(2002)	found	Web-based	and	traditional	paper-based	surveymethodologies	were	equally	valid	and	reliable	for	social	science	research.	A	significant	differenceoccurred	in	the	response	rates	between	two	equal	groups	of	agricultural	educators;	however,	nodifferences	were	found	between	the	groups'	opinions	on	computer	usage	in	an	educational	setting.The	Web-based	group's	response	rate	exceeded	the	traditional	group,	72	to	7,	in	the	first	week	ofdata	collection.	These	results	provided	strong	evidence	for	using	Web-based	data	collectionmethods	in	social	science	research	when	time	and/or	financial	constraints	pose	barriers	torelevant,	timely,	effective	program	development	processes.Before	developing	new	programs,	it	is	important	to	be	mindful	of	gathering	and	using	Extensionstakeholder	input	in	developing	the	program	materials.	One	of	the	first	steps	in	designing	adulteducation	curricula	is	to	conduct	a	needs	assessment	(Sork	&	Caffarella,	1989).	Knowles,	Holton,and	Swanson	(1998)	provided	two	assumptions	about	adult	learning	that	are	critical	in	the	needsassessment	phase.	These	assumptions	are	the	need	to	know	and	the	learner's	self-concept.Essentially,	adults	need	to	know	why	they	need	to	learn	something	new.	Also,	adults	will	resist	andresent	(learner's	self-concept)	situations	in	which	they	feel	others	are	imposing	their	will	on	them(Knowles,	Holton,	&	Swanson).	Such	assumptions	about	adult	learning	provide	important	reasonsfor	using	stakeholder	input	during	curriculum	development.Decreasing	state	funds	for	Extension	programming	have	forced	many	states	to	seek	alternativemethods	to	continue	providing	quality	educational	programs	for	their	clientele	at	the	county	level.Extension	clientele	input	for	developing	instructional	modules	in	the	Turf	for	Texans	MasterGardener	Program	was	sought	using	innovative,	cost-effective	data	collection	methods.Purpose	and	ObjectivesThe	purpose	of	the	study	described	here	was	to	gather	stakeholder	input	for	the	most	FrequentlyAsked	Questions	(FAQs)	in	turfgrass	management	curricula	for	the	Turf	for	Texans	MasterGardener	Program.	The	following	objectives	guided	this	study.1.	 Identify	FAQs	for	nine	turfgrass	instructional	modules	in	the	Turf	for	Texans	Master	GardenerProgram.2.	 Rank	the	importance	of	the	identified	FAQs.3.	 Rank	participants'	agreement	levels	of	the	identified	FAQs	for	inclusion	in	the	turfgrassinstructional	modules.Methods	and	ProceduresDescriptive	survey	methodology,	with	a	Delphi	technique,	was	used	in	this	study.	Web-basedsurvey	data	collection	methods	(Ladner,	Wingenbach,	&	Raven,	2002)	were	used	after	obtainingapproval	to	conduct	the	study	through	the	Texas	A&M	University	Institutional	Review	Board(#2002-0276).The	target	population	(N	=	339)	consisted	of	all	Texas	County	Extension	Agents,	programcoordinators,	and	volunteers	who	participated	in	a	Texas	Master	Gardener	Program	during	2003.	Aproportional	stratified	sample	from	11	Texas	Cooperative	Extension	Service	districts	was	obtainedby	contacting	two	agents	from	each	district,	who	in	turn,	chose	one	coordinator	and	one	volunteerwith	at	least	1	year	of	experience	from	their	respective	Master	Gardener	Programs.	All	participantswere	sent	formal	letters	requesting	their	participation	in	the	study.	The	sample	consisted	of	22agents,	22	program	coordinators,	and	22	volunteers	(n	=	66).The	first	instrument,	posted	on	a	secure	Internet	site,	consisted	of	nine	open-ended	questionsdesigned	to	obtain	a	wide	range	of	responses.	Using	their	own	Master	Gardener	experiences,respondents	were	asked	to	identify	the	top	five	FAQs	for	turfgrass	management	in	each	of	nineTurf	for	Texans	instructional	modules.	The	identified	FAQs	were	used	to	develop	content	for	themodules.	Electronic	mail	reminders	were	sent	to	non-respondents	to	complete	round	one;	all	datawere	collected	in	3	weeks.	A	total	of	20	agents,	4	coordinators,	and	12	volunteers	(n	=	36)	from	33counties	in	the	11	districts	responded	to	round	one,	resulting	in	a	55%	response	rate.	Findingsfrom	this	study	should	not	be	generalized	beyond	the	limited	number	of	respondents.A	team	of	Extension	turfgrass	specialists,	graduate	students,	and	agricultural	education	facultymembers	condensed	and	combined	initial	responses	into	statements	without	altering	their	originalmeanings.	A	panel	of	experts	from	the	Departments	of	Soil	and	Crop	Science	and	AgriculturalEducation	reviewed	the	instrument	for	face	validity.	The	statements	were	posted	on	a	secureInternet	site	for	use	in	round	two.In	the	second	round	of	data	collection,	respondents	were	instructed	to	read	each	FAQ	for	eachmodule	and	rate	the	level	of	importance	(Likert-type	scale:	1	=	Not	Important	-	4	=	VeryImportant)	for	including	the	FAQ	in	its	respective	turfgrass	instructional	module.	Electronic	mailnotices	requesting	participation	in	round	two	were	sent	to	all	66	participants.	All	66	participantswere	asked	to	complete	all	three	rounds	because	of	their	vested	interest	in	the	consensus-buildingprocess.	A	total	of	16	agents,	7	coordinators,	and	12	volunteers	(n	=	35)	responded,	resulting	in	a53%	response	rate.	All	data	were	collected	in	2	weeks.Upon	conclusion	of	data	collection	in	the	second	round,	all	statements	were	ranked	according	totheir	grand	mean	scores,	sorted	by	level	of	importance,	and	posted	in	a	third	instrument	on	asecure	Internet	site.	The	third	instrument	allowed	respondents	to	rate	their	agreement	levels(Likert-type	scale:	1	=	Strongly	Disagree	-	4	=	Strongly	Agree)	with	the	importance	levels	for	eachFAQ	in	each	turfgrass	instructional	module.	Electronic	mail	notices	requesting	participation	inround	three	were	sent	to	all	66	participants.	A	total	of	15	agents,	5	coordinators,	and	10	volunteers(n	=	30)	responded,	resulting	in	a	46%	response	rate.	All	data	were	collected	in	10	days.Descriptive	statistics	were	derived	for	each	instructional	module.	ANOVA	tests	were	used	todetermine	significant	differences	among	subgroups	in	this	consensus-building	process.	Instrumentreliability	was	assessed	using	Cronbach's	alpha	coefficients	in	rounds	two	and	three.	Results	arepresented	in	Table	1.Table	1.Cronbach's	Alpha	Coefficients	for	ReliabilityModule Round	II Round	IIIIntroduction	to	Texas	Lawn	Care 0.83 0.74How	Lawn	Grasses	Grow 0.82 0.89Grass	Species	and	Varieties	Adapted	for	Texas 0.77 0.91Turfgrass	Establishment 0.85 0.92Mowing 0.81 0.78Cultural	Practices	for	Established	Texas	Lawns 0.92 0.91Nutrient	Management 0.86 0.91Irrigation	Matters	in	Texas 0.84 0.91Pests	and	Integrated	Pest	Management 0.89 0.87FindingsDue	to	space	limitations,	only	grand	means	from	the	third	(final)	round	of	the	Delphi	arepresented.	For	more	detailed	information	from	the	first	and	second	rounds	of	this	study,	readersshould	contact	the	authors	at	c-mayfield@tamu.edu	or	g-wingenbach@tamu.edu;	information	alsois	available	regarding	responses	by	agents,	volunteers,	and	coordinators.Thirty-six	respondents	with	Texas	Master	Gardener	Program	experiences	ranging	from	less	than	1to	over	20	years	(M	=	4.73),	identified	the	top	five	FAQs	for	turfgrass	management	in	their	TexasMaster	Gardener	Programs.	Overall,	a	total	of	115	FAQs	were	identified,	ranked,	and	prioritized	bystakeholders.	The	top	three	FAQs	for	each	module	are	depicted	in	Table	2.	Results	are	sorted	bydescending	grand	means.Table	2.Descriptive	Statistics:	Turf	for	Texans	Master	Gardener	Program	(n	=	30)FAQs MaModule	One:	Introduction	to	Texas	Lawn	CareWhat	determines	if	a	lawn	is	healthy? 3.14Are	there	benefits	of	having	turf	in	my	landscape? 3.11What	are	the	environmental	benefits	of	turf? 3.07Module	Two:	How	Lawn	Grasses	GrowWhat	are	the	differences	between	warm	and	cool	seasongrasses?3.31What	techniques	can	I	use	to	plant	grass? 3.28Why	do	you	sod	some	grasses	and	others	you	seed? 3.25Module	Three:	Grass	Species	and	Varieties	Adapted	for	TexasWhat	factors	should	be	considered	when	selecting	a	lawn	grass? 3.47How	do	I	decide	which	grass	is	best	suited	for	my	area? 3.47Which	grass	variety	is	best	suited	for	me	in	my	area	of	Texas? 3.47What	is	the	most	drought-tolerant	turfgrass? 3.47Module	Four:	Turfgrass	EstablishmentWhy	should	I	have	a	soil	test? 3.67What	is	the	best	way	to	prepare	the	soil	for	a	new	lawn? 3.63How	much	and	how	often	should	I	irrigate	my	new	lawn	until	itbecomes	established?3.60Module	Five:	MowingWhat	are	the	effects	of	improper	mowing? 3.50What	are	the	mowing	heights	for	different	grasses? 3.47Should	I	catch	or	leave	the	clippings? 3.40Module	Six:	Cultural	Practices	for	Established	Texas	LawnsDoes	the	practice	of	leaving	grass	clippings	on	my	lawncontribute	to	thatch?3.33What	is	a	good	indication	that	I	may	need	to	aerate	my	lawn? 3.27What	is	the	difference	between	scalping	and	de-thatching? 3.23Module	Seven:	Nutrient	ManagementWhen	do	I	need	to	fertilize? 3.60How	often	should	I	fertilize? 3.57How	much	fertilizer	should	I	apply? 3.57Module	Eight:	Irrigation	Matters	in	TexasHow	often	should	I	water	my	turfgrass? 3.76What	is	a	good	indicator	that	my	lawn	needs	watering? 3.73How	much	water	does	my	lawn	need? 3.67Module	Nine:	Pests	and	Integrated	Pest	ManagementWhat	common	Texas	turfgrass	diseases	might	attack	my	lawn? 3.67What	common	Texas	insects	attack	lawns? 3.60How	can	I	determine	if	I	have	a	disease	problem	or	an	insectproblem?3.57Note.	Four-point,	Likert-type	scales	measured	levels	of	importance.	a1=StronglyDisagree,	2=Disagree,	3=Agree,	4=Strongly	Agree.The	consensus-building	process	of	this	Delphi	technique	was	useful	in	helping	respondentsprioritize	the	most	important	FAQs	for	each	of	the	nine	modules.	No	significant	differences	werefound	among	respondents'	agreement	levels	of	the	FAQs	in	eight	of	the	nine	modules.	Only	inModule	5	(Mowing)	was	there	a	significant	difference	between	rankings.	Program	coordinatorsagreed	with	the	FAQ	"how	often	should	mower	blades	be	sharpened,"	more	than	did	agents	inround	three.Conclusions/RecommendationsFrom	the	findings,	it	can	be	concluded	that	lawn	health,	differences	between	warm	and	coolseason	grasses,	turfgrass	selection	factors,	soil	tests,	effects	of	improper	mowing,	grass	clippings,when	to	fertilize,	frequency	of	irrigation,	and	lawn	diseases	were	deemed	the	most	important	FAQsfor	inclusion	in	the	turfgrass	curricula.Although	the	identified	and	ranked	FAQs	for	the	instructional	modules	proved	useful	in	developingcurricula	for	the	Turf	for	Texans	Master	Gardener	Program,	the	authors	believe	the	most	importantfinding	was	derived	from	the	methodology	used	to	gather	stakeholder	input.	The	Delphi	technique,administered	through	online	data	collection	techniques,	provided	effective	means	to	determinestakeholders'	needs	in	designing	turfgrass	management	curricula.	Participants	were	able	toincorporate	their	opinions	(round	one),	value	judgments	(round	two),	and	agreement	levels	(roundthree)	in	a	consensus-building	process	for	the	FAQs	used	in	the	turfgrass	managementinstructional	modules.A	practitioner's	checklist	for	using	this	data	collection	process	includes:1.	 Asking	respondents	to	provide	information	relevant	to	the	programming	objective.	(Round	I)2.	 Condensing	responses	into	statements,	being	careful	not	to	change	the	original	meanings	ofthe	responses.3.	 Gathering	respondents'	importance	level	ratings	for	each	identified	statement	(importance	ofits	inclusion	in	the	programming	objective).	(Round	II)4.	 Rank-ordering	statements	according	to	their	indicated	levels	of	importance.5.	 Collecting	respondents'	agreement	levels	on	the	rank-ordered	importance	of	each	statementfor	its	inclusion	in	the	programming	objective.	(Round	III)Additionally,	stakeholder	input	was	gathered	in	an	economical,	shortened	frame	(6.5	weeks),confirming	the	Web-based	surveying	methods	proposed	by	Ladner,	Wingenbach,	and	Raven(2002).	The	Delphi	technique	used	in	the	study	provided	consistency	in	the	data	collectionprocedures,	as	proposed	by	Somers,	Baker,	and	Isbell	(1984).By	including	stakeholders'	input	to	build	consensus	on	relevant	topics	for	Extension	programs,Extension	personnel	address	the	need	to	know	and	learner's	self-concept	assumptions	raised	byKnowles,	Holton,	and	Swanson	(1998).	This	could	allow	for	more	rapid	adoption	of	Extensionprograms.	It	can	also	allow	Extension	personnel	to	focus	greater	attention	on	developing	relevanteducational	materials	for	their	clientele.We	recommend	these	methodologies	(Delphi	technique	and	Web-based	data	collection	methods)be	used	by	Extension	personnel	when	seeking	stakeholder	input	for	instructional	materialsdevelopment.	Using	these	methodologies,	Extension	personnel	can	gather	stakeholder	input	in	ashortened	time	frame	with	minimal	cost	and	inconvenience	resulting	in	a	high	quality	Extensionprogram.ReferencesDebecq,	A.	L.,	Van	de	Ven,	A.	H.,	&	Gustafson,	D.	H.	(1975).	Group	techniques	for	programplanning.	Glenview,	Illinois:	Scott,	Foresman	and	Company.Knowles,	M.,	Holton,	E.,	&	Swanson,	R.	(1998).	The	adult	learner:	The	definitive	classic	in	adulteducation	and	human	resource	development	(5th	ed.).	Houston,	TX:	Gulf	Publishing	Co.Ladner,	D.,	Wingenbach,	G.,	&	Raven,	M.	(2002).	Internet	and	paper-based	data	collection	methodsin	agricultural	education	research.	Journal	of	Southern	Agricultural	Education	Research,	52(1),	40-51.Ludwig,	B.	(1997).	Predicting	the	future:	Have	you	considered	using	the	Delphi	methodology?Journal	of	Extension	[On-line],	35(5).	Available	at:	http://www.joe.org/joe/1997october/tt2.htmlSomers,	K.,	Baker,	G.,	&	Isbell,	C.	(1984).	How	to	use	the	Delphi	technique	to	forecast	trainingneeds.	Performance	and	Instruction	Journal,	23(4),	26-28.Sork,	T.	J.,	&	Caffarella,	R.	(1989).	Planning	programs	for	adults.	In	S.	Merriam,	and	P.	Cunningham,(Eds.),	Handbook	of	adult	and	continuing	education	(pp.	233-245).	San	Francisco:	Jossey-BassPublishers.Copyright	©	by	Extension	Journal,	Inc.	ISSN	1077-5315.	Articles	appearing	in	the	Journal	become	the	property	of	theJournal.	 Single	 copies	 of	 articles	may	 be	 reproduced	 in	 electronic	 or	 print	 form	 for	 use	 in	 educational	 or	 trainingactivities.	Inclusion	of	articles	in	other	publications,	electronic	sources,	or	systematic	large-scale	distribution	may	bedone	only	with	prior	electronic	or	written	permission	of	the	Journal	Editorial	Office,	joe-ed@joe.org.If	you	have	difficulties	viewing	or	printing	this	page,	please	contact	JOE	Technical	Support©	Copyright	by	Extension	Journal,	Inc.	ISSN	1077-5315.	Copyright	Policy

Assessing Stakeholder Needs: Delphi Meets the Internet

https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=4642&context=joe

Assessing Stakeholder Needs: Delphi Meets the Internet

Abstract

Similar works

Full text

Available Versions

Clemson University: TigerPrints