This is the periodic update on the evaluation of the Nurse Family Partnership program in North Carolina focused on the priority health outcomes of women and infants. During this period, we refined our methodology to more precisely estimate the effect of NFP participation on the health of women and children. In this report, we will report the estimated effect of participation on birthweight, gestational age, NICU admission, and breastfeeding initiation. We will also discuss the differential treatment effect of participation by maternal race as well as variation in estimation between statewide, county and hospital level analysis.As previously reported, this study focuses on NFP participants in North Carolina and proximal health outcomes as well as health care costs. This study is limited by its relatively small sample size used to analyze uncommon outcomes, suggesting the ability to detect programmatic effects may be limited. In other words, because of the relatively small sample, a priori we might expect to conclude there is no effect when there truly is

Mark Holmes

Regina Rutledge

English

IssueLab

March	2016	1	Evaluation	of	the	Nurse	Family	Partnership	in	North	Carolina		March	2016		Mark	Holmes	and	Regina	Rutledge	Department	of	Health	Policy	and	Management	UNC	Gillings	School	of	Global	Public	Health	Contact:	mark_holmes@unc.edu		Key	Findings			1.	NFP	participation	was	associated	with	reduction	in	the	rate	of	“very	preterm”		births	(28‐31	weeks)	and	NICU	admission	in	the	hospital	clustered	model.		Both	these	outcomes	are	known	to	be	associated	with	healthier	infants	and	reduced	costs.			2.	NFP	participation	has	a	sizable	and	significantly	different	effect	on	women	based	on	their	race	and	ethnicity.	When	disaggregated	by	race,	all	specific	categories	of	preterm	birth	were	significantly	reduced	for	both	White	and	Black/African	American	women.	The	effect	of	NFP	on	Black/African	American	women	was	double	that	of	White	women.			3.	The	most	appropriate	control	population	is	the	hospital	clustered	model	using	repeated	standard	errors.	Confidence	intervals	were	consistently	smallest	using	this	methodology.								Overview	and	Purpose		 This	is	the	periodic	update	on	the	evaluation	of	the	Nurse	Family	Partnership	program	in	North	Carolina	focused	on	the	priority	health	outcomes	of	women	and	infants.	During	this	period,	we	refined	our	methodology	to	more	precisely	estimate	the	effect	of	NFP	participation	on	the	health	of	women	and	children.	In	this	report,	we	will	report	the	estimated	effect	of	participation	on	birthweight,	gestational	age,	NICU	admission,	and	breastfeeding	initiation.	We	will	also	discuss	the	differential	treatment	effect	of	participation	by	maternal	race	as	well	as	variation	in	estimation	between	statewide,	county	and	hospital	level	analysis.		As	previously	reported,	this	study	focuses	on	NFP	participants	in	North	Carolina	and	proximal	health	outcomes	as	well	as	health	care	costs.	This	study	is	limited	by	its	relatively	small	sample	size	used	to	analyze	uncommon	outcomes,	suggesting	the	ability	to	detect	programmatic	effects	may	be	limited.		In	other	words,	because	of	the	relatively	small	sample,	a	priori	we	might	expect	to	conclude	there	is	no	effect	when	there	truly	is.				Review	of	Methods	Sample		 This	evaluation	matched	NFP	program	participation	with	birth	certificate	data	to	examine	potential	effects	on	maternal	and	infant	health	outcomes.		Information	on	1027	NFP	clients	was	obtained,	including	36	that	had	no	valid	delivery	date	(these	were	discarded).		Using	maternal	birthdate,	infant	birthdate	and	county	of	residence,	908	of	the	991	remaining	records	(92%)	NFP	participants	were	matched	to	their	infant’s	birth	record.	Among	those	that	did	not	match	exactly,	additional	efforts	were	March	2016	2	made	to	match	records	with	potential	entry	errors	of	birthdates;	no	additional	records	could	be	matched	confidently.		While	birth	certificate	and	NFP	participation	data	were	available	for	year	2009‐2013,	comparability	of	measures	over	the	time	period	presented	challenges.	In	2010,	North	Carolina	implemented	components	of	the	nationally	revised	Standard	Certificate	of	Live	Birth,	thereby	altering	the	way	many	social,	demographic	and	health	attributes	were	recorded	for	women	and	infants.	Among	those	indicators	that	were	changed	were	maternal	education,	neonatal	intensive	care	unit	(NICU)	admission,	maternal	smoking	habits,	and	maternal	body	mass	index.	In	light	of	the	importance	of	these	attributes	in	comparing	NFP	participants	to	similar	non‐participants	and	incomparability	pre	and	post	2010,	the	first	two	years	of	participation	data	and	birth	records	were	excluded	from	analysis.		Finally,	the	comparison	sample	was	adjusted	to	resemble	participants	in	Nurse	Family	Partnership.	Women	with	higher	order	births	(any	previous	live	birth)	and	those	that	did	not	have	Medicaid	as	the	primary	source	of	payment	at	the	time	of	delivery	were	excluded.	Multiples	(from	both	the	NFP	and	comparison	sample)	were	excluded	to	avoid	biasing	gestational	age	and	birthweight	estimates.		Figure	1	shows	the	sample	construction	that	yielded	an	analytic	sample	of	564	NFP	participants	and	61,429	controls.		Figure	1.	Analytic	Sample	Design						Table	1.	Weighted	sample	means	and	proportions	of	covariates	Recorded	Births	2009‐2013599,385Matched	NFP/Birth	Record	(NFP):	908Unmatched	Birth	Record	(Control):598,477Births	in	2011‐2013NFP:	672Control:352,753SingletonNFP:	645Control:	340,657First	birthNFP:	633Control:141,602Medicaid	fundedNFP:	564 Control:	61,249Other	fundedNFP:		69Control:80,274Higher	order	birthNFP:	12	Control:198,953MultipleNFP:	27Control:	12,093Births	in	2009‐2010NFP:	236Control:	245,735Unmatched	NFP	Records119	NFP	Records	2009‐20131,027March	2016	3	NFP	Participants	 Controls	Maternal	age	(years)	 21.93	 22.10	Under	20	(%)	 33.99	 31.97	Hispanic	(%)	 10.81	 9.22	White	(%)	 45.55	 48.76	Black	(%)	 39.99	 37.61	American	Indian/Alaskan	Native	(%)	 2.09	 2.15	High	school	education	or	less	(%)	 26.74	 25.95	Married	(%)	 19.15	 20.53	WIC	participant	(%)	 91.20	 85.74	Smoker,	preconceptive	(%)	 20.89	 21.14	Obese	(%)	 26.32	 24.77			Analysis	There	are	multiple	ways	to	measure	the	effect	of	the	program.		For	example,	we	could	estimate	“What	would	happen	to	the	average	pregnant	woman	if	they	received	NFP?”,	“What	would	the	outcomes	be	for	the	NFP	recipients	been	if	they	had	not	received	NFP?”,	or	“What	would	outcomes	be	if	the	non‐NFP	recipients	received	NFP?”		These	analyses	each	pose	different	questions	that	could	produce	highly	variable	outcomes.	For	this	analysis,	we	have	estimated	the	“average	effect	on	the	treated”.	Using	NFP	participants,	we	predict	what	their	outcomes	would	have	been	without	the	intervention	by	looking	at	“similar”	women	who	did	not	receive	NFP.			Likewise,	there	are	multiple	ways	to	identify	“controls”.		We	adopted	three	strategies	to	assess	comparability	across	three	levels	of	aggregation:	a)	Statewide	uses	all	women	in	North	Carolina,	b)	Hospitals	uses	only	controls	delivering	in	hospitals	used	by	NFP	participants	that	year,	and	c)	Counties	use	only	controls	delivering	in	those	counties	also	containing	NFP	participants	that	year.1		These	alternative	methods	offer	different	controls,	and	thus	results	should	differ.	If	they	all	provide	similar	results,	this	is	support	for	the	evidence.	Based	on	these	decisions,	inverse	propensity	for	treatment	weights	(IPTW)	scores	were	generated	to	assist	in	the	comparison	between	the	NFP	participants	and	the	control	sample.2	This	method	adjusts	the	control	sample	to	more	closely	resemble	the	NFP	participants	using	key	attributes	such	as	maternal	age,	race,	ethnicity,	education,	and	marital	status.		IPTWs	aim	to	reduce	the	standardized	differences	between	the	treatment	(NFP)	and	control	population	to	less	than	10%,	thereby	making	comparative	analysis	possible;	our	approach	met	this	threshold	for	all	key	covariates	(maternal	age,	race	and	ethnicity,	maternal	education,	marital	status,	smoking	behaviors	and	obesity	status).			A	doubly‐robust	logistic	regression	with	clustered	standard	errors	was	used	to	examine	the	effect	of	Nurse	Family	Partnership	participation	on	several	maternal	and	infant	outcomes.	This	approach	protects	against	errors	in	the	propensity	weighting	as	well	as	mis‐modeling	by	using	the	same	covariates	in	both	weighting	and	as	control	covariates.(1)	Clustered	standard	errors	were	also	used	to	account	for	unobserved	similarities	in	populations	within	hospitals	and	counties.	Finally,	a	bootstrapping	technique	was	used	to	run	100	repetitions	of	the	model	to	obtain	more	precise	standard	errors.			 	In	addition	to	examining	treatment	effects	for	the	full	sample	of	participants,	we	measured	how	the	program	may	have	affected	women	differently	by	maternal	age,	race	and	ethnicity.		We	created	differential	treatment	estimates	for	women	under	20	years	of	age,	women	20	years	of	age	and	older,	white	women,	black	women,	and	Hispanic	women	using	the	hospital	clustered	model.		This	approach	is																																																									1	This	is	similar	to	the	approach	used	by	South	Carolina	in	their	preliminary	evaluation	of	NFP.	2	An	excellent	resource	is	Austin	P.	An	Introduction	to	Propensity	Score	Methods	for	Reducing	the	Effects	of	Confounding	in	Observational	Studies.		Multivariate	Behav	Res.	2011	May;	46(3):	399–424.		2011.	doi:		10.1080/00273171.2011.568786	March	2016	4	important	if	we	expect	the	treatment	effect	to	vary	by	maternal	characteristics	(e.g.,	more	effective	for	younger	mothers).					Results	This	evaluation	examined	a	series	of	eight	health	outcomes	believed	to	be	affected	by	participation	in	a	perinatal	support	program	such	as	Nurse	Family	Partnership.		These	outcomes	included		 preterm	birth	(including	any	preterm	birth,	and	moderate	preterm	birth,	very	preterm	birth	and	extreme	preterm	birth	separately),		 low	birthweight	(including	any	low	birthweight,	moderate	low	birthweight	and	very	low	birthweight	separately),		 NICU	admission	and		 breastfeeding	initiation	recorded	at	hospital	discharge.			We	controlled	for	(a	quadratic	model	of)	maternal	age,	race/ethnicity,	education,	marital	status,	WIC	participation,	adequacy	of	prenatal	care	and	smoking	behaviors	before	and	during	pregnancy.			 Table	2	summarizes	the	results	across	all	three	types	of	control	specifications	(state,	county,	and	hospital).			In	both	the	statewide‐control	and	county‐control	models,	NFP	did	not	have	a	statistically	significant	effect	on	any	outcome	other	than	the	rate	of	very	preterm	birth	(Figure	2a	and	2c).	In	the	hospital–control	model,	NFP	statistically	reduced	the	rate	of	very	preterm	birth	and	NICU	admissions	(Figure	2b).			However,	given	the	clear	trends	across	models	suggesting	NFP	may	have	been	improving	outcomes	but	our	sample	was	not	powered	to	show	statistical	significance,	we	further	unpacked	the	sample	by	maternal	race/ethnicity.			 Figure	3	shows	the	effect	of	NFP	participation	on	the	outcomes	by	maternal	race	and	ethnicity.	We	restricted	this	analysis	only	to	White,	Black/African	American,	and	Hispanic	women	due	to	sample	size.	Our	findings	show	a	clear	statistically	significant	trend	in	reducing	all	specific	stages	of	preterm	birth	for	White	and	Black/African	American	women.	In	addition	to	showing	sizable	reductions	in	preterm	birth,	the	effect	size	is	markedly	larger	and	statistically	different	for	Black/African	American	women	as	compared	with	White	women.						 In	addition	to	maternal	race	and	ethnicity,	we	explored	how	maternal	age	(less	than	20	years	old	and	20	years	and	older)	may	have	resulted	in	differential	treatment.	While	none	of	the	results	showed	significance,	the	data	trended	to	suggest	women	20	years	and	older	benefited	more	from	NFP	participation	than	their	younger	counterparts.	There	was	not	enough	variation	in	marital	status	among	NFP	participants	to	test	for	potential	variation	on	this	characteristic.			Implications	of	Findings		 The	study	has	a	number	of	key	findings:	The	greatest	evidence	for	NFP	effectiveness	comes	when	using	controls	from	within	the	hospitals	used	by	NFP.			This	is	similar	to	the	pattern	of	findings	seen	by	SCDHEC’s	preliminary	analysis,	where	statewide	controls	showed	little	evidence	but	increased	specificity	of	the	control	enhanced	the	evidence	of	an	effective	program.		This	is	intuitive,	as	NFP	participants	are	likely	far	more	similar	to	mothers	at	NFP	hospitals	than	to	mothers	across	the	state.			Given	the	IPTW	approach,	the	hospital‐control	method	is	similar	to	the	matched	hospital	method	in	Comparison	Group	3.	NFP	appears	to	be	effective	at	reducing	the	rate	of	NICU	and	the	risk	of	very	preterm	birth.		The	results	from	the	hospital‐control	model	suggests	it	cut	the	rate	of	NICU	by	approximately	20	percent	(from	7.5	to	5.9)	and	the	risk	of	very	preterm	birth	by	two	thirds.		These	are	considerable	effects.	When	stratified	by	race/ethnicity,	NFP	appears	to	be	most	effective	for	African‐American	mothers.				By	subaggregating	the	observations,	we	identified	a	sizable	difference	in	effectiveness	across	race/ethnicity,	with	the	largest	improvement	occurring	in	African‐American	mothers.	These	positive	March	2016	5	treatment	effects	are	not	detectable	without	disaggregating	the	data	modeling	specifically	by	race	and	ethnicity.	Our	findings	support	continuing	to	assess	the	effects	of	Nurse	Family	Partnership	on	the	state,	county	and	hospital	level	but	also	within	subgroups.	Further	analysis	of	the	variation	in	treatment	effects	will	be	possible	when	we	begin	working	with	the	Medicaid	claims	data.					 										 	March	2016	6		Table	2.	Probability	of	Outcome	Dependent	Upon	NFP	Participation,	controlling	for	maternal	age,	race/ethnicity,	education,	marital	status,	WIC	participation,	adequacy	of	prenatal	care	and	smoking	before	and	during	pregnancy		Probability	of	Outcome	Without	NFP	Probability	of	Outcome	With	NFP	Difference	attributable	to	participation	in	NFP	 95%	Confidence	Interval	Model	A:	Statewide	Control	Model	 	 	 	 	 	All	Low	birthweight	(<2500g)	 10.89%	 11.44%	 0.55%	 2.28%	 ‐1.18%	Low	birthweight	(1500‐2499g)	 10.04%	 9.70%	 ‐0.34%	 0.99%	 ‐1.67%	Very	low	birthweight	(<1500g)	 1.32%	 1.76%	 0.45%	 1.13%	 ‐0.24%	All	Preterm	(<37	weeks)	 12.47%	 12.50%	 0.04%	 1.93%	 ‐1.86%	Moderate	preterm	(32‐36	weeks)	 10.93%	 10.86%	 ‐0.08%	 1.55%	 ‐1.71%	Very	preterm	(28‐31	weeks)	 1.59%	 0.66%	 ‐0.92%	 ‐0.67%	 ‐1.18%	Extremely	preterm	(<28	weeks)	 0.61%	 0.87%	 0.26%	 0.65%	 ‐0.13%	NICU	admission	 6.31%	 6.13%	 ‐0.17%	 1.20%	 ‐1.54%	Breastfeeding	initiation	 61.89%	 66.00%	 4.10%	 6.54%	 1.67%	Model	B:	Hospital	Control	Model	All	Low	birthweight	(<2500g)	 12.58%	 11.66%	 ‐0.92%	 0.86%	 ‐2.70%	Low	birthweight	(1500‐2499g)	 11.37%	 10.21%	 ‐1.16%	 0.29%	 ‐2.61%	Very	low	birthweight	(<1500g)	 1.74%	 1.64%	 ‐0.10%	 0.55%	 ‐0.75%	All	Preterm	(<37	weeks)	 13.59%	 12.64%	 ‐0.95%	 1.05%	 ‐2.96%	Moderate	preterm	(32‐36	weeks)	 11.45%	 11.13%	 ‐0.32%	 1.47%	 ‐2.11%	Very	preterm	(28‐31	weeks)	 2.13%	 0.71%	 ‐1.42%	 ‐1.13%	 ‐1.70%	Extremely	preterm	(<28	weeks)	 0.89%	 0.76%	 ‐0.12%	 0.22%	 ‐0.47%	NICU	admission	 7.46%	 5.93%	 ‐1.53%	 ‐0.08%	 ‐2.98%	Breastfeeding	initiation	 64.29%	 66.55%	 2.26%	 4.65%	 ‐0.13%	Model	C:	County	Control	Model	All	Low	birthweight	(<2500g)	 12.55%	 11.69%	 ‐0.86%	 0.93%	 ‐2.65%	Low	birthweight	(1500‐2499g)	 11.40%	 10.19%	 ‐1.20%	 0.24%	 ‐2.65%	Very	low	birthweight	(<1500g)	 1.69%	 1.66%	 ‐0.03%	 0.63%	 ‐0.69%	All	Preterm	(<37	weeks)	 13.49%	 12.67%	 ‐0.82%	 1.19%	 ‐2.83%	Moderate	preterm	(32‐36	weeks)	 11.37%	 11.14%	 ‐0.23%	 1.56%	 ‐2.02%	Very	preterm	(28‐31	weeks)	 0.87%	 0.78%	 ‐0.09%	 0.26%	 ‐0.44%	Extremely	preterm	(<28	weeks)	 2.13%	 0.72%	 ‐1.42%	 ‐1.14%	 ‐1.70%	NICU	admission	 7.35%	 5.99%	 ‐1.36%	 0.10%	 ‐2.82%	Breastfeeding	initiation	 64.45%	 66.49%	 2.05%	 4.45%	 ‐0.35%									March	2016	7	Figure	2a.	Estimated	effect	of	Nurse	Family	Partnership	on	select	health	outcomes	with	95%	confidence	intervals	–	STATEWIDE	CONTROL					Figure	2b.	Estimated	effect	of	Nurse	Family	Partnership	on	select	health	outcomes	with	95%	confidence	intervals	–	HOSPITAL	CONTROL				Figure	2c.	Estimated	effect	of	Nurse	Family	Partnership	on	select	health	outcomes	with	95%	confidence	intervals	–	COUNTY	CONTROL	All	low	birthweight	Moderate	 low	birthweight	Very	low	birthweightAll	pretermExtremely	pretermVery	pretermModerate	pretermNICU	admissionBreastfeeding	 initiation‐0.05 0 0.05 0.1NFP	decreases probabilityof	event← ← ← NFP	increases probability	of	event→ →→All	low	birthweight	Moderate	 low	birthweight	Very	low	birthweightAll	pretermExtremely	pretermVery	pretermModerate	pretermNICU	admissionBreastfeeding	 initiation‐0.05 0 0.05 0.1NFP	decreases probabilityof	event← ← ← NFP	increases probability	of	event→ → →March	2016	8																										 	All	low	birthweight	Moderate	 low	birthweight	Very	low	birthweightAll	pretermExtremely	pretermVery	pretermModerate	pretermNICU	admissionBreastfeeding	 initiation‐0.05 0 0.05NFP	decreases probabilityof	event← ← ← NFP	increases probability	of	event→→ →March	2016	9		Figure	3.	Changes	in	outcome	attributable	to	NFP	participation,	by	race/ethnicity	(hospital	control)				All	low	birthweight	Moderate	low	birthweight	Very	 low	birthweightAll	pretermExtremely	pretermVery	pretermModerate	pretermNICU	admissionBreastfeeding	initiation‐0.1 ‐0.05 0 0.05 0.1  White ●Black/African	 American   HispanicMarch	2016	10	Next	Steps				 This	next	stage	of	the	evaluation	will	use	the	existing	data	merged	with	Medicaid	claims	to	further	examine	the	effects	of	Nurse	Family	Partnership	participation	on	health	outcomes	and	healthcare	expenditures.	The	claims	data	will	offer	a	richer	source	of	information	on	the	severity	of	poor	birth	outcomes	as	well	additional	diagnoses	to	further	control	the	model	and	isolate	estimated	treatment	effects.	In	addition	to	further	evaluating	NFP’s	impact	on	health,	we	will	use	the	claims	to	assess	real	costs	to	Medicaid	as	well	as	model	the	financial	implications	of	averted	poor	birth	outcomes	and	NICU	admissions.						References		1.		 Funk	MJ,	Westreich	D,	Wiesen	C,	Stürmer	T,	Brookhart	MA,	Davidian	M.	Doubly	Robust	Estimation	of	Causal	Effects.	Am	J	Epidemiol.	2011	Apr	1;173(7):761–7.		2.		 Kotelchuck	M.	An	evaluation	of	the	Kessner	Adequacy	of	Prenatal	Care	Index	and	a	proposed	Adequacy	of	Prenatal	Care	Utilization	Index.	Am	J	Public	Health.	1994	Sep;84(9):1414–20.		3.		 Bloch	JR,	Dawley	K,	Suplee	PD.	Application	of	the	Kessner	and	Kotelchuck	Prenatal	Care	Adequacy	Indices	in	a	Preterm	Birth	Population.	Public	Health	Nurs.	2009	Sep	1;26(5):449–59.		4.		 Gorber	SC,	Schofield‐Hurwitz	S,	Hardt	J,	Levasseur	G,	Tremblay	M.	The	accuracy	of	self‐reported	smoking:	A	systematic	review	of	the	relationship	between	self‐reported	and	cotinine‐assessed	smoking	status.	Nicotine	Tob	Res.	2009	Jan	1;11(1):12–24.		5.		 Aurrekoetxea	JJ,	Murcia	M,	Rebagliato	M,	López	MJ,	Castilla	AM,	Santa‐Marina	L,	et	al.	Determinants	of	self‐reported	smoking	and	misclassification	during	pregnancy,	and	analysis	of	optimal	cut‐off	points	for	urinary	cotinine:	a	cross‐sectional	study.	BMJ	Open.	2013	Jan	1;3(1):e002034.		6.		 Ip	S,	Chung	M,	Raman	G,	Chew	P,	Magula	N,	DeVine	D,	et	al.	Breastfeeding	and	Maternal	and	Infant	Health	Outcomes	in	Developed	Countries.	153.	Rockv	MD	Agency	Healthc	Res	Qual	Evid	ReportTechnologyAssessment.	2007;		7.		 MONTGOMERY	DL,	SPLETT	PL.	Economic	Benefit	of	Breast‐Feeding	Infants	Enrolled	in	WIC.	J	Am	Diet	Assoc.	1997	Apr;97(4):379–85.		8.		 Tuttle	CR,	Dewey	KG.	Potential	cost	savings	for	Medi‐Cal,	AFDC,	food	stamps,	and	WIC	programs	associated	with	increasing	breast‐feeding	among	low‐income	Hmong	women	in	California.	J	Am	Diet	Assoc.	1996	Sep;96(9):885–90.			

Evaluation of the Nurse Family Partnership in North Carolina

https://www.issuelab.org/resources/28289/28289.pdf

Evaluation of the Nurse Family Partnership in North Carolina

Abstract

Similar works

Full text

Available Versions

IssueLab