Trustee Evaluation ToolkitTrustees care deeply about impact. Understanding results is part of their fiduciary duty. As foundations strive to improve performance, advance accountability and share knowledge, their desire for evaluation -- reliable data on organizational effectiveness -- grows. Based on discussions with trustees, we've heard that current evaluation approaches don't always generate useful information. In too many cases, foundation evaluation practices don't align with trustee needs. Trustees across the United States believe there are ways to improve how we determine the effectiveness of social investments. FSG Social Impact Advisors, with funding from the James Irvine Foundation, interviewed dozens of foundation trustees, CEOs and evaluation experts to uncover critical issues and exciting ideas related to evaluation. This "toolkit" shares highlights from these interviews, and explores innovative new approaches

Eva Nico

Mark R. Kramer

Samantha Nobles-Block

English

IssueLab

e v a l u a t i o n  k i t  f o r  t r u s t e e s m a y  2009What’s the Difference? How Foundation Trustees View EvaluationFSG SOCIAL IMPACT ADVISORS    .     PAGE 2    .    THE JAMES IRVINE FOUNDATIONEVALUATION MATTERSTrustees KnowTrustees care deeply about impact. Understanding results is part of their fiduciary duty.  As foundations strive to improve performance, advance accountability and share knowledge, their desire for evaluation — reliable data on organizational effectiveness — grows.  EVALUATION ISN’T MEETINg OUR NEEdS. Trustees wish that current approaches generated more useful information. In too many cases, foundation evaluation practices don’t align with trustee needs. IT cAN wORk bETTER. Trustees across the United States believe there are ways to improve how we determine  the effectiveness of social investments. Many are already using proven, practical  approaches today.FSG Social Impact Advisors, with funding from The James Irvine Foundation, interviewed dozens of foundation trustees, CEOs and evaluation experts to uncover critical issues and exciting ideas related to evaluation. This document shares highlights from these interviews.See Page 6 to learn about the complete  Evaluation Kit for Trustees.FSG SOCIAL IMPACT ADVISORS    .     PAGE 3    .    THE JAMES IRVINE FOUNDATIONI’M HERE TO MAkE A dIFFERENcE.Trustees are personally and passionately motivated to make a difference. While some hold a perspective that “if the grant sounds good, we’ve done our job,” the vast majority feel a weighty responsibility to themselves, their staff and their communities. They want to learn from past results to ensure that foundation resources are being used to achieve the greatest possible effect.EVALUATION IS PART OF OUR JOb.Trustees take their fiduciary duties very seriously. And, they see evaluation as an important part of fulfilling these duties. They feel that spending foundation assets wisely is just as essential  as investing and managing them wisely.wHEN IT cOMES TO EVALUATION, OUR AcTIONS dON’T ALwAyS MATcH OUR cONVIcTIONS.While trustees say evaluation is important, many admit that it gets lost in the shuffle as they press onward with projects. Those interviewed say it would help to set clearer, more concrete goals and strategies that build in evaluation from the start. They say evaluation deserves more time, attention and resources than it currently receives.MANy EVALUATION APPROAcHES dON’T dO THE JOb.The way foundations pursue evaluation often doesn’t meet trustee needs. Conducted by academics and social scientists, many evaluations result in long reports where key insights are buried and lost to busy trustees from the worlds of business, politics and nonprofit leadership. Some are summarized so briefly that their meaning is diluted. Some evaluation findings are purely retrospective and do not inform future grantmaking decisions.  They often come too late — after the next round of grants is already out the door. Poorly aimed, packaged and timed reports chip away at the usefulness of evaluation, leading some trustees  to view the practice as an excessive administrative cost. AbOUT EVALUATIONWhat Trustees SayIf something isn’t working, we need to know it. we need to know that we’re not wasting money.Mariam Noland, Trustee John S. and James L. Knight FoundationThe field is filled with evaluation reports that are unused, in part because their implications are not adequately translated  to practice.Fay Twersky, Director of Impact Planning & Improvement Bill & Melinda Gates FoundationIt’s so interesting and exciting to keep working on what’s in front of us… it’s tough to make yourself stop and look back.William Getty, Trustee and CEO Claude Worthington Benedum FoundationThere is a fiduciary responsibility that boards play, and evaluation should inform the judgments of trustees about the reach and impacts of the foundation’s work. Dr. Kent McGuire, Trustee Wachovia Regional Foundation and California HealthCare FoundationFSG SOCIAL IMPACT ADVISORS    .     PAGE 4    .    THE JAMES IRVINE FOUNDATIONTHE TRUTH cAN bE UNcOMFORTAbLE.Trustees, staff members and CEOs are all personally invested in foundation projects and programs. They want them to succeed, and because reputations and legacies factor into this desire, they sometimes find it painful to face and disclose grim results. Evaluators who are hired by foundations also hesitate to share negative findings candidly with their clients. Though these conflicts of interest often go unspoken, they are real, and they threaten the existence and utility of evaluation. EVALUATION SHOULd bE PRAcTIcAL ANd FOcUSEd ON LEARNINg.While some trustees critique today’s evaluations as ill-timed and unfocused, others see potential. They envision a new type of evaluation: It’s forward looking and directly tied to upcoming decisions. It’s multifaceted and pragmatic in practice. It reports back in real time  to allow for midcourse corrections. It feeds organizational learning and  offers insights that other foundations across the field can run with for a progress-accelerating ripple effect.gIVE ME A cLEAR, bIg PIcTURE — wITH NUMbERS.Many trustees are strategists. They have experienced analytical rigor in business and academia, and they know it helps them make tough calls on when to hold on and when to get out. But it’s a mistake to mire them in details. To make informed decisions, trustees need salient facts about large investments, most often at the program strategy or foundation level and less often about individual grants. They also want information on relevant external trends. Stories are good for conveying emotion and context, but  for many trustees, numbers paint a more telling picture. Social impact may be difficult to describe with quantitative data, but according to trustees, this is no excuse not to measure. IT’S MORE AbOUT RESULTS THAN REcOgNITION.Getting credit for a job well done is nice, but it shouldn’t be the reason for doing evaluation. Most trustees want evaluations to tell them whether the foundation’s efforts have contributed to the goals they set out to achieve. Some observed that no one organization has the resources to make the difference, so it’s more important to focus on the progress being made than on whether the change can be attributed to their foundation’s intervention.FROM EVALUATIONWhat Trustees WantOur board is realistic, and the trustees know that in some areas we will never be able to attribute success solely to our contribution.Christy Pichel, President,  Stuart Foundationyou don’t wait until a project  is complete to see what  you’re getting.Lise Yasui, Trustee William Penn FoundationThe board wants high-level findings, and information on the rigor of the evaluation. They look at it from 30,000 feet… in their role as strategists. Risa Lavizzo-Mourey,  Trustee and CEO Robert Wood Johnson FoundationFoundation boards and staff  often experience a conspiracy  of graciousness.Eugene Cochrane, President The Duke EndowmentFSG SOCIAL IMPACT ADVISORS    .     PAGE 5    .    THE JAMES IRVINE FOUNDATIONHELP ME cHAMPION EVALUATION.When trustees have accurate expectations for what’s achievable (and what’s not) through evaluation, they can help drive demand for it and hold foundations accountable for results. Trustees can help make evaluation a worthwhile endeavor by: •	Asking	questions	early	on	about	program	design,	goals	and	milestones, and how the evaluation will measure success•	Expecting	staff	to	use	data	in	shaping	plans	and	guiding	implementation•	Making	time	at	board	meetings	to	discuss	the	results	of	past	 grants and the implications for the future•	Using	evaluation	results	to	inform	judgments	about	resource	allocations and strategic decisions•	Being	efficient	about	information	requests:	asking	for	only	the	data	they’ll use and taking advantage of information other funders have  already gleaned from granteesFROM EVALUATIONWhat Trustees WantThe trustees want to learn about the type of grants they should be making, and how effective our investments  are. They want to know the impact of what we’re  doing and see tangible, bottom-line outcomes.   Dr. James Knickman, CEO New York State Health Foundation{continued}FSG SOCIAL IMPACT ADVISORS    .     PAGE 6    .    THE JAMES IRVINE FOUNDATION Is your foundation’s use of evaluation relevant to your decision making? Are your board conversations about evaluation productive? Is it time to take time to improve evaluation?EVALUATION kIT FOR TRUSTEESresources: learn about evaluation practicestools: examine your evaluation process Let’s Make Evaluation workA Planning Guide for Foundation TrusteesUnravel common evaluation issues and identify strategies others have used that might work for your foundation. Create a custom action plan for improving performance through evaluation.Let’s consider EvaluationA Self-Assessment Tool for Foundation TrusteesComplete this simple survey to capture trustee points of view on evaluation purpose, method and cost at your foundation.SnapshotsHow Foundation Trustees Use Evaluation Explore brief case studies that show how foundation trustees are successfully employing different types of evaluation for a variety of purposes. Let’s discuss EvaluationA Framework for Trustee Conversations and A Facilitator’s GuideGenerate dialogue on evaluation and compare trustee points of view with research findings from the field. A companion piece helps the facilitator plan and lead the discussion.what’s the difference? How Foundation Trustees View EvaluationGet highlights of key findings from interviews about evaluation with dozens of foundation trustees, CEOs and evaluation experts. AbOUT EVALUATION?What Do You SayFSG SOCIAL IMPACT ADVISORS    .     PAGE 7    .    THE JAMES IRVINE FOUNDATIONe v a l u a t i o n  k i t  f o r  t r u s t e e sSOURcES The Evaluation Kit for Trustees is a project of FSG Social Impact Advisors, with funding from The James Irvine Foundation. It is based on interviews with foundation trustees, CEOs and evaluation experts from across the country, as well as findings documented in From Insight to Action: New Directions in Foundation Evaluation, a report on emerging approaches to evaluation in the philanthropic field produced by FSG and funded by The William and Flora Hewlett Foundation. Irvine engaged Williams Group to develop this kit to help foundations and their trustees act upon the research findings.FSg Social Impact Advisors is an international nonprofit consulting and research organization dedicated to accelerating social progress by advancing the practice of philanthropy and corporate social responsibility. (www.fsg-impact.org)The James Irvine Foundation is a private, nonprofit grantmaking foundation dedicated to expanding opportunity for the people  of California to participate in a vibrant, successful and  inclusive society. (www.irvine.org)williams group helps people and organizations do better  through communication. The firm plans, designs and manages strategic communications programs for a variety of nonprofit  and corporate clients. (www.wgsite.com)AdVISORy bOARdWe would like to thank the members of this project’s Advisory Board for providing their guidance  and feedback.Gale Berkowitz Evaluation Director David and Lucile Packard FoundationPaul Brest President William and Flora Hewlett FoundationMarty Campbell Vice President for Programs The James Irvine FoundationAnne Vally Special Initiatives Officer The James Irvine FoundationJim Knickman President and CEO New York State Health FoundationKen Moore Director of Evaluation and Technology, Trustee  Gordon E. and Betty I. Moore FoundationEdward Pauly Director of Research and Evaluation Wallace FoundationSteven Schroeder Director The James Irvine Foundation and    Robert Wood Johnson FoundationFSG SOCIAL IMPACT ADVISORS    .     PAGE 8    .    THE JAMES IRVINE FOUNDATIONe v a l u a t i o n  k i t  f o r  t r u s t e e sINTERVIEwEESThe following foundation trustees, CEOs and evaluation experts participated in interviews for this project.Annie E. casey FoundationTony Cipollone, Senior Advisor and Vice President for Assessment & AdvocacyThomas Kelly, Manager of EvaluationAssociation of baltimore Area grantmakersBetsy Nelson, Executive Directorcalifornia Healthcare FoundationWalter Noce, DirectorMark Smith, President and CEOGene Washington, Director (also Trustee of the Robert Wood Johnson Foundation)The california wellness FoundationGary Yates, President and CEO, Directorclaude worthington benedum FoundationWilliam Getty, CEO and TrusteeNewt Thomas, TrusteeThe duke EndowmentEugene Cochrane, Presidentgordon E. and betty I. Moore FoundationKen Moore, Director of Evaluation & Technology, TrusteeThe grable FoundationGregg Behr, Executive DirectorHarry c. Trexler TrustMalcom Gross, TrusteeRobert Wood, TrusteeHouston Endowment, Inc.Larry Faulkner, President and DirectorIndependence FoundationPhyllis Beck, TrusteeSusan Sherman, President and CEOBart Silverman, TrusteeThe James Irvine FoundationJim Canales, President and CEO, DirectorToby Rosenblatt, DirectorSteven Schroeder, Director (also Trustee of the Robert Wood Johnson Foundation)Janice wood consulting, Inc.Janice Wood, Principal and Evaluation ExpertJohn S. and James L. knight FoundationRobert Briggs, TrusteeJames Crutchfield, TrusteePaul Grogan, Trustee (also President and Trustee of The Boston Foundation)Alberto Ibargüen, President and TrusteeMariam Noland, Trusteekendall FoundationTed Smith, Executive DirectorNational Philanthropic TrustEileen Heisman, President and CEO, TrusteeNeighborworks Tracey Rutnik, Evaluation ExpertNew york State Health FoundationJim Knickman, President and CEOPatrizi Associates Patricia Patrizi, Principal and Evaluation ExpertFSG SOCIAL IMPACT ADVISORS    .     PAGE 9    .    THE JAMES IRVINE FOUNDATIONe v a l u a t i o n  k i t  f o r  t r u s t e e sINTERVIEwEES {continued}The Pew charitable TrustsLester Baxter, Chief Evaluation OfficerRebecca Rimel, President and CEO, DirectorThe Ralph M. Parsons FoundationWendy Garen, President and CEOGayle Wilson, DirectorRobert wood Johnson FoundationRisa Lavizzio-Mourey, President and CEO, TrusteeRockefeller brothers Fund, Inc.Stephen Heintz, President and TrusteeThe Skillman FoundationCarol Goss, President and CEO, TrusteeSkoll FoundationRoger Martin, DirectorSally Osberg, President and CEO, DirectorStuart FoundationStuart Lucas, DirectorChristy Pichel, President Unihealth FoundationMary Odell, Presidentwachovia Regional Foundation Lois Greco, Senior Vice President and  Evaluation OfficerEleanor Horne, DirectorC. Kent McGuire, Directorwallace Foundation Edward Pauly, Director of Research and Evaluationwalter and Elise Haas FundPamela David, Executive Directorwilliam Penn FoundationMichael Bailin, DirectorFeather Houstoun, PresidentChris James-Brown, DirectorLise Yasui, Directorwoodcock FoundationAlexandra Christy, Executive DirectorStuart Davidson, Trustee (also Trustee of the Acumen Fund, REDF, Rockefeller Philanthropy Advisors, and  Phalarope Foundation)Steven Liebowitz, Program FellowTo learn more, please visit www.fsg-impact.org/ideas/item/trustee_evaluation_tools.html ©2009 tHe James irvine FounDation anD FsG social impact aDvisors. tHis Document may be reprinteD or pHotocopieD For Free Distribution, WitH attribution to its creators.

What's the Difference? How Foundation Trustees View Evaluation

https://www.issuelab.org/resources/2655/2655.pdf

What's the Difference? How Foundation Trustees View Evaluation

Abstract

Similar works

Full text

Available Versions

IssueLab