I’m sitting here, reflecting on the Innovation Network’s “State of Evaluation 2012” report.
I encourage you to download it and read it for yourself; start with pages 14 and 15. These two pages display infographics that summarize what funders (also known as “grantors,” or if you’re Bob Penna, as “investors”) and nonprofits (also known as “grantees”) are reporting about why they do evaluation and what they are evaluating.
Regardless of whether you call it evaluation, impact assessment, outcomes management, performance measurement, or research – it’s really, really difficult to ascertain whether a mission-based organization is delivering the specific, positive, and sustainable change that it promises to its stakeholders. Many organizations do an excellent job at tracking outputs, but falter when it comes to managing outcomes. That’s in part because proving a causal relationship between what the nonprofit does and the specific goals that it promises to achieve is very costly in time, effort, expertise, and money.
But assuming that a mission-based organization is doing a rigorous evaluation, we still need to ask: what is done with the findings, once the analysis is complete?
What the aforementioned infographics from the “State of Evalution 2012” tell me is that both grantors and grantees typically say that the most important thing they can do with their outcome findings is to report them to their respective boards of directors. Considering the depth of the moral and legal responsibility that is vested in board members, this is a pretty decent priority. But it’s unclear to me what those boards actually do with the information. Do they use it to guide the policies and operations of their respective organizations? If so, does anything change for the better?
If you have an answer to the question of how boards use this information that is based on firsthand experience, then please feel to post a comment here.
The Nonprofit Technology Network (NTEN) has released a report prepared by Idealware on the current state of nonprofit data. Highly recommended!
Some of the news it contains is scary. In our sector, we currently aren’t very successful at collecting and analyzing the most crucial data. For example, only 50% of the respondents reported that their nonprofit organizations are tracking data about the outcomes of clients/constituents.
According to the survey respondents, there are daunting barriers to tracking and using data:
- issues related to collecting and working with data (27 percent of responses).
- lack of expertise (24 percent of responses)
- issues of time and prioritization (22 percent of responses).
- challenges with technology (23 percent).
Page 13 of the report features a chart that I find especially worrisome. It displays of types of data that nonprofit organizations should or could be using, with large chunks falling into three chilling categories:
- we don’t know how to track this
- we don’t have the technology to effectively track this
- we don’t have the time/money to effectively track this
In the case of data about outcomes, 17% lack the knowledge, 20% lack the technology, and 22% lack the time or money (or both) to track it.
Are you scared yet? I confess that I am. Perhaps half of all nonprofits surveyed don’t know – and don’t have the resources to find out – whether there is any causal relationship between what their activities and the social good that they are in business to achieve.
And that’s just programmatic outcomes. The news is also not very encouraging when it comes to capturing data about organizational budgets, constituent participation in programs, and external trends in the issue areas being addressed by nonprofit organizations.
So much for the bad news. The good news is that now we know.
It takes some courage to acknowledge that the baseline is so low. I applaud Idealware and NTEN for creating and publishing this report. Now that we know, we can address the problem and take effective action.