Drowning in data, drafting a data checklist, and asking “WHY?”

Two poster boys of nonprofit data sanity: Bob Penna (l) and Steve Pratt (r).

Now that TNB Labs is up and running, we’re receiving a lot of requests from nonprofit organizations who are perplexed about how to manage the data that they have, before they plunge any further into data analytics or think about acquiring a new data analysis tool.  This has given me a lot of opportunities to reflect on how difficult it can be for people whose expertise lies elsewhere to orient themselves to data governance.

Steve Pratt‘s blog article “Drowning in Data?” has been a huge inspiration.  In it, he explains the importance of data inventories, and offers to send the Root Cause template to anyone who requests it.  I highly recommend that you send an email to info@rootcause.org, and ask for a copy.

At the same time, as I went over Steve’s template, I had a nagging feeling that we needed something even more elementary.  Remembering my friend Bob Penna‘s exhortation of a few months before, about asking “who, when, where, what, how, and why,” I quickly drafted a data checklist that focused on those basic questions.  When I sent it to Bob, he very quickly returned it with some excellent enhancements; the most brilliant one was to start the checklist with the question “WHY?”  As he very sensibly pointed out, if you can’t come up with a good reason why you are collecting, analyzing, reporting, and archiving information, you might as well stop there.  In the absence of a persuasive answer to the question “why?” there’s no need to ask “who, when, where, what, and how;” in fact there’s no reason to collect it at all.

With that wisdom in mind, I have tweaked the draft of the data checklist, and herewith present it to you for feedback. This version is the result of a Penna/Finn collaboration:

You can view it by clicking on this link.

Before you take a look at it, I recommend reading “Drowning in Data?”  After you’ve perused the spreadsheet, I recommend reading Bob Penna’s book, “The Nonprofit Outcomes Toolbox.”

 

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License.

What I learned about outcomes management from Robert Penna

Yesterday, along with a number of colleagues and friends from Community TechKnowledge, I had the privilege of attending a training by Robert Penna, the author of The Nonprofit Outcomes Toolbox.

As you probably  know, I’ve been on a tear about outcomes measurement for a few months now; the current level of obsession began when I attended NTEN’s Nonprofit Data Summit in Boston in September.  I thought that the presenters at the NTEN summit did a great job addressing some difficult issues – such as how to overcome internal resistance to collecting organizational data, and how to reframe Excel spreadsheets moldering away in file servers as archival data.  However, I worked myself into a tizzy, worrying about the lack, in that day’s presentations, of any reference to the history and literature of quantitative analysis and social research.  I could not see how nonprofit professionals would be able to find the time and resources to get up to speed on those topics.

Thanks to Bob Penna, I feel a lot better now.  In yesterday’s training, he showed me and the CTK team just how far you can go by stripping away what is superfluous and focusing on what it really takes to use the best outcomes tools for job.  Never mind about graduate level statistics! Managing outcomes may be very, very difficult because it requires major changes in organizational culture – let’s not kid ourselves about that.  However, it’s not going to take years out of each nonprofit professional’s life to develop the skill set.

Here are some other insights and highlights of the day:

• Mia Erichson, CTK’s brilliant new marketing manager, pointed out that at least one of the outcomes tools that Bob showed us could be easily mapped to a “marketing funnel” model.  This opens possibilities for aligning a nonprofits programmatic strategy with its marcomm strategy.
• The way to go is prospective outcomes tracking, with real time updates allowing for course correction.  Purely retrospective outcomes assessment is not going to cut it.
• There are several very strong outcomes tools, but they should be treated as we treated a software suite that comprises applications that are gems and applications that are junk.  We need to use the best of breed to meet each need.
• If we want to live in Bob Penna’s universe, we’re going to have to change our vocabulary.  It’s not “outcomes measurement – it’s “outcomes management.” The terms “funder” and “grantmaker” are out – “investor” is in.

Even with these lessons learned, it’s not a Utopia out there waiting for nonprofits that become adept at outcomes management.  Not only is it difficult to shift to an organizational culture that fosters it, but we have to face continuing questions about how exactly the funders (oops! I should have said “investors”) use the data that they demand from nonprofit organizations.  (“Data” is of course a broad term, with connotations well beyond outcomes management.  But it’s somewhat fashionable these days for them to take an interest in data about programmatic outcomes.)

We should be asking ourselves, first of all, whether the sole or primary motivation for outcomes management in nonprofits should be the demands of investors.  Secondly, we should be revisiting the Gilbert Center’s report, “Does Evidence Matter to Grantmakers? Data, Logic, and the Lack thereof in the Largest U.S. Foundations.” We need to know this. Thirdly, we should be going in search of other motivations for introducing outcomes management.  I realize that most nonprofits go forward with it when they reach a point of pain (translation:  they won’t get money if they don’t report outcomes). 

During a break in Bob’s training, some of my CTK colleagues were discussing the likelihood that many nonprofit executives simply hate the concept of outcomes management.  Who wants to spend resources on it, if it subtracts from resources available for programmatic activities?  Who wants to risk finding out (or to risk having external stakeholders find out) that an organization’s programs are approximately as effective as doing nothing at all?  Very few – thus the need to find new motivations, such as the power to review progress and make corrections as we go.  I jokingly told my CTK colleagues, “the truth will make you free, but first it will make you miserable.”  Perhaps that’s more than a joke.