Visualizing the role of data for mission-based organizations

First of all, I want to say that I distrust the term “data-driven organization,” even though some very highly respected colleagues use it in their writings and their conversations.

I firmly believe that if your organization is driven by data, you’re stopping too soon.

It’s important to roll that data (which is raw material) into information (which has been sorted and analyzed), to roll that information into knowledge (which has been enhanced by understanding of context), and to roll that knowledge into wisdom (which been enhanced by experience and intuition). From there you can proceed to good decisions, and ultimately to mission success; moreover, at this point you of course now have more data.  From there, it’s an opportunity for continuous improvement and possibly even further innovation.

My challenge right now is to come up with a clear image to convey this.  The one that springs naturally to my mind is linear, but trusted advisors seem to favor a more cyclical illustration.

Please take a look at these two logos (which were created by yours truly), and tell me which one gets my message across most effectively:

Linear logo

Cyclical logo

And please feel free to post comments here (or send me email) to elaborate on your thoughts about this!

Outcomes measurement for nonprofits: Who does the analysis?

I invite you to participate in this survey, bearing in mind that it is for recreational purposes, and has no scientific value:

There are many reasons that this survey is of dubious value, for example:

• I’m using a convenience sample of respondents, so there’s no way to generalize about the findings.

• As the survey’s principal (and only) investigator, I’m hopelessly biased in favor of certain responses and against others; no effort has been made to create a neutral instrument.

• No pilot testing has been done to ensure that the choices offered are both exhaustive and mutually exclusive.

The list could go on, but I’ll leave it at that.  Although most of my training is in qualitative social research, I have taken undergraduate and graduate level courses on quantitative research, and the points I made about what’s wrong with my survey are what I could pull out of memory without consulting a standard text on statistics.

In other words, when it comes to quantitative analysis, I know just enough to be dangerous.

Meanwhile, I worry about nonprofit organizations that are under pressure to collect, analyze, and report data on the outcomes of their programs.  There are a lot of fantastic executive directors, program managers, and database administrators out there – but it’s very rare for a nonprofit professional who falls into any of those three categories to also have solid skills in quantitative analysis and social research methods.  Nevertheless, I know of plenty of nonprofit organizations where programmatic outcomes measurement is done by an executive director, program manager, or database administrator whose skill set is very different from what the task demands.  In many cases, even if they come up with a report, the nonprofit staff members may not even be aware that what have done is presented a lot of data, without actually showing that there is any causal relationship between the organization’s activities and the social good that they are in business to deliver.

Let’s not be too hasty in deprecating the efforts of these nonprofit professionals.  They are under a lot of pressure, especially from grantmaking foundations, to report on programmatic outcomes.  In many cases, they do the best they can to respond, even if they have neither the internal capacity to meet the task nor the money to hire a professional evaluator.

By the way, I was delighted to attend gathering this fall, in which I heard a highly-regarded philanthropic professional ask a room full of foundation officers, “are you requiring $50,000 worth of outcomes measurement for a $10,000 grant?” It’s not the only question we need to ask, but it’s an extremely cogent one!

I’d love to see nonprofit professionals, philanthropists, and experts in quantitative analysis work together to address this challenge.

We should also be learning lessons from the online tools that have already been developed to match skilled individuals with nonprofit professionals who need help and advice from experts.  Examples of such tools include the “Research Matchmaker,” and NPO Connect.

We can do better.  It’s going to take time, effort, money, creativity, and collaboration – but we can do better.

The state of nonprofit data: Uh-oh!

The Nonprofit Technology Network (NTEN) has released a report prepared by Idealware on the current state of nonprofit data.  Highly recommended!

Some of the news it contains is scary.  In our sector, we currently aren’t very successful at collecting and analyzing the most crucial data.  For example, only 50% of the respondents reported that their nonprofit organizations are tracking data about the outcomes of clients/constituents.

According to the survey respondents, there are daunting barriers to tracking and using data:

• issues related to collecting and working with data (27 percent of responses).
• lack of expertise (24 percent of responses)
• issues of time and prioritization (22 percent of responses).
• challenges with technology (23 percent).

Page 13 of the report features a chart that I find especially worrisome.  It displays of types of data that nonprofit organizations should or could be using, with large chunks falling into three chilling categories:

• we don’t know how to track this
• we don’t have the technology to effectively track this
• we don’t have the time/money to effectively track this

In the case of data about outcomes, 17% lack the knowledge, 20% lack the technology, and 22% lack the time or money (or both) to track it.

Are you scared yet?  I confess that I am.  Perhaps half of all nonprofits surveyed don’t know – and don’t have the resources to find out – whether there is any causal relationship between what their activities and the social good that they are in business to achieve.

And that’s just programmatic outcomes.  The news is also not very encouraging when it comes to capturing data about organizational budgets, constituent participation in programs, and external trends in the issue areas being addressed by nonprofit organizations.

So much for the bad news.  The good news is that now we know.

It takes some courage to acknowledge that the baseline is so low.  I applaud Idealware and NTEN for creating and publishing this report.  Now that we know, we can address the problem and take effective action.