Tag Archives: programmatic outcomes

Let’s revisit the concept of failure-friendliness

Eight years ago, I wrote a blog article about failure-friendliness in nonprofit technology. It was very much inspired by my friend and colleague, Dan Scharfman. Since Dan died this week, and this is also a week when I have been thinking hard about the obstacles that nonprofit organizations face in tracking their outcomes, it seems appropriate to reprise the article here and now. Having coped with the need for failure-friendliness in nonprofit technology for years, I see that my understanding is still superficial when it comes to the difficulties that nonprofits have in acknowledging programmatic failure. I invite your thoughts on how we can be more transparent about and more open to learning from failure. Meanwhile, special thanks go out to Beth Kanter, for her outstanding blog articles on this topic.

FAIL stamp

Wed 26 Jan 2005 05:41 PM EST

The term “failure-friendly organization” was first introduced to me by a colleague I revere – Dan Scharfman of Baird Associates.

My first impression was that he was an unlikely champion of failure, since Massachusetts is well-supplied with nonprofit organizations that consider the technology services that he has provided to them very successful indeed.

However, many of us in the nonprofit sector have seen the following things happen with major implementations or upgrades:

  • The technology doesn’t work, or doesn’t work nearly as well as it should;
  • The intended users won’t have anything to do with the technology;
  • Major changes in technology in the outside world quickly render the organization’s choices obsolete;
  • Programmatic priorities change, and the technology is all but irrelevant;
  • The organization has not factored in the shocking cost of customizing, tweaking, maintaining, and upgrading the technology.

Although techies vary greatly in their attitudes about projects that don’t work out, we also tend to make tacit assumptions that everyone concerned understands that we are not engaged in an exact science but in an evolving process.

Techies also tend to regard failure as pretty interesting – as a good source of information about what ought to be fixed when Version 97.53.01 of the software is released.  We also enjoy working on cool tools, even if such tools don’t actually deliver the outcomes desired by those who are underwriting the project.  This form of process orientation can be less than endearing to decision-makers in nonprofit organizations.

Oddly enough, nonprofit workers tend to be very good at process orientation when they are on familiar ground.

Sometimes this process orientation is a grim necessity, with governmental agencies strictly mandating, auditing, and enforcing protocols that nonprofits must follow in order to maintain their tax-exempt status, accreditation, or contracts for services.  These are headaches that would impel just about any organization or individual to worry a great deal about operating according to plan and documenting the process, rather than ensuring a specific outcome. This of course is a very “functional” (or “instrumental“) form of process orientation.

A more “expressive” form of process orientation is also frequently seen in nonprofit organizations – manifesting as a desire to be flexible and responsive to changing situations, or as a desire to arrive at decisions through consensus.  However, it can be difficult to extend that attitude to technology, which tends to be difficult for non-specialists to comprehend, time-consuming, and expensive.

Another challenge is that organizations and individuals (including yours truly) can be reluctant to cut their losses, and say, “This isn’t working.  Let’s stop, figure out why, and decide on some next steps.”  Of course, in some settings, the decoded version of this message is “Let’s find someone to blame and punish…maybe YOU.”

Yikes!

Is there any solution in sight?  I only wish I had something certain and simple to offer.  Here are a few ideas, although none of them come with guarantees of success:

  • Techies need to understand the nonprofit organizational cultures in which they are operating.  Progress toward this goal is possible if the techies listen, ask questions, and listen some more.  These conversations should start early in the planning phase.
  • Nonprofit workers need to understand how technology innovations and implementations happen in real life, and have a reasonable idea of what factors can lead to unexpected outcomes in technology projects.  Progress is possible if – yes, you guessed it – the nonprofit workers listen, ask questions, and listen some more.
  • Everyone needs to cooperate in creating incentives for spotting, discussing, and correcting errors rather than evading their detection.  I freely admit that I always find it easier to do these things when the mistake was made by someone else, but am always striving to do better.

I wish I could remember who it was that first said to me, “This is not about one person against another. This is about our team against the problem.”  Anyone who can say that is a saint, a boddhisatva, a tzadik, or an unusually effective manager.

“We count our successes in lives”

Brent James

Brent James is one of my new heroes.  He’s a physician, a researcher, and the chief quality officer of Intermountain Healthcare’s Institute for Health Care Delivery Research.

We had a very inspiring telephone conversation this afternoon, about whether the lessons learned from evidence-based medicine could be applied to nonprofits that are seeking to manage their outcomes.  We also swapped some stories and jokes about the ongoing struggle to document a causal relationship between what a health care organization (or a social service agency, or an arts group, or an environmental coalition, for that matter) does and what the organization’s stated aims are.  In fact, documenting that an organization is doing more good than harm, and less harm than doing nothing at all, continues to be a perplexing problem.  The truth may be less than obvious – in fact, it may be completely counter-intuitive.

In this phone conversation, we also waded into deep epistemological waters, reflecting on how we know we have succeeded, and also on the disturbing gap between efficacy and effectiveness.

It’s not merely a philosophical challenge, but a political one, to understand where the power lies to define success and to set the standards of proof.

I doubt that this is what William James (no relation to Brent, as far as I know) had in mind when he referred to success as “the bitch-goddess,” but there’s no doubt that defining, measuring, and reporting on one’s programmatic success is a bitch for any nonprofit professional with intellectual and professional integrity.  It’s both difficult and urgent.

What particularly struck me during my conversation with Brent was his remark about Intermountain Healthcare:

“We count our successes in lives.”

On the surface, that approach to counting successes seems simple and dramatic.  The lives of patients are on the line.  They either live or die, with the help of Intermountain Healthcare.  But it’s really a very intricate question, once we start asking whether Intermountain’s contribution is a positive one, enabling the patients to live the lives and die the deaths that are congruent with their wishes and values.

These questions are very poignant for me, and not just because I’m cancer patient myself, and not just because yesterday I attended the funeral of a revered colleague and friend who died very unexpectedly.  These questions hit me where I live professionally as well, because earlier this week, I met with the staff of a fantastic nonprofit that is striving to do programmatic outcomes measurement, and is faced with questions about how to define success in a way that can be empirically confirmed or disconfirmed.  Their mission states that they will help their clients excel in a specific industry and in their personal lives.  They have a coherent theory of change, and virtually all of their criteria of professional and personal success are quantifiable.  Their goals are bold but not vague. (This is a dream organization for anyone interested in outcomes management, not to mention that the staff members are smart and charming.)  However, it’s not entirely clear yet whether the goals that add up to success for each client are determined solely by the staff or by the client or some combination thereof.  I see it as a huge issue, not just on an operational level, but on a philosophical one; it’s the difference between self-determination and paternalism.  I applaud this organization’s staff for their willingness to explore the question.

When Brent talked about counting successes in terms of lives, I thought about this nonprofit organization, which defines its mission in terms of professional and personal success for its clients.  The staff members of that organization, like so many nonprofit professionals, are ultimately counting their successes in lives, though perhaps not as obviously as health care providers do.  Surgeons receive high pay and prestige for keeping cancer patients alive and well – for the most part, they fully deserve it.  But let’s also count the successes of the organization that helps a substantial number of people win jobs that offer a living wage and health insurance, along with other benefits such as G.E.D.s, citizenship, proficiency in English, home ownership, paid vacations, and college educations for the workers’ children. Nonprofit professionals who can deliver that are also my heroes, right up there with Brent James.  While we’re holding them to high standards of proof of success, I hope that we can find a way to offer them the high pay and prestige that we already grant to the medical profession.

The state of nonprofit data: Uh-oh!

The Nonprofit Technology Network (NTEN) has released a report prepared by Idealware on the current state of nonprofit data.  Highly recommended!

Some of the news it contains is scary.  In our sector, we currently aren’t very successful at collecting and analyzing the most crucial data.  For example, only 50% of the respondents reported that their nonprofit organizations are tracking data about the outcomes of clients/constituents.

According to the survey respondents, there are daunting barriers to tracking and using data:

  • issues related to collecting and working with data (27 percent of responses).
  • lack of expertise (24 percent of responses)
  • issues of time and prioritization (22 percent of responses).
  • challenges with technology (23 percent).

Page 13 of the report features a chart that I find especially worrisome.  It displays of types of data that nonprofit organizations should or could be using, with large chunks falling into three chilling categories:

  • we don’t know how to track this
  • we don’t have the technology to effectively track this
  • we don’t have the time/money to effectively track this

In the case of data about outcomes, 17% lack the knowledge, 20% lack the technology, and 22% lack the time or money (or both) to track it.

Are you scared yet?  I confess that I am.  Perhaps half of all nonprofits surveyed don’t know – and don’t have the resources to find out – whether there is any causal relationship between what their activities and the social good that they are in business to achieve.

And that’s just programmatic outcomes.  The news is also not very encouraging when it comes to capturing data about organizational budgets, constituent participation in programs, and external trends in the issue areas being addressed by nonprofit organizations.

So much for the bad news.  The good news is that now we know.

It takes some courage to acknowledge that the baseline is so low.  I applaud Idealware and NTEN for creating and publishing this report.  Now that we know, we can address the problem and take effective action.

%d bloggers like this: