Tag Archives: outcomes management

In search of my next vocation!

"Excelsior!" Cartoon by James Thurber

“Excelsior!”   (Cartoon by James Thurber)

After five very productive years at Tech Networks of Boston (TNB), I am now looking for my next professional challenge. I’m ready for a career shift! I’ve notified the leadership at TNB, so this is not a covert search.

If you know about any job opportunities at organizations that need someone with my skill set, I’d love to hear about them. In my next job, I’d like to focus on some or all of the following:

  • Weaving networks among nonprofit organizations in order to build collaboration, peer learning, and communities of practice.
  • Building the capacity of philanthropic and nonprofit organizations to achieve and document their desired outcomes.
  • Fostering equity, inclusion, social justice, and corporate social responsibility.
  • Aiding philanthropic and nonprofit organizations in seamlessly matching resources with needs.
  • Establishing best practices in the strategic use of information and communication technologies among mission-based organizations.
  • Facilitating candid dialogue and successful collaborations between grantmakers and grantees.

I invite you to peruse my LinkedIn profile and my résumé, and to get in touch with me about any contacts or opportunities that you’d like to suggest.

Please help me find new ways to serve organizations and individuals who are working to make the world a better place!

Deborah Elizabeth Finn – résumé – June 2018

 

 

 

 

Every nonprofit needs a theory of change for its technology. . .and for its evaluation process

if then

I’ve spent a lot of my professional life (thus far) thinking about the missions of nonprofit organizations, and about information/communication technologies for nonprofits.

In the past few years, it’s become fashionable to talk about the importance of a “theory of change” for nonprofits.  This is merely a way of underlining the importance of making an explicit statement about the causal relationship between what a nonprofit organization does and the impact that it has promised to deliver.  I applaud this!  It’s crucial to say, “if we take all of the following resources, and do all of the following actions, then we will get all of the following results.”  An organization that lacks the capacity to marshal those resources and take those actions needs to reconsider, because it is on track to fail. If its capacity is not aligned with its commitment, it should acquire the resources or change its commitment to results.  Of course, it some cases, it will merely need to revise its theory of change.  In any case, it will have to work backward from its mission, and understand how each component contributes to achieving it.

This kind of thinking has lead to a lot of conversations (and a lot of anxiety) in the nonprofit sector about performance measurement, outcomes management, evaluation, and impact assessment.

I’d love to have some of this conversation focus on the information/communication technologies that nonprofit organizations are using.  In other word, it’s time to be explicit about a theory of change that explains in detail how every component of the technology an organization uses contributes (directly or indirectly) to its ability to deliver a specific kind of social, cultural, or environmental impact.

Likewise, I’d love to have the conversation address the ways in which the efforts of a nonprofit organization’s performance measurement, outcomes management, evaluation, or impact assessment team contributes (directly or indirectly) to its ability to deliver the kind of impact that it promised its stakeholders.

 

 

TNB Labs has launched, and its first priority is data services for nonprofits!

tnb labs logo july 2016

 

Greg Palmer and I are very pleased to announce the launch of TNB Labs, a company dedicated to providing nonprofit organizations with high quality data services and program management.  TNB Labs works with organizations to audit and assess data management methodology, develop and implement data standards, and provide structural oversight through data governance to organizations of all sizes.

Through our partnership with Tech Networks of Boston, we have had a unique opportunity to listen to hundreds of stakeholders at TNB’s Roundtables uncover a need to elevate the role of data in nonprofit organizations.  TNB Labs is focused on improving the capacity of nonprofit organizations at every stage of the outcome management process.

The immediate priorities of TNB Labs are:

1) Providing Master Data Management (MDM) services to nonprofit organizations in support of their missions, focusing on data governance, data quality, data modeling, data visualizations, and program evaluation.

2) Providing workforce program management for Desktop Support Technicians (DST), Data Support Analysts (DSA), and Data Analytics/Data Evaluation entry level professionals.

3) Managing the TNB Roundtable series, which is now jointly owned by Tech Networks of Boston and TNB Labs.

TNB Labs is led by Greg Palmer (chief executive officer), and Deborah Elizabeth Finn (chief strategic officer).  The other co-founders are Bob Master (former CEO of Commonwealth Care Alliance) and Susan Labandibar (founder of Tech Networks of Boston).

TNB Labs is here to solve your problems.  Please contact us with any questions and comments you have about TNB Labs, or to learn more about data management or program management services that might be helpful to your organization.

Best regards from Deborah and Greg

Greg Palmer
gpalmer@tnblabs.org
508.861.4535

Deborah Elizabeth Finn
definn@tnblabs.org
617.504.8188

TNB Labs, LLC
PO Box 2073
Framingham, MA 01703
www.tnblabs.org

 

 

It’s not just a half-day outcomes management training for nonprofit executives – it’s an occasion for rejoicing!

snoopy happy dance

For more than two years, I have been worrying aloud about the lack of training for nonprofit professionals who want to lead their organizations in implementing outcomes management and data visualization.  Today I’m rejoicing, because Tech Networks of Boston opened registration for a free (and sales-pitch-free) half-day outcomes management training for nonprofit executives.

It’s happening in April because some wonderful allies have stepped up – such as TNB’s co-hosts, the Mel King Institute and the College of Public and Community Service at the University of Massachusetts, and the wonderful Kathryn Engelhardt-Cronk of Community TechKnowledge, who will serve as our trainer.

This isn’t the full series of three day-long trainings on outcomes management and outcomes data visualization that I had originally envisioned, and that I still hope we can organize.  If we are able to do that, the other trainers will be the equally wonderful Beth Kanter and Georges Grinstein.  Right now, I’m looking at plans for Kathryn’s half-day outcomes management training as a miracle in itself, but also as the thin edge of the wedge.  (If you prefer more up to date jargon, you can call it a “proof of concept.”)

Of course, my thinking has become even more grandiose since I originally came up with the idea of a three-day outcomes/data viz training series.  Now I’m thinking in terms of a “Massachusetts Institute of Nonprofit Technology,” in which the first initiative would be a degree program in nonprofit data analysis.

Let’s take this training opportunity, which will be brief in comparison to the more elaborate programs that I’ve envisioned, and build on it!

 

 

How much fun is the Nonprofit Technology Conference? This much fun. (Plus some thoughts about shifting from tactical to strategic support of nonprofit organizations.)

Deborah is delighted by the artist's rendition of a concept of Tech Networks of Boston's. The photo was taken at the Netsuite.Org booth, at the 2015 Nonprofit Technology Conference .

Photo by Peggy Duvette of Netsuite.Org.

The good folks of Netsuite.Org had a great idea for their exhibit area at the Nonprofit Technology Conference this year.  They asked attendees to describe their technology visions in three words.  I chose “shared” “data,” and “outcomes.” and an artist quickly drew up a visual to express this.  (Unfortunately, I did not note down her name; I hope I can find it in order to give her proper credit for her work.)  The photo shown above was taken by Peggy Duvette, and as you can see, I was delighted to see this concept, which is part of Tech Networks of Boston’s strategic thinking, become part of the patchwork quilt of ideas that were being expressed.

Here’s a close-up of the TNB concept:

I (Deborah) took this photo at the Netsuite.Org booth, at the 2015 Nonprofit Technology Conference. Alas, I did not note down the name of the artist who did this drawing.

I took this photo at the Netsuite.Org booth, at the 2015 Nonprofit Technology Conference. Alas, I did not note down the name of the artist who did this drawing.

At TNB, we are thinking more and more about collaborative technology management – not just in terms of how we work with our nonprofit clients, but also about how clusters of NTAPs and nonprofits can work together toward a shared long term goal.   We have great relationships (and in many cases, shared nonprofit clients) with some great local nonprofit technology assistance providers, such as Annkissam* and 501Partners.  The three NTAPs are already collaborating on a series of sales-pitch-free evenings in which local nonprofit professionals are offered pro bono tech consultations.

However, the potential exists to do so much more, especially considering how many clients we share.

Wouldn’t it be great if the three NTAPs could offer their shared clients the following:

1) Seamless integration of TNB, AK, and 501P’s services.

2) Shared best practices for clusters of nonprofits with similar programs, operations, or missions.

3) Coordinated outcomes measurement and management for nonprofits that have overlapping constituencies.

The joy of #15NTC is in realizing that although we are just three NTAPs in one region, we are part of a wider movement.  In fact, if you were to look at the entire collection of artist’s renderings that were done at the Netsuite.Org exhibit area, you’d see that many nonprofit organizations are on the cusp of dreaming this dream.  Most of in the nonprofit sector understand that for lasting positive change in the world, one program at a single nonprofit organization is not enough.  The future is in sharing and coordinating our work.  What if nonprofit technology assistance providers started with that challenge, rather than the challenge of keeping a network server from crashing?  The emphasis would shift from the tactical support of nonprofits to the strategic support of their missions.  And by “missions,” I don’t mean vague statements; I mean specific (and even quantifiable) positive changes that nonprofit profits have committed themselves to delivering to their stakeholders.

Because mission achievement is why we all get up in the morning to do our jobs.

And because building a nonprofit technology movement that supports mission achievement is the best possible reason for participating in the Nonprofit Technology Conference.

 

* I also serve Annkissam directly as a consultant.

 

 

 

The Massachusetts Institute of Nonprofit Technology: Let’s Do This!

Massachusetts Institute of Nonprofit Technology

 

We need a Massachusetts Institute of Nonprofit Technology, and I can tell you what degree program we need to establish first:  Bachelor of Nonprofit Data.

The inspiration for this comes from many conversations with many people, but I’d especially like to credit Susan Labandibar, Julia Gittleman, and Laura Beals for pointing out, in their different ways, that one of the most pressing real-life challenges in nonprofit technology today is finding people who can bridge between the outcomes / impact assessment / evaluation / research team (on one hand) and the information systems team (on the other hand) at a nonprofit organization.

Not that I’m a professional full-time data analyst myself, but if I were, I’d find the numbers, and start doing the math:

  • How many brilliant computer scientists are graduating right here in Massachusetts every year from our best high schools, colleges, and universities?
  • Of those graduates, what percentage have strong skills in database design, database development, database management, or data analysis?
  • Of those who have strong data skills, what percentage would be eager to use their geek skills for good, if they were offered an attractive career ladder?

That’s our applicant pool for the Massachusetts Institute of Nonprofit Technology.  (Or MINT, if you prefer.)

Now, let’s figure out the absolute minimum of additional knowledge that these computer science graduates would need in order to be the kind of data analysts who could bridge between the outcomes / impact assessment / evaluation / research team and the information systems team  at a nonprofit:

  • Outcomes measurement
  • Outcomes management
  • Impact assessment
  • Evaluation
  • Social research methods
  • Knowledge management
  • Organizational cultures of nonprofits
  • Nonprofit operations
  • Organizational cultures of philanthropic foundations

That’s our basic curriculum.

If we want to expand the curriculum beyond the basics, we can add these elective subjects:

  • Nonprofit budgeting
  • Group dynamics
  • Ethics
  • Etiquette
  • Negotiation
  • Project management
  • Appreciative inquiry
  • Meeting facilitation

All of these electives would pave the way for other degree programs, in which they would also be extremely useful:

  • Bachelor of Nonprofit Systems Engineering
  • Bachelor of Nonprofit Web Development
  • Bachelor of Nonprofit Help Desk Support
  • Bachelor of Nonprofit Hands On Tech Support
  • Bachelor of Nonprofit Social Media

I already have my eye on some great local colleagues who could be the faculty for the Bachelor of Nonprofit Data program.  In addition to Susan, Julia, and Laura, I’d want to recruit these folks:

Please note that three members of the TNB team top the list of potential faculty members.  Why?  Because I work there, and because TNB has set a Big Hairy Audacious Goal of developing the careers of 1,000 technology professionals. This undertaking would be very congruent with its vision!

However, setting up the Massachusetts Institute of Nonprofit Technology must be a collaborative effort.  It will take a strong network of colleagues and friends to make this happen.

Do you think that this is needed?  Do you think my plan needs a lot of work?  Do you have any ideas or resources that you’d like to suggest?  Please feel free to use the comments section here to share your thoughts.

Visualizing the role of data for mission-based organizations – Round II

I am much obliged to all the good folks who have posted suggestions and feedback about my first attempt to create an image that would represent my thinking on the role of data in mission-based organizations.  Likewise, those who emailed me their thoughts deserve thanks!
I’ve created a revised version that incorporates some of the feedback.  Before you take a look at it, please bear in mind that:

  1. I am not a graphic designer.
  2. I am not attempting to create a graphic that illustrates everyone’s ideas about the role of data in a mission-based organization.  I am merely trying to illustrate my ideas.
Visualizing the role of data for mission-based organizations - Round II

Visualizing the role of data for mission-based organizations – Round II

Item #2 on the list notwithstanding, I am enjoying very much the opportunity to learn more about what others in the field think about (and visualize) when they ponder the role of data in our sector.  Once again, I invite you to post your reflections, suggestions, and questions in the comments section here on this blog.

“Forgive and remember”

Forgive and remember

File this under “lessons about failure that the nonprofit sector can learn from medical sociology.”

Forgive and Remember: Managing Medical Failure, by Charles Bosk, is a classic of medical sociology, an analysis of how coping with failure is built into the training that surgeons receive in hospital rounds, mobidity and mortality conferences, and other settings.

Please note that I am not claiming that surgeons themselves have a lot to teach us about dealing with failure, because my experience is that while their sub-culture does have rituals and protocols that they enact privately, they still have a way to go in transparency and accountability to others.

This has been my experience in three instances of major surgery:

  1. Surgeon did not follow instructions given by the specialist physician managing my condition.  Acknowledgement: Partial.  Apology: No.
  2. Surgeon did not inform me that the tumor to be removed might be malignant and require addition surgery until I was under anesthesia. Acknowledgement: Yes, after I complained. Apology: Yes, after I complained.
  3. Surgeon did not respect my request regarding administration of anesthesia:  Acknowledgement: Yes, after I complained. Apology: No.

Not that I am bitter.

But let’s face it:  most of us are highly invested in showing the world that we are skillful, trustworthy, and deserving of whatever prestige is ascribed to us.  As a patient, I naturally blamed the surgeons, not only for their errors in judgement, but for the instances in which they failed to acknowledge or apologize for their mistakes.  As a fellow human being, I completely empathize with their reluctance.  I imagine that that reluctance is more acute among professionals who have to cut people open.  Their work is obscenely invasive but often lifesaving,  and therefore must maintain an impeccably trustworthy reputation.

That’s why Brent James is one of my heroes, along with surgeons and physicians like him who are putting it on the line for evidence-based practice.  There will be no accountability, transparency, or improvement in health care unless successes and failures are accurately documented.  Those results must then be carefully analyzed, made available to the public in appropriate ways, and used to improve their efforts.

As with medicine, so with other mission-based organizations.  We need to track outcomes, acknowledge failures, and then do better.  If it takes a pink feather boa and an amusing ritual for nonprofits to get there, I’m all for it, though I’m not expecting surgeons to adopt the feather boa.

As for the slogan, “forgive and remember,” I think of it as both a spiritual and a practical precept.  We not only need to forgive ourselves and others when we have failed – we also need to bear the lessons of failure in mind.  Both individuals and organizations not only need to keep learning, but to take appropriate action to protect those who are at risk.

I used to work in violence prevention, and for me, one of the most heart-rending aspects of it was the well-documented difficulty in stopping offenders from repeatedly battering their loved ones.  In some cases, they simply didn’t see their behavior as abusive, or their loved ones didn’t see any alternative to accepting abuse.

As I reflect on that today, it drives home very painfully the lesson that we cannot always change others, or even control a specific behavior of theirs.  The old cliche that they “have to really want to change” is true, and it’s also true that not everyone who wants to change can do so. This is the really difficult side of facing failure for nonprofit organizations – in some cases, there may be no alternative to severing ties with individuals or organizations, if the organization is going to face its failure and move on.  It’s going to take more than a pink feather boa, a “joyful funeral,” or a FailFaire to get past that.  When the well-being of vulnerable people is on the line, there are cases where forgiving and remembering is crucial, but it isn’t enough.


Bonus item:

Q:  How many psychotherapists does it take to change a lightbulb?

A:  Only one, but the light bulb really has to want to change.

Adventures in failure (and ritual studies): The “joyful funeral”

Beth Kanter and I are not twins who were separated at birth, but we have some things in common.

So perhaps if we were twins, I wouldn’t be the evil twin (I hope) but the lagging twin.  She’s succeeding at writing in a very engaging and helpful way about failure, and I am definitely benefiting from that.  Thinking about how to acknowledge failure flows very naturally from my current absorption in outcomes management for nonprofit organizations.

Beth recently published a blog article on “Six Ways Nonprofits Learn from Affordable Losses or Little Bets to Improve Impact” that appealed to me greatly, mostly because some the practices described have a ritual component.

I was especially excited when I saw that Beth had included the “Joyful Funeral” custom that was created by Moms Rising.  I had heard rumors of this ritual in nonprofit management circles, but couldn’t remember the details.  Fortunately, Beth’s article includes a cool video, in which she interviews Ashley Boyd about what it really entails.

Now, unlike Beth, I studied sociology of religion as a graduate student, and have a longstanding interest in ritual studies.  Regardless of one’s religious beliefs and affiliation – or lack thereof – it’s easy to see that ritual often has great power in assisting human communities that are confronted by change or loss.

Let’s look at the characteristic stages of a “rite of passage:”

  • Detachment or withdrawal from the status quo
  • Transition
  • Reincorporation into the social group

Likewise, consider a purification ritual, in which the transition in question is from an “unclean” to a “clean” state.

I propose that we think of a joyful funeral as a combination of passage and purification.  The individual or organization has an opportunity to mark the change (which may also be a loss) from a viable initiative to a failure, to acknowledge shortcomings, to mourn, to be supported by the community, and to achieve closure, and to begin the next stage of life.

Many people are left cold by any kind of ritual, and others are put off by the links between elaborate ritual and religious institutions from which they are alienated.  For that reason, I would never argue that a joyful funeral (or any of the other celebrations of failure that Beth describes) should be attempted by everyone.  But for many of us, a ritual can be a comfort, especially if it doesn’t demand that we buy into a dogma or denomination.  A ritual can also be goofy and fun.

I like the idea of building laughter without humiliation into a ritual acknowledgement of failure. It’s less scary and less punitive than a solemn occasion, and better for strengthening ties among the team and making it fun to learn from mistakes.  For this reason, I recommend the “DoSomething PinkBoa FailFest” to beginners in the art of failing and moving on.


Bonus item:  a joke for people who take ritual a little too seriously.

Q:  What’s the difference between a liturgist and a terrorist?

A:  You can negotiate with a terrorist.


 

Why are you always talking about “saving the whales?” (Redux)

This is an article that appeared in my first blog in January 2005.  Many thanks are due to the Wayback Machine, which enabled me to salvage it. 

undefined
11:46AM (EST) on January 20, 2005

Whenever I am speaking or writing on the topic of mission-based technology planning, I seem to end up talking about  “saving the whales.”  Several people have asked me (or teased me) about this, so here are some reasons:

  • I like whales. It’s quite possible that they have forms of language, cognition, and social structure that are as rich as – but completely different from – what humans have developed. For someone with my moderate-progressive values, saving them from extinction seems like an obviously good idea.
  • It sounds like a very straightforward mission, but is actually fraught with complexity and nuance.  I have a sort of fantasy scenario about the disconnect between serious techies and nonprofit professionals.  The techies might be brought in to help a nonprofit organization that exists to save the whales; they get very excited telling the nonprofit team about how they are going to equip all the staff and volunteers of the nonprofit with personal digital assistants (with global positioning systems, naturally) and program them so that the PDAs will start beeping whenever a whale is washed ashore within ten miles of the person with the PDA.  Everyone on staff delves deep into considering whether there’s room in the budget for this exciting but possibly complicated and expensive technology.  Then the Cyber-Yenta does her best to call time-out and remind both the techies and the nonprofit workers that their organization does not actually “save the whales” by going down to the beach with big nets to drag them back into the water.  In fact, what they do is “save the whales” by working for laws and policies that protect the whales.  Can they think of a way that PDAs with GPS would help them do that?  Usually, the answer is no, because legislators and policy-makers won’t consent to being tagged and tracked like wildlife by lobbyists.  In this fantasy scenario, the optimum outcome is that everyone goes back to thinking about technology that actually supports their mission, strategy, and tactics.
  • It’s a great way to introduce the concept of realistic outcomes measurement to a nonprofit organization that is struggling with it.  The introductory question can be:  how many whales do you save a year?  This is actually very difficult to calculate, but is absolutely crucial before moving on to advanced questions such as: how many MORE whales will you save a year with that new technology implementation?

 

Let’s revisit the concept of failure-friendliness

Eight years ago, I wrote a blog article about failure-friendliness in nonprofit technology. It was very much inspired by my friend and colleague, Dan Scharfman. Since Dan died this week, and this is also a week when I have been thinking hard about the obstacles that nonprofit organizations face in tracking their outcomes, it seems appropriate to reprise the article here and now. Having coped with the need for failure-friendliness in nonprofit technology for years, I see that my understanding is still superficial when it comes to the difficulties that nonprofits have in acknowledging programmatic failure. I invite your thoughts on how we can be more transparent about and more open to learning from failure. Meanwhile, special thanks go out to Beth Kanter, for her outstanding blog articles on this topic.

FAIL stamp

Wed 26 Jan 2005 05:41 PM EST

The term “failure-friendly organization” was first introduced to me by a colleague I revere – Dan Scharfman of Baird Associates.

My first impression was that he was an unlikely champion of failure, since Massachusetts is well-supplied with nonprofit organizations that consider the technology services that he has provided to them very successful indeed.

However, many of us in the nonprofit sector have seen the following things happen with major implementations or upgrades:

  • The technology doesn’t work, or doesn’t work nearly as well as it should;
  • The intended users won’t have anything to do with the technology;
  • Major changes in technology in the outside world quickly render the organization’s choices obsolete;
  • Programmatic priorities change, and the technology is all but irrelevant;
  • The organization has not factored in the shocking cost of customizing, tweaking, maintaining, and upgrading the technology.

Although techies vary greatly in their attitudes about projects that don’t work out, we also tend to make tacit assumptions that everyone concerned understands that we are not engaged in an exact science but in an evolving process.

Techies also tend to regard failure as pretty interesting – as a good source of information about what ought to be fixed when Version 97.53.01 of the software is released.  We also enjoy working on cool tools, even if such tools don’t actually deliver the outcomes desired by those who are underwriting the project.  This form of process orientation can be less than endearing to decision-makers in nonprofit organizations.

Oddly enough, nonprofit workers tend to be very good at process orientation when they are on familiar ground.

Sometimes this process orientation is a grim necessity, with governmental agencies strictly mandating, auditing, and enforcing protocols that nonprofits must follow in order to maintain their tax-exempt status, accreditation, or contracts for services.  These are headaches that would impel just about any organization or individual to worry a great deal about operating according to plan and documenting the process, rather than ensuring a specific outcome. This of course is a very “functional” (or “instrumental“) form of process orientation.

A more “expressive” form of process orientation is also frequently seen in nonprofit organizations – manifesting as a desire to be flexible and responsive to changing situations, or as a desire to arrive at decisions through consensus.  However, it can be difficult to extend that attitude to technology, which tends to be difficult for non-specialists to comprehend, time-consuming, and expensive.

Another challenge is that organizations and individuals (including yours truly) can be reluctant to cut their losses, and say, “This isn’t working.  Let’s stop, figure out why, and decide on some next steps.”  Of course, in some settings, the decoded version of this message is “Let’s find someone to blame and punish…maybe YOU.”

Yikes!

Is there any solution in sight?  I only wish I had something certain and simple to offer.  Here are a few ideas, although none of them come with guarantees of success:

  • Techies need to understand the nonprofit organizational cultures in which they are operating.  Progress toward this goal is possible if the techies listen, ask questions, and listen some more.  These conversations should start early in the planning phase.
  • Nonprofit workers need to understand how technology innovations and implementations happen in real life, and have a reasonable idea of what factors can lead to unexpected outcomes in technology projects.  Progress is possible if – yes, you guessed it – the nonprofit workers listen, ask questions, and listen some more.
  • Everyone needs to cooperate in creating incentives for spotting, discussing, and correcting errors rather than evading their detection.  I freely admit that I always find it easier to do these things when the mistake was made by someone else, but am always striving to do better.

I wish I could remember who it was that first said to me, “This is not about one person against another. This is about our team against the problem.”  Anyone who can say that is a saint, a boddhisatva, a tzadik, or an unusually effective manager.

“We count our successes in lives”

Brent James

Brent James is one of my new heroes.  He’s a physician, a researcher, and the chief quality officer of Intermountain Healthcare’s Institute for Health Care Delivery Research.

We had a very inspiring telephone conversation this afternoon, about whether the lessons learned from evidence-based medicine could be applied to nonprofits that are seeking to manage their outcomes.  We also swapped some stories and jokes about the ongoing struggle to document a causal relationship between what a health care organization (or a social service agency, or an arts group, or an environmental coalition, for that matter) does and what the organization’s stated aims are.  In fact, documenting that an organization is doing more good than harm, and less harm than doing nothing at all, continues to be a perplexing problem.  The truth may be less than obvious – in fact, it may be completely counter-intuitive.

In this phone conversation, we also waded into deep epistemological waters, reflecting on how we know we have succeeded, and also on the disturbing gap between efficacy and effectiveness.

It’s not merely a philosophical challenge, but a political one, to understand where the power lies to define success and to set the standards of proof.

I doubt that this is what William James (no relation to Brent, as far as I know) had in mind when he referred to success as “the bitch-goddess,” but there’s no doubt that defining, measuring, and reporting on one’s programmatic success is a bitch for any nonprofit professional with intellectual and professional integrity.  It’s both difficult and urgent.

What particularly struck me during my conversation with Brent was his remark about Intermountain Healthcare:

“We count our successes in lives.”

On the surface, that approach to counting successes seems simple and dramatic.  The lives of patients are on the line.  They either live or die, with the help of Intermountain Healthcare.  But it’s really a very intricate question, once we start asking whether Intermountain’s contribution is a positive one, enabling the patients to live the lives and die the deaths that are congruent with their wishes and values.

These questions are very poignant for me, and not just because I’m cancer patient myself, and not just because yesterday I attended the funeral of a revered colleague and friend who died very unexpectedly.  These questions hit me where I live professionally as well, because earlier this week, I met with the staff of a fantastic nonprofit that is striving to do programmatic outcomes measurement, and is faced with questions about how to define success in a way that can be empirically confirmed or disconfirmed.  Their mission states that they will help their clients excel in a specific industry and in their personal lives.  They have a coherent theory of change, and virtually all of their criteria of professional and personal success are quantifiable.  Their goals are bold but not vague. (This is a dream organization for anyone interested in outcomes management, not to mention that the staff members are smart and charming.)  However, it’s not entirely clear yet whether the goals that add up to success for each client are determined solely by the staff or by the client or some combination thereof.  I see it as a huge issue, not just on an operational level, but on a philosophical one; it’s the difference between self-determination and paternalism.  I applaud this organization’s staff for their willingness to explore the question.

When Brent talked about counting successes in terms of lives, I thought about this nonprofit organization, which defines its mission in terms of professional and personal success for its clients.  The staff members of that organization, like so many nonprofit professionals, are ultimately counting their successes in lives, though perhaps not as obviously as health care providers do.  Surgeons receive high pay and prestige for keeping cancer patients alive and well – for the most part, they fully deserve it.  But let’s also count the successes of the organization that helps a substantial number of people win jobs that offer a living wage and health insurance, along with other benefits such as G.E.D.s, citizenship, proficiency in English, home ownership, paid vacations, and college educations for the workers’ children. Nonprofit professionals who can deliver that are also my heroes, right up there with Brent James.  While we’re holding them to high standards of proof of success, I hope that we can find a way to offer them the high pay and prestige that we already grant to the medical profession.

My current daydream: The marriage of outcomes management apps with data visualization apps

The marriage of outcomes management with data visualization

Given my current preoccupation with both outcomes management and data visualization for mission-based organizations, perhaps it’s not a surprise that I’m daydreaming about integrating applications that were designed for these two tasks.

This daydream was inspired by a recent conversation with Patrice Keegan, executive director of Boston Cares (a HandsOn Network affiliate).  She is keenly interested in both outcomes and data visualization, and she leads a nonprofit of modest size that collaborates not only with many local partners but also with a national network of sister organizations that facilitate short-term volunteering.  In other words, Boston Cares provides a gateway to volunteerism for individuals, corporations, and community-based nonprofits, and then shares best practices with its counterparts across the United States.

What better poster girl could there be than Patrice, for my Cause, which is making it not only possible but easy for her to take her outcomes analyses and turn them in visuals that tell the story of the social impact of Boston Cares?

Moreover, what good is a cause and a poster child, without a poster?  Here’s mine:

Patrice Keegan of Boston Cares
Special note to software developers in the nonprofit sector:  please take a look at that bright, shining face, and give your efforts to the cause.

Harsh truths

Illustration from "Six Harsh Truths That Will Make You A Better Person" by David Wong    http://www.cracked.com/blog/6-harsh-truths-that-will-make-you-better-person/
Perhaps as a counter-balance to my previous blog post, I’ve been reflecting on an article by David Wong that appeared this month in Cracked.Com: “Six Harsh Truths That Will Make You A Better Person.”

While I don’t agree with all of Wong’s assertions, there are a few that I think we should take to heart in the nonprofit sector.  I present them here as bullet points:

  • “The World Only Cares About What It Can Get from You”

Wong points out two corollaries to this:  “…society is full of people who need things….”  and “(e)ither you will go about the task of seeing to those needs by learning a unique set of skills, or the world will reject you, no matter how kind, giving and polite you are.

He’s right, or at least he should be.

(We all know of high-profile nonprofit professionals whose main achievement is making people feel warm inside.  They don’t contribute to any lasting and positive change, but they get plenty of photo opportunities.  If this were an entirely fair universe, the world would be rejecting them, and instead it applauds them.  However, they are the exceptions.)

Most of us are plebs in the nonprofit sector, and we are obliged to solve problems in order to justify our continued employment.  That’s the way it should be.  At the end of the day, if our organization’s mission is to save whales, then we should be judged on how our efforts add up to success in saving whales.  That might involve sticking to methods that have stood the test of time, or to finding ways to save more whales with fewer resources, or to finding ways to ensure that once we save them they stay healthy for the long run.   We have to keep our eye on results, and not give free passes to people who are merely impassioned without being effective.

By this I don’t mean that personalities and processes should be completely discounted.  It’s important to be considerate, ethical, and fair.  It’s also important to value the processes enough to learn from both failures and successes.  But having a pure heart is not enough, either as a process or an outcome, when your self-proclaimed mission is to save the whales.

  • “What You Produce Does Not Have to Make Money, But It Does Have to Benefit People”
‘Nuff said, right?  This the nonprofit sector, right? It’s not just that people ask “what’s in it for me?” when they donate, it’s that we’re living in an WIIFM universe. If what’s in it for us, as nonprofit professionals, is the dubious glamor of altruism, I suppose that’s alright, but we still have an obligation to make our work more than a vanity project.  Again, if it were a fair world, then effective projects would get more love than vanity projects.  But even though it’s unfair, there’s still a principle involved that obligates us to be as effective as possible in benefiting the world, even when that’s less glamorous.
  • “Everything Inside You Will Fight Improvement”

Thank you, Mr. Wong.  This is true indeed – not just for individuals, but for nonprofit organizations.  This one of the most compelling reasons why outcomes management is such an uphill battle in our sector.  It’s not just that we don’t like being judged by our results; it’s that we don’t like having to change.

I’m not using the first person plural (e.g., “we”) here in a abstract, vague, editorial way.  I’m using it, because I’m talking about flaws in which I fully partake.  When Wong says, “(t)he human mind is a miracle, and you will never see it spring more beautifully into action than when it is fighting against evidence that it needs to change,” he’s certainly speaking to my condition.

Here is his list of the powerful defense mechanisms of which I am (and perhaps you are) capable of fielding when we are challenged or criticized:

              “Intentionally Interpreting Any Criticism as an Insult”

              “Focusing on the Messenger to Avoid Hearing the Message”

             Focusing on the Tone to Avoid Hearing the Content”

              Revising (My) Own History”

             “Pretending That Any Self-Improvement Would Somehow Be Selling Out (My) True Self”

Of course, my personal favorite is the last mentioned. When I start wallowing in it, I do my best to remember that there are more important (and perhaps more valid) ethical principles at stake than expressing what I take to be my essential nature.

In fact, it’s our obligation to submit to public scrutiny and criticism when we work in the nonprofit sector.  (At least, it is in the U.S.; I can’t say much about other countries.)  As George McCully points out in his book Philanthropy Reconsidered, we are engaged in private initiatives for the public good, and the public has a right to evidence that we deserve the trust that is vested in us.  They deserve to know that we are striving to serve them with both processes and results that are valid, and that we are quickly learning from processes and results that do not yield strong positive benefits over the long run.

How grantmakers and how nonprofits use information about outcomes

State of Evaluation 2012: Evaluation Practice and Capacity in the Nonprofit Sector, a report by the Innovation Network

I’m sitting here, reflecting on the Innovation Network’s “State of Evaluation 2012” report.

I encourage you to download it and read it for yourself; start with pages 14 and 15. These two pages display infographics that summarize what funders (also known as “grantors,” or if you’re Bob Penna, as “investors”) and nonprofits (also known as “grantees”) are reporting about why they do evaluation and what they are evaluating.

Regardless of whether you call it evaluation, impact assessment, outcomes management, performance measurement, or research – it’s really, really difficult to ascertain whether a mission-based organization is delivering the specific, positive, and sustainable change that it promises to its stakeholders. Many organizations do an excellent job at tracking outputs, but falter when it comes to managing outcomes. That’s in part because proving a causal relationship between what the nonprofit does and the specific goals that it promises to achieve is very costly in time, effort, expertise, and money.

But assuming that a mission-based organization is doing a rigorous evaluation, we still need to ask:  what is done with the findings, once the analysis is complete?

What the aforementioned infographics from the “State of Evalution 2012”  tell me is that both grantors and grantees typically say that the most important thing they can do with their outcome findings is to report them to their respective boards of directors.  Considering the depth of the moral and legal responsibility that is vested in board members, this is a pretty decent priority.  But it’s unclear to me what those boards actually do with the information.  Do they use it to guide the policies and operations of their respective organizations?  If so, does anything change for the better?

If you have an answer to the question of how boards use this information that is based on firsthand experience, then please feel to post a comment here.

What I learned about outcomes management from Robert Penna

Robert Penna

Yesterday, along with a number of colleagues and friends from Community TechKnowledge, I had the privilege of attending a training by Robert Penna, the author of The Nonprofit Outcomes Toolbox.

As you probably  know, I’ve been on a tear about outcomes measurement for a few months now; the current level of obsession began when I attended NTEN’s Nonprofit Data Summit in Boston in September.  I thought that the presenters at the NTEN summit did a great job addressing some difficult issues – such as how to overcome internal resistance to collecting organizational data, and how to reframe Excel spreadsheets moldering away in file servers as archival data.  However, I worked myself into a tizzy, worrying about the lack, in that day’s presentations, of any reference to the history and literature of quantitative analysis and social research.  I could not see how nonprofit professionals would be able to find the time and resources to get up to speed on those topics.

Thanks to Bob Penna, I feel a lot better now.  In yesterday’s training, he showed me and the CTK team just how far you can go by stripping away what is superfluous and focusing on what it really takes to use the best outcomes tools for job.  Never mind about graduate level statistics! Managing outcomes may be very, very difficult because it requires major changes in organizational culture – let’s not kid ourselves about that.  However, it’s not going to take years out of each nonprofit professional’s life to develop the skill set.

Here are some other insights and highlights of the day:

  • Mia Erichson, CTK’s brilliant new marketing manager, pointed out that at least one of the outcomes tools that Bob showed us could be easily mapped to a “marketing funnel” model.  This opens possibilities for aligning a nonprofits programmatic strategy with its marcomm strategy.
  • The way to go is prospective outcomes tracking, with real time updates allowing for course correction.  Purely retrospective outcomes assessment is not going to cut it.
  • There are several very strong outcomes tools, but they should be treated as we treated a software suite that comprises applications that are gems and applications that are junk.  We need to use the best of breed to meet each need.
  • If we want to live in Bob Penna’s universe, we’re going to have to change our vocabulary.  It’s not “outcomes measurement – it’s “outcomes management.” The terms “funder” and “grantmaker” are out – “investor” is in.

Even with these lessons learned, it’s not a Utopia out there waiting for nonprofits that become adept at outcomes management.  Not only is it difficult to shift to an organizational culture that fosters it, but we have to face continuing questions about how exactly the funders (oops! I should have said “investors”) use the data that they demand from nonprofit organizations.  (“Data” is of course a broad term, with connotations well beyond outcomes management.  But it’s somewhat fashionable these days for them to take an interest in data about programmatic outcomes.)

We should be asking ourselves, first of all, whether the sole or primary motivation for outcomes management in nonprofits should be the demands of investors.  Secondly, we should be revisiting the Gilbert Center’s report, Does Evidence Matter to Grantmakers? Data, Logic, and the Lack thereof in the Largest U.S. Foundations.We need to know this. Thirdly, we should be going in search of other motivations for introducing outcomes management.  I realize that most nonprofits go forward with it when they reach a point of pain (translation:  they won’t get money if they don’t report outcomes). 

During a break in Bob’s training, some of my CTK colleagues were discussing the likelihood that many nonprofit executives simply hate the concept of outcomes management.  Who wants to spend resources on it, if it subtracts from resources available for programmatic activities?  Who wants to risk finding out (or to risk having external stakeholders find out) that an organization’s programs are approximately as effective as doing nothing at all?  Very few – thus the need to find new motivations, such as the power to review progress and make corrections as we go.  I jokingly told my CTK colleagues, “the truth will make you free, but first it will make you miserable.”  Perhaps that’s more than a joke.

%d bloggers like this: