Balancing the Mission Checkbook

The Value of Evaluation – What we Learned About Our Work

Kate Barr September 28, 2015

“They’re going to do whatever they can to help you succeed.”

Earlier this year, we engaged a consultant to conduct an in-depth evaluation of our programs. We wanted to learn more about the real impact that our work has with our nonprofit clients. We set out to answer the following questions, and more:

  • What kind of impact do we have on the financial health of our clients?
  • What does working with NAF do for our clients’ stability?
  • Does working with NAF improve our clients’ management, governance, and strategy?
  • What could we do differently?

The quote above crystallizes the responses from the evaluation—and there’s no higher praise in our world! The evaluators with our consulting partner, Aurora Consulting, also summarized that NAF’s combination of an essential service (loans to nonprofits) and supportive service (in the form of advice and management and strategy support) “contribute to real impacts in the sector and a relationship that we call true partnership.”  We couldn’t ask for better news!

The research was focused on the 200 nonprofits that received a loan from NAF in the past six years.  The consultants distilled their data analysis, interviews, and surveys to these conclusions and interpretations.

Below, you’ll find some of my favorite excerpts from the report, as well as a few direct quotes from clients. If you’d like to dig into the full report, it’s posted here.

On “Technical Assistance” AKA Supportive Service or Financial Advice

In the day-to-day servicing of a loan, technical assistance is just one part of a collection of practices, values, and qualities that NAF loan officers use to build relationships and strengthen the recipient nonprofit. Because of NAF’s mission, their service is different than the service offered by other lenders.

The relational components of any service delivery can seem mushy and intangible. We often call them soft skills for this reason. However, we found that supportive service has tangible qualities and impacts. Loan officers practicing supportive service are flexible, frank, approachable, and rigorous. They aren’t afraid to be tough on clients while also being supportive and committed. In the short term, supportive service helps nonprofits achieve success with their loan. In the long term, we found that supportive service can help improve a nonprofit’s overall financial health. This report describes the tangible qualities and impacts of supportive service.

On NAF Qualities Valued by Clients

 The interviews and surveys agreed that certain qualities were evident in the way NAF works with nonprofit clients. We organized the qualities identified by clients into two categories: effective service and soft qualities.

NAF is committed to the success of the nonprofits they lend to and demonstrates this by getting to know clients, being rigorous in their lending practices, and being flexible when situations change. NAF understands their clients.

“NAF has a genuine human component. They are motivated by our missions. They do what they do so we can do what we do. They put us through rigor so that we can do what we do.”

NAF’s soft qualities show their strong commitment to and passion for the nonprofit sector. NAF inspires trust from clients and motivates them to do their best work.

“It’s not just technical assistance, they are fostering financial resilience in the nonprofit sector.”

On Impact—Beyond Financial Health

NAF asked us to not use the typical financial ratios as a way to determine the financial or organizational health of their clients because these only describe one particular aspect of financial health. Instead, our challenge was to uncover impacts resulting from the supportive service that NAF provides. We report the impacts using the following organizing sections: knowledge, behavior, attitude, growth and stability.

“We got so many business and nonprofit concepts out of our work with [NAF] that weren’t part of my theater major.”

Knowledge : Because of NAF’s deep knowledge of the nonprofit sector and their supportive service, clients learn not only about effectively managing their loan, but also gain a better understanding of their own financial situation and important fundamental business concepts.

Behaviors: After working with NAF, clients improved their financial rigor, improved their financial reporting and gained a better ability to plan and adapt.

Attitude: For some organizations, working with NAF means a positive shift in the way they think about finance or the finances of their organization.

Growth and Stability: Receiving supportive service from NAF provides valuable stability and supports meaningful growth in nonprofit organizations like Kid’s Charter School.

“[After we worked with NAF,] when we had a strategic planning meeting we were talking about vision, not paying the bills.”

On Our Essentiality 

Clients said that NAF gave them a loan that other lenders either didn’t offer, found too risky, or didn’t have the expertise to service. For this reason, we call NAF’s loans an essential product in the nonprofit sector. NAF is able to offer an essential product because they are both knowledgeable about the nonprofit sector and focused on the client’s mission rather than just financial return. These qualities differentiate NAF from other lenders and financial partners.

What will we do this with report?

While evaluations are useful for case statements and funder reports, our real reason for doing this was to ask some big questions about our programs and then to take action on what we learned. So what will we do now?

  • Continue committing a significant amount of time and resource towards “soft skills”/supportive service
  • Communicate the value and impact of supportive service to clients and potential clients
  • Consider what other organizations could benefit from a partnership with us
  • Use what we heard to guide our plan for growth and our own sustainability—who else could we work with that needs what we offer?
  • Use what we heard to inform the way we talk about ourselves to potential partners
  • Continue developing educational programming to support the learning needs of our clients

We encourage all nonprofits to find a way to ask the important questions about the value and impact of their programs and to act on what they learn.  We’re proud of what we’re able to help our nonprofit partners accomplish, and feel honored to be a part of their incredible work. And on a personal level, I’m deeply gratified to know that our work has both the tangible and intangible effects that we have always aimed for.  Special thanks to the team at Aurora Consulting for their great work and for being a true partner to NAF for this project.


Note on evaluation details and method:

In order to examine the impacts and qualities of the bundled loan and services model, we used a grounded survey method and invited all NAF clients who had a loan between 2009 and 2015, the past six years, to participate (200 total). A grounded survey method combines the benefits of both interview and survey data collection to answer complex questions. First, we use in-depth interviews with a small number of clients to identify important themes. Then, we test those themes with all clients through a survey. Client participation was very good. Three quarters of the clients (75%) completed the survey. We found the respondents to be representative of the whole group.  The full evaluation report is available on the website.

Kate Barr believes that every nonprofit financial question relates to strategy, structure and mission impact. She enjoys interpreting financial information to find stories numbers can tell. She loves writing, teaching, and talking with interesting people.