Evaluation as collaboration

Assessment doesn’t necessarily mean judgment. Measuring success should be a collaborative inquiry.

Evaluation doesn’t have to be scary. One way to ease anxiety and foster openness to this important practice is to recast it as a learning experience, says Susan Weber, coordinator of the Evaluation Project for the Religion Division of Lilly Endowment Inc.

“Evaluation is a way to learn and grow through the ministry experience,” she said. “It’s not meant to threaten anyone or to judge anything.”

Weber works with Religion Division grantees helping them build their own capacity for evaluation. Her approach to helping religious and non-profit organizations offers insights that clergy and lay people can use in other contexts as well.

She suggests church leaders view evaluation as a collaborative inquiry focused on the question: “How can we learn from this and make ministry even better to serve who we want to serve?”

Asking good questions

Begin by asking good questions. Good questions help focus the scope of the evaluation (What do we want to learn about this ministry?), guide the design of the evaluation plan (What information is needed to answer our evaluation questions?) and, once the data and information is collected, determine its meaning in relationship to the ministry (What are we learning?)

Before these questions can be identified, an organization needs to define its own marks for success. For example, what single change would you hope to see? The change could be in attitudes, behavior, knowledge, skills or action, she said.

When Weber works with organizations, she encourages the involvement of all key stakeholders. Stakeholders are persons or groups who have a vested interest in the ministry’s success and/or are involved in the ministry’s design and implementation.

If the project were an adult daycare center sponsored by a local congregation, for example, a group of stakeholders might identify one meaningful single change to be helping the elderly maximize their full potential to function independently through socialization and interaction. Other signs of success might focus on the value of the program and support given to caregivers. In turn, the “signs of success” enable the stakeholders to focus on the evaluative questions important to them.

  • What unique value does our program bring to caregivers? To the elderly?
  • How does our adult daycare provide supportive services that ensure the elderly who live alone continue in mainstream society?
  • How do caregivers and/or the elderly know about our services?
  • What services are missing from our program that we need to consider?
  • How does our adult daycare provide meaningful respite and support for caregivers of older adults?

 

Gathering, interpreting and learning from data and information

Collect information and data that will most appropriately answer the evaluative questions. For example, in an adult daycare, three types of focus groups might be convened: groups of caregivers, participating senior citizens, and care providers and volunteers to answer the questions generated above. Surveys of these same groups would provide more in-depth information.

The information that is collected is then organized, analyzed and shared with the stakeholders. So in this case, the adult daycare’s board of directors and staff would discuss the evaluative findings in terms of what they are learning about how programs and services could be improved, marketing enhanced, or how to create a case for support to donors and organizations.

Beyond the board and staff, the organization should consider how to disseminate findings to audiences who either have an interest in the ministry or who can offer a perspective that would provide added value to the findings. Written articles, speaker series, or one-on-one conversations are excellent ways to promote what an organization has learned from an evaluation. The residual value is that it promotes your good name and good work.

Finally, for its greatest effectiveness, evaluation should be ongoing -- and inclusive. With one evaluation completed, an organization now has baseline data and information to build on and learn from. Additionally, it has created a community of learners who now know evaluation practices and value the rich conversation opportunities it creates.

“Evaluation that is effective is not something that is done by a few people,” Weber said. “It should be woven into the culture of the organization.” That is, a culture that values learning, curiosity, growth and improvement, and accountability.