Reconciliation and Evaluation; Now what?


(J Noureddin) #1

May 26-May 29 The Canadian Evaluation Society’s 2018 Conference was hosted in Calgary where the theme was “Co-creation” and the focus was to explore new and not-so-new approaches to evaluation in the context of Indigenous knowledge and methods. An impassioned keynote of Indigenous evaluators reminded the packed room that there has been countless studies, reports and research published backing the need to indigenize evaluation practices. What then, they asked, is the hold-up in implementation?

Over the three days, there was lots of interesting conversations around when and how the sector can begin looking to Indigenous ways of measuring and evaluating as valid practices and not simply, work-arounds or “exceptions” to rule; Indigenizing evaluation is not just a way of being more “inclusive” but, a must in the field’s path towards reconciliation.

Big questions around what constitutes evidence and what role, if any, non-Indigenous evaluators play in measuring towards an Indigenous communities’ goals and needs, all contributed to an amazing learning experience. Dr. Nicole Bowman, one of the keynotes and a leading voice in the global initiative to connect evaluation efforts around the world with her “EvalIndigenous” work group, finished off her powerful speech by suggesting all read “Pagans in the Promised Land: Decoding the Doctrine of Christian Discovery” by Steven T. Newcomb.

(Stacey McDonald) #2

@JNoureddin, could you share some examples of the practices that you mention? What are some of these practices or methods that we should no longer consider as ‘work-arounds’ or ‘exceptions’, but instead recognize as valid?

(J Noureddin) #3

No specific Indigenous practices or methods were shared as being the ultimate way of measuring and evaluating but rather, two principles were mentioned repeatedly as rules of thumb:

  1. Do not assume that a practice/method which works in one community will work in other communities; Indigenous communities are unique and each have their own ways of doing and knowing
  2. The Indigenous ways of collecting evidence and sharing knowledge is valid. Full stop.

The underlying message was that the categorization of data and evidence used to distinguish between Indigenous truths and non-Indigenous truths is arbitrary and discriminatory.
The take aways: Non-Indigenous evaluators must…
-Make space for Indigenous evaluation practices and methods
-Accept Indigenous evaluation practices and methods as valid and equal
-Support Indigenous evaluators to provide the expertise and take the lead

(Phil Nowotny) #4

As service provider to the homeless with a significant part of our clients being indigenous this is an important but nontheless challenging topic for us. From the very limited humble approached we have endeavoured, the take away for me is that 2 factors are the crux:
a) time
b) knowledge/resources

Re a)
Indigenous geared evaluation to me seems more time intensive, particularly due to many elements overlapping what we could call qualitative research
Re b)
Access to qualified/trained people and also the resources to pay them is the even bigger challenge.

But there’s hope:
Together with Prof. Tracy Coates from Ottawa U we commissioned a student project to investigate options into indigenous appropriate programme evaluation. Please feel free to contact me if you would like to have the report.
Their outcome pointed towards the Waawiyeyaa Evaluation tool which is Anishinaabe informed

For OTF, I think wherever indigenous service users are involved, I’d earmark a portion of the evaluation budget for that purpose OR provide relevant resources like in your eval training toolkits/webinars; even better a repository.

In any case, more resources are welcome - please share widely!!!

(J Noureddin) #5

Thank you for sharing Phil. Sufficient time and resources surfaced in our discussions numerous times as factors that contribute significantly to the effectiveness of an evaluation and specifically, when working with Indigenous communities. As participants, we were invited to think more deeply around our definitions of common terms like “effective”, “qualitative”, “outcome” and how these words can and do signify different things to different communities. We were challenged to think outside the box; “Re-imagine what a meeting looks like. Helping to haul water can provide just as many meaningful opportunities to learn, if not more, than a scheduled meeting can”

(Stacey McDonald) #6

@pnowotny have you used this evaluation tool? I’d love to hear about your experience with it if you have.

(Phil Nowotny) #7

Hi Stacey, not yet and unlikely to happen anytime soon. Did I mention resources as a constrain x-/?

(Stacey McDonald) #8

@pnowotny yes, i did see that. I know that this is a common challenge, and one that is hard to overcome once a project has begun.

While this may not help right now, don’t hesitate to include the money you need for your evaluation as part of future grant applications. At OTF, organizations can request up to 10% of their grow grant for their evaluation. Many organizations don’t take full advantage on this, and that may be because OTF needs to be clearer about that, which is why I wanted to share it here.