Do you ever ask yourself: "Do I really need to collect all this data?"


(Stacey McDonald) #1

More times than i’d like to recall, I start writing a survey or interview questions, and next thing i know it’s twice as long as I intended. I’m curious, i want to know things. I also know that asking everything is really not a great idea. People don’t like long surveys. Response rates go down, and data quality suffers. Analyzing extra/unnecessary qualitative data takes a lot of time! I try to ask myself, and others that I work with, what will i do with each piece of information, how will that inform future action, and if we even act on it? If there answer is no, then it gets cut.

It really is hard to cut things down to asking only those things that will help us answer the key evaluation questions we have. Right now, in fact I’m designing a questionnaire that’s probably too long. Any tips out there to help make those tough cuts, or do you lean on the side of more is better?

(Gillian Kerr) #2

Hi, Stacey - Does OTF have recommended measures for its programs? I recall that you used to recommend the Arizona Self-Sufficiency Matrix and other instruments but I can’t find them in the Knowledge Centre or OTF site.

(Stacey McDonald) #3

Hi @Gillian_Kerr! Sorry for not responding sooner. Son is home sick this week, so I’m just getting to this now.

There are a couple of places on the website where you can get the info, both under the What We Fund tab:

  1. Click on Action Area in left hand menu and then open the PDF for the Grant Results, Metrics and Survey Tools for each individual Action Area

  2. Click on the Investment Streams in the left hand menu and then on Grow. On that page there are two PDFs that you might want to look at: Priority Outcomes, Grant Results and Metrics, this also has the links to the standardized tools (like the one you mentioned) and Grow Grants Survey Tools.

I hope this answers your question! if not, please feel free to get in touch!

(Gillian Kerr) #4

Thanks, Stacey. I have been looking at the Dutch version of the Self-sufficiency matrix and it looks a bit more relevant to Canada than the Arizona version (it assumes basic health care is accessible to everyone, it adds domestic violence and safety, and separates out parenting measures from the general adult measures - It would be interesting to compare possible instruments and choose the ones that are as short as possible - as you were saying in your original post. I’ll bring this up in our evaluation cafe discussion.

(Stacey McDonald) #5

@Gillian_Kerr Thank you so much for sharing this! I’ll definitely look into it. We’re aware that some of the tools may not be the best fit, and we will be starting a process in the coming year to identify ways that we can improve the implementation of our strategy, and this will include data collection. Please keep the suggestions coming!

(Gillian Kerr) #6

Oh wow, I’ve collected dozens of good instruments that I think would be relevant to OTF-funded agencies, all of which are free or open source. We’re building them into our evaluation framework, but they could be used separately. If we have time let’s talk about this when we meet.

(Stacey McDonald) #7

@Gillian_Kerr I would love to continue this conversation! Would you like to start a discussion topic related to standardized tools and start sharing of these?

(Gillian Kerr) #8

Yes, would be happy to! Would this be an example of having a posting that could be updated and edited, as an ongoing reference in the discussion?

(Stacey McDonald) #9

@Gillian_Kerr Yes! That would be wonderful.