Welcome to the Measurement Hub! Bienvenue au carrefour Évaluation!


(Stacey McDonald) #1

Salut! Hello!

I’d like to welcome everyone to the Measurement Hub. For my first post in this forum, I’d like to introduce myself and share why I’m so excited to kick off the start of this discussion.
I’ve been at OTF for 6 years, working on the foundation’s policies, strategy, and performance measurement. Currently, my focus is on helping to build the evaluation capacity of Ontario’s non-profit sector and support the implementation and measurement of OTF’s outcomes-based funding approach.
I grew up in Sudbury, where most of my family still lives. I moved to Toronto in 2008 to complete a Master in Public Policy at the University of Toronto, then a few months after graduation, I started working at OTF. Since then I’ve seen the countless ways that using data can improve what we do, and the decisions we make. Sometimes the numbers tell us exactly what we expect; other times our gut feelings and assumptions are just totally wrong!
Finding the right things to measure, collecting the data, making sense of it, learning from it, and sharing what was learned with others can be straightforward or frustratingly complex. However difficult it may be at times, I firmly believe that measuring performance and continuously evaluating our work is vital to understanding how we’re doing and improving the way we do it. Measurement ensures we’re doing the best we can to help build healthy and vibrant communities across Ontario.
Since this work is new to many of us, and we’re developing our practice together, I really look forward to engaging and learning together in this space. I’d love to hear from all of you about why you’re interested in measurement, so please introduce yourselves to the group!
Stacey


(Ashraful Hasan) #2

Thanks, Stacey,

Echoing your words, as we’re developing our practice together, I also am really looking forward to engaging and learning together. So, let me introduce myself and explain why (it’s a long story, so take a sip of your coffee before you read it):

I am Ashraful Hasan; you can call me Ash. I came from Bangladesh and was first introduced to the evaluation and measurement through International Development Projects there. As my academic background is in Economics and Finance, the only evaluation approach I knew then was Randomized Control Trial (RCT) and various experimental and quasi-experimental framework. I came to Canada and completed my Master here in International Economics and Finance from Ryerson University. At the end of my study, as I was thinking about my career, I somehow stumbled upon Evaluation Cafe’. I was curious and went in one of their event and by the stroke of luck, it was right before 2013 CES conference in Toronto. So, I volunteered there, and my life changed on the final day of the conference. On that day, the keynote speaker was John Gargani. His main idea was the future of evaluation needs to learn from impact measurement and broader discourse happening in Economics and Finance (it’s me !!). Then, Sanjeev presented a synthesis of key ideas of the conference, and that was we need to learn from what is happening across boundaries and particularly from developing country experiences (me, again, yay!!). I immediately knew what I need to do and what would be my career.

Since then I started volunteering at CES and took all the courses and workshops available, went to all the conferences and events I could go to, and shadowed many kind mentors (Anne Miller, Biljana, Paul, Kelly, Gillian to name a few) I have along this journey. Now, I am the lead consultant for an evaluation firm, Impactrio, working on impact measurement, gamification, platform evaluation, sustainability reporting and many other exciting topics. I started creating new tools ( “wheel for change” for social enterprise evaluation, policy evaluation canvas, etc.), contributing to the conversation, and mentoring others. So, conversing and engaging with others is how I started my career, how I learn, and how I grow as an evaluator. OTF knowledge Centre is an excellent opportunity for me and for many others to connect and create swarm intelligence.


(Stacey McDonald) #3

Hi Ash! Welcome to the Knowledge Centre! Thank you so much for sharing your background. I’m so looking forward to hearing more from you. Please feel free to invite others to chat in this space.


(Stacey McDonald) #4

(Gillian Kerr) #5

Hi, Stacey - I was trying to find one of the posts I wrote in March or so with some suggested resources for evaluation, but I can’t find them. Did some of the old posts get deleted? I’d like to add a couple of new resources including a good World Bank site on evaluation.


(Stacey McDonald) #6

Hi @Gillian_Kerr! Welcome back to the Knowledge Centre!

Based on user feedback and technical issues, we had to start over and re-develop the site using an entirely new platform. Unfortunately, that means all posts in the beta site were lost.

Please share any resources you might have!!!


(Gillian Kerr) #7

Got it. I can re-post resources but I can’t see a location on the Knowledge Centre for posting links and files for sharing. Does that functionality exist? If not, I can look for another place to post resources and then share the link here.


(Stacey McDonald) #8

@Gillian_Kerr you can post links in a hub post (if on a new topic, ideally start a new topic discussion/post). We currently don’t have a library of sorts to upload materials and are discussing options. I’d love to hear any thoughts or suggestions.


(Gillian Kerr) #9

@smcdonald I was in a meeting of evaluators yesterday (the regular bi-monthly ‘evaluators cafe’ sponsored by the Canadian Evaluation Society) and we were wondering whether the OTF knowledge centre would be interested in having us join and discuss evaluation topics in a forum where agencies could participate, ask questions etc. It might be useful to the community. Our evaluation group includes nonprofits (like LogicalOutcomes) and internal staff within nonprofits who are responsible for evaluation, as well as consulting evaluators. Topics would include things like how to budget an evaluation, how to work with agencies to develop a good evaluation plan etc. If you were interested in ‘hosting’ this group within the knowledge centre (it would be open to everyone to join and participate in, or just read) we would need a space to keep shared resources and find them easily, by category or description, both links and files. What do you think?


(Stacey McDonald) #10

@Gillian_Kerr I was hoping to make that meeting yesterday, but alas with a babe at home, making evening meet-ups is difficult. I told Paul recently I’d try to organize something over lunch, if there’s any interest (or maybe at 4?).

I love the idea of using this space for the evaluation community to connect online beyond the cafe meetups. We could agree on a common tag/hashtag for all posts for the community, which allows us to view on one page all discussions with a common tag. We do have a resources section that needs to be developed, so let me get back to you about that - i’m looking into options. Maybe i could try to organize an evaluation cafe meet-up around this?


(Ben McNamee) #11

I love this idea Gillian!


(Stacey McDonald) #12

@Gillian_Kerr I just found out that within any post, you can upload a document (including a PDF) if it doesn’t already live elsewhere. The sharing and discussion around different existing resources can happen within the discussion forum.

However, if after a time the discussion leads to the development of a new (or improved) product (let’s say a guide to how to develop an evaluation budget), then OTF can translate it, put it in an accessible format, and then add it to the resources section of this site. Does that seems like it could work with what you had in mind?


(Gillian Kerr) #13

Wonderful. Paul Bakker thought it would be great to encourage agencies to participate in ongoing conversations with a group of evaluators who specialize in nonprofit work - it’s such a cool idea, isn’t it?


(Gillian Kerr) #14

Hmm. We could try it out. The idea of doing an evaluation meetup on this is terrific - let’s suggest it to Paul.

The problem with hashtags is that it needs a level of discipline for all posters that is difficult when it’s not part of a daily routine, and it’s hard to locate files within a forum. I’m not sure it’s worthwhile starting the process without a place to consolidate and update resources. For example, the World Bank has recently posted an evaluation resource at https://dimewiki.worldbank.org/wiki/Main_Page. Better Evaluation (https://www.betterevaluation.org/) has lots of resources, some of which are better than others so it would be good to provide a curated list of sub-links. You can’t do that within discussions.


(Stacey McDonald) #15

Thanks @Gillian_Kerr,

In terms of hashtags, we are here to help with that! As this is a fully-moderated space, we can add or remove tags where necessary when members of the community may not tag appropriately. Additionally, our forum is fully searchable by tag, user, etc., and will return results, including attached PDFs, when searched by keyword.

There’s also nothing stopping you from creating a new post that contains a fully-HTML formatted/curated collection of hyperlinks to resources that exist elsewhere, and to edit that original post over time to add more content or remove it. It can be a living list in that sense, controlled by you and/or others.

While the sector is free to share links, curate lists, create discussion topics, and even upload PDFs, this activity will always remain within the Knowledge Centre’s discussion forum.

We aren’t trying to build a resource library of other folks’ content (such as in your Better Evaluation example) – though as mentioned previously, our intention is to respond to the needs of our virtual communities of practice to produce OTF-branded knowledge products that we will host in our Resources section. That section will grow over time, and we can’t do that without hearing from you in the forum! Seeing what resources shared there resonate most with the community will go a long way toward helping us prioritize what resources to focus on producing.


(Gillian Kerr) #16

I really like your idea of a curated post:

Could we start a discussion topic (like ‘evaluation resources’ or something) that allows members to suggest resources, and then have a few of us pin a series of curated posts at the top summarizing the most useful resources? That would probably meet our needs and keep it informal. As you say, if some tool emerges from our discussions that is worth posting in another area of Trillium, it could be nominated to OTF staff, translated etc., but that’s a different process.


(Stacey McDonald) #17

Hi @Gillian_Kerr, you can absolutely start a discussion topic called ‘Evaluation Resources’ and we can pin it to the top of the message board so it doesn’t get lost.

We can go about this two ways. 1) make it a wiki that anybody could edit or 2) identify specific users that could edit, and adjust their ‘trust levels’ so that they could upload things like PDFs (which i just found out someone that hasn’t posted before can’t do, as a security measure against spamming).


(Gillian Kerr) #18

This sounds perfect. Re permission levels, let’s discuss at the evaluation cafe; we can codesign it together and then make changes as you get feedback from the whole OTF community. I’m excited about this - thanks for figuring out how we can have a public transparent conversation on evaluation topics!


(Amy MacFarlane) #19

Hello! Great to be a part of the group and hoping we can be of great resource!

Our RecRespite Metrics https://recrespite.com/recmetrics/ work comes from a team of professional recreation therapists who have the skill to identify ways of removing barriers for those with disabilities so they are able to participate in their communities. This educational and skilled background gives us the insight to the barriers they are faced with and how (through short and long term goals) we can create more inclusive, accessible opportunities. We measure the impact of community projects in partnership with non profits. Striving for social change and long term sustainability of programs for those who otherwise face barriers to participation. We have been evaluators on a number of very successful community projects from seed grants to grow grants.

I would love to connect with anyone here (amy.macfarlane@recrespite.com) that might want to work together in this capacity or share resources!

Thank you @smcdonald for putting this forum together! :smile: