Evaluating Your Program
Ongoing quality improvement is a hallmark of effective mentoring programs. Accurately assessing your program’s successes and ongoing challenges allows you to celebrate and build on what you do well and improve those aspects of the program that are not working. You will probably want to gather evaluation data related to both the mentoring program’s processes and outcomes. Process information lets the program staff know whether the program is being implemented as intended or whether mid-course adjustments are needed. Outcome information allows program staff to measure how well the program has achieved its short- and long-term objectives, and more specifically, to track the “extent to which program participants experience the benefits intended.”1
Some examples of process-related evaluation questions include:
- How many mentors and students were matched (if one-on-one) or participated?
- What was the duration of the program? (e.g. number of weeks, months, semesters)
- Was student attendance adequate?
- Was mentor attendance adequate?
- How long was each meeting?
- Were adequate facilities available for each meeting?
- What kinds of activities did mentors and students participate in?
- How many mentors left before the completion of the program?
- How many students left before the completion of the program?
- How would you characterize the relationships between the students and their mentors?
Some examples of outcome-related evaluation questions include:
- Did students demonstrate an increased awareness of career opportunities in emerging technologies?
- Did students demonstrate an increased enthusiasm for potential career opportunities in emerging technologies?
- Did students enroll in and successfully complete prerequisite courses in science, mathematics or technology?
- Did the rate of retention in emerging technologies programs increase among female students?
- Did students report increased confidence in their ability to perform scientific and mathematical operations successfully?2
The ultimate success of your program depends on how well you are able to assess its effectiveness, address any weaknesses and demonstrate that it is meeting its goals and objectives. Your local college probably has a department of institutional research that could help you in designing and implementing your evaluation. Before talking to them, prepare by answering the following questions:
- Will you use an outside evaluator or someone from your staff to do the evaluation?
- What program processes would you like to measure in your evaluation?
- What program outcomes would you like to measure in your evaluation?
- What indicators of program implementation viability and volunteer fidelity, such as training hours, meeting frequency and relationship duration.
- What instruments (e.g. questionnaires, surveys, interviews) will your program use to collect evaluation data and measure outcomes?
- From which program stakeholders (e.g. mentors, students, parents, educators, local industry representatives, partner organizations) will you collect data?
- How will you and program stakeholders reflect on the evaluation findings and use them to improve the quality of the program?
- How might you use the evaluation results as a recruitment and marketing tool?
1.Outcome Measurement: What and Why—An Overview. United Way of America: Alexandria, VA, 2002.
2.Adapted from Yes, You Can: Establishing Mentoring Programs to Prepare Youth for College. US Dept of Education Partnership for Family Involvement in Education, 1998.
