LARNet; The Cyber Journal of Applied Leisure and Recreation Research 

Mentoring At-Risk Youth Through In-School Recreation:
Issues in Program Evaluation
(Aug 2004)
Don Dawson
Angela Gray
Kealey Hester

Leisure Studies Program
School of Human Kinetics
University of Ottawa
Ottawa, Ontario, Canada

contact:
Kealey Hester
School of Human Kinetics
University of Ottawa
1819- 190 Lees Ave
Ottawa,Ontario, Canada
K1S 5L5
e-mail: kealey_hester@canada.com

Abstract

Mentoring is an increasingly popular approach to dealing with the needs of at-risk youth. Mentors are generally older, more experienced people who serve as good role models and offer positive encouragement to youth. At-risk youth are young people who are deemed to be at-risk of doing poorly in school, getting in trouble in the community, and otherwise not achieving their full potential. Among the variety of program innovations there are numerous recreational interventions and school-based initiatives. One such specialized program, Mentoring At-Risk Youth Through In-School Recreation, is examined. In the program, university students volunteer as mentors for at-risk children and youth in local schools. The mentors assist the at-risk students academically in the classroom and organize recreational activities for them during the school day.  Initial evaluations conducted with mentors show the program to be successful in improving the self-esteem, social skills and academic performance of at-risk students. However, these evaluations face a series of obstacles, and need to be further developed and expanded.

Key Words: mentoring; at-risk; youth; recreation; school; program evaluation.


Introduction
Recreation contributes to many facets of community and cultural life. It has been increasingly applied to improving community health and personal well-being. As Witt and Crompton (2003) point out, beginning in the mid-1990s, there has been an unprecedented expansion in recreation programs designed to reduce at-risk youth deficit behaviors. A series of recent recreation-based projects have been initiated to bring the benefits of recreation to at-risk youth in a variety of settings. At-risk youth face many hurdles in their lives and it is hoped that appropriate, targeted recreational interventions will assist in improving their quality of life. One particular approach that integrates recreation into the lives of at-risk youth is mentoring. Mentoring programs of various types are now widely provided by many organizations and they are becoming more accessible to many at-risk youth across the USA and Canada. The specialized approach of mentoring at-risk youth through recreation has now found its niche, particularly in the educational arena.

This paper examines one such mentoring program: the Mentoring At-Risk Youth Through In-School Recreation program that has been in place for several years in Ottawa, Canada. It is a cooperatively run program involving a not-for-profit community association, a local university, and area public schools. Essentially, university students act as mentors in schools with a high incidence of at-risk pupils, helping in the classroom and organizing recreational activities. The goals of the program are to provide good role models and positive encouragement to at-risk youth in order to increase their self-esteem, to help them get along better with peers and teachers, and to inspire them to reach their full potential in school and beyond. The primary focus of this report is the challenges inherent in evaluating the program’s effectiveness. Evaluations serve not only to aid in improving program delivery, but also to sustain stakeholder commitment and secure future funding.


Mentoring At-Risk Students

Mentoring has a long history dating back almost 5000 years in Africa.  At that time, mentors were seen as guides to younger people, responsible for their social, physical, intellectual, and spiritual development (Carr, 1999). According to Baran (1992) mentoring is a one-to-one relationship between two people differing in age, with the older person as the mentor and the younger as a protégé or mentee.  This supportive relationship is directed toward the development or mental growth of the younger member.  Baran sees mentorship as a powerful tool in the shaping of young people’s lives. Education has an enormous impact on the future of today’s youth, and mentorship programs are seen to be potentially of particular beneficial for at-risk youth as an intervention in the schools (Baran, 1992).  If implemented correctly, mentors can help put at-risk children back on the right track at school.

Mentoring programs are offered through schools, communities, businesses, and higher education institutes and serve young people in a variety of situations and stages of development.  School-based mentoring is an increasingly popular mentoring design, used, for example, by Big Brothers/Big Sisters as an alternative to their traditional Big Brother/Big Sister program. This style of mentoring pairs a volunteer and a youth who meet at the youth’s school during school hours. Mentors will generally spend some time counseling the young student on academic matters, but the intent is that they will devote most of their together time to other activities. School-based mentoring is advantageous because it attracts volunteers who may otherwise be unable to participate in community-based programs due to scheduling or being unable to commit enough time (Herrera, 1999). There is the expectation that in-school mentoring programs, limited to the school year and with weekly or bimonthly meetings at a fixed site in the school, will lead to greater numbers of volunteer mentors.

The term “at-risk” as applied to youth has several definitions from which one can gather an understanding of this broadly used term.  Stumbo (1999) held that those children and youth who have characteristics such as low self-esteem, minimal social skills, ineffective coping and stress management skills are most at-risk.  Stumbo emphasized that at-risk youth often lack the social skills needed to maintain friendships. McWhirter,  McWhirter,  McWhirter, and McWhirter (1998) offer their definition of the term at-risk directly in relation to education. At-risk youth are students who are at-risk of school failure to which also links many other problems expressed by young people (McWhirter et al., 1998:61). Mentoring at-risk youth within the educational context focuses primarily on the child’s school performance, but recognizes that many other emotional and behavioral objectives are important as well.

Kronick’s (1997) discussed how youth’s at-risk status is a function of the inappropriateness of their developmental environments in meeting their needs as a person, and that a focus on these deficient environments may be more productive than a focus on individual deficit behaviors. At-risk youth are defined as a category of persons whose personal characteristics, conditions of life, situational circumstances, and interactions with each other make it likely that their development and/or education will be less than optimal (Kronick, 1997:p5).  Mentoring programs are seen as one of the developmental environments that can motivate at-risk youth to stay in school and to strive toward greater achievement.

McIntyre, Haggar, and Wilkin (1994) provided a more specific definition of mentoring within an educational setting. Mentoring is presented as a mechanism of counselling, educating and socializing the student into the school environment, through which student enjoyment and success in school can be increased. In-school mentoring can be multi-disciplinary, and approaches such as those emphasizing recreational interventions and/or tutoring initiatives are encouraged as long as they provide benefits to the at-risk child or youth. Again, these benefits may be social or emotional, as well as directly academic.

McWhirter et al. (1998) point out that when teachers have larger classes, less attention can be given to at-risk students who are experiencing difficulty in coping within the classroom. Teachers cannot devote enough extra time to meet the greater needs of at-risk pupils. Consequently, they often are disruptive, sometimes withdrawn, but always falling behind academically. This, in time, has a negative impact on their self-esteem and relationships with peers.  With mentoring programs in schools, the mentors are there to help both the teacher and at-risk students who need more attention and guidance.

Research on mentoring at-risk students in many educational settings has found that they are more likely to feel academically competent, achieve higher grades, are less likely to begin using alcohol or drugs, and are more likely to be enrolled in post-secondary education after high school (Grossman, 1998). Mentors report that they have made a difference in reducing the negative feelings their protégés/mentees had about themselves and decreasing the amount of trouble they got into at school. Mentors become advocates to their protégés and assist in improving relationships with teachers and other academic staff. Their grades improved subsequently. Mentors also facilitate relationships with other children, improving peer relations (Herrera, 1999).

Sipe and Roder (1999), in a Big Brothers/Big Sisters report, provide evidence that at-risk youth benefit from in-school mentoring in many areas, including being less likely than their peers who were not involved in the program to hit others or to skip class. Participants also achieved better grades at the end of the study and felt more capable of doing well in school. Further research by Big Brothers/Big Sisters reported that youth who participated in their school-based mentoring program developed higher levels of self-confidence, were better able to express their feelings and had improved relationships with adults and peers (Sipe & Roder, 1999).

A national, meta-evaluation of Big Brothers/Big Sisters in-school mentoring programs demonstrated that sustained youth-adult connectedness could facilitate resilience in young people (Tierney, Baldwin-Grossman, & Rech, 1995).  In these relationships, at-risk youth believed the mentor was available for them.  With this support they stayed in school, felt more competent about doing their schoolwork, achieved higher grades, and reduced drug use. Moreover, it was proposed that such on-going contact needs to be part of schooling, because it is essential to the sustained climate of the “real-talk” relationship between mentors and protégés that facilitates a sense of connectedness which leads to the acquisition of information as a lifelong learning process (Brown, D’Emidio-Caston, & Bernard, 2001).  Within these school-based mentoring programs at-risk youth have the opportunity to create relationships with reliable people who are available for them.
In a study by Booth (1993) high school students completed questionnaires before and after a sustained in-school mentoring program. The students reported increases in their levels of confidence and attributed that to their mentors being supportive, accessible, sympathetic, and positive.  As a result, Booth (1993) concluded that mentors were instrumental in increasing students' confidence in class participation, positive response to teaching, and enthusiasm in approaching school work.

An extensive study of at-risk youth in mentoring programs in the Birmingham, Alabama area reported that national achievement tests showed that by the time these students had reached senior high school, they were performing better than comparable at-risk students not benefiting from any mentoring (Cafferty & Spangenberg, 1983). Yet, they were still two to three years behind in skills proficiency for their grade level.  It was concluded that intervention was needed much earlier in the students’ lives if the problem was to be addressed at its core.  Indeed, it is not uncommon for middle school students to drop out before many at-risk programs begin (McWhirter et al., 1998).  In fact, teachers in the primary grades can often easily identify those students who will later be at-risk of failing in high school (Slavin, Karwei, & Wasik, 1994).  These findings demonstrate the need for schools to provide intervention programs at the elementary level. Consequently, a new program of counselling and tutoring in basic skills for elementary and early secondary school students was established (Cafferty & Spangenberg, 1983).  The in-school program operated after school hours in severely depressed neighbourhoods of Birmingham and focussed on students with basic skills below grade level, poor attendance records, and low participation in school activities.  These students were judged to be most at-risk of dropping out of school because most exhibited a strong sense of failure.  Pre-testing before the start of the program and post-testing evaluations after two years of the program demonstrated that the majority of students did make significant (x2 probability less than 0.05) quantitative gains in their grade level, attendance record and participation in school activities (Cafferty & Spangenberg, 1983).  The Birmingham experience highlights two important programming facets of in-school mentoring. First, it shows how essential evaluations are in the delivery of a good program. Second, it reveals that intervention at the elementary level can result in even greater success than mentoring that only begins at the high school level.

Another example of successful elementary level mentoring intervention is the School to Aid Youth (STAY) program which is aimed at creating a positive attitude toward school and building self-esteem, thereby increasing at-risk children’s ability to learn (McWhirter et al., 1998).  This program served first grade students who were identified as being at risk during their year in kindergarten.  Project STAY engages mentors in one-on-one tutoring as well as interacting in small groups of students. The program also stresses communication between teachers, mentors and parents.  A ten-year study of the STAY program demonstrated that 80% of the STAY graduates were performing at or above grade level.  The key to these positive results, according to McWhirter et al. (1998) is the program’s success in building up students’ self-esteem.


Recreation Interventions
Mentoring is also found in a variety of recreational programs for youth in that activity leaders regularly offer their young charges guidance and support, as well as acting as role models. In a comprehensive review of recreation service delivery in the Province of Ontario, Canada, the Parks and Recreation Federation of Ontario, composed of professional recreation practitioners, and the Ontario government’s Ministry of Tourism and Recreation (1992) concluded that leisure opportunities for youth provide positive lifestyle choices and alternatives to self-destructive behaviour, and deter negative social behaviours such as vandalism and minor criminal offences. Recreation participation, the review found, reduces the alienation, loneliness and anti-social feelings that many at-risk youth experience. In this light, recreation activities are seen as  preventive in that they serve to release tension, address youthful desires for risk, challenge and competition, and can also act as safe outlets for aggression (Ministry, 1992).

In a 1995 study, members of the American Academy for Parks and Recreation were asked how college and university recreation curricula should respond to the major trend of addressing the needs of youth at-risk through recreation programs.  Results indicated that the Academy members held a strong belief that students should be exposed to real life situations and the challenges of youth-related programming (Hultsman & Little, 1995).  In addition, it was suggested that students should have more courses and internships that deal directly with youth. Mentoring within a recreational context is one such initiative.

Stumbo (1999), following an extensive review of successful intervention activities for at-risk youth, advocates recreation-based mentoring programs, offered through the schools, specifically reaching out to children who display characteristics such as low self-esteem, minimal social skills, ineffective coping and stress management skills, and a sense of loss of control. She suggests that such interventions will result in improved academic performance as well as assisting these students in developing friendships through improved social skills.

In 1993 the Dade County, Florida Park and Recreation Department joined local businesses in the implementation of the Good Life Mentoring Program in three schools, involving approximately 150 youthful participants (Blitzer & Wolff, 1993).  The recreation-based program gathered together volunteer mentors who were willing to serve as positive role models for at-risk youth at the elementary and middle school levels. Since its inception, there had been no extensive evaluations of the Good Life Mentoring Program.  While preliminary interviews with program administrators and participants indicated that it is highly valued and much appreciated by the surrounding communities, it has been plagued by transportation, personnel, and financial problems that are limited the program’s development.  Without punctual evaluations demonstrating the benefits of the program, securing much needed funding is difficult and the program’s future faced uncertainty.

The city of Anaheim developed the Save-A-Youth (S.A.Y.) Project whose short-term objectives include:
(a) To keep youth in school;
(b) To return youth to school;
(c) To assist youth in achieving passing grades (reduce incidence of failing);
(d) To assist youth in reducing drug and gang activity; and,
(e) To assist youth in completing informal/formal probation (reduce repeat offences).

Project S.A.Y. consisted of outreach, paraprofessional counselling, as well as recreation and sports to encourage youth participation in positive activities. Within the project was the Kids-in-Action program, a prevention component for children five to twelve years of age that focused on self-esteem and confidence building, keys for treating at-risk youth.  It offered recreation and sports programs in local community centres and in elementary schools.

Both Project S.A.Y. and Kids-in-Action underwent extensive evaluations of their programs, varying from one to three times per year.  These included participant surveys, school personnel and parent surveys, evaluations of program staff and sites, and police personnel surveys.  Due to the positive results of these evaluations, in 1994 the City of Anaheim allocated $250,000 towards Project S.A.Y. activities (Witt & Crompton, 1996).  This is an example of the importance of evaluations in assuring a program’s future by demonstrating program effectiveness.

Successful program evaluations are crucial to delivering and sustaining quality, in-school mentoring programs. The issues related to the evaluation of a particular Mentoring At-Risk Youth Through In-School Recreation program are outlined below.


The Program
In 1999, the Canadian Association for the Advancement of People (CAAP), an Ottawa area not-for-profit community association, approached local school boards with the idea of using university students as mentors in designated elementary and secondary schools to interact with at-risk students in the classroom and during recreational activities run by the university students. The objectives of the Mentoring At-Risk Youth Through In-School Recreation program was to make school a more positive experience for at-risk children and youth, to increase their self-esteem, and to improve their social skills and academic performance. The University of Ottawa was then convinced by the mentoring program proposal, and agreed to participate by setting up a credit course in conjunction with the program. The program and course were formally initiated in 2000. The mentors were undergraduate students from the University of Ottawa who were enrolled in a parallel full year academic course while they mentor in the CAAP program.

Approximately 30 students each year became mentors in the local schools identified as having a large population of at-risk youth. Mentors volunteered in one of the five elementary schools, from kindergarten to grade 6, a middle school of grades 7 & 8, or one of the two secondary schools, grades 9 to 12. Between 3 to 5 mentors were assigned to each of the participating schools, each one was assigned to a classroom and teacher at the beginning of the school year. The Mentoring At-Risk Youth Through In-School Recreation program was then conducted over the school year, starting in September, and concluded when the university students completed their winter semester in April.

Prior to entering the schools, prospective mentors received training in Workplace Hazards (as mandated of all volunteers in any provincial institution), Child Abuse Prevention, Behavior Management, Allergies, and Medication. They also had to complete and pass a police records check.  Once in the schools, mentors were placed in a classroom based on the school’s needs and the mentor’s strengths and experience. The mentors helped in the classroom by focusing primarily on potential protégés, those at-risk students identified by the supervising teacher as being either the neediest, or who may benefit most from the program. Mentors also organized school-based recreational activities for students before and after school, as well as at lunch or recess. Mentors organized activities such as a lunchtime basketball league, a games room, video game or computer clubs, card trading clubs, or arts groups. Some initiated unstructured play groups at recess time. The mentors’ goal was to be a good role model both academically and socially for the at-risk students they befriended and with whom they tried to develop close mentor-protégé relationships.

Each mentor was required to complete a minimum of 4 hours of mentoring per week for the duration of the school year. The training/support program for the mentors consisted of weekly sessions at the university under the supervision of a professor. These classes examined mentoring, at-risk children and youth, the benefits of recreation and related topics. During scheduled discussion periods, mentors had the opportunity to voice their concerns or discuss the challenges they are facing with their protégés and were able to seek feedback and suggestions from the other mentors or program supervisors. Mentors were often privy to personal information entrusted to them by their protégés and therefore, gained valuable feedback in these confidential weekly sessions on how to best support their protégés. Mentors were also responsible for completing field reports that detail their experiences as mentors and their personal observations in the schools.

The program balanced three stakeholder groups that contributed to its success. First, the university, which provides the student mentors, training and on-going support for the mentors in the credit course. Second, the community group coordinated with the university and the participating school boards. The community group also placed and supervised the mentors in the schools that best suit their experience and interests. Third, the schools provided the potential protégés (i.e., at-risk children and youth) and supported the mentors within the classrooms and recreational programs.

Unlike many other mentoring programs, the program used a spontaneous pairing method, allowing the mentors from the university and the protégé to choose one another during the classroom time or during recreational activities organized by the mentor.  No pre-program matches were made, and under these circumstances, each university student could have become a mentor to one or more possible protégés. Mentor-protégé relationships were allowed to form naturally with the mentor and classroom teacher identifying possible protégés and then introducing the mentor to the children or youth.

The spontaneous pairing approach embraced by this mentoring program had many benefits for participating at-risk youth, including the self-selection of a mentor that they feel comfortable with, trust, and with whom they identify. However, certain issues did arise with this form of mentoring when attempting to evaluate the effectiveness of the mentoring on the protégés. Due to the gradual and spontaneous nature of the mentor-protégé self-selection process, it was difficult to administer a pre- test and establish a control group before the start of the program. At the program’s outset, protégés and mentors had not yet identified each other, as no formal mentor-protégé pairs are arranged by program administrators. As at-risk students were not required to participate in program initiatives, but rather do so voluntarily, it was also impossible to pre-determine which of them would ultimately choose to become involved with the program.


Evaluating the Program
Four groups of subjects were potentially available to participate in the evaluation of the effectiveness of the Mentoring At-Risk Youth Through In-School Recreation program; the mentors, the teachers, the protégés, and the parents of the at-risk youth. The preliminary evaluations outlined below involve only the mentors. A few teachers had filled out and commented on draft evaluations specifically designed for them, but the evaluation component was still at the developmental stage. The direct participation of at-risk youth and/or their parents in program evaluation was awaiting school board ethics approval and should be in place in a year or two.  Nevertheless, the mentor-based evaluations were crucial components in any evaluation scheme because of their close face-to-face involvement with their protégés.
The evaluation of the mentoring program has been undertaken on three separate occasions. The university student mentors developed the first evaluation in the initial 2000-2001 academic year. This initial attempt at program evaluation employed an instrument formulated by the mentors in the program. The evaluation, self-administered at the end of the program, measured the impact the mentors reported having on their protégés.

The questionnaire (see Appendix 1) provided for short answer and closed questions to assist mentors (N=32) in evaluating their impact on their protégés. In the first evaluation, 87% of mentors described their relationships with protégés as either good or great, while all mentors reported evidence of positive effects of their mentoring on their protégés. These improvements were primarily made in work habits, peer interaction and attitude towards school. Self-image and scholastic performance were also indicated as areas where improvement could be observed. Nearly half of the mentors, however, did indicate that despite their mentoring efforts, not all of the at-risk youth improved their relationships with peers and teachers, their work habits, scholastic performance or self-image. Nevertheless, mentors did feel they had a positive impact, with 63% reporting to be either effective or very effective in their mentoring, and 94% felt they made some impact or a great impact on the lives of their protégés. All the mentors who completed the evaluation reported they would continue mentoring if circumstances permitted. Finally, when asked what the most valuable part of the mentoring experience had been, mentors concentrated on working with the protégés and becoming an important person in the child’s life, making kids smile and laugh, and being a friend.  The use of a self-developed evaluation allowed mentors, at the end of the program, to reflect upon and evaluate the particular aspects of the program of most importance to them.

In the next year of the mentoring program, 2001-2002, a second evaluation was introduced. The mentors completed the self-developed evaluation of the previous year, but also responded to an evaluation of mentoring outcomes suggested by the Enter Mentor program guide (Baran, 1992). This evaluation focused principally on the specific improvements and benefits evidenced by the protégés during the program, as observed by their mentors. Baran’s list of benefits to protégés provided the basis for this second evaluation of the Mentoring At-Risk Youth Through In-School Recreation program. An ex post facto questionnaire was administered to mentors following the completion of their participation in the program (see Appendix 2). The questionnaire used a ten point scale (0-10) to measure the benefits accrued by the protégés according to the mentors’ perception. Mentors (N=28) completed a separate questionnaire for each of their protégés (N=32). The questionnaire used a variety of items to measure (a) pro-social behavior, (b) academic performance, (c) self-concept and (d) motivation, to determine the perceived impact of the mentoring program on the protégés.

Mentors reported improved pro-social behavior, with 87% stating improved communication and 94% revealing their protégé experienced improved interactions with peers. Academically, all of the mentors reported protégés to have fewer distractions and 94% reported better concentration. Self-concept was seen to have improved for all protégés, with mentors evaluating improved self-confidence and improved self-esteem to have been enhanced by the program. Lastly, mentors reported increased motivation; with 94% of protégés being more enthusiastic and over half were happier at school. Moderate improvements were also observed in most of the other areas of pro-social behavior and academic performance. The use of an ex post facto evaluation is particularly appropriate in this program because it allows for the natural relationships to form gradually, with evaluations completed only after the program, thus avoiding the pre-selection/pre-test difficulties outlined above.

Most recently, a pre-test/post-test evaluation was attempted with the mentors in 2002-2003, in addition to the 2000-2001 mentor-developed evaluation and the 2001-2002 Enter Mentor (Baran, 1992) inspired evaluation. The pre-test/post-test instrument was developed based on a review of the literature on mentoring, examining previous studies of benefits to protégés, and evaluations of similar programs targeting at-risk youth. Early in the program, mentors filled-out the pre-test for all potential protégés, as identified by school personnel, and then completed a post-test at the end of the program.

This evaluation attempted to evaluate each protégé’s status early in their involvement in the mentoring program and later on, following the time spent with their mentor. Mentors (N=30) completed a one-page evaluation (see Appendix 3) of their protégé(s) after the first two weeks of the program. Mentors rated protégés using items related to their (a) scholastic performance, (b) peer interaction and (c) social conformity on a five-point scale. The mentors were then asked to re-evaluate their protégés at the conclusion of the mentoring program using the same questions and scale. Results were then analysed to assess whether a positive progression had been made in each of the areas, comparing pre-test scores to those on the post-test for each protégé.

Results for the protégés for whom both the pre-test and post-test scores were available (N=24) indicated that improvements could be clearly observed in several areas. The most impressive improvement was made in the area of communication with the mentor. Fully 83% more of protégés were discussing their out-of-class problems with the mentor at the end of the program than at the early stages. The protégés also discussed issues they experienced at home 75% more at the end of the program and 67% more discussed their relationships with their friends with the mentor.  Academically, 67% of protégés improved in the areas of participation in group activities in class, completing homework on time, complaining less about finishing assignments and obeying the teacher’s directions. Mentors also reported protégés to raise their hand more often in class, to express happiness at completing a task in class, and that students became more honest in their interactions with the mentor.

A major challenge associated with this last method of evaluation was that no pre-matching of mentors and protégés was in place. The program uses a spontaneous pairing system, in which the mentor and protégé relationship develops naturally from their association and interaction in recreational or academic settings. The spontaneous pairing approach allows for the mentor-protégé relationship to emerge naturally as the pairs self-select themselves. This yields an authentic mentoring relationship which avoids the contrived nature of a planned dyad wherein an identified at-risk youth is pressed into a relationship with a pre-determined mentor they have not previously met.

After two weeks of interacting with students, mentors completed the 2002-2003 evaluation pre-test for all potential protégés; any students identified by the school or teachers as being at-risk and who displayed an early interest in program activities. It is to be noted that not all so-identified students chose to continue their participation in the recreational and academic activities initiated by the mentors, and they ultimately did not enter into a mentor-protégé relationship. Thus, pre-tests were completed for some students who did not become protégés. As well, some students not pre-identified as possible program participants did emerge as active protégés in the mentoring program, and, for them, no pre-test was available. As a result, although there were 30 mentors in the 2002-2003 program, both pre- and post-tests were completed for only 24 protégés.

The above factors also mitigate against establishing a matched comparison group of at-risk youth who will not participate in the program. In fact, many activities run by the mentors are open to any and all students who wish to participate. Again, at-risk students are not slotted into the experimental or treatment group or the control or comparison group prior to the delivery of the program in the schools. Similarly, the level of participation in the program is determined by the protégés themselves, resulting in some protégés participating far more extensively with their mentor than other protégés do with other mentors. Consequently, it would be expected that some students would benefit from the program more than others based on their degree of engagement within the recreational and academic mentoring activities.


Discussion
Mentoring through recreation is an innovative intervention strategy that shows promise in addressing the needs of at-risk youth. In an educational setting, such programs have goals of getting children and youth to enjoy their school experience more, in addition to ameliorating their academic performance. In achieving these goals, mentors work to enhance the self-esteem and social skills of at-risk students such that they have a better self-image, more confidence and improved relations with teachers and peers. As with any new program initiative, evaluations are required to demonstrate that stated program goals are, in deed, being met. Program evaluations, then, need to be routinely conducted. Existing evaluations of mentoring programs, while increasing in number, have tended to be sporadic and often largely anecdotal (Merriam, 1993). In particular, programs featuring a recreational component have not been widely evaluated (Witt & Crompton, 2003).

As a means of highlighting some of the challenges facing the evaluation of programs aimed at mentoring at-risk youth through recreation, the issues surrounding one such program were examined. The Mentoring At-Risk Youth Through In-School Recreation program in Ottawa, Ontario, Canada served as an illustrative example. Three different program evaluation efforts were carried out in the first three years of the program, from 2000 to 2003. In the first two years ex post facto evaluations were conducted using mentor assessments of protégé progress. Both indicated positive impacts and a variety of perceived improvements in the at-risk students’ self-concept, pro-social behaviour, and academic motivation and performance. A more rigorous evaluation of this type of recreation intervention should approximate, as nearly as possible, an appropriate experimental design. That is to say, a design incorporating a pre-test as well as a post-test and using a control group. The third (2002-2003) evaluation did attempt to include a pre-test for program participants.

The philosophy and structure of the Mentoring At-Risk Youth Through In-School Recreation program allows for spontaneous mentor-protégé relationships to emerge naturally out of the interaction of mentors and at-risk youth in academic and recreational settings. While this approach gradually yields unforced, genuinely self-selected mentor-protégé relationships, it renders the selection of a control/comparison group and administration of pre-tests problematic because it is not known before hand which at-risk youth or children will actually participate in the mentoring program. These are exactly the obstacles encountered in carrying out the otherwise positive third year evaluation of the program; no comparison group was readily available, and both the pre-test and post-test were not completed for some program participants.

A solution to this problem may be to carry out pre-tests and post-tests for all students in a class or grade. This would include all program participants and non-participants, because all students, not just those at-risk students who are initially thought to be potential participants, would be tested. Subsequently, students could then be divided into comparison groups according to their degree of participation in the mentoring program. For example: no participation at all; partial participation; and full participation. This would ensure pre-tests and post-tests for all students and enable program evaluators, with input from mentors, to separate the at-risk students into non-participants and participants, or to sort them according to their degree of participation.

It would be difficult for mentors to adequately observe and accurately evaluate all students in pre- and post-tests because of the sheer numbers involved that could be up to 30 students in a class, and the depth of personal knowledge required to evaluate each student. To assist in these tests, teachers could also fill out the pre-test/post-test instrument. The combined scores of teachers and mentors would provide more complete student assessments than found in the earlier evaluations that relied solely on the perceptions of mentors. As well, pre-test/post-test instruments administered directly to students themselves could be developed to measure self-esteem and other at-risk characteristics. There is also the possibility of getting parents involved by having them fill out pre- and post-tests concerning their children’s development.

The above evaluation improvements, pre-and post-tests for all students, regrouping students as to their degree of program participation, and using teacher, student and parent as well as mentor assessments are certainly feasible individually and in combination. As of the present, the schools involved in the Ottawa Mentoring At-Risk Youth Through In-School Recreation program need to be convinced that the application of the full extent of these measures is important to program success and that they would not be too intrusive when implemented. It must be recognized that program evaluation, in order to be successful, is an on-going process that requires stakeholder commitment as well as time and energy from all concerned.  It appears that the next step for the Mentoring At-Risk Youth Through In-School Recreation program will include the pre-test and post-test of all pupils in the classroom to make a comparison group of non-participants available and to have both tests for all members of the treatment group of program participants. The full participation of teachers in both the pre- and post-testing will supplement the observations of the mentors.

While the Mentoring At-Risk Youth Through In-School Recreation program is but one example of the burgeoning array of mentoring programs being developed for at-risk youth, the issues it has had to address in trying to evaluate its effectiveness are illustrative of those facing similar programs across North America. Like other mentoring projects with a recreational component, it shows great promise, but this promise will only be realized when convincing, well designed evaluations demonstrate that its benefits to at-risk youth are both substantial and tangible.


References

Baran, D.  (1992).  Enter Mentor: A Program Guide.Ottawa: Human Resources Development Canada.

Blitzer, L.E. & Wolff, R.M. (1993).  The Good Life Mentoring Project Program Manual. Miami: Florida International University.

Booth, M.  (1993) The effectiveness and role of the mentor in school: The students’ view. Cambridge Journal of Education 23(2):185-197.

Brown, J.H., D’Emidio-Caston, M., & Bernard, M. (2001).  Resilience Education. Thousand Oakes: Corwin Press Inc.

Cafferty, S.J., & Spangenberg, G. (1983).  Backs Against the Wall: Urban-Oriented Colleges and Universities and the Urban Poor and Disadvantaged.
            New York: Ford Foundation.

Carr, R. (1999). Peer Resources: Directory of mentor arts and mentorship.  (Online) http://www.peer.ca/peer.html.

Grossman, J.B.  (Ed.). (1999). Contemporary Issues in Mentoring. (Online)  Public/Private Ventures. http://www.ppv.org/content/mentoring.html.

Herrera, C.  1999,  School-Based Mentoring: A First Look into its Potential. (Online) Public/Private Ventures. http://www.ppv.org/content/mentoring.html.

Hultsman, W., & Little, B. (1995). Recommendations for a Youth-At-Risk Initiative: Comments from the Academy.  Journal of Park and Recreation Administration
           13(1):10-25.

Kronick, R.F.  (1997).  At-Risk Youth: Theory, Practice, Reform. New York: Garland Publishing Inc.

McIntyre, D., Hagger, H. & Wilkin, M.  (1994).  Mentoring: Perspectives on School-based Teacher Education.  London: Kogan Page Limited.

McWhirter, J.J., McWhirter, B.T., McWhirter, A.M., & McWhirter, E.H. (1998).  At-Risk Youth: A Comprehensive Response (2nd ed.). Pacific Grove:
           Brooks/Cole Publishing.

Merriam, S. (1993).  Mentors and Protégés: A Critical Review of the Literature. Adult Education Quarterly 33(3):161-173.

Ministry of Tourism and Recreation. (1992).  The Benefits of Parks and Recreation: A Catalogue.  Toronto, Ontario: Parks and Recreation Federation of Ontario.

Sipe, C.L., & Roder, A.E. (1999).  Mentoring School-Age Children: A Classification of Programs. (Online)  Public/Private Ventures.
           http://www.ppv.org/content/mentoring.html.

Slavin, R.E., Karwei, N.L., & Wasik, B.A. (1994).  Preventing Early School Failure: Research, Policy, and Practice. Boston: Allyn & Bacon.

Stumbo, N.J. (1999).  Intervention Activities for At-Risk Youth.  Pennsylvania: Venture Publishing Inc.

Tierney, J.P., Baldwin-Grossman, J., & Resch, N.L. (1995). Making a Difference: An Impact Study of Big Brothers/Big Sisters. Philadelphia: Public/Private
           Ventures.

Witt. P.A. & Crompton, J.L. (2003). Positive youth development practices in recreation settings in the United States. World Leisure Journal 45 (2):4-11.

Witt, P.A., & Crompton, J.L. (1996).  Recreation Programs that Work for At-RiskYouth.  Pennsylvania: Venture Publishing Inc.


Appendices
1back



2back



3back