Achievements and Efficacy of Youth Development Programmes

Achievements and Efficacy of Youth Development Programmes from the report: Youth Development Structured Programmes - A Review of Evidence.

At its inception, youth development was a concept and movement united around two central axioms, as opposed to being a field or an approach.1 These axioms were, according to Connell et al, that programme thinking is inadequate as a basis for policy thinking and that developmental thinking should organise youth policy in general and youth interventions and settings in particular.2 By the late 1980s and early 1990s, there was a substantial body of research that demonstrated that interventions assuming 'the problem' was a deficit in a young person had produced "weak, transient, or no results".3 Such interventions had failed to take account of the complexity of young people's lives or the environment in which they continued to function.

Converging evidence from the adolescent development field, from resiliency studies and applied social research provided a credible platform for the movement, bringing "new substance and credibility to a set of ideas that were already intuitively appealing".4 At the centre of this thinking was the idea that young people were "assets in the making - their development dependent on a range of supports and opportunities coming from family, community and the other institutions that touch them".5

The resultant youth development field and body of practice is one which has, according to Benson and Saito, "high face validity amongst practitioners working in such settings as schools, agencies and youth-serving organizations".6 Early successes include encouraging policy makers and programme designers to focus on young people’s strengths and their general development, and bringing "justifiable and needed attention to youth serving organisations that have long taken a developmental approach, even if they did not call it that".7

Among many others, however, Benson and Saito observe that youth development practice has moved considerably ahead of the scientific foundations of the work.

As we review the research literature, we find kernels of encouragement for establishing youth development as a viable approach. But we see little evidence of the kind of systematic enquiry necessary to guide, shape, refine and fuel the approach.8

Benson and Saito draw a parallel between this situation and that faced previously by the prevention field. In about 1960, they suggest, prevention became an approach that promulgated programmes, professions and professionals. Accountability issues emerged and the field is now being underpinned by a unifying prevention science. Youth development is now, they suggest, at the crossroads faced by the prevention field decades earlier.9

In 2000, Public/Private Ventures (PPV) wrote that the state of knowledge about positive youth development depends on your perspective. "On the one hand, from the perspective of commonsense, it is clear that active attention to a youth's developmental needs has a high probability of paying off in terms of increasing a youth's successes in life and decreasing his or her serious problems".10 On the other hand, the body of evidence about effective interventions is quite limited. "There are many small studies, but few are large and methodologically stringent enough to persuade a sceptic."11 It is quite possible, PPV suggested, for someone to 'support' youth development but not be convinced that social programmes can do much to accomplish it.

The World Bank was considerably blunter in its 2007 Report on World Development:

...programme officials, policy makers and the community in general have only limited information about how best to implement programmes..., about which programme features are the most effective, and even which programmes yield the greatest benefits. Therefore, in practice, most youth programmes are selected and designed on the basis of anecdotal evidence, interviews with potential participants, and the programme designer's personal predilection.12

However limited the evidence base for intervening remains, it is nonetheless increasing. There is accumulating evidence that individual social programmes can in fact produce the skills/competencies/assets etc that are associated with positive development.13

Perhaps one of the strongest sources of support for youth development programmes comes from the reviews and meta-analyses that have been conducted across different programme types. The Social Development Research Group at the University of Washington carried out one of the most substantial early reviews of positive youth development programmes.14 The reviewers located 77 youth development programmes that met their criteria, which included the availability of a rigorous evaluation that involved either a control group or a strong comparison group and automatic exclusion of any programme that involved treatment of a diagnosed disorder or behavioural problem (which made them a prevention programme rather than a youth development one). They ultimately drew on 69 programmes, and more closely on a subset of 25 programmes, that had been designated as 'effective' via evaluation.

The review found that the programmes had variously resulted in positive youth development outcomes including, but not restricted to, improved interpersonal skills, quality of peer and adult relationships, self control, problem solving, cognitive competencies, self efficacy and academic engagement and achievement; and prevented problem behaviours, in the areas of alcohol and drug use, aggressive and violent behaviour, truancy, high risk sexual behaviour and smoking.

The reviewers concluded that, while a range of strategies had produced these results, there were a number of themes common to success. These themes involved methods that strengthened various forms of competence, built self efficacy, shaped messages from families and communities about standards for positive behaviour, increased healthy bonding with adults, peers and younger children, expanded opportunities and recognition for engaging in positive behaviour and activities, provided structure and consistency in programme delivery, and intervened with young people for at least nine months. Eccles and Gootman have subsequently mapped their own work involving personal and social assets and features of effective youth development settings (discussed earlier in this report) back to that of Catalano et al's, and found a high degree of alignment between core concepts.15

There were two general strategies evident in the most effective programmes: skill building and environmental/organisational change. Skills building included social and cognitive skills as well as life skills like decision making, self management, and coping skills. Reflecting the nature of studies chosen, environmental/organisational strategies identified included efforts to influence teacher practice in the classroom and the influencing of peer norms and perceptions.

Moving from meta-analyses such as Catalano et al's that look across programmes to particular types or classes of youth development activities, the evidence becomes considerably weaker. With the exception of mentoring, which has a relatively substantial and robust evidence base,16 much of the evidence for efficacy relies on non-experimental studies at the strongest end through to participant or provider self reports at the other. There is, however, some rigorous evaluation work in the area of cadet-type programmes, of which NZCC and YSC are part.

Cadet-style Programmes

NZCC and YSC fall within a class of youth development activity termed 'cadet-style programmes'; the critical elements of which are, according to the Australian Council on Education, Employment, Training and Youth Affairs:

  • training aimed at developing specific skills
  • opportunities to develop personal characteristics and talents
  • an opportunity to provide some kind of community service.17

There are a considerable number of cadet-style programmes described in the literature. While no programmes were identified that precisely matched the MYD-funded NZCC and YSC programmes, Australia's Green Corps programme seemed to come closest. Green Corps, classified as a nation-wide youth development programme by the Australian government,18 provides Australians aged 17-20 years with the opportunity to volunteer their time and effort to conserve, preserve and restore Australia’s natural environment and cultural heritage. Individual projects involve groups of ten young people and last for twenty six weeks. The aims of Green Corps are to:

  • provide high quality, genuine environmental outcomes
  • provide youth development opportunities for young Australians, including improved employment and educational outcomes
  • promote connections between young Australians and their communities.19

Participants receive an allowance, a uniform, and are provided with training which includes:

  • accredited training leading to a Certificate 1 level in a field relating to the project activities
  • on-the-project training to provide practical skills
  • first aid training
  • occupational health and safety training
  • career counselling.20

A fairly basic evaluation was conducted of Green Corps in 1999, approximately two years after the programme commenced.21 The evaluation reported that programme participants had relatively strong education levels at programme commencement, with 59% having completed secondary education; 61% had been registered as unemployed prior to participation. About five percent were indigenous, which is slightly higher than the proportion of the population in this age group. Using self report and observational measures, the evaluation found participants gained skills and work experience, developed personal and social skills, increased self esteem and maturity, and increased environmental awareness.22

A key aspect of the Green Corps model, and one of the points where the programme appears to diverge from NZCC/YSC, involves the strong expectation that young people will obtain accredited training through programme participation. This training must be provided by a training officer recognised under the Australian Qualifications Framework.

This emphasis on formal qualifications in Australian youth development programmes is also evident in the policy work carried out by the Australian National Training Authority (ANTA) in 2002 to identify a way of recognising participation in youth development activities within the overarching national qualifications framework.23 ANTA proposed a multi-faceted approach to recognition which included "formal recognition through established education and training systems and processes such as mapping skills developed to training packages and school curricula, as well as investigation into qualifications in generic and leadership skills".24 ANTA also recommended that further work be undertaken to develop a 'Youth Participation Certificate', the aim of which was to provide "a universal form of recognition based on participation rather than skill outcomes for all youth activities and to act as a base for more formal recognition options".25

At the current time, the Australian Youth Participation Certificate has not been operationalised; the result of "implementation issues" encountered.26 As part of the preparatory work, however, a tool kit titled 'Above and Beyond' was developed for the youth development field, designed to help youth development groups "officially and formally accredit and recognize their work".27 Part of this toolkit included a booklet on how to develop 'meaningful' youth participation certificates,28 developed in response to youth feedback highlighting the need for improvements in the way youth development organisations approach to recognition and certification.29

The American-based cadet-style programmes, such as Americorps and the American Youth Corps, differ more substantially from NZCC/YSC. In general, they tend to be much longer in duration, include more formal and substantive vocational and education components, pay wages or some form of cash incentive, and result in formal qualifications, for example, major secondary school qualifications.


A recent eight-year impact study of Americorps found graduates were more connected to, empowered about, and active in their communities than the comparison group.30 Graduates were also more likely to have gone on to work in the public sector, particularly those from racial and ethnic minority groups, and reported higher levels of life satisfaction. No difference was found in education levels obtained, although the study did note that about one quarter of both the graduate and comparison groups were still enrolled in higher education so this finding could not yet be considered definitive.

An experimental study of the American Youth Corps, which includes the California Conservation Corp on which NZCC and YSC are modelled, included a cost-benefit analysis of established Corps programmes and impact assessment on participant and community outcomes across a broader range of established and newer Corps programmes.31 32 The cost-benefit analysis indicated a net monetary benefit to society of $1.04 over and above costs, for each hour of service.

Participant impacts were examined at 15 months following random assignment to individual Corps programmes across the following nine outcome domains:

  • civic, social and personal development
  • current and planned community service
  • current or planned involvement in other social service
  • voting behaviour
  • education and training achievements and plans
  • employment and earnings
  • involvement with risk behaviour
  • educational aspirations and expectations
  • work performance.

The most significant impacts related to employment and earnings, with the participant group more likely to have worked for pay and worked more hours than the control group. Much of this impact is attributable to working while in the Corps. Participants were also less likely to have been arrested, with participant arrest rates prior to the follow-up point nearly one third lower than the control group. Participants were also less likely to have obtained a technical certificate or diploma, suggesting that participation may have been a substitute for further education, at least in the short term. Across the other outcome measures, impacts were positive but not significant.

The overall impacts masked important differences across subgroups. No significant differences were found in impacts in relation to participant age or whether they had completed high school, nor in relation to the length of stay in the programme. The most significant impacts were on African-American males, who scored higher on measures of personal and social responsibility, civic behaviour, employment participation and earnings, educational qualifications obtained and were more likely to have increased their educational aspirations than the control group. Hispanic males also had more increases in total hours worked since programme enrolment and had more promotions at work. Negative impacts were evident for white males' employment and earning' outcomes and perceived control of work outcomes.

Impacts on females were generally positive but more limited than for males. African-American females were more likely to have worked since programme enrolment and less likely to be pregnant out of marriage. Hispanic females were also more likely to have worked since enrolment, to have higher educational aspirations and to have received a raise at their current job. White females were more likely to have earned an associate's degree, had higher educational aspirations, and lower alcohol consumption levels.

Overall, participants were highly satisfied with their programme experience. Satisfaction was expressed even amongst participants who were involuntarily terminated from the programmes. Overall, only about a third reported having completed the programme. Another third lest for reasons associated with negative experiences, with the remaining third leaving for employment or education reasons and personal problems.

Benefits to communities stemming from the service activities carried out by the Corps members included, during the 14 month period covered by the evaluation, over 1 million hours of service worth almost $14 million across the eight programmes. Almost 80% of the sponsors of the service projects were highly satisfied/satisfied with the quality of the service provided and virtually all indicated they would be willing to work with the Corps programme again. Nearly three-quarters of the service beneficiaries perceived improvement in their quality of life resulting from programme services.

Mentoring

Mentoring is a formal mechanism for providing young people with a positive relationship with a caring adult. The basic logic is that a structured and sustained relationship with a caring adult will be a protective factor for a young person, with the adult providing support, guidance and assistance that may otherwise be absent, in part or wholly, from their life.33 34 Mentoring programmes vary in their goals, emphasis and structure. Some have very broad youth development goals while others are focused on narrower, more specific goals such as prevention or reduction of specific problem behaviours, or improvements in certain activities such as academic performance. Mentoring may be delivered as a stand-alone activity or as one component of a multi-strand programme and ranges from highly to very informally structured.35 36 37 Reports of group mentoring programmes are also available in the literature, although there is some debate as to whether this does actually constitute mentoring or some other form of intervention.38

The seemingly most commonly cited of mentoring programmes is the US's Big Brothers Big Sisters mentoring programme. Evaluation of the school-based programme, using a random-assignment methodology, found that mentoring had positive effects on a range of areas in young people's lives including school attendance and performance, expectations for future academic activity, and prosocial behaviour.39 A less commonly-cited aspect of the study is that findings apply for a period of up to 18 months after mentoring began, which means this study cannot be used as evidence that mentoring promotes sustained growth or change.40

Jekielek et al similarly found that mentoring programmes had a positive impact on young people's educational participation and attainment, health and safety, social and emotional wellbeing.41 Qiao and McNaught's evaluation of Project K, a 14 month, multi-strand programme aimed at Year Ten students that includes a significant mentoring component, also found significant positive impacts on participants. These included improved ability to master academic activities, to form and maintain peer relationships and social assertiveness in the classroom, to make good career decisions, alongside short term improvements in students' ability to ask for adult help, information and support.42 DuBois, Holloway and Valentine et al also found evidence that mentoring is effective, although the effect sizes were relatively small, and suggested effective programmes included on-going training for mentors, structured activities, frequent contact between mentors and mentees, and parental involvement.43

In their meta-analysis of 39 mentoring programmes for high risk youth, Tolan et al found positive, albeit again modest, effects for delinquency, aggression, drug use and academic performance. They also did not find any differences in outcome depending on whether the mentoring occurred on its own or whether it occurred either as part of a multi-component intervention or alongside other interventions.

Other studies have found that mentoring had either no effect on the outcome areas measured or, in cases where the mentoring relationship broke down, harmful effects on young people's self esteem and alcohol consumption.44 Liabo and Lucas conclude that mentoring does not seem useful for young people at risk of permanent school exclusion, with poor school attendance, involved in criminal activities, with histories of aggressive or disruptive behaviour, or those already involved with welfare agencies.45

Service Programmes

The literature also includes evidence to demonstrate the positive effects that volunteering programmes can have on developmental assets or outcomes. Volunteering or youth service involves "an organised period of substantial engagement, where young people are contributing to their local, national, or world community, in exchange for minimal or no monetary contribution to the participant".46 This contrasts with most youth programmes and activities, where public services are provided to young people.47

The inequality of opportunities for participation in community service activities has been highlighted through research. Opportunities for "having a voice and participating in community affairs are lower for urban youth living in areas of concentrated poverty than for their peers living in middle class communities".48 According to Hart and Atkins, these gaps in civic opportunities and competencies stem not only from the smaller amount of financial resources available but also from the lower ratio of adults in impoverished, 'child saturated' areas.49

Service-focused youth development programmes, one way of filling this gap, have been linked with reductions in problem behaviours, higher intrinsic motivation for work and lower individualistic focus on careers, greater reflection on and subsequent adjusting of future priorities, the formation of volunteering habits, and improved life skills, educational participation and performance and employment outcomes.50 51 52 53 Service programmes also provide a range of benefits to the host organisations, where applicable, and to the communities in which the activities occur.54


Outdoor Adventure Programmes

Outdoor adventure activity has also been linked with a range of positive developmental outcomes. At its most basic, outdoor adventure activity involves using the outdoors as the setting for activities that seek to effect some form of change/growth in participants. Outwards Bound and Spirit of NZ are two of New Zealand's more well-known outdoor adventure programmes. Common features of adventure programmes include a wilderness or backcountry backdrop; small group size (typically fewer than 16); various mentally and/or physically challenging tasks; frequent and intense interactions usually involving group problem solving and decision making skills; a non-intrusive, trained leader; and a duration of two to four weeks.55

Positive programme outcomes described in the literature include an enhanced sense of trust, personal and social responsibility and positive self perception, greater knowledge, skills and abilities, increased understanding of a positive peer culture and ability to develop positive peer relationships, and enhanced social skills.56 57 58 Outdoor adventure can also help participants perceive their world differently and enhance their appreciation for the natural environment.59 Building strong attachments between people and the natural environment can "give rise to spiritual experiences in which people feel a sense of connection with a larger reality that helps give meaning to their lives".60 It can also counter the human cost of alienation from nature, the 'nature-deficit disorder', such as diminished use of senses, attention difficulties, and higher rates of physical and emotional illnesses.61

Adventure programmes are not inherently good, however. Kiewa, for example, observes that, while the learning experience provided by adventure activity will always be powerful, it may not always be positive. Kiewa suggests the necessary components of a positive adventure experience include: an experiential approach; a simple and meaningful reality; need for cooperation; intensity of feeling; opportunities to process experiences; success; choice; and a humane climate.62

Evaluations of outdoor adventure programmes demonstrate the variability in outcomes between different studies, programmes and individuals. Hattie et al's meta analysis, for example, is a key contribution to this work.63 As with other types of activities, there is still much to be learned about which elements of a programme are the most beneficial, as well as how to facilitate the transferral of gains made during a programme to participants' home environments.


Arts and Culture Programmes

The literature suggests participating in art, music, drama and dance programmes can build young people's confidence, self esteem, self discipline, and social and teamwork skills. It can also help young people to be more open-minded, better able to make friends and to deal with difficult experiences, and be more creative in their thinking.64

Cultural programmes, such as those run through the UK's Heritage Lotteries Fund (HLF), can also foster positive developmental outcomes. In 2003, the HLF launched the Young Roots grant programme to engage young people in developmentally-focused heritage activities. An evaluation of 69 very different Young Roots projects, conducted relatively early on in the grants programme's lifespan, found that young people participating in the various projects had:

  • gained new skills in communication, team working and leadership, as well as technical, creative, practical and heritage skills
  • developed self-confidence through having to negotiate with adults, persuade others to be involved in their projects, team working, and assuming responsibility for aspects of the projects
  • developed an understanding of heritage, which included an increased understanding of cultural identity, their community's history, and their place within contemporary society
  • improved social inclusion, and awareness and tolerance of cultural differences amongst young people, and the trialling of new activities were other achievements attributed to the programme.65

At the same time, participating heritage organisations reported gaining expertise in working with and involving young people, which subsequently helped them to increase interest amongst young people more generally in heritage organisations, while communities gained resources they could continue to use, such as heritage trails, gardens, improvements to nature reserves, exhibitions and leaflets, and improved inter-generational understanding through cross-generational projects leading to greater community cohesion.66


Footnotes

1 Connell et al, 2000, op cit, p282

2 Connell et al, 2000, op cit

3 Connell et al, 2000, op cit, p282

4 Ibid

5 Connell et al, 2000, op cit, p283

6 Benson and Saito, 2000, op cit

7 Connell et al, 2000, op cit, p284

8 Benson and Saito, 2000, op cit, p134

9 Benson and Saito, 2000, op cit

10 Public/Private Ventures, 2000, op cit, p12

11 Public/Private Ventures, 2000, op cit, p13

12 Cunningham et al, 2008, op cit, p122

13 Public/Private Ventures, 2000, op cit

14 Catalano, R. F., Berglund, M. L., Ryan, J. A. M., Lonczak, H. S., and Hawkins, J. D (1999), Positive youth development in the United States: Research findings on Evaluations of positive youth development programmes. Seattle, Washington: University of Washington

15 Eccles and Gootman, 2002, op cit

16 Public/Private Ventures, 2000, op cit

17 Ministerial Council on Education, Employment, Training and Youth Affairs, 2000, op cit

18 Ibid

19 See www.greencorps.org, retrieved May 2009

20 See www.greencorps.org, retrieved May 2009

21 At that time, the programme seemed to have more of an ETE focus than a youth development one, as is currently the case, which may explain the lack of attention awarded to broader developmental outcomes in the evaluation

22 Department of Education, Training and Youth Affairs (1999), It’s why you’re swinging the mattock: An evaluation of the Green Corps programme. Canberra: Department of Education, Training and Youth Affairs

23 Australian National Training Authority (2002), Due credit: Examining the potential to recognise the skills achieved by young people participating in youth development programs. Melbourne, Vic: ANTA

24 Australian National Training Authority (2002), Youth participation certificate: Draft discussion paper. Prepared by ANTA for the Commonwealth Department of Family and Community Services, Australia, p2

25 Ibid

26 Personal communication with Anne Hugo, Information Manager, Australian Clearinghouse for Youth Studies, University of Tasmania, Australia, 26 May 2009

27 Hon Larry Anthony, MP, Parliamentary speech Launch of the resource kit: Above and beyond – Recognising Youth development in Australia, presented Parliament House, Canberra, 23 June 2004

28 Department of Family and Community Services (2004), Guide 1: A guide to developing youth participation certificates, http://pandora.nla.gov.au/pan/57238/20060525-0000/thesource.gov.au/above_and_beyond/Guide_1.pdf

29 Personal communication with Anne Hugo, 26 May 2009, op cit

30 Corporation for the National and Community Service (2008), Still serving: Measuring the eight-year impact of Americorps on Alumni. www.nationalservice.gov, retrieved May 2009

31 Jastrzab et al, 1996, op cit

32 No author (n.d.), Youth Corps (American Conservation and Youth Service Corps), www.childtrends.org/Lifecourse/programs/YouthCorps.htm, retrieved May 2009

33 Cunningham et al, 2008, op cit

34 Jelielek et al, 2002, op cit

35 Cunningham et al, 2008, op cit

36 Liabo, K. and Lucas, P (2006), One-to-one mentoring programmes and problem behaviour in adolescence. What works for children group: Evidence nugget, www.esrcsocietytoday.ac.uk, retrieved March, 2009

37 Jelielek et al, 2002, op cit

38 Herra, C., Vang, Z. and Gale, L. Y (n.d.), Group mentoring: A study of mentoring groups in three programs. Public/Private Ventures, www.ppv.org, retrieved May 2009

39 Herrera, C., Kauh, T. J., Cooney, S. M.,Grossman, J. B., and McMaken, J (2008), High school students as mentors: Findings from the Big Brothers Big Sisters school-based mentoring impact study. Public/Private Ventures. www.ppv.org, retrieved May 2009

40 Walker, G (2007), Mentoring, policy and politics. P/PV Brief, www.ppv.org, retrieved April, 2009

41 Jelielek et al, 2002, op cit

42 Qiao, C and McNaught, H (2007), Evaluation of project K. Wellington: Centre for Social Research and Evaluation, MSD

43 DuBois, D., Holloway, B., Valentine, J. and Cooper, H (2002) cited in Boaz, A. and Pawson, R (2005), The perilous road from evidence to policy: Five journeys compared. Journal of Social Policy, 34(2), p175-194

44 Liabo and Lucas, 2006, op cit

45 Ibid

46 Sherraden, M (2001) cited in Cunningham et al, 2008, op cit, p63.

47 Cunningham et al, 2008, op cit

48 Hart, D. and Atkins, R (2002) cited in Finlay et al, 2007, op cit, p67

49 Ibid

50 Cunningham et al, 2008, op cit

51 Finlay et al, 2007, op cit

52 HM Treasury and Department for Children, Schools and Families, 2007, op cit

53 Hair et al, 2003, op cit

54 Ministerial Council on Education, Employment, Training and youth Affairs, 2000, op cit

55 Hattie et al, 1997, op cit

56 Russell, K. C., Hendee, J. C. and Phillips-Miller, D (2000), How wilderness therapy works: An examination of the wilderness therapy process to treat adolescents with behavioural problems and addictions, USDA Forest Service Proceedings RMRS-P, 15(3), p207-217

57 Garst et al, 2001, op cit

58 Schoel et al, 1988, op cit

59 Garst et al, 2001, op cit

60 Schroeder, H (1996) cited in Garst et al, 2001, op cit, p48

61 Louv, R. (2006). Child in the woods: Saving our children from nature-deficit disorder. Chapel Hill, NC: Algonquin Books of Chapel Hill

62 Kiewa, J (1994), Self-control: The key to adventure? Towards a model of the adventure experience. The Haworth Press

63 Hattie et al, 1997, op cit

64 HM Treasury and Department for Children, Schools and Families, 2007, op cit

65 Heritage Lotteries Fund (2005), Learning from Young Roots: Evaluation of the Young Roots Grant Programme. www.hlf.org.uk, retrieved April 2009

66 Ibid