Conclusions and Recommendations

Conclusions

The evidence reviewed demonstrates that effective youth development programmes can have a positive impact on youth development.  There seems, however, to be a mismatch between the current evidence about programme accomplishments and the enthusiasm and passion that proponents of the youth development field express for it.  Certainly there are amazing transformations that take place over the course of individual programmes and no doubt most youth development practitioners can truthfully cite examples of young people's lives moving on a more positive trajectory during and subsequent to intervention.  Overall, however, the evidence reviewed suggests the impact of youth development programmes, when done well, tends to be positive but modest against the outcomes measured.

This picture of programme accomplishment seems largely consistent with the accomplishments of other disciplines or fields of practice that seek to effect change in the lives of young people.  The literature suggests that ETE programmes have had limited, or modest, effects on employment levels and earnings of young people.¹ Areas like youth justice and offending have equally struggled to develop programmes where positive gains in areas like motivation, efficacy and pro-social behaviour are maintained post programme, or even generalised back into young people's settings where those programmes are residential.²

That notwithstanding, our knowledge of 'what works' in youth development is still very much in the early stages; a process not helped by the enormous variability in outcome frameworks across different programmes and the even greater variability in the measures used for testing them.  (I have a personal suspicion, for example, that we have yet to adequately capture the positive effects that actions like 'increased connections with positive settings' have on a young person's life over time).  A further impression is that there is considerable work still to be done before there is consistent application of what is known about effective practice across the youth development sector.  What appears to represent a modest contribution today may in time deliver more substantial returns, especially as the quantity and quality of evaluation and synthesis work increases and that knowledge is more consistently translated into practice. 

This review contends that 'modest value', if indeed that's what it truly is and not an artefact of our evaluation designs to-date, is not 'no value'.  For young people 'on the cusp', a well-timed, well-designed and well-delivered youth development programme that firmly anchors them to a positive setting may well be what makes the difference between life spent on a downwards trajectory versus one on a positive trajectory.  Being firmly anchored to positive settings and carrying out 'work-affirming' activities (eg maintaining a structured routine, developing a sense of belonging, mattering, and being able to contribute meaningfully to a community) may also be what facilitates the smooth entry or return to the labour market for some of the more vulnerable young people unable to secure employment in the current economic climate. 

What this conclusion does highlight, however, is that greater realism may be needed about what youth development programmes can achieve, at least at this point in time.  This seems reasonable when you consider that disadvantaged young people have often spent a decade or more in the school system and still not learned the literacy, numeracy and life skills that will help them obtain employment.³ To expect a 20 week (or possibly shorter) programme to accomplish this is "somewhat astonishing".⁴

The challenge for MYD is ensuring the programmes it funds reflect current knowledge of 'what works', and more broadly, ensuring sufficient standardisation occurs across the programmes to support accountability and efficiency, whilst still allowing providers enough flexibility to be able to respond to the divergent needs of individual localities and participants.  There is clearly an inherent tension between flexibility and standardisation.  On the one hand, a 'one size fits all' programme will never meet the needs of all young people and providers must be able to adapt their activities to reflect the particular young people that are participating in their programme at any one time.  On the other, programmes need to consistently reflect what is known about effective youth development practice and consistently achieve what was intended with the public money that funds them, which implies a degree of standardisation alongside comprehensive, effective monitoring processes. 

Regardless of the challenge involved, there is now enough known about best practice in youth development work to make the application of that knowledge in practice a reasonable expectation.  This review would contend, in fact, that anything less that full application of that knowledge could open the sector and the Ministry to charges of negligence in respect to the value that is expected to be derived from public money.  At the same time, it is recognised that a gap does exist between best practice and what is practiced currently by the youth development sector in New Zealand.  This gap, which in some cases may be quite substantial, will take time and effort to close.

Recommendations

MYD has an important role to play in facilitating the application of best practice across the youth development sector.  It appears the changes needed to support effective practice in the structured youth development programmes are twofold.  Action is needed to ensure best practice principles for youth development are consistently applied by those receiving MYD funding, but more fundamentally, action also appears necessary to ensure the programmes designed and subsequently purchased by MYD are consistent with more generic programme design and delivery good practice principles.

In terms of the generic programme design and delivery issues, MYD needs, firstly, to clarify its own expectations regarding various aspects of programme design and delivery and, secondly, to align its own practices with those expectations.  Specific actions MYD can undertake include:

• confirm the intended programme participant group and communicating this information to providers⁵
• more narrowly define the areas of young people's lives that MYD's structured programmes are expected to effect change in
• explicitly articulate the logic by which the programmes are expected to achieve intended outcomes and communicate this logic to providers to help guide their choices of activities 
• ensure that programme deliverables, including outcomes at programme exit and at three months, reflect this scope and logic
• provide rationales for the programme deliverables outlined in service contracts to help providers understand why they are being asked to perform particular activities and to guide their choices of related activities
• improve the tools MYD requires providers to use for individual deliverables⁶
• build a monitoring framework, together with meaningful measures, that reflects the intended programme scope and deliverables, desired activities and practices, and intended outcomes.

In terms of the application of best practice youth development principles, MYD can:

• encourage providers to use a wider or different range of activities, taking account of appeal to young people, ability to provide or create needed experiences, and the limited information that is known about their ability to facilitate desired developmental outcomes⁷
• require providers to conduct activities that will build participants' connections with positive people that endure beyond the duration of the course 
• require providers to conduct activities that will help participants identify and move towards their longer term goals, rather than simply their short term goals 
• require providers to demonstrate both conceptually and practically how their activities provide meaningful developmental opportunities⁸
• require providers to demonstrate both conceptually and practically how their service projects benefit the community
• require providers to demonstrably incorporate into their programmes features of activities/their settings associated with effective practice
• require providers to articulate and apply a model of engagement and learning practice that optimises the learning and growth that occurs from activities
• consider variations to the current standard programme model of 20 weeks, near-full time activity with eight to twelve young people⁹
• consider including an additional component in programmes to support the portion of young people who remain without positive adult support post programme.

Footnotes

¹ Martin and Grubb, 2001, op cit

² For a summary of evidence, see, for example, Zampese, L (2002), When the bough breaks: A literature based intervention strategy for young offenders. Christchurch: Department of Corrections

³ Higgins, 2003, op cit

⁴ Grubb, D. (1999) in Higgins, 2003, op cit, p21

⁵ Whether this reflects the profile outlined in the current report or some other sub-group

⁶ For example, the goal setting and personal planning tools

⁷ Activities do not need to be conservation- or outdoor-based to be developmentally useful

⁸ Especially in terms of how they increase aspirations, build enduring connections, and better-place them to carry out activities beyond the duration of the programme

⁹ Given there seems no evidential reason to endorse this particular structure over others