Making it count: Improving how we measure effectiveness in youth justice

In 2017, the Research Evidence into Policy Programmes and Practice (REPPP) project began studying administrative data processes in youth justice systems. John Reddy investigate

Administrative data is information collected by service providers and primarily is used for record-keeping and risk assessment; case management; monitoring and evaluating programme performance; and ensuring service provider accountability. The policy question was ‘how useful is data we keep in the youth justice system in terms of informing the tax-payer about how well the youth justice system is operating?’

We first reviewed the Irish system – tracing the development of the State’s responses to youth crime and presenting an analysis of current policies and priorities. The multi-agency system we have today focuses on diverting young people away from crime and involvement in the criminal justice system and rehabilitating young offenders. Youth justice agencies provide welfare, development, and educational programmes that aim to improve youth behaviour, reduce recidivism, and, when necessary, prepare young people for re-entry into society. Data is collected from young people as they interact with justice services. Agencies analyse and report data gathered using the An Garda Síochána’s Pulse system, risk assessment and case management procedures, and from case review processes.

To examine how Ireland’s system compares to international best practice, we selected six jurisdictions – Washington and Pennsylvania in the USA, The Netherlands, England and Wales, Sweden, Scotland. These jurisdictions were identified in criminal justice literature and in United Nation’s reports as having effective youth justice systems, with well-developed data collection and system measurement processes. We assessed data processes by reviewing government published material and justice agency reports and interviewing justice experts from each system.

In systems, data is collected using a range of administrative processes. These include police databases, risk assessment and case management procedures, youth crime and recidivism monitors, court and detention review processes, and national youth crime surveys. Typically, state-supported agencies coordinate data processes, assess interventions and publish research in order to provide information about youth crime and justice programmes. In most systems, data inputted into national data systems is used in the compilation of practice reports and research, statistical updates, assessments of evidenced-based interventions, and practice toolkits. These outputs are published via system databases and justice websites.

The study found that it is important that data processes provide the capacity to assure the quality and performance of youth justice interventions and ensure service providers meet evidence standards. This is best achieved when data is analysed and reported on a system-wide basis. A more complete understanding of youth justice becomes available when data generated daily by agencies is integrated and subjected to regular and standard analyses. An effective data system is one that draws on up-to-date information from each part of the system. Service management and practitioners also require ‘rounded’ assessments so they can better target resources, services and programmes. This may include information about youth crime and re-offending trends, youth background and family circumstances, their interaction with justice and welfare agencies, youth experiences and attitudes to crime, and which youth justice interventions are working best.

What was common across the international systems studied was that implementing effective and integrated data systems required innovative and genuine state and local level partnerships. This typically meant careful negotiation around data access and balancing system goals and local priorities. In systems with more effective data systems, coordinating data bodies encouraged compliance with administrative data processes primarily by providing current and accessible (user-friendly) criminal justice databases. Data processes need to be considered inclusive and meaningful if they are to be used consistently (and correctly) by practitioners. This is best achieved when practitioners view data processes as informing their practice and improving their capacity to provide positive outcomes for young people.

Ireland, like the international systems, mainly collects information about what resources are spent on youth justice and what activities these resources support. However, there is less information from all the jurisdictions studied on whether these resources and activities have made a positive difference. The second part of this study focusses on data use in the Irish system and how administrative processes may be best used to inform youth justice policy and practice.

John Reddy is a Youth Justice Researcher in the School of Law, University of Limerick

For full and in-depth coverage, see the current printed edition of Garda Review.

subscribe button