A few years ago my colleague, James Bolas, the Executive Director of the Coalition for Homeless Youth (http://www.nychy.org), told me there were rumblings that suggested RHY programs had not sufficiently demonstrated whether and how they affect the lives of the young people they serve. In fact, while there is some research on specific programs and behavioral interventions for RHY studied within single settings, it is true that no studies have yet described the programs that serve RHY more broadly, or examined their potential effects on youths’ outcomes. In response, the Coalition for Homeless Youth and our research team at the NYU College of Nursing combined forces to conduct such a study, which is funded by the William T. Grant Foundation and now in its third and final year.
Studying the effectiveness of RHY setting is surprisingly complicated. We decided to conduct a cross-sectional (that is, single point-in-time) study as a first step in a larger program of research. Grounded in the Youth Program Quality Model and using qualitative and quantitative methods, we are assessing program quality and RHY’s outcomes from the perspectives of RHY (focused on those aged 16-24 years old), program administrators, and front-line and mid-level staff, as well as conducting observations of programs using the Youth Program Quality Assessment (http://www.cypq.org/assessment). We are focusing on 29 randomly selected program settings in New York State, focusing on Transitional Living Programs (TLPs), drop-in centers (DICs), and multi-program organizations, in rural, suburban, and urban areas across the state.
RHY’s individual goals and readiness to achieve their goals will vary, but we focused on six youth outcomes which we agreed were reasonable given the missions of the programs and the atypical development of RHY, namely:
- Engagement in school/training/employment
- Whether the setting helps RHY get or stay involved in school/training/employment
- The number of days on which alcohol or drugs were used in the past three months
- Whether the setting helps RHY avoid or manage substance use
- Involvement in the street economy in the past three months
- Whether the setting helps RHY avoid the street economy, if they so wish
We recently finished our first analysis of the data set, which we presented in March of 2016 at the NN4Y conference.
Much of this, presented below, is familiar to youth experts. Yet we believe there is great value in empirically documenting what providers and other stakeholders may already know about RHY programs, but isn’t quite yet found in the research literature, as well as advancing the literature on settings for RHY, and highlighting future research questions.
First we described the RHY in the study.
We interviewed 463 RHY across 29 settings. We found that the characteristics of youth were similar to other studies of the population. They were:
- 19 years old on average
- About half were female (at birth)
- Most were African American/Black and Latino/Hispanic,
- About half were LGBQ/non-heterosexual
- 10% were transgender
Youth had been out-of-home for a number of years, and had numerous poor health indicators and risk factors (e.g., housing instability, depression, anxiety, past incarceration).
Next we looked at how youth rated the quality of the settings.
RHY rated the settings quite positively. Using a structured measure developed by Youth Development Strategies, Inc. called the “Supports and Opportunities Scale” (SOS) (http://www.ydsi.org/ydsi/index.html), youth rated the quality of organizations favorably, with the average rating equivalent to a letter grade of “B” (with ratings ranging from “C+” to “A-“).
We examined youths’ perspectives on specific characteristics of the settings, from the most to least favorable characteristics.
The SOS assesses a range of program setting characteristics, consistent with the Youth Program Quality Model. From the perspectives of RHY, the characteristics of settings ranked in order of the highest to the lowest were:
- Physical safety (“A-“)
- Emotional support (“A-“)
- Practical support (“B+”)
- Peers have knowledge of youth (“B+”)
- Emotional safety (“B+”)
- Growth and progress is supported (“B+”)
- Guidance is provided (“B+”)
- Sense of belonging (“B”)
- Adults have knowledge of youth (“B”)
- Programs are interesting (“B”)
- Programs foster the youth’s own decision-making (“B-“)
- Programs foster youth leadership (“C-“)
These findings validate the programs’ successes in creating safe and supportive environments, the foundation of effective work with RHY. Given that youth enter settings with caution, and typically have difficulties forming trusting relationships, the recognition of the high levels of support and safety in this study is worth noting, and represents a success on the part of the settings.
The ratings on the lower end of the scale raise questions about the importance of a youth-focused approach. Is it necessary to involve youth in decisions and in governance of the setting in order to achieve high setting quality? We will continue to explore this question and elicit expert opinion as we go forward.
We created a combined rating of setting quality, integrating data from RHY (described above), with coded observations of programs, and coded data collected from program administrators.
Again, settings were rated quite positively using this combined rating. Setting quality ranged from a grade of “C” to “A,” with an average grade of “B,” and 76% of settings fell into the A-to-B range. So…again, most settings evidenced high quality, and none were found to be below average. There was reasonable agreement among RHY ratings, observations, and interviews with program administrators, providing evidence for the validity of the assessment of setting quality.
Our study advances what is known about how to assess the quality of settings that serve RHY, and provides evidence that, while setting quality varies somewhat across the different settings, overall the settings for RHY evidence high levels of program quality, and no settings were rated as below average.
In part II of this blog post, we will:
- Examine whether setting quality makes a difference in youth outcomes, and
- Describe findings from the qualitative data regarding the top six characteristics of high-quality settings, the barriers that settings face to achieving high quality, and solutions that settings have found to overcome these barriers.
CITATION: Gwadz, M., Bolas, J., and the RHY Impact Collaborative Research Team. (March, 2016). Settings serving runaway and homeless youth: First analysis of quality, impact, mechanisms, barriers, and actionable strategies. Paper presented at the National Network for Youth National Summit on Youth Homelessness, Washington, DC.