In a recent blog post we began to describe a research study we are conducting to address an important, but complex, question, namely, how do we know whether the settings that provide services to runaway and homeless youth (RHY) make a difference in youths’ lives?
To continue this conversation, we now want to talk about whether the quality of a setting makes a difference in youth outcomes, as one means of exploring whether and how settings impact youth. We also describe findings from qualitative data we have collected regarding the top six characteristics of high-quality settings.
First, we described how youth are doing on the six main outcomes we previously described.
- 81.0% were engaged in school/training/employment in the past three months
- 61.7% said the setting helps them get or stay involved school/training/employment
- The average number of days on which alcohol or drugs were used in the past three months was 16.72 (Standard deviation [SD] = 27.74)
- 47.0% said the setting helps them avoid substance use
- 36.7% were involved in the street economy in the past three months
- 66.4% said the setting helps them avoid the street economy
Looking at the above, we see that RHY in this sample are doing pretty well in some respects, including compared to other studies of this at-risk population. But on the other hand, these data also raise some concerns about the well being of the youth. For one, rates of involvement in the street economy are a concern, given the potentially dangerous and traumatic nature of those activities. We also need to know more about the level of educational achievement and types of jobs RHY have obtained. For example, under-employment is common among “typically developing” adolescents, and, we can assume, for RHY as well. We will continue to drill down in the data sets to better understand these outcomes and their implications.
We explored the hypothesis that youth in transitional living programs (TLPs) do better than youth in drop-in centers (DICs) on the six main outcomes, in large part because stable housing serves as a protective factor.
We found youth in TLPs showed better outcomes than those in DICs on three of the six main outcomes. Youth in TLPs were significantly more likely:
- To be engaged in school/training/employment (92% vs. 79%)
- To say the setting helps them get or stay involved in school/training/employment (75% vs. 60%)
- To use alcohol and drugs on fewer days than their peers in DICs (mean = 9 days [SD = 21.05] vs. 18 days [SD =28.46])
There were no significant differences between youth in TLPs and DICs on the other three outcomes we are studying.
Our cross-sectional study does not allow us to determine if RHY in TLPs do better because of the program, or if they do better because TLPs require higher-functioning RHY (and/or the higher-functioning RHY are able to access, engage with, and benefit from a TLP). In fact, there are some differences in the location of TLPs (more likely to be in Upstate New York) and demographic characteristics of RHY compared to those in DICs (more likely to be female and White in TLPs). Theses regional and demographic differences complicate the interpretation of findings somewhat. However, we have data on the length of youths’ involvement in the programs, which we can use in future analyses to explore these questions, although we suspect the true answer is that a combination of these factors is at play.
The next question we asked was whether setting quality matters with respect to youth outcomes, that is, is higher setting quality associated with better RHY outcomes.
We found that the quality of a setting does make a difference for youth on three of the six outcomes, where a higher quality of the program increases the odds that the youth reported the setting:
- Helps them get or stay in school/training/work
- Helps them avoid or manage substance use
- Helps them avoid the street economy.
We did not see a relationship between setting quality and the other three outcomes. Yet most were involved in school/training/work, suggesting a “ceiling effect,” that is, a situation where scores are high, thus limiting the ability to examine relationships among variables. Another possibility is that larger social, economic, or structural factors exert more influence than programs in some cases. We found rates of involvement in the street economy were significant, and not associated with setting quality, and this may reflect the primacy of economic need which programs may not be able to address, particularly if youth are not stably housed. However, it is worth noting that even when RHY are involved in the street economy, programs generally help youth to conduct these activities safely, consistent with a Harm Reduction approach.
Using qualitative data, we explored the characteristics of high-quality settings across the 29 organizations we studied.
We found six main characteristics of high-quality settings:
- An over-arching core philosophy and mission have been articulated
- Staffing, and staff support, foster appropriate and positive youth-staff relationships, the heart of effective settings.
- The setting is youth-centered with respect to youth involvement and governance, and youths’ setting their own goals and expectations.
- Both instrumental support and emotional support are provided and these work synergistically to engage and support RHY.
- Settings help youth develop both short- and long-term goals, despite primacy of youths’ crises.
- Settings engage in on-going reflection and quality evaluation and improvement.
We are currently preparing a manuscript for publication that describes and teases out each of these characteristics, with quotes from RHY and staff. We will also identify barriers that programs face in achieving high quality, and some solutions that program settings have found to overcome these barriers
Of course, every study has limitations and we acknowledge some of ours, including its cross-sectional design, and the fact that outcomes for RHY are assessed by self-report. On the other hand, assessments of RHY were anonymous, which may improve the veracity of youths’ reports, and we used existing, reliable measures. Nonetheless, we will address these potential shortcomings by looking for areas where the various data sources show convergence, and plan to conduct a longitudinal study in the near future to better untangle these relationships.
Although no research study is definitive, these study findings make a substantial contribution to the body of evidence in support of the daily critical and life-saving role that RHY programs play in the lives of these youth at-risk.
CITATION: Gwadz, M., Bolas, J., and the RHY Impact Collaborative Research Team. (March, 2016). Settings serving runaway and homeless youth: First analysis of quality, impact, mechanisms, barriers, and actionable strategies. Paper presented at the National Network for Youth National Summit on Youth Homelessness, Washington, DC.