When families are struggling, they may benefit from interventions designed to help improve an aspect of how they function. Together with support such as case management, access to economic resources, and peer partners and mentors, clinical interventions can be critical to improving a family’s safety, permanency, and well-being. Child- and family-serving agencies, whether community-based organizations or government-based child protection agencies, have a legal and moral obligation to carefully consider how to implement, adapt, and offer interventions that will be the most effective to the families they serve. Toward that end — and because the Family First Prevention Services Act (Family First) provides funding for the delivery of evidence-based programs and practices to prevent a child’s entry into foster care — child protection agencies and their community-based partners are expanding their use and implementation of evidence-based programs and relying more than ever on data to inform program, policy, and contracting decisions.1,2,3
Key Questions
Implementing evidence-based programs and services — whether directly or via contract — can help child- and family-serving agencies use resources more efficiently, increase consumer satisfaction, and improve safety, permanency, and well-being outcomes for children and families involved with (or at risk for involvement with) the child welfare system4,5 To be effective, however, evidence-based programs must address an identified need, be implemented with program fidelity; and be compatible with the values of the specific population it will serve (racially, ethnically, culturally, and linguistically).
Deciding which evidence-based practice to implement requires engaging the community and lived experts, and analyzing current and historical needs assessment data, using the following strategies:
Identify the problem to solve, and the specific population to be served. The first step is to identify a specific issue to address. Although agency leaders may have some sense of the outcomes to focus on, they first need to analyze data to determine whether existing practices or programs are meeting agency goals and, if so, for which populations or sub-populations. Qualitative data is particularly important, such as feedback from community members and those receiving agency services now or in the past. Community needs assessments and participatory research are some key methods for gathering this information. Other data sources include internal agency monitoring and review data, as well as external oversight data, such as data obtained during an outside review or audit.6
Determine why the issue or service need exists. After identifying the issue to be addressed, the next step is to figure out root causes of the issue or service need. Root cause analysis is conducted by continuing to ask why an issue or service need exists and examining the data to better understand the factors associated with the problem. In addition to data from an agency’s case management system, possible data sources include literature reviews, focus groups with lived experts and other stakeholders, surveys, and case record reviews. The Child Welfare Capacity Building Collaborative has created a video series to assist agencies in problem exploration.
Chapin Hall’s Family First Evidence-Based Practice Exploration & Cost Tool
Chapin Hall created an Excel-based resource, the Family First Evidence-Based Practice Exploration & Cost Tool, to facilitate discussions among planning teams and ensure that diverse perspectives are represented when assessing and selecting an intervention or array of interventions. The tool, which includes detailed implementation and cost information on numerous evidence-based programs, can assist with exploration of models, benefit-cost exploration, and assessment of readiness for implementation. The “Model Selection” survey tool can be used to match program fit and usability based on target population, program goals, evidence rating, program intensity and duration, program cost, staffing and training requirements, and other factors. Survey items were adapted from the National Research Implementation Network’s Hexagon Discussion and Analysis Tool and the Capacity Building Center for States’ Readiness Assessment Tool.
While research indicates that many evidence-based programs are effective at improving the safety, permanency, and well-being of children and families involved with the child welfare system, the findings may apply only to certain populations of children, families with specific characteristics, or a subset of specific agency contexts.7,8 Other factors that contribute to a program’s suitability for a given population include the specific issue the program or practice addresses, cultural appropriateness for the population served, the fit between the program and the culture of the family-serving agency, historical and current implementation challenges, and availability of resources.9,10 Improving the fit between an evidence-based program and the agency implementing it increases the likelihood of a program’s effectiveness.11
Once the target population has been identified, along with the specific issue to be addressed, evidence-based program selection can be improved by employing the following strategies:
Understand how different sources present evidence of effectiveness. Selection requires analysis of program effectiveness. Multiple websites and clearinghouses are available to aid in the identification of evidence-based programs and practices.12 Some clearinghouses simply present the evidence that exists, while others have defined standards of evidence and provide a rating for the program based on those standards. For example, some clearinghouses rate programs that were evaluated multiple times with rigorous evaluations as “evidence-based,” while those that have been evaluated using a less rigorous design are rated as “promising practices.”13
Consider agency context. Simply analyzing information about the effectiveness of programs implemented in the past is not sufficient. Additional information about the environmental context, specific implementation methods, and how the program can be adapted for diverse contexts and populations is also necessary.14
Information about how different contextual variables affect outcomes is not often available from clearinghouses.15 Agency leaders, staff, and lived experience partners should consider interviewing the program developer or purveyor, or consulting with another agency that already has implemented the program, to gain a better understanding of contextual factors before committing to implementation.
Consider culturally specific models developed by and for specific communities. While many agencies serve families of color and LGBTQ+ families, perhaps even as their majority population, the majority of evidence-based programs are developed for and studied with white cisgender individuals and families. Some programs can be adapted for other communities, but those that are designed for a specific community utilize approaches and strategies that are particularly designed to benefit that community, and should be prioritized whenever possible. Chapin Hall’s brief, Elevating Culturally Specific Evidence-Based Practices, defines what it means to for an intervention to be culturally-specific and includes detailed descriptions of several evidence-based programs.
Questions to Assess Evidence-Based Program Fit
There is no failsafe method to ensure that every program an agency decides to implement is a good fit. Likewise, there are no assurances that a chosen program will automatically improve safety, permanency, or well-being for the identified population, especially in isolation, as we know that children and families also benefit from access to case management and concrete and peer supports. That said, clearly identifying the problem to be solved and its root causes, using practices and programs with reliable research evidence, properly training and coaching staff, and ensuring the chosen program addresses agency needs can help support better outcomes for the children and families that agencies serve.
1 Content of this brief was informed through consultation with researchers at Chapin Hall: Clare Anderson, Senior Policy Fellow; Brian Chor, Research Fellow; Jennifer O’Brien, Senior Policy Fellow; and Gailyn Thomas, Policy Fellow.
2 Pew-MacArthur Results First Initiative. (2014). Evidence-Based Policymaking: A guide for effective government.
3 Tseng, V. (2012). The uses of research in policy and practice and commentaries. Society for Research in Child Development, Social Policy Report 26(2).
4 DuMont, K., Wulczyn, F., Anderson, C., Samuels, B., Weiner, D., Danielson, D., Killos, L., Maher, E., O’Brien, K., Pecora, P.J., & White, C. R. (2016). Assembling the pieces: Research, policy, and practice in child welfare. Chapin Hall, Casey Family Programs, and William T. Grant Foundation.
5 Pew-MacArthur Results First Initiative. (2014).
6 Permanency Innovations Initiative Training and Technical Assistance Project. (2016). Guide to developing, implementing, and assessing an innovation. Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau.
7 Pecora, P. J., Klein, R., Simon, J., Foster, L. J. J., Gallagher, A., Keller, C. & Baumgardner, S. (2023). Interventions shown to be effective with children and families of color and LGBTQ2SI+ persons who may be served with Family First funding. (Second Edition). Seattle, WA: Casey Family Programs.
8 Hyland, S. T., & O’Brien, J. (2023). Evidence-based programs desk guide 2023. Chapin Hall at the University of Chicago.
9 Tseng, V., & Nutley, S. (2014). Building the infrastructure to improve the use and usefulness of research in education. In A. Abukari, M. Byrne-Jimenez, & P. W. Miller (Eds.), Using research evidence in education (pp. 163–175). Springer International Publishing.
10 Capacity Building Center for States. (2023). Change and implementation at a glance: Intervention selection and design/adaptation. Children’s Bureau, Administration for Children and Families, U.S. Department of Health and Human Services.
11 Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4–23.
12 Horne, C. S. (2016). Assessing and strengthening evidence-based program registries’ usefulness for social service program replication and adaptation. Evaluation Review, 41(5).
13 Pew-MacArthur Results First Initiative. (2014).
14 Horne, C. S. (2016).
15 Horne, C. S. (2016).