How are child welfare leaders applying safety science to critical incident reviews?

The death of a child is devastating to families and communities.¹ When the tragic death is related to abuse or neglect, there is often tremendous pressure to react quickly and assign individual responsibility and blame. This may result in firing of staff and perhaps the resignation of the child welfare leader, but rarely do these reactive and punitive actions result in lasting systemic change. More often, such responses limit how much a system learns from the incident to keep children safe and prevent future fatalities. Furthermore, this reaction may even do harm by derailing or redirecting resources, which can lead to increased staff turnover and defensive practice. In addition, new policies and procedures often are enacted that overload the system and make it less effective in carrying out its mandate of ensuring the safety of children that come to its attention. These quick, knee-jerk responses to individual events, often exacerbated by media and public pressure, do little to prevent the more than 1,700 child maltreatment fatalities that occur in the United States each year.

A better way

Momentum has been building for applying safety science principles to critical incident reviews (CIRs) in the child welfare context. Grounded in both systems thinking and psychological safety, a safety science framework² can enhance accountability of child protection agencies, using the same method employed by other complex organizational environments requiring high levels of safety, including aviation and medical systems. In 2011, Casey Family Programs launched a national effort to improve safety and prevent child maltreatment fatalities in partnership with various stakeholders including child welfare leaders, policymakers, academics and practitioners.  

In January 2018, child welfare agencies from 20 jurisdictions participated in the Tennessee Safety Culture Summit about the application of safety science in child welfare. These efforts culminated in 11 jurisdictions and Casey Family Programs forming the National Partnership for Child Safety (NPCS), a quality improvement collaborative focused on applying safety science in child welfare, specifically to prevent future maltreatment fatalities and improve child safety through innovations in child protection. The NPCS continues to grow (24 jurisdictions in 2020), with members participating in safety science-derived quality improvement activities, including applying a standardized platform for CIRs and sharing data.  

The safety science field is valuable when applied to critical incident and child fatality reviews because it is evidence-based, systems-focused, and promotes learning and change through an approach that: 

1. Transitions from individual blame to overall systemic accountability. 
2. Applies systemic methods of learning and investigation.
3. Addresses complex systemic issues rather than focusing on the application of quick, simplistic fixes such as firing staff. 

A safety science approach goes hand in hand with the development of a safety culture, which must be cultivated throughout the agency — from top to bottom — and with external partners, including constituent partners and those with lived experience, to promote learning. Tools must be created to collect data and identify and understand systemic barriers to child safety, and recommendations must be made to effectuate change. The safety science approach leads to instructive, retrospective learning when applied to systemic critical incident reviews, but also necessarily includes a prospective component focused on mindful organizing and other team-based strategies to function as a proactive strategy to prevent future harm. This brief shares the experiences and learnings from four jurisdictions that are part of the NPCS: New Hampshire;³ Georgia; New York City; and Los Angeles County. 

Evidence-based

Safety science is an evidence-based field of discipline and successful CIRs use scientific methods to inform system reform, including the collection and analysis of data. Through application of the safety science approach, CIRs identify and explore the system factors contributing to improvement opportunities, which articulate the gap between what families need and what families receive from the child welfare system. 

Assessment tool

The NPCS uses tools, such as the Safe Systems Improvement Tool (SSIT) and the System Analysis Tool (SAT),⁴ which are structured to assess, organize, and rate information gathered when exploring systemic barriers to child and family safety and well-being. By reaching beyond “human error” to identify the complex and various contributing factors related to an incident, the tools produce data that inform improvement strategies. The two key purposes of the tools in the context of a CIR are to record and explore the “system’s story” of the critical incident, and communicate and advocate for a quality improvement strategy.⁵ Jurisdictions can create their own tailored tools, but should include four core elements:⁶

• Quality improvement: Allows for the identification of systemic improvement opportunities that set quality improvement actions into motion. 
• Outcomes measurement: Ability to translate ratings into data that can be analyzed at an individual or aggregate level. 
• Communication: Turns complex relationships and systems into a common language that facilitates discussion.
• Culture carrier: Reinforces a culture of safety that focuses on systems, not individuals.

Shared data

One of the important goals of the NPCS is sharing data across jurisdictions to capitalize on a more comprehensive dataset aimed to protect children from future harm. When each jurisdiction looks at its own data in isolation, the numbers are small and less informative. Having a national collaborative approach to data collection allows for the identification of meaningful trends and patterns, enhancing the visibility of areas for child safety improvements. Sharing data across states also makes it possible to disaggregate and examine the data by race/ethnicity to track disparities at different decision points, potentially identifying larger issues, such as implicit bias, that can be used to inform systemic or policy reforms that advance racial justice.

Sharing data locally also is important for identifying potential system-level improvements. It allows for tracking data to identify trends in practice, reporting, response, and follow-up. For example, if each critical incident was reviewed separately, patterns would be difficult to detect and the ability to identify improvement strategies would be lost. 

Mapping of issues

The tools (SSIT and SAT) are designed to help assess and record systemic contributors to improvement opportunities. Improvement opportunities can be visually represented on a “systems map” to facilitate multi-disciplinary conversation about the systemic factors that led to how and why decisions were made and how those decisions, plans, and actions may have influenced the trajectory of the incident. The mapping allows for expanding the examination of an incident to explore it thoroughly. Mapping does not identify cause, blame, or make value statements about the work, but rather explores rationales for decisions made, demands and pressures on the staff, systemic interactions, and variability or drift in case practice. The process must be blameless to be effective. Reviewers work from the premise that every decision made was thought to be the best possible in the given situation, and that those decisions were made in the context of existing system flaws and vulnerabilities. Mapping highlights the obstacles that impacted the safety and well-being of children and families, and allows for a constructive conversation regarding how the system could be better designed to support families and prevent critical incidents. 

Staff debriefings

While staff who were not directly involved with the family at the center of the critical incident typically do the systems mapping, a key component of CIRs are one-on-one, supportive debriefings with the staff who worked directly with the family prior to the critical incident. Staff debriefings, sometimes called “human factors debriefings,” serve as facilitated opportunities for staff involved in the incident to share, process, and learn in a blameless and voluntary environment. Through this process, staff have an opportunity to share their experience working with the family, and can describe the complex intrapersonal, interpersonal, and agency factors that affected decision-making. In this way, staff are given a voice in the review process and, ultimately, in generating recommendations for system reforms.

Systems-focused

A systems-focused model expands the scope of learning from an individual case to a comprehensive analysis that explores system-wide opportunities for improvement. With CIRs, systemic challenges that serve as barriers to child safety are brought to light, and meaningful recommendations are generated to ameliorate them. 

Create a safety culture

A culture of fear undermines and inhibits the potential for learning. To implement a safety science approach successfully, child welfare leaders must balance individual responsibility with system accountability, and create an environment that promotes psychological safety for staff by eliminating fear of punitive actions, making concerted efforts to support them, and allow their voices to be heard in every review process. A lack of trust can bury information critical to child safety and place additional pressure on an already stressed system. Safety culture helps staff feel comfortable sharing information and letting issues surface by being transparent, collaborative, and self-reflective. 

Safety culture cannot be decreed: it must be modeled, accepted, and embraced by all levels of leadership. Leadership charts a path for general acceptance of safety science principles throughout a child protection agency⁷ and must be consistent, sincere, and committed to not targeting individuals when things go awry. 

Before Georgia’s Division of Family & Children Services (DFCS) started using a safety science review approach, caseworkers were advised by supervisors not to talk in reviews due to fear of negative consequences. Following implementation, reviews now are perceived as helpful. Caseworkers embrace the process as positive, acknowledging that they appreciate knowing they have contributed to making the system better and walk away from the experience with new knowledge. DFCS and Georgia’s Office of the Child Advocate (OCA) also support staff who may be emotionally impacted by the incident and in need of support. 

Similarly, Los Angeles County Department of Children and Family Services (DCFS) had an adversarial review process prior to implementing a safety science approach, whereby Internal Affairs was involved in the review process and the District Attorney could go as far as to charge a caseworker criminally. DCFS developed trust by sitting down with regional administrators, caseworkers, and all other relevant staff to explain individually how this new open and honest review process would work. Gaining trust took time and energy, and included establishing a review process protocol for participants regarding the safety of the process,⁸ but ultimately led to acceptance and faith in the new approach. 

Shift to systems accountability

A focus on individuals absent a critical examination of how the system as a whole promotes child safety will not prevent future harm. Eliminating shame and blame in CIRs does not excuse or minimize accountability for tragic events; it expands the focus so responsibility is assigned to the system as a whole. 

New York City’s Administration for Children’s Services (ACS) recognizes that case practice can be imperfect, and child protection teams may encounter a variety of challenges on any one case. ACS operates from the presumption that well-intentioned case practice decisions were made in light of systemic obstacles, isolating individual accountability only when “willful misconduct” is clear. The goal of the reviews is to surface obstacles as learning opportunities, understand the rationale behind decisions, and provide insight into what could have been done differently. Ultimately, the review process has helped promote the principles of safety science across the organization, launching a safety culture campaign, and shifting both the culture and the language used to describe case practice. The CIRs are a continuous process of learning, creating a mechanism for ongoing system refinement, with each review surfacing new areas for improvement.

Collaborate with external partners

Child welfare is a complex system with many external partners, and the CIR process should reflect that complexity by including multi-disciplinary perspectives, data-sharing, and a collaborative examination of various points in the system where support and safety could be enhanced. The child protection agencies in both Georgia and New Hampshire have significant partnerships with their respective Office of the Child Advocate (OCA), but collaborate in different ways. In Georgia, the Division of Family and Children Services and OCA work together to hold one review. In New Hampshire, OCA leads its own CIR process — the System Learning Review and System Review mapping — alongside and in support of multiple layers of CIRs, including internal reviews by New Hampshire’s Division for Children, Youth, and Families, and the state Child Fatality Review Committee comprised of multi-disciplinary experts. When all system partners collaborate, there is easier access to information and each external partner can bring a different lens to identifying system flaws, including communication barriers between systems. For example, medical and legal jargon can be difficult for child welfare staff to interpret. A review that identifies confusion as a basis for a decision can lead to a systemwide commitment to use language understandable to all. 

Promotes learning and change

CIRs grounded in safety science can lead to recommendations that result in meaningful change and improved safety for children.  The system-focused tools (SSIT and SAT) explore systemic contributors to improvement opportunities, and the collected data leads to action steps that result in systemic fixes. The jurisdictions profiled in this brief applied safety science principles to CIRs and found greater success engaging staff, identifying systemic issues to address, and effectuating concrete systemic change. Examples of recommendations and outcomes to date from the safety science CIRs include: 

• Proactive and timely responses to safety concerns, including immediate, real-time safety considerations for other members of the household where the critical incident occurred. Information should be fresh, accessible, and not excessively academic.
• Changes in case practice, which may involve hiring clinical specialists to educate staff on working with families exposed to domestic violence or struggling with mental health issues. In the New Hampshire Office of the Child Advocate’s 2019 System Learning Review Summary Report, smaller caseloads and greater support from supervisors were recommended steps to address high levels of caseworker stress, and enhance staff capacity and well-being.
• Creation of a feedback loop to share findings and continue discussion. In New York City, the review team reports back to the local office involved in the review with timely recommendations that can lead to more discussion.
• Streamlined policy requirements to reduce extensive or burdensome documentation requirements that may overwhelm staff and keep them from other case practice responsibilities. New York City ACS worked to remove policies that were unnecessary and streamline remaining ones to be more efficient, eliminating 15 policies in all and increasing productivity.
• Public service announcements and safety campaigns to engage the public and educate them about specific supports or steps to take to prevent fatalities, such as lack of sleep safety or an increase in child drownings in the jurisdiction.
• Collaborative efforts with external partners to improve communication. In Los Angeles County, DCFS worked with medical hubs that provide child evaluations to educate physicians about how to use plain language to avoid misinterpretation. As part of this initiative, doctors also make themselves more available to child welfare staff, who are invited to call them if they need more clarification.
• Plans for action published in annual or periodic reports with reform recommendations and timelines for implementation. Both New York City ACS and New Hampshire’s Office of the Child Advocate publish annual reports pursuant to local laws.
• Data system modifications, such as the data system created in New York City to track recommendations and the steps necessary to accomplish them. 

Looking ahead

The Family First Prevention Services Act sets forth requirements that each state provide a description of the steps it is taking to collect information about child maltreatment fatalities and implement a plan to prevent them. This presents an important opportunity for states to improve the quality of the CIR process and utilize it to improve child safety and prevent future deaths. Implementing a standardized CIR process grounded in safety science is a critical step toward fulfilling these requirements and is supported in recommendations made by the Commission to Eliminate Child Abuse and Neglect Fatalities.

1 Content for this brief was developed based on interviews with: Michael Cull, Associate Professor and Associate Director, and Tiffany Lindsey, Clinical Instructor and Policy Analyst, Center for Innovation in Population Health, University of Kentucky, Dec. 11, 2020; Noel Hengelbrok and Scott Modell, Collaborative Safety, LLC, Dec. 4, 2020; Zeinab Chahine, Managing Director, Casey Family Programs, Dec. 3, 2020; Moira O’Neill, Director, and Emily Lawrence, Associate Child Advocate and Counsel, State of New Hampshire Office of the Child Advocate, Feb. 5, 2021; Diane Iglesias, Senior Deputy Director, and Cynthia Wong-Blye, Division Chief, Risk Management Division, Los Angeles County Department of Children and Family Services, Jan. 26, 2021; Mary Havick, Deputy Division Director of Child Welfare; Rateicia Davis, Child Fatality Specialist, Lee Biggar, Senior Director of Knowledge Management, Steven Reed, Director of Quality Management, Martha Dukes, Manager Child Death/Serious Injury Review Team, Lashundra Stevenson, Child Fatality Specialist, Georgia Division of Family & Children Services; Ryan Sanford, Investigator, Rachel Davidson, Director, Georgia Office of the Child Advocate; and staff from Georgia Division of Family & Children Services and Office of the Child Advocate, Feb. 5, 2021; Andrew White, Deputy Commissioner, Division of Policy, Planning, and Measurement, Ancil Payne, Assistant Commissioner to the Office of Quality Improvement, and Ellen Howard-Cooper, Associate Commissioner to the Office of Quality Improvement, New York City Administration for Children’s Services. Feb. 12, 2021.
2 Cull, M., Rzepnicki, T.L., O’Day, K., & Epstein, R.A. (2013). Applying Principles from Safety Science to Improve Child Protection. Child Welfare, 92(2), 179-196.
3 New Hampshire’s Division for Children, Youth, and Families (DCYF) participates in the National Partnership for Child Safety, but the state’s Office of the Child Advocate (the agency interviewed for this brief) does not.
4 Developed by Collaborative Safety, LLC
5 Cull, Michael, Lindsey, Tiffany, & Epstein (2019). Safe Systems Improvement Tool: National Partnership for Child Safety Version. Chicago: Praed Foundation.
6 Cull, M. & Lindsey, T. (2020). Safe Systems Improvement Tool (SSIT) User Tip Sheet. Chicago: Praed Foundation.
7 Deaver, A.H., Cudney, P., Gillespie, C., Morton, S., & Strolin-Goltzman, J. (2020). Culture of Safety: Using Policy to Address Traumatic Stress Among the Child Welfare Workforce. Families in Society: The Journal of Contemporary Social Services 101(4), 428-443.
8 Los Angeles County Division of Children and Family Services. (2017). Lessons Learned Collaboration. Internal agency report: unpublished.
9 Hengelbrok, N., Modell, S., Cheetham, T., & Nyce, J.M. (2019). A Systems Approach to Child Death Review. Child Welfare, 97(4), 45-63.