How can behavioral science be used to improve the assessment of risk and safety?

Strategies to address hotline calls

Reducing inappropriate calls and contacts

Hotlines receive many calls that concern matters beyond the purview of child welfare. Finding ways to discourage unnecessary contacts while ensuring that proper calls are not excluded is a constant challenge. In many states, this issue is exacerbated by the fact that all contacts must be logged and local policy includes a supervisory review of all cases that are not put forward for investigation. This means that if the fraction of inappropriate calls is high, a supervisor could spend more time reviewing unnecessary contacts than truly concerning cases.

Research in similar settings shows that even a slight pause in response time to phone calls can lead to a dramatic reduction in the proportion of inappropriate calls. For example, an analysis of calls to a local non-emergency reporting line in South Wales² shows that inappropriate calls are roughly halved by a three-second pause – or roughly a single ring on the line. The proportion of inappropriate calls was reduced even more dramatically when the caller waited for at least six seconds for the call to be answered; around 40 percent of calls answered within one second were inappropriate, compared to just 10 percent of those that waited at least six seconds. It would appear that just hearing the phone ring is enough to prompt many inappropriate callers to drop off the line, whereas the more serious callers understandably are not so easily dissuaded.

Improving the quality of reports and the efficiency of hotline calls

Even callers whose reports eventually move on to investigation may not have the right information on hand before they start their hotline call. This puts cases at risk of being wrongly rejected and takes up time that could be spent on other activities or calls. Simple cues while the caller is on hold may be enough to ensure that cases can be assessed faster and more thoroughly when the conversation starts. Research from actual physical queueing shows that environmental signals, such as queue guides or carpet runners, can prompt those in line to begin preparing for the task at the end of the line.³ The theory on why this works is that people zone out and fail to notice the passage of time while in line; small changes — whether physical or sensory — can jog people to attention and prompt them to think about what they need to do next. Using audio signals, such as a change in announcer voice or music, to trigger getting the necessary information ready may be more effective than simply telling those on hold to prepare.

Strategies that impact initial screening decisions

Reducing over-referral to investigations

One of the major differences in child welfare hotlines around the country is with staffing. Some hotlines, like Arkansas, employ newly graduated social workers, using the position as an introduction to case work practice. Others, like San Diego County, employ social workers with considerable experience in the field, viewing this experience as critical to proper assessment of a child maltreatment report. All kinds of factors, from workforce composition to the level of specificity in the state’s definition of maltreatment, shape these staffing decisions. However, regardless of the staffing model, those who work on the hotline will generally over-refer or under-refer at least some fraction of their cases. Over-referral burdens staff tasked with investigating in the field, while under-referral may leave children at risk. Research from other fields shows that individualized feedback on the outcome of decisions can be a critical factor in helping staff to improve their assessment skills. For example, giving doctors in the habit of over-prescribing antibiotics a single instance of feedback that they were making different decisions from their peers was highly effective in correcting this behavior.⁴ Such mechanisms can be easy to implement in hotline environments that use newer technologies where referrals are often tracked through investigation and beyond, meaning that hotline staff members can be sent automated reports on their effectiveness and how they compare to their peers, and the downstream outcomes for children and families.

Reducing bias in case reviews

Research on decision science shows that processes that demand sequential decisions on independent cases can result in biased assessments. For example, in recruitment, evidence shows that an assessment of a candidate can be influenced by the quality of the candidate whose resume was reviewed just prior. Seeing an excellent candidate right before penalizes the next applicant, while seeing a bad candidate right before gives that same applicant’s chances a boost.⁵ This phenomenon is demonstrated below through two sets of grayscale charts.⁶

First, take 10 seconds to examine the horizontal chart below:

Now, without referencing the horizontal chart, look at the vertical gray squares below and try to determine the corresponding shade number from the horizontal chart:

Can you confidently say which number each shade corresponds to? In fact, when each shade is shown one by one, a person’s frame of reference adjusts to the last square the person saw: the judgment is literally colored by the example that came right before it. While case assessment is certainly more complex than ranking shades of gray, the point remains: the order of the cases, the state of attention spans, and the weight placed on different factors can all impact a person’s judgment, meaning that a person might not draw the right conclusions. This is relevant for staff in both hotline and investigation teams dealing with multiple cases day in and day out. Technological assistants, such as predictive tools that ignore the order in which the case arrived and include the factors that really mattered from the case history, can be a great help in offsetting the bias that results from ordering effects.⁷ While these technologies are not yet available everywhere and may not be for some time, the good news is that relatively simple changes to process design can help, especially for reviews or quality checks where there may be less time pressure and more control over the order in which information is seen.

One approach is comparative judgment.⁸ To understand this, now consider a different way of assessing color comparisons. Which of the colors in the pair below is darker?

How about this pair?           

It is much easier to make a judgment when there is a single point of comparison. A larger number of rapid comparative judgments like this can quickly flush out the objective rank order of colors to reproduce the scale. In one child welfare agency, the night supervisor for the hotline had a secondary task to review every case that came into the hotline over the course of the day to verify recommendations. In these instances, a comparative judgment approach that presents cases side-by-side for comparison may be more effective at establishing the right priority order than sequential review. Of course, there may be logistical challenges in implementing such an approach and it may not be a fit in all settings. At a minimum, ensuring that supervisors or those conducting secondary reviews of cases review in a different order than the initial worker may help break biases in assessment resulting from ordering effects. For example, if a caseworker deemed a borderline case as low risk because the previous case was especially high risk, then a supervisor is less likely to repeat the mistake if review of those same set of cases is done in a different order.

Strategies that impact investigations and assessments

Improving process adherence and quality information gathering

Social workers who assess household safety during investigation are often required to balance complex tasks involving expert judgment with a number of simple activities, such as checking for running water and working electricity in the home. Beyond this, they also have to pay attention in the moment while documenting what they observe during their visit. Although note taking and checking a faucet are arguably simpler than assessing whether a home environment is safe for a young child, these basic tasks can be easy to forget without effective and timely prompts. This issue is not unique to child maltreatment investigations. Research shows that the way in which staff — from pilots to surgeons — are guided through routine tasks can make the difference in the overall quality of their work. Hospitals, for example, have successfully reduced deaths and complications resulting from surgery by implementing checklists that remind staff to perform simple actions like hand-washing or checking that wounds are free of surgical accessories before suturing.⁹ Similarly, changing the form physicians use to write prescriptions was enough to reduce the number of errors and increase the amount of key information, such as the doctor’s phone number.¹⁰

Too often, social workers are given either printed or electronic case notes but nothing in the way of a structured visitation guide or space in the printed or electronic form to record notes. Simply providing a checklist, prompting questions, and room for documenting the visit could go a long way to improving the quality of information captured, reducing the need for follow-up visits, and minimizing work when translating notes into the system.

Conclusion

Prioritizing safety, keeping children from disruption whenever possible, and managing the enormous volume of referrals is incredibly difficult. However, applying small tweaks at each stage of the process can make a big difference in improving the efficiency and effectiveness of work done from intake to assessment.