Retrospective Study for the Use of the Arnold Public Safety Assessment (PSA)

Key Findings

The purpose of this study was to test the effectiveness of the Arnold Ventures’ Public Safety Risk Assessment (PSA) in Vermont. The Arnold PSA measures the risk of a person failing to appear for a court date (FTA) and engaging in new criminal activity (NCA) or committing a new violent crime (NCV) while out on bail. The PSA relies on criminal histories, the current charged offenses, and the age of the defendant to score the likelihood of a person engaging in the measured behavior. CRG researchers applied the PSA to a cohort of Vermont 3,742 defendants using regression analysis to determine whether the PSA could accurately predict a defendant’s FTA, NCA, and NCV. Analysis found:

• Overall, the rate of FTA (~11%) for the Vermont cohort was low. FTAs are only recorded in Vermont criminal histories if a warrant is issued for the defendant’s failure to appear. Interestingly, the FTA rate for Windham County was 22.94% which was almost 20 percentage points higher than the county with the lowest FTA rate (Lamoille, 4.26%).
• Including violations of conditions of release, 31.4% of people in the cohort committed a new crime while out on bail. This drops to 25.5% when VCOR is excluded. Unlike the FTA scale, the NCA considers the age of the defendant and weighs more heavily on those under the age of 22. Notably, age was significantly related to race in the study’s cohort with 20% of Black defendants being under the age of 23 compared to 15% of White defendants.
• Of the 3,742 people in the cohort, 10% were arrested for committing a new crime of violence while out on bail. Like the NCA, the NCV considers the age of the defendant and Black members of the cohort were again overrepresented as 6% of Black defendants were under the age of 21 compared to 2.6% of White defendants.

Racial Equity

Criminal histories do not measure the culpability of an individual. They measure the criminal justice system’s response to an individual. From arrest to release and parole, the criminal justice system’s response is a series of discretionary actions by police, prosecutors, judges, and corrections officials. Criminal histories record the outcome of those discretionary activities. A person in Vermont may be charged with disorderly conduct while the same conduct might result in a charge of assault in a different jurisdiction. One individual may be charged with a felony, when the same conduct would result in a misdemeanor charge for another individual. These decisions are recorded in criminal histories and then scored in risk assessments, as though they were indicative of a person’s culpability or past behavior. They may not be.

Scholars have argued that over policing of people of color and the systemic exclusion of people of color from socio-economic parity results in risk assessment based on criminal histories and/or socio-economic factors, being a proxy for race. (Harcourt, 2015), (Starr, 2014 & 2015). To explore these critiques, one must test the results of a risk assessment or examine decision-making for bias. (DeMichele, 2018). The argument is that if the risk assessment does not result in disparate impact for people of color, then the tool will not contribute to systemic racism. However, we agree with scholars who argue that there is another layer of bias embedded in using the records of a racist system that created an unmeasurable bias in risk scores and outcomes. (Eckhouse, Lum, Conti-Cook, & Cicciloni, 2019). It is important for stakeholders to understand these arguments when evaluating whether the PSA would further Vermont’s goal of racial equity.

Discussion and Conclusion

The PSA may have performed poorly in Vermont because the scales are meant to predict issues which are not prevalent among Vermont’s justice-involved population. The overall rate of failure to appear (FTA) for the cohort (people arraigned on felony charge in 2016-2017) was 11%. This is low. New crimes of violence while out on bail were also low, with 10% of the cohort being arrested or arraigned with a new crime of violence. While about 25% of the cohort committed a new criminal offense (excluding Violations of Conditions of Release), the PSA was not able to accurately predict who would commit a new crime. Given the PSA’s inability to accurately predict defendants’ future behaviors and the potential for bias decisions and policies to influence the tool’s scores, we do not recommend the adoption of the PSA in Vermont.

Background and Data

This research was conducted at the request of the Vermont stakeholder group of the National Criminal Justice Reform Project (NCJRP). The NCJRP was supported by the National Governors Association, Arnold Ventures, and the National Criminal Justice Association. This report was funded by the Bureau of Justice Statistics.

The Court Adjudication Database maintained by CRG was used to identify a cohort of individuals with at least one felony charge in 2016 or 2017. These are individuals who were most likely to be held without bail or held on a very high bail. Individuals who were incarcerated during the period in between their arrest and disposition were excluded from the cohort. Vermont criminal histories were obtained in addition to out-of-state and federal criminal histories. The final cohort included 3,742 Vermonters.