Overview of Risk Assessments

All You Need to Know About Risk Assessments

by Ken W. Good

A.    Introduction

The use of a Risk Assessment Tool has been pushed for the last four years as the gold standard for bail reform.  These tools were heralded as the antidote for reforming an alleged “broken” system.  The bail industry and others have been skeptical of risk assessments and has suggested that risk assessment tools should be viewed with caution.  These concerns have been proven correct.  These tools have not withstood the test of time. Essentially all of the initial advocates pushing for the adoption of risk assessment tools now oppose their use.  Why?  Because research in the last several years has all been negative toward their use.  A pamphlet summarizing all of the research and other studies which have concluded that risk assessments should not be used is attached as Exhibit A to this notebook.  Most recently, the Pretrial Justice Institute which originally pushed for risk assessments to be used across the country, has released a press release saying that they were “wrong” to support risk assessments and they now oppose their use.  This press release is attached as Exhibit B to this notebook.

In the last twoTexas  legislative sessions, bills were  filed seeking to mandate some type of risk assessment tool for use in Texas.  Additionally, the proposals included resolutions seeking a constitutional amendment authorizing preventative detention.  Therefore, there are two forms of reform using risk assessment tools: the use of a risk assessment tool coupled with a constitutional amendment authorizing preventative detention; and the use of a risk assessment tool without preventative detention. 

Originally risk assessment tools were sold as an automated tool which can determine the defendant’s risk quickly.  But this is not true.  There is no central database which will automatically provide a criminal history.  Therefore, this information has to be found by hand before it can be entered into the app for the resulting score.  Consequently, in some situations, the risk assessment tool is based only on the information disclosed by the defendant. Otherwise, someone will have to research and verify the defendant’s criminal history.  In our smaller counties who is going to do this?  The Sheriff’s Association has opposed this burden being placed on their staff.  Further, the vast majority of small counties do not have a pretrial bond office (which is the department which performs this task in large counties).

The use of Risk Assessment Tools have been controversial from the beginning.  The Governor of New Mexico cautioned against their use early on.  Some of these concerns arose because the app makes mistakes at times and these mistakes usually involve a determination that someone who is high risk is low risk.  Finally, logic would say that once a defendant is determined by the app to be high risk, then the defendant will be branded high risk from that point forward when the defendant comes into contact with the criminal justice system.

Additionally, Risk Assessment Tools are inconsistent with the Governor’s push to have criminal histories reviewed before bail is set.  The current versions of the tools only review convictions in a short past period and do not consider other matters which would be disclosed on a criminal history such as arrests and other current charges.  The Risk Assessment Tool does not consider other pending charges because they have not resulted in a conviction which would seem to artificially skew the risk that the defendant would re-offend downward.

B.    Risk Assessment Tool Combined with Preventative Detention

    i.     Summary

• Requires Constitutional Amendment to authorize Preventative Detention.
• This is usually referred to as the New Jersey Model.  In New Jersey the language of the constitutional amendment was sold as giving more discretion to the trial court, but the wording did the opposite and took discretion away from the trial court and made it more difficult to exercise that discretion.
• The Pretrial Justice Institute has concluded that New Jersey is an example of failed bail reform.

    ii    Discussion

 The original intended use of the risk assessment tool is coupled with a constitutional amendment to allow preventative detention.  This would seem contrary to the purpose of addressing jail overcrowding because once preventative detention is authorized in mass, it  usually results in more people being held pretrial without release.  An example of this would be the federal system which was changed to allow preventative detention during the Reagan Administration.  These changes were implemented with the goal of reducing jail overcrowding, but had the opposite impact. Before the changes approximately 35% of pretrial arrestees in the federal system were detained until trial.  Currently, approximately 70% of arrestees in the federal system are detained awaiting trial.

The use of a Risk Assessment Tool coupled with preventative detention is an example of  the reform that has been implemented in New Jersey.  The New Jersey Constitution was amended to authorize  preventative detention.  Under this system, the risk assessment tool is used to determine two things:  risk of committing a new crime and risk of failure to appear.  A score of low risk allows simple release.  A score of high risk requires detention subject to a mini trial within a short time after arrest.  A score of medium risk requires conditions and could require surety bail (although the New Jersey Attorney General ruled it could only be used after everything else had been tried).  Additionally, the amendment to the constitution raised the evidentiary standard from abuse of discretion or preponderance of the evidence to clear and convincing evidence making it more difficult for judges to detain anyone.  The constitutional amendment was sold to the public as giving more discretion to judges.  But the change in the standard which authorized holding defendant had the opposite result.  It became more difficult for judges to hold defendants. 

Further, PJI, the Pretrial Justice Institute  now claims that New Jersey, that implemented the use of a risk assessment model with the support of PJI, is an example of failed reform.  See Exhibit B attached to this notebook.  

We have compiled a pamphlet summarizing the most recent studies regarding risk assessments which all have concluded that they should not be used as a part of Criminal Justice Reform.  See attached Exhibit A to this notebook.

C.    Risk Assessment Tool without Preventative Detention

    i.    Summary

• A risk assessment which is not tied to a constitutional Amendment is simply additional information.
• The criticism is that the underlying data would be more beneficial to the magistrate than a risk score.
• All of the original supporters of risk assessments (with the exception of the Arnold Foundation) no longer support their use.
• The largest technology companies in the world have issued a statement saying that courts should not use risk assessments.
  • The argument is that risk assessments are pretty accurate at predicting what a group will do, but is terrible at predicting what an individual will do.
  •  Also, to predict what a group will do, the risk assessment relies on essentially historical data which is ends up being  weighted against certain minority groups.

    ii    Discussion

The second type of use of risk assessment tools is where there the risk assessment tool is not coupled with preventative detention.  In this situation, the tool is just one more piece of information to be given to the magistrate to consider in setting bail but it is not coupled with preventative detention. Additionally, the standard of review is not changed for appealing the judges determination.   Critics of this type of use of the risk assessment tool maintain that it would be better to provide the magistrate with the underlying information instead of a score on an app.  Additionally, in the last few years, the science has turned against risk assessment tools.

Kim Ogg, the District Attorney of Harris County has testified before the Texas Legislature that a risk assessment tool does not fit well with Texas current bail system.  What she meant by this was that an assessment of risk does not integrate well with a system where the judge is required to set a bail amount taking into account several factors.  Critics maintain that a magistrate would be in a better position to exercise their discretion with the information that was put into the tool and not the resulting raw score.  In the last two legislative sessions, this is the type of risk assessment tool which ultimately was  pushed by advocates for change.  Currently, OCA has developed PRAISTEX. OCA describes the tool as follows:

The Pretrial Risk Assessment Information System (PRAISTX) is a web application developed by the Office of Court Administration (OCA) to support the use of the Laura and John Arnold Foundation’s Public Safety Assessment (PSA™). The PSA™ is a pretrial risk assessment that provides judges with reliable and neutral information about defendants to inform pretrial release decisions. If used properly, the PSA assigns a risk assessment score for two key areas: the likelihood a defendant will fail to appear and the likelihood a defendant will commit a new crime while on release. The PSA will also flag defendants who present an elevated risk of committing a violent crime if released before trial.

Under Texas law the magistrate is required to set bail based upon the factors set out in article 17.15 of the Texas Code of Criminal Procedure.  These statutory factors require consideration of more than the two factors considered by the risk assessment tool.  Therefore, a reliance on the tool alone would not be in compliance with Texas law.  

 Further, the risk assessment tool does not take into consideration when a defendant is claiming poverty.  Therefore, it does not comply with the appellate decisions issued by the 11th Circuit and the 5th Circuit that require procedures to be established which allow defendants to seek a reduction in bail based upon claims of poverty.  Consequently, there are questions about whether there is much value in a risk assessment tool.  It provides additional information, but that information does not comply with article 17.15 or the appellate decisions on what magistrate must do in setting bail. Therefore, the claims of automation and time savings are not true.

D.    If Risk Assessments Have Been Disproven, Then Why are Certain Advocates Still Pushing For Their Use

We anticipate that there will be yet another push this session for the legislature to enact a bill which will require one of the two forms of a risk assessment tools discussed above to be used in Texas.  Such a push raises substantial questions that the supporters of this move will not be able to answer.  First, why are they continuing to push for a tool that all but one of its original supporters now oppose.  Second, why are they supporting something that the largest technology companies claim should not be used and which claim that the original tool was rushed and not adequately researched before its use.  Google and the other technology groups stated that while this type of tool may be effective in predicting group actions, these tools are not accurate and should not be used to predict the actions that an individual may take.  Third, numerous scholarly research articles have now been published and not one of them supports the use of risk assessment tools.  They conclude the opposite that they should not be used as a part of criminal justice reform.  Fourth, the Pretrial Justice Institute which originally was a strong advocate for the use of risk assessment tools, now has the opposite position and maintains that New Jersey which adopted this tool is an example of failed bail reform.

So why would any group continue to support the use of risk assessments?  The reason is because they have nothing else to offer other than the argument to “release everyone” on a PR bond  which has been an absolute failure.  The Arnold Foundation, now reclassified as Arnold Ventures is approaching various jurisdictions in an attempt to conduct a pilot program in eleven jurisdictions across the country to resurrect their own pretrial risk assessment tool.  The Texas Office of Court Administration has worked closely with the Arnold Foundation and now Arnold Ventures.  David Slayton has even approached counties to recommend that they partner with Arnold Ventures  with the foundation providing a grant to the county.  While the Arnold Foundation historically has  worked closely with the Pretrial Justice Institute, their current opposing positions on the use of risk assessments have led to them to part ways.

It is unclear how adopting a risk assessment for use in Texas will make the system better since the tool is under attack and no one other that Arnold Ventures is advocating its use.  Further, even the Arnold Foundation has not pointed to any research that supports the use of its tool.

Arnold Ventures appears to be under the mistaken belief that if the control of the tool is taken away from probation and given to a pretrial group under the direction of the judges the results will be better.  However, this argument is deficient for several reasons.  First, pretrial supervision is a part of probation only in a minority of counties that currently have such a group (the number is small to begin with).  Second, the problem is not who is in charge of the tool, but the tool itself.  If Arnold Ventures is not able to support the tool it is asking Texas to adopt, then Texas should not be willing to use this tool based upon the currently available data which uniformly states that these types of tools should not be used as a part of criminal justice reform.