In the previous post in our series we discussed Court Case Management Systems (CCMS) Dashboards that support case process, management information, and decisions. In other words, information that makes our “court process factory” more efficient. And the key purpose of case management is to organize data (and insure completeness) so that it can in turn be converted into information for cases to be adjudicated and decisions rendered.
But what’s next? We think that one part of the answer is to integrate and extend the judicial decision support functionality into the CCMS itself. We explain below…
---
Court Case Management Systems Part 18: Decision Support
by James E. McMillan and John T. Matthias, NCSC
It is exciting to report that there have been a lot of projects initiated in judicial decision support in recent years. But the creation and testing of these systems, as you will read, have been most often done in tests that aren’t connected to the core court case management information systems. This is not surprising since new ideas must be piloted and tested to determine their feasibility and effectiveness. We believe that it is now time to change this approach so that full efficiency can be realized.
But first before we delve into the judicial decision support projects and ideas, we must try to answer the question: Why does decision support need to be integrated into the CCMS? We think that some answers are:
- We believe that the CCMS should serve as the portal for court processes and case information. Fragmentation of systems and services very often results in uneven adoption and confusion for users as they may not know if and when decision support information is available. In other words, including decision support in case management is, as it is often noted in retail sales, creates a “one stop shopping” experience.
- Access control and security of information, especially documents, is best serviced by the CCMS. This is because ideally all of the parties, judges, court staff and associated entities involved in the case are registered with it. And as discussed before (see link below), the current status of those entities with the case should also maintained in the CCMS determining what, when and to whom this information is available.
- Documents are the courts’ “big data.” - Now this is where things get interesting and perhaps concerning. As anyone who has been on the Internet since 1998 understands, documents can be computer-indexed and searched. The added beauty of court CCMS information is that the documents in most cases are associated with descriptive data of the item. With E-filing, they are increasingly becoming submitted in machine readable formats such as PDF. Therefore the CCMS data plus associated documents can serve as the bulk data repository for current and historical analysis and research by the analytical engines.
Database tools used by CCMS can also be programmed for anonymization of research queries that can in turn address privacy concerns. New database systems can also perform many other queries and analysis for trends, inference, and validation studies.
One must understand that these capabilities are coming to the legal profession and public whether one likes this idea or not. A current and prescient example is the use of US Federal Court PACER data by commercial vendors. It is already being used for analysis and case outcome predictions and therefore it is prudent for all courts to study and understand this trend, as it is already happening. - Cross-case, cross-party, and cross-court data linkage. Depending on the status of the case and parties, it is important to know whether it is allowable or not to be able to view and use case histories. One could also use case management systems to supplement criminal history records (as several states such as New Jersey already provide), as the courts could have the entire and complete sentencing record. These kinds of linkages are easier at the state level if the state court system has a unified CCMS; this function is more problematical in states with decentralized courts which use a variety of CCMS’s.
We have always believed that better information equals better decisions. More information may or may not be better information. But it does create the opportunity to test different approaches. It also provides an approach to deal with increasing amounts and complexity of data so that it can be turned into information.
So why are courts and justice systems looking toward data analytics and decision support for help? Several papers have pointed at the following reasons:
- Reducing decision bias. One of my favorite parts of Brad Pitt’s “Moneyball” movie is the discussion among the American baseball talent scouts that a particular player would not be good because he has an “ugly girlfriend,” as a sign that he lacks confidence. Brad Pitt rolls his eyes at that comment which is the supposed “expert opinion.” And the rest of the movie (and in real life for the Oakland baseball team) illustrated that the expert hypothesis was incorrect, and that good analytical data resulted in record setting team performance.
- Fair and unbiased justice is the goal of every justice system and the legal system that supports it. Therefore we must be on guard for bias and use the data that we have to avoid it.
- Risk reduction. The courts are necessarily involved in many of society’s most difficult decisions such as removing children from a home, confiscating money or property, imprisoning a defendant, or adjudicating a complex set of facts and issues. The risk of making the wrong decision is considerable. So the various systems of justice use process and information and evidence-based practices to reduce risk. But there is potentially so much information available now that it may not be humanly possible, without computer support, to have the time to thoroughly analyze the information. In the history of technology and the law, at some point the intentional non-use of technology is considered negligence.
- Cost reduction (primarily through reduction in jail populations for detention and short-term sentences). Improved risk analysis has the potential to reduce the numbers of persons who are detained. This in turn reduces facility overcrowding that makes them safer for the inmates. And over time this can reduce the need to build or enlarge jail holding facilities. (Parole systems are always separate from court systems, but the same benefits could be achieved with prison populations.)
- Long term societal benefits. Statistically valid application of equal sentencing/ outcomes can improve the image of the justice system in society as being a fair system. Unequal application of justice has been one spark to civil unrest in the recent past in the USA.
- Process improvement. Finally, improvements in data collection and analysis can provide guidance for court and overall justice system operations. It can also focus efforts on specific decision-making steps that in turn reduce or expand work time for the participants.
Before going further, we must emphasize that we are not discussing or advocating automatic machine-driven decision making. Algorithms (computer programs) currently make millions of buy and sell transactions per second in stock markets around the world without any human interaction. Unlike those systems, in this article we are exploring recasting the presentation of data from static two-dimensional paper based approaches to more dynamic and useful forms.
We also must warn that many of the systems discussed below are either in their infancy or are speculative based upon technology trends. Risk analysis and evaluation systems in particular are subject to a significant amount of concern if they have not been validated. We have seen similar worries before with other new technologies. And what we have seen with other technologies is that practical concerns influence development of the system and make it better. Thus we are including them in our list below.
Further, an article in Harvard Business Review, January-February, 2016 edition titled “Algorithms Need Managers Too” (page 98) warns that “people treat algorithms and the machines that run them the same way they’d treat an employee, supervisor, or colleague. But algorithms behave very differently from humans, in two important ways.” … “Algorithms are extremely literal” and “Algorithms are black boxes.” The article correctly observes that “(r)ecognizing these two limitations … is the first step to managing them better.”
In other words, along with our examples below, we expect judges to manage the use of these tools as a means to an end, and to continue to be the final decision maker.
With that said, we can now discuss the ideas and potential benefits of decision support systems for the courts in the remainder of this article.
Technology Drivers
One last general trend observation: Automated systems can do a lot more work in a few seconds than people could hope to accomplish in a year. The examples below use many different data presentation approaches, including graphical approaches and different data organization approaches.
In the legal business this is manifest in the development and use of e-Discovery software systems. And these systems can consume and use all manner of data (including paper documents that are OCR’d, and also pictures). They then apply rules-engines to organize and provide search and data presentation in a myriad of ways. The following two short video/commercials for two e-Discovery products show some of what these systems do.
- Everlaw (2 minutes) - https://www.youtube.com/watch?v=gnrm02RQyvI
- Recommind Axcelerate 5 (1.5 minutes) - https://www.youtube.com/watch?v=YgdXIpRiQx4
Eight Decision Support Tools
Eight court decision support tool projects are described in the following sections. We believe that the ideas and concepts pioneered in these projects can be considered and planned for future integration into the CCMS environment.
1. Risk Assessment
With the leadership of former New Jersey Attorney General, Ms. Anne Milgram, the Arnold Foundation has supported development and testing of “the Public Safety Assessment (PSA), a pretrial risk-assessment tool that is designed to assist judges in making release/detention determinations.”
“The PSA was created using a database of over 1.5 million cases drawn from more than 300 U.S. jurisdictions. We analyzed the data to identify the factors that are the best predictors of whether a defendant will commit a new crime, commit a new violent crime, or fail to return to court. These factors are related to a defendant’s criminal history and current charge. They do not include factors that could be discriminatory such as race, gender, level of education, socioeconomic status, and neighborhood. The PSA is more objective, far less expensive, and requires fewer resources to administer than previous techniques. And because it was developed and validated using data from diverse jurisdictions from across the country, it can be used anywhere in the United States. It is currently being used in 29 jurisdictions, including three entire states—Arizona, Kentucky, and New Jersey—as well as three of the largest cities and two of the largest jail systems.”
http://www.arnoldfoundation.org/initiative/criminal-justice/crime-prevention/public-safety-assessment/The Conference of Chief Justices has supported the development of Evidence Based Pretrial Release. And the NCSC has a great deal of interest in this area and has created a Pretrial Justice Center for Courts microsite. This is an excellent place to learn more and monitor developments in this area.
However, risk assessment systems are not without critics in its current form and capabilities. Two important reviews, (in our opinion) inaccurately titled: “Should Prison Sentences Be Based On Crimes That Haven’t Been Committed Yet?” by FiveThirtyEight.com, and Pro Publica’s “Machine Bias” raise valid questions that need to be addressed as these systems develop.
FiveThirtyEight.com’s article in particular provides some ideas for the future development of these systems:
“Richard Berk, a University of Pennsylvania statistician, said the most widely used tools are “a generation behind a lot of the developments that are going on in computer science and statistics.” Berk has been at the national forefront of efforts to bring risk assessment into the modern era. He has developed assessment tools that use a more advanced statistical discipline known as machine learning. In essence, Berk feeds a huge amount of data into a computer and lets a program figure out which variables matter and how much. He argues that his approach generates predictions that are both more accurate and more finely tuned — distinguishing, for example, between violent and nonviolent crimes.” …https://statistics.wharton.upenn.edu/profile/553/ -- for access to articles, go to “My Personal Page”Please see the additional information links at the end of this article for more.
2. Smart Sentencing and Resource Allocation
Judge Michael Marcus pioneered resource availability and allocation such as for drug treatment and intensive probation programs in Portland, Oregon. Similar approaches have been used in the past by the pioneering Midtown Manhattan Community Court information system. Michael’s Smart Sentencing website is still online at: http://smartsentencing.com/whatwrks.html
3. Timeliness of Information and Linkage between Matters
This concept graphically displays the age of information provided, and then models and displays the information-sharing linkages. Is the information current or old? Are the linkages simply there or not?
A graphical representation of the age of data could be either by color coding the data (red text for example would be older than 1 year) or else as a color warning on the field or for the entire form. The color coding does not necessarily mean that the decision should be any different. Instead it is intended to prompt the users to verify it with the person or with other data sources for accuracy.
Data linkage can be best explained with a short story. Early one morning I met with a judge who was upset. She explained that she was not told that the person she ordered to pay back child support earlier in the month had stopped paying his other two obligations. She simply didn’t know that there was an earlier order against the person, and the parties failed to produce this information. The critical information for her decision was both not presented and then too late, in this instance.
Therefore, a system that links persons to all cases including criminal, civil, domestic and potentially to juveniles who have cases in the courts would be very useful in avoiding such situations. But the system may also note when a link is not available. In that example perhaps a traditional personal query by court staff is needed in order to check the link.
4. Identification Certainty
This decision support capability would report whether the person before the judge has been 100% identified via Biometrics such as fingerprint, iris, or facial recognition. Many times a person’s identity is based only upon their personal testimony, unsupported by government verification. Or else, the person’s identification documents are questionable.
 |
| AFIS Nebraska State Patrol System |
And interestingly, from the times we live in (therefore not arguing whether this is a good thing or not), with the decrease in digital privacy, there is also the possibility of using social media/ digital persona as part of the identification process by prosecution and police, and reported as such in the status.
5. Balance of Justice
The “balance of justice” concept comes from a test system developed in Scotland. The system provided information to the judge regarding sentencing decisions in similar convictions across the jurisdiction. In other words, the concept is that in order for equal justice to be achieved, sentences should ideally be relatively consistent within the jurisdiction. But as we know, it is often true, a particular defendant’s sentence may be more or less severe just because they randomly drew a harsh or lenient judge or were adjudicated in a particular location.
The Scottish system provided the range of sentencing and the norms of punishment. It also allowed judges to justify/document the reasons for deviation from the norm. A more complete description of this systems approach was written in the Journal of Artificial Intelligence and Law (v6: 203-230, 1998) – “The Application of Judicial Intelligence and ‘Rules’ to Systems Supporting Discretionary Judicial Decision-Making” by Professor Cyrus Tata.
6. Law Resources
This idea comes from several states that have electronic bench books. We believe that it is possible to embed these legal resources into the case management system. This would allow the CMS to automatically link between the case matter and the relevant sections of the law and bench book materials -- for example, to make checklists that have been developed by judges easily accessible.
An example of this is described in an article on North Carolina’s work in this area in the NCSC 2015 Trends in State Courts Report available here, “Building a Better Bench Book.”
7. Family and Interpersonal Relationship Mapping
Most of us working in courts understand the difficulty that judges have with complex family situations. And thus it is no surprise to hear that judges can become confused about a person’s role in the family structure, particularly when family relationships can quickly change due to the stress of being involved in a domestic dispute.
This is where the concept of Genogram diagrams could potentially help. As shown below, the diagram links persons by connecting lines that denote emotional relationships.
The diagram is then used to graphically display a person’s interpersonal relationships with other as shown in the example below that was published in an American Probation and Parole report: “Implementing the Family Support Approach for Community Supervision” (starting at page 21) in 2008.
There have been successful tests of the Genogram graphical relationship mapping in family courts. One practical issue is which participant in the process is able to produce the diagram; often the social worker who works for the state or county knows the family situation best but may not have the tools or motiviation to produce a diagram. But with new software tools we believe it is increasingly possible to build these maps dynamically from the data resources available.
8. Case Management System Embedded Decision Support
Earlier in 2016 we published an article here in the Court Technology Bulletin: “Court Case Management Events and Decision Mapping.” In that article we noted that the case events (triggering mechanism) of CCMS could be expanded to include a decision table. And a specific example of a decision table is eligibility in Georgia for reduction of motor vehicle insurance premiums:
Use of decision tables and other embedded decision support would allow the CCMS more flexibility and automation in actions taken such as task scheduling and document creation after an event is captured. Specifically it has the potential to simplify the number of tasks triggered from the case even since there would be a one-to-many output instead of a one-to-one (we often refer to this as the falling dominoes) approach.
9. E-Bench.
Finally, we need to pull everything together for the judge. Some decision support can be embedded directly into the CCMS functionality. But we also think that we are on the path to include a great deal of additional functionality in an “E-bench” type interface for the judge.
This is discussed in the COSCA/NACM Judicial Tools paper and other articles on judicial E-bench systems here and here.
Conclusion
We believe that court case management systems have many more possible capabilities that can be developed in future years. It is an exciting time and we all look forward to seeing what develops.
Next in our CCMS series (which is nearly done after four years): Other User Interface Needs and Options
The full CCMS series is listed and linked here (please scroll down to the bottom of the page).
Notes - Additional articles on risk assessment systems:
- http://www.nytimes.com/2014/08/11/opinion/sentencing-by-the-numbers.html?ref=todayspaper&_r=1
- http://www.arnoldfoundation.org/more-than-20-cities-and-states-adopt-risk-assessment-tool-to-help-judges-decide-which-defendants-to-detain-prior-to-trial/
- https://jasonbfischer.wordpress.com/2014/06/13/moneyballing-criminal-justice/
- http://www.justicesystempartners.org/projects/
No comments:
Post a Comment