Friday, November 14, 2014

Court Case Management Systems Part 16 – Reports and Statistics

Originally, court case management systems were justified based on their ability to produce court management reports and statistics for policy makers.  They have generally achieved these goals but there is more that CCMS should be able to do.

---

Court Case Management Systems Part 16 – Reports and Statistics

By James E. McMillan and John T. Matthias

This is a complex subject.  Therefore we will very likely miss some important factors from particular viewpoints in this discussion.  But noting that, we will bravely try to start with some general concepts.

The Goals

First, in starting this article we want to define the difference and the reason/goals between reports and statistics.  For purposes of discussion in this article let’s generally create two categories:
  • Reports = Management or operational performance -- CCMS output that is used to actively manage day-to-day decisions about court organization and operations, using financial and non-financial performance measures.  An upcoming court schedule and a list of NSF checks are two such court operations reports.
  • Statistics = Policy-oriented -- CCMS output that is most often a time-oriented “snapshot” count of cases, numbers of cases in a particular status, and the like.  Statistics may feed further analysis such as the cost-benefit of implementing a policy like a problem-solving court or a one-day-or-one-trial program.  Statistics are often contained in monthly or quarterly reports that courts submit to a higher authority. But statistics also involve case status and demographics to categorize and make sense of information.  And both case status and demographics are dynamic as definitions and criteria change based upon the questions asked by researchers and/or policy makers.
We believe that often the goals of statistics and reports are mixed and result in confusion for both policy makers and judges/court managers.  There is a continuum of kinds and uses of numerical information about courts, with management/ operations reports on one end, and statistical policy-oriented reports on the other end.  On this continuum neighboring examples are not greatly different from each other, but the extremes are quite distinct. Put another way, operational data such as trial date certainty has mixed uses and can be used to allocate daily courtroom resources and also to drive policy decisions. We want to make some clear distinctions as to the purposes of using data and thus the data required from the CCMS.

Courts Count Events

If one gets to the essence of traditional court statistics one will find that they are based on case events (and tasks) as we described in Part 6 of this series.  Let us explain.  A filing of a case is an event recorded on the date it occurred.  In the same way, hearings, case conclusions, and case reopening are also case-level events recorded in the CCMS.  Now if you think about that, courts record many other events.  But we rarely count them as part of our statistical reports.
  
Is not issuing a summons also a case event?  We know that this work gets recorded in CCMS as well as in paper registries in the past.  This kind of event within a case consumes staff time, and in this instance also paper, envelope, and postage resources.  So it should be counted and reported as part of the court’s overall workload report.  But beyond that, not all events are equal, so events that consume judge or staff time should be “weighted” so that the time and resources can be credited to the court for the work that was done.  Therefore both the “raw count” and “weighted count” can be included in a court’s statistical report.

Why?  Because in order to support the court’s budget with policy makers, courts need to be able to accurately reflect all of the real work that is performed.  Case counts are just one overall summary of work that in the past was relatively easier to report.  But case counts only tell part of the story.  With a modern automated CCMS, counting everything in the system, including events of all kinds, is just as easy as counting only some things.  

Everything in a CCMS is Data and Can be Counted

The other precept for this way of thinking about court reports and statistics is that any data field captured in a CCMS, including the text and image contents of documents, can be searched and quantified.  In Part 10 we discussed searching both data fields in the database and text in documents as part of a new CCMS system, thanks to new search technology and XML-enabled databases (accepting XML as input and rendering XML as output).  And since we are focusing on the future, let’s assume that all of the court’s documents will be in some form of electronic format.  Please note that this leverages another legal technology trend, turning it into factors for planning here.  First, this applies primarily to pleadings, motions and proposed orders, but also to exhibits in electronic format. Second, e-discovery is driving Optical Character Recognition capabilities even with handwritten documents.

Therefore, if everything is potentially statistical data, we need to start to think about what data in the CCMS needs to be in which form?  This is important because in the past, courts had to add data fields to the relational database in order to capture data, often policy-related data to count things.  Unfortunately this approach was only as successful as the courts were able to add data fields, change data entry screens and add reports. Without a legislative mandate with funding to support this process, it has been difficult for courts to capture new data for analysis because this had to compete for attention with other projects, and courts do not often receive any direct benefit (or funding for staff and time) for this additional work. Thus it is a low priority.

For policy makers much of this need related to research for improving caseflow of criminal and social services cases.  But since this data can change over the life of the case -- think of a child being tracked for a decade or more -- a CCMS which is not highly configurable is not designed to accommodate initiation and evolution of this kind of data capture. But, if we can count things inside court forms and documents that are updated naturally in the CCMS case document file by the litigants and case participants, then we believe that both operational and policy data needs can be addressed and, in most instances, fewer specialized statistical data fields will be needed.

Again, as pointed out in Part 10 in this series, CCMS events registered in the database are often the documents / forms submitted to or created by the courts.  And those documents are identified by the event codes and/or the form identifier itself.  So adding a data field to a required form provides and exposes the data to use and analysis. Obviously events and therefore the data field values in documents can be counted as well as timelines/ differentials as part of statistical analysis.

Case Status

Another important concept that deserves discussion is that of case status.  We all know that case status can change throughout the life of a matter brought before a court. Is it in a status of case preparation, active, inactive, or closed?
 
Whether the case status is active and therefore the case is under management of the court, is a key operational and statistical differentiator.  In other words, is the court in control of it and can move forward with the case or not?  

Case status in the CCMS should be recorded and changed either via the case documents received or created by the court.  For example, in some instances cases are not pursued by the litigants.  After some defined time a reminder task should be sent to the case manager who in turn can make the appropriate inquiries.  If the case is still legally active, but say the person has left the jurisdiction, that fact should be noted in the case file in a document, the status changed to inactive, and case aging suspended.  One can see that this would move the case from pending active and pending inactive.  At some point if the case does not receive any additional action, then the matter should be statistically closed in the CCMS while legally remaining open.  In other words, this event process provides management information so that the courts can realistically allocate their resources.  And this approach can also address the perceived need to count all “legally active” case matters filed in the court.  But these counts should be reported separately so that they might be addressed in future policy changes, such as to timelines and grounds for dismissal.

A quick example.  In one country the electric utility and hence the court was not able to pursue action on theft of electricity simply because the electrical meters and utility poles did not have any kind of identification number.  So the cases sat in the court as pending matters year after year.  Once it was determined that the cases could not be pursued by the plaintiff electric utility, they were able to withdraw them from the court.  Having a large number of cases as pending inactive would have alerted the courts and policy makers much earlier to this problem and remediation action taken.  In this instance the electric utility painted identification numbers on the utility poles and meters so that they could file cases that the court could actually adjudicate in the future.

Status Creates Context

The status of the case also creates the context in which case management and statistical reports are viewed.  Work done on inactive cases such as reissuing a summons may not seem important as it may not result in any positive action toward closing the case.  But it is important that this work be performed and counted.  Availability of status data can incentivize the court to responsibly and proactively attempt to adjudicate the matters that have been brought before it.  Conversely if the inactive case count status continues to grow, there is often a legal or procedural issue that should be dealt with by legislation or court rule.

Context is particularly important for judges as they are, fairly or not, evaluated on their case disposition counts.  Again, if the cases cannot be pursued, such as when defendants have absconded, thus moving the case to an inactive status, then the judge should not be viewed as being unproductive.  Another example is in countries where cases are sent by judges to “experts” for review and input.  The time period status for that expert work must be captured and closely monitored to identify delay and potential corruption activity.

Participant Demographics

Participant demographics is statistical information that is needed or perceived to be needed by policy makers, and also provides context for policy and operational decisions.  Today, this data can be obtained via electronic forms submitted by the litigants or from social service government agencies and law enforcement.  As a policy for privacy protection, many courts are starting to confine demographic data to one form that is then assigned a higher security access level.  This form can also be encrypted to reduce identity theft issues.
  
But there is a second level to this discussion for policy and analysis purposes that was also envisioned by the US Department of Justice, Office of Justice Programs Privacy Technology Focus Group in 2005 (see Working Team Two’s section in this PDF document). The personal and demographic data from the forms can potentially be “anonymized” and placed into a separate database.  Researchers would then be provided with ability to work with the anonymized database thus allowing policy research to proceed while reducing risks to an individual’s privacy rights.
 
While many of you might not think that this should be part of the CCMS, it is important to begin thinking about this in order to support your policy reporting needs.

Process-Oriented CCMS Data

At this point we must acknowledge Ms. Janet Cornell’s pioneering paper, “Court Performance Measures: “What You Count, Counts!” (Court Manager, Volume 29, Issue I)  (Page 15)  See: https://nacmnet.org/publications/index.html for membership and subscription information

In her paper she expanded on the concept that there are additional potential counts that are now possible.  They are:
  • Traditional monthly statistics on cases filed and adjudicated with the court
  • Expanded monthly workload measures with month-to-month comparisons
  • Use the National Center for State Courts’ CourTools measures (with locally applied methodology)
  • Creation of a counting process to identify ‘touch points’ whereby the court identified the number of times that interactions occur per case
  • A process to count workload times, by activity on a case
  • A task inventory project, known as the “bucket list,” in which court tasks and work volumes are tallied and assessed; and
  • Reporting of court accomplishments, noted and grouped in High Performance Courts strategy areas or quadrants
It is possible for the CCMS to help to identify and count Ms. Cornell’s “touch points.”  For example, it is common that queries of the CCMS either via a court website or public terminal can be counted. The number of queries and financial transactions handled by the front counter staff can also be counted since they would log into the CCMS with the “Front Counter” role designation.  And telephone calls into the court can be connected (IP-Telephone) with queries to the CCMS.  Time per transaction can also be captured as an additional metric.  But time is not as important as service delivery in measuring success, and therefore courts might consider adding the ability to ask the caller or website visitor whether they received the information they needed.  And if not, what was the problem so that the system can be improved in the future?

Management Reports

The focus of court management reports should be to quantify the court’s workload and workflow.  Task reports by individual and group role are particularly useful for previewing upcoming work and potential issues that can arise from uneven assignment.  Uneven workload can naturally happen (a judge “wins the lottery of hard cases”) or can be the result of policy or personnel capabilities (and many other factors).  The point is to have a system that continually provides the monitoring information needed for management action, both tactically and strategically.

Management reports also examine trends which, on the operations/ policy continuum, are closer to statistics for policy purposes, though trends also have operational implications.  Is there a seasonal change in the number or kinds of case matters?  Did law enforcement change a policy or resource focus?  For example, Ms. Janet Cornell tells of her city deciding to implement traffic speed cameras. This had a huge impact on their caseload that she was able to show the “before and after” statistics. The city later rescinded the program because of public response, and the court had statistics to show the effects of the policy change.

Some other judicial friends of ours have used their CCMS to examine sentencing practices.  If the goal is fair and equal justice, this is a very legitimate use of trend data.

Statistical Reports

Statistics are counts that are greatly influenced by timeframes and case status.  Depending on the date that the statistical report counts are computed using a court database may result in different numbers from day to day.  This is because a case may be reinstated between day one and day two.  Thus it has moved from a status of closed to a status of active.  And if active, it is part of the pending caseload count.  So when thinking about how the CCMS should work with statistics one must keep these “dynamics” in mind.

As Ms. Cornell quotes “What you count, counts” (that we first heard from Dean Ernest C. Friesen at the 1988 Court Technology Conference). The CCMS should provide the ability to count everything in the database.  And it should be able to do those counts with knowledge of their status and demographic context.

Conclusion

Again, this is a very large subject matter.  We greatly look forward to your input on this subject and will be happy to publish your comments below or even separate articles here in the CTB.

Next Time

Dashboards and decision support

CCMS Series Index

For all earlier articles in the CCMS Series, see the web page index by clicking here.

No comments:

Post a Comment