Utah ODR Usability Report

 

The Innovation for Justice Program ( www.law.arizona.edu/i4j ) at the University of Arizona James E. Rogers College of Law has issued a new and insightful report titled: The Utah Online Dispute Resolution Platform: A Usability Evaluation and Report (full report available online in PDF for download here)

---

For the study “The i4J Program partnered with the Utah Administrative Office of the Courts and the Pew Charitable Trusts to conduct observation-based usability testing and to identify how the Utah ODR platform could be improved or enhanced, with a focus on functionality and usability.”

The summary notes “Five Actionable Recommendations”:

"Ease the Transition from Paper to Platform. Employ the best practices of URL formation, website naming, and UI design, and highlight key information on the paper forms to assist website users.

Our baseline test of the Utah ODR platform demonstrated that typical ODR users experienced significant difficulty making the transition from the affidavit and summons to the ODR website, and experienced visibility and accessibility issues once on the homepage. Our redesigned affidavit and summons aims to help participants more easily navigate the transition from paper to the online platform through clear information hierarchy that highlights key information needed for future steps and a QR code that connects to the ODR platform. We recommend a short and simple URL for the ODR platform that uses a familiar domain name and easy path, and is not case-sensitive. In addition, we recommend an official website name on the summons that is discoverable through a web search apart from typing a direct link. The ODR homepage should include the term “ODR” and the phrase “online dispute resolution” on the website homepage. Apply best practices in user interface design and accessible design on the ODR site by adopting a higher contrast color palette; applying a clear, consistent information hierarchy throughout the site; and avoiding text on images.

Streamline the Registration Process. Apply the best practices of web form design by providing consistent system status visibility, error prevention, and matches between the system and the real world to make the registration process easier for website users to complete.

Sixty-four percent of Utah defendants never log in to the ODR platform, so it is essential that the registration and login process not present unnecessary hardships that inadvertently prevent potential users from successfully engaging with the court. Study participants struggled to navigate the registration and login process on the Utah ODR platform, which included many steps requiring repeated referencing of the paper summons and affidavit, and lacked sufficient error-prevention and recovery measures. Common issues included typing in a name or case number incorrectly, re-entering the password because the participants didn’t notice the system requirements, and failing to understand key terms, such as the distinction between plaintiff and defendant or business and individual. We recommend reducing the number of steps required for registration, giving users a progress bar, displaying system requirements for acceptable passwords, verification codes, and email addresses and providing error-prevention feedback to prevent users from attempting to submit incorrect information. In addition, “tooltips” that open lightboxes can provide visual instruction regarding the location of the requested information on the documents. 

Simplify Document Upload and Review. Implement intuitive design choices by integrating document upload functionality into the chat interface and using best practices for system status visibility and error prevention to improve document management for website users. 

The ability for users to upload, share, preview and sign documents is an essential element of the ODR process. This was one of the most difficult tasks for study participants because the process to upload documents in ODR was not intuitive. Many users struggled to simply locate files on their phone for upload, and could not identify the path to reviewing and signing a settlement agreement. One participant became so frustrated that they opted to leave before completion of the test. We recommend allowing users to share documents directly within the chat feature, simplifying the upload process by using familiar icons and text labels, and adding system status feedback to indicate that a document upload is in progress and has successfully completed. In addition, a simplified “Settlement Agreement Review” process would allow users to clearly review and confirm settlement details on the platform before previewing and signing the document. Users should also have the option of downloading and printing documents before and after signing. 

Improve ODR Information and Help. Anticipate and address common questions about ODR throughout the user experience and apply best practices for visual hierarchy, user onboarding, and effective FAQ design to make it easier for website users to access critical information. 

A majority of participants reported being frustrated with the UTAH ODR platform due to the difficulty of finding information about ODR, including information about how it worked, whether participation was mandatory, and how to contact someone for more assistance. Study participants expressed concern about the low perceived legitimacy of the affidavit, summons and ODR platform. In addition, the baseline required users to successfully complete four steps in order to navigate from the affidavit and summons to online information about how to use the ODR platform. We recommend structuring the information that defendants need to make informed decisions about engaging with ODR so that it is clear and easy to locate, and prominently featuring the State of Utah seal on the affidavit, summons, and homepage. A welcome message on the homepage could introduce the online dispute ODR platform and how it operates. Adding an “FAQ” button to the homepage and main navigation menu would provide users with a consistent method for finding help information at any time. We also recommend an overview guide for first-time visitors and a welcome overview video outlining how ODR works, with closed captions in several languages for accessibility and inclusion. 

Clarify Legal Information and User Options. Employ best practices for visual information hierarchy, define keywords and clarify roles, and simplify the claim response experience to help defendants better understand their legal rights and feel more confident in their actions.

Concerns about the lack of access to, or understanding of, legal information were expressed by participants again and again throughout the multi-phase testing process, and thus were shown to be an issue underlying many of the difficulties that participants faced with the ODR experience. We recommend ensuring that the information that defendants need to make informed decisions about their legal rights is clear and easy to locate on the affidavit, summons and throughout the ODR platform. Using a visual information hierarchy to clearly label and highlight defendant options, providing definitions for relevant legal terms used throughout the ODR platform, and adding a claim response tool that helps defendants craft and customize their responses are all strategies designed to connect users with substantive legal information. In addition, clarifying the role of the ODR Facilitator by explaining it in the overview guide, and ensuring that the ODR Facilitator begins each chat with an introductory message explaining their role and the participant’s options, can add further transparency to the online process."

In conclusion, they found that “users are ready for online courts”.  They further concluded that “The challenge—and opportunity—now is to integrate human-centered design into the expansion of ODR so the technologies employed are useful and intuitive for all users.”

Congratulation to both the Utah State Courts on their system and to the University of Arizona for compiling this important review.