Monday, November 12, 2012

CMS Data Conversion and Archiving with XML

There is an attractive alternative to converting data when replacing, updating, or archiving your court case management system.

Almost every court has suffered the pain in time and cost of converting data from an older system when implementing a new case management system.  I question the wisdom of moving data from an old system for many reasons.  Some problems observed are:

1. The older CMS data fields almost never match the new system.  So old event (docket/registry) codes have to be changed when converted or, the new system has to allow for both the old and new codes in the new validation tables.  In other words, your data has changed.

2. Missing data fields.  More than likely your shiny new CMS comes with new data fields and capabilities.  So when you convert data from the old system into the new, those fields are now empty.   Or, one has to define some generic value in the conversion program that is meaningless in the new system.  And I have personally seen in the instance where the data fields are left empty a system  “explode” with errors asking your users to comply with its needs (like a bad science fiction television episode).

Please comply with my data requirements.

3. Only the active case data may be needed.  And this could also include cases with default judgments or failure to appear fines.  But as you will see below, with archiving in XML, that data will still be searchable and available and it won’t be cluttering up your new CMS.

4. Archiving data.  It just simply doesn't make sense to keep 20 years of red light violations in your CMS.  Clearing them out will make your system run more quickly and your statistics more accurate.

So what is the alternative?  I think that answer lies in our good data friend, XML.  So let’s look at the benefits of saving your old system data in an XML archive.

In an excellent article, Structured Data Archiving using XML Databases by Jeroen van Rotterdam, the author notes the following significant number of benefits of saving data and retiring old systems:
Completely eliminating legacy servers/applications leads to the following cost reductions:
  • Maintenance costs of software licenses
  • Maintenance on aging and obsolete hardware and database
  • IT staffing costs keeping legacy applications running
  • Business staffing costs to maintain knowledge of older systems for data access
  • Energy costs and data center infrastructure costs
It also eliminates the following risks:
  • Maintaining unsupported software versions
  • Maintaining unsupported operating systems
  • Maintaining unsupported hardware
  • Loss of access to data through lack of application knowledge
  • Loss of data or data access through system failures or corruption
And provides important compliance and governance benefits:
  • Secures data from multiple sources in a central repository
  • Enables rapid search and access to all data making responses to discovery or reporting requests more efficient.
  • Allows data to be managed and deleted against compliance retention policies.
The author also provides a Return on Investment (ROI) calculation at the end of the article.

Another article, Archiving Data Using XML by Arup Nanda from the July/August 2006 edition of Oracle Magazine describes in technical details how one could create an XML archive (with code) and also how to restore data to a relational database.  He lists five requirements for an XML archive.  They are:
1. The data must be archived based on its age.
2. The archived data must be dropped from the main database without much performance impact.
3. Reinstating the archived data must be quick and easy.
4. The solution must allow for table structures to change.
5. It must be possible to search inside the archived data without restoring it to the main database.
And he also states the fact that data types, database columns, and system validation codes change over time.  So the article provides a strategy for dealing with the reality.

Two last points.

First, courts do not have to create a single XML file for all the data.  One can create a case and a person table with cross-reference or any other number of permutations.  XML data takes such a small amount of storage space (unless of course documents are involved) that one can create multiple special files as needed.

And second, courts must document what the data fields and codes mean as part of the archiving process.   But this work is still much cheaper to create than data conversion involving weeks if not months of persons programming, testing, performing data entry, and verifying a converted system.

And for reference, one more good article on the subject is Database Archiving: A Critical Component of Information Lifecycle Management, April 21, 2004 in Database Journal.

No comments:

Post a Comment