Wednesday, August 13, 2014

Data Conversion = Pain

For a number of years I have tried to advise courts not to convert data from their legacy systems into their new ones.  Here are my reasons why.


First, in this article I am not talking about live, active case data but rather closed inactive cases that courts seem to insist must be converted into any new case management system they have acquired.

Regarding active cases; I think that it generally makes sense to let those cases be managed in the old system for a period of time to let them be completed without conversion.  This reduces the technical and the verification workload for data conversion.  Unfortunately this may not be possible if you are suffering from a broken system.  Understood.

Second, notwithstanding the caveat above, I would like to posit my reasons for NOT converting your case data.

1. It is expensive to do.  It often this requires programming and a good amount of testing and staff visually validating the results.  And what real long term benefit do you get?  Please at least ask the question.

2. Technically, the reason that programming is required is that the data from the old system will very often not validate in your new system’s data fields.  You bought the new system to do more things for you.  Why would you want it to work like your old system?  As a result often the data field validation is turned off for conversion.  And this in turn results in the less reliable data in your new system.

3. Other than name and case lookup and perhaps judgment/sentence – why do you need the rest of the old case data?  Again, please ask the question to yourself and your team on this.

4. I think there is now better technology to handle this – convert the data to flat XML files.  All the data is preserved.  All the data can be searched.  Fast, simple and no problems with data validation.  And if you don’t want your old data to mix with your new system, you can buy or install a separate XML database.  See this Wikipedia article on XML_databases for more.

5. You can possibly keep your old system running.  Virtual machines allow for old computer environments to be maintained in your new server.  I have written in the past here in the CTB about my adventures with creating an MS-DOS 5.0 virtual machine on my laptop to run a CCMS that was created in the 1980's.  It worked fine.  Actually more than fine as it ran very quickly on the new computer hardware.  See this Wikipedia article for more on Virtual Machines.

Thank you for reading my rant.  And do please share your reasons for your court's data conversion in the comments below.  I (and I am sure others) would love to learn about your experiences.

1 comment:

  1. Another idea was to just move the old data into a DW for querying. Handy for organizations that have such infrastructure. I like the idea of asking the why questions on the scope of conversion. Interesting thought on what actually needs to be converted (Active vs Inactive). More great questions to have answers for when looking at such a data migration effort.