Considerations for data conversion

If you are considering converting data from any previous source to HIFIS 4, there are several key considerations you should take into account.

Consent

Most legislation surrounding the collection, use, and disclosure of personal information states that when a client consents to have their information collected, it can be used and disclosed only for specific purposes. When converting from a non-shared data system to a shared data system, most communities therefore need to develop and implement new consent forms that include a clause stating that the client’s information can be shared with other agencies.

Let’s assume, then, that your community is developing a new consent form for use with HIFIS 4.

If you have a new consent form that states that information can be shared, and you begin using it as you move to HIFIS 4, that means all your previous clients have been signing consent forms that do not allow their data to be shared.

Let’s further assume that you’ve been using your previous data system for some period of time, like 5 years. In that time, you’ve worked with a number of clients – hundreds, or even thousands – who have since been housed and are no longer interacting with your homeless-serving system. Or who have moved on to another city. Or who are currently spending time in an institution.

It is nearly impossible for your community to ensure that you have valid consent to share personal information for all your past clients, if you weren’t initially collecting consent to do so from the start. What this means is you’re going to have some sub-section of clients in your current database whose data cannot be converted while still satisfying the terms of the consent they’ve provided.

Similarly, you may have previously been collecting consent that was valid for a period of time, like one year, and could have a large number of past clients who have expired consent. So for them, you have no permission to do anything at all with their data.

If you want to proceed with data conversion, you’ll need a way to ensure that you’re adhering to the legislation that allows you to collect personal information. This may mean converting only a select group of clients (those with appropriate consent), or it may mean converting all data but blocking access to the data for those who did not previously have permission to access it.

Tip: It’s strongly recommended you consult with your Legal Support if you’re considering converting your previous data.

Data Clean-Up

A second topic to consider is the issue of data clean-up. At its simplest form, you may be considering taking one database and converting it from one format to another. This alone carries some risks of certain fields not transferring properly or data getting corrupted in the transfer.

You are going to need to do at least one test conversion, and then perform a number of tests to see whether the conversion worked properly or not, identify what data pieces did not convert adequately, and address those issues. This is an iterative process, and there’s no telling how long this step will take. Resources – in particular IT support – will need to be assigned to this task.

The problem is further compounded if you are considering taking two or more existing databases and merging them into one HIFIS 4 database. This is a likely scenario, considering that many communities might have two or more shelters, and the most common arrangement is for each shelter to be using an isolated HIFIS 3 installation.

If you’re considering merging multiple databases, you’re going to have the added challenge of addressing duplicate clients. Even at its very best, software (in general) can only successfully identify duplicate people some of the time. Unfortunately, HIFIS 4 is not the best software at identifying and addressing duplicate records.

There are several challenges with locating duplicate records. You might collect minimal information on clients and only have their name and age, or even approximate age. With limited information on each client, it’s hard to tell whether there’s an actual duplicate or two distinct clients with the same name. There could easily be two 40-year-old James Smiths or two 30-year-old Emily Martins. In fact, among my own social circle I know two different Daniel MacDonalds who live in the same city, have the same hobbies, and are the same age!

Even if you collect more than just the bare minimum, there’s still a challenge. For example, two clients with a date of birth of 1980-03-06 and 1980-06-03 might look to a computer like two different clients, but one of the files might have been a data entry error from a staff who doesn’t remember if date of birth should be entered as YYYY-DD-MM or YYYY-MM-DD.

Then there’s spelling errors and nicknames. For example, is William Jonson the same as Will Johnson? What about David Thomas-Brown and Dave Brown (and David Thomas)? Then you also could very well have clients who have changed their names after a marriage, divorce, or gender transition.

The point is, after conversion your merged database is going to be messy. You can manually merge duplicate clients in HIFIS 4, but you can only do so one at a time, so it can take a while if you have a large database. Several communities that have successfully merged databases have done so with the assistance of many dedicated resources, such as assigning extra personnel to data clean-up after merge.

Tip: It’s recommended that if you’re considering converting your database, you have a frank conversation with your project team and your IT Support and determine how many resources are available to this step. If you find yourself at or over your budgeted resources, it may be a good idea to abort the idea of data conversion and start HIFIS 4 from a clean, blank database.