There are numerous obstacles in making healthcare data useful for downstream analysis for decision making. There may be no bigger challenge than taking data that has been captured from disparate healthcare EMR systems and cleansing and normalizing them into something uniform for consumption. This is a result of the way that data is captured in EMR systems.
It is important to understand why data could be non-standard to determine what to do about it.
- The source allows free form text to be entered such as a patients address
- Source allows fields to be optionally populated
- Source captures data at different granularities across the workflow such as an ordered medication vs the administered medication vs medication on patient med list
- Multiple sources will have different internal codification of values such as gender
- Sources has note or comment fields that are by its nature non standard
Each distinct challenge could require a different approach to determine how to best address them.
But before you set off on looking for tools to fix the problem, its vital to come up with a methodology to understand the scope and a prioritization.
- Identify the key data fields that are used to drive business decisions and rank them by business value
- Profile these key data fields to understand the characteristics of the distinct data that is populated. What % is non standard? What % can easily be standardized?
- Determine the complexity and effort to standardize each of the key data fields.
- Prioritize a backlog based off the business value of the data and the effort to standardize.
The key is accepting that it is an ongoing to effort to data standardization and you want to bite chunks off at a time. Not all data has equal business value and need to determine where best to use your resources. There are tools that will make addressing these challenges easier but you will need to understand the problems and have a sober awareness of the areas that will be challenging. The goal should be to improve the usefulness and not looking to achieve perfection.