As a technologist raised in Kansas as a child, I am used to examining the horizon for changes in the weather. We could watch the large thunderstorms build up during the heat of the day into 40,000 foot monsters full of hail and tornadoes. I was reading an Ars Technica article on how pervasive computing and cloud will change the nature of IT recently, and the article pointed out the coming “big data” storm. The article argues that like the heat of a Midwestern day growing and driving the thunderstorm, key technologies have reached a tipping point to create the perfect “big data” storm.
Just like storms, big data includes three big trends of volume, velocity and variety. Data sets were predicted to reach 1 petabyte (1 million gigabytes) and without recent advances in hardware and software, especially virtualization and cloud storage, would be unmanageable in terms of volume. Velocity is especially evident in healthcare 
where near real-time information is not just a goal but becoming a requirement. Velocity demands very fast processing to analyze data, potentially aggregate it, and visualize results. Variety of data is a challenge in healthcare as well, with new and unstructured forms like video, text and sensor readings. The proliferation of medical devices and sensors alone can create data that literally overwhelm the typical healthcare organization’s ability to analyze and take action based on the information, thus creating a dangerous storm.
If our “big data” universe is going to continue to grow in these frightening ways, then the question is taming the storm or creating ways to analyze and consume it. The recent trend in healthcare over the last few years is data fusion or bringing data from multiple sources into one place for analysis. Data fusion is very useful for combining costs with procedural outcomes in healthcare to work on reducing costs and improve patient results, for example. Another popular trend in healthcare is time series analysis, which is measuring or predicting one or more values over time. A great example of time series analysis is predictive preventable readmissions in healthcare, where the history of readmissions by disease is used to model or predict the potential for preventable readmissions. In the future, time series analysis should lead to ensemble learning where multiple predictive models are used to improve forecasting of results. Ensemble learning will fine tune advanced analytics to support decision-making on complex medical problems like cancer treatments, for example. Machine learning, or systems like IBM’s Watson, can create contextual models of data and associations between data elements to predict the future or solve problems using large amounts of unstructured data.
Today, like an observer standing on the ground looking at a large storm, we are only seeing or analyzing a very small portion of the fast growing digital universe. As the cloud drives down the cost of storing information for analysis, the tools to visualize the data in meaningful ways will become important. Think about color Doppler radar and the ability it provides today to find hail or tornadoes embedded in the swirling storm. The traditional approach of building business intelligence systems to answer pre-determined questions falls flat in the big data world where the patterns of the data, like colors on the radar, must be tagged, analyzed and acted upon. Right now, data scientists or people with advanced statistical skills are needed to mine these vast data stores, but data visualization tools are advancing to meet the challenge. Making big data easier to consume will drive new discoveries, improve healthcare outcomes and find disease management strategies that simply aren’t visible today.
What is your organization doing to prepare for the storm? More importantly, what business opportunities will your healthcare organization miss without color radar?
