Using On-demand Data With Dynamic Data Writeback In Spotfire

I have worked on a few clinical applications where adding little functions to existing applications can greatly improve an individual’s job function and subsequently benefit the company as a whole. Looking for these little gems isn’t time consuming or difficult as long as we are aware of the system’s capabilities and are able to capitalize on the novel (or not so novel) use of functionality that is often right at our fingertips.

Take, for instance, TIBCO’s Spotfire which is a popular and powerful tool for visualizing data in many industries. For some industries, such as life sciences, being able to record the activities or notes from the system users is critical for the effective management of business processes, especially in cross-department or cross-system interactions.

One such example is the clinical review of safety and efficacy data. While reviewing safety data, reviewers may have questions relating to outliers or suspect results seen in a clinical trial. In these cases, being able to communicate the actual case data with the individuals responsible for maintaining the data is critical to assessing potential issues with the study drug. Using two different systems, one for visualizing/identifying potential issues and the other for recording the request for information or follow-up, is inefficient and can potentially introduce time delays or an inaccurate translation of the data.

The integration and use of Spotfire can be a timesaver for clinicians and can prevent transcription errors when manually entering data into other systems based upon observations from the Spotfire visualizations. Spotfire, of course, is known for its ability to quickly show data in many formats, from simple data grids to more sophisticated tree maps, box –plots and 3D scatter charts.

The dynamic nature of Spotfire is to link these visualizations with one another and provide the reviewer with a sophisticated tool of identifying outliers. Now, how can we effectively communicate the outlier, or outliers, to another system? The solution is writeback.

 Using this feature, a user can select the data of interest and with a simple click of a button write the information to a database.

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

As a very simple example, above is a screen shot of a simple 3D scatter plot showing an outlier with no dose frequency. Selecting the value in the scatter plot also selects the associated rows in the data grid, showing other related data. A follow-up Text Area has also been added to the visualization, allowing the user to identify the type of follow-up to be performed along with any comments specific to the request.

By clicking the Followup Marked Records button, data from the marked records in the visualization can be used to insert new follow-up records into the database for use by other systems. This same follow-up information can dynamically be available to the user within Spotfire to view the history of follow-up actions and status against them.

To enable this functionality, I created a set of procedures in an Oracle package in the database supplying clinical data for the visualization, but the SQL procedures and data can be stored in any appropriate database.

The SQL logic is nothing special – just typical upsert logic to create and update the appropriate records in the follow-up tables using parameters that pass in the appropriate reference data and supporting data from the user recording the follow-up.

What’s interesting part is the interface with Spotfire. I did two things: create some common logic to record follow-up information in a generic fashion and place buttons in appropriate visualizations that could identify the appropriate marked records to be related to the follow-up for a specific page or type of data and pass it to the common routines. Code reuse is an important part of providing a maintainable code library. Below is a snippet from the common script for recording the follow-up.

In the case of our example for the outlier Intervention follow-up, calling the common routine above with specific tables and columns looks like this:

It passes along the information link containing the marked records and related tables to be refreshed when the insert of the new records is complete.

The concept is simple – provide visualizations that let the user explore and mark data of interest and also provide functionality that allows the user to take action upon observations that they make on the selected data. The integration with the multiple systems is invisible to the user in this case – they are just reviewing the data in a powerful (and fun!) tool.

Spotfire allows me to add a few of these “little things” that can make it easier for a user to get their job done while at the same time improving user efficiency and data reliability, and allowing the user to spend more time at what makes them valuable – their expertise in reviewing clinical data.

To learn how we can help with your data analytics and warehousing needs, please contact us.