In today’s data-driven world, the demand for insightful and well-structured reports has never been greater. The advent of generative AI has opened exciting possibilities for automating the report-writing process. In this blog post, we will take you through our experience in creating an application that leverages generative AI to assist in generating reports from tabular data. From exploring various AI models to overcoming challenges and realizing the strengths of this technology, we will provide an in-depth overview of our journey.
Exploratory Phase
Our journey began with an exploration of various AI models and tools available in the market. We evaluated models from Hugging Face, ChatSonic, Azure Cognitive Services, InsightsR, Pandas profiling, and more. We also delved into advanced specialized software like Power BI and Tableau. However, most of these options either lacked a fully functional API, came with excessive costs, or did not align with our specific goals. After careful consideration, we decided to harness the power of GPT-3.5, in partnership with Azure Open AI, to bring our vision to life.
First Approach
Initially, we attempted to streamline our process by automatically pre-processing the data using a specialized model before feeding it into the Language Models (LLMs) for report generation. Unfortunately, this approach proved unsuccessful.
Limitations
It is important to acknowledge the limitations we encountered. GPT-3, and LLMs in general, have constraints when it comes to processing data. They struggle with even relatively small datasets and face challenges in performing basic mathematical operations like addition and multiplication. Since these models are not specifically designed or trained for direct data analysis tasks, the results generated were inaccurate most of the time.
Refocusing on Strengths
In response to these limitations, we decided to refocus on the strengths of GPT-3 and similar models:
- Generating Insightful Questions: GPT-3 has the remarkable ability to generate insightful questions that aid in extracting information from data.
- Automated Code Generation: It can automatically generate code snippets that extract valuable information from the data, streamlining the analysis process.
- Report Generation: GPT-3 can produce comprehensive and well-structured reports by utilizing specific information derived from data analysis.
Current Approach
Our application now follows a clear and user-friendly workflow:
- Data Input: Users provide clean data in tabular form with descriptive field names. If needed, they can map these names to corresponding data types.
- Question Generation: The AI generates a specified number of questions based on the data.
- User Interaction: Users have the option to review, edit, or add new questions, tailoring the analysis to their needs.
- Answer Generation: The application displays answers to the questions generated. If a user is dissatisfied, they can request a regeneration.
- Report Generation: Once users are satisfied with both questions and answers, the application compiles a written report.
- Chart Integration: Users can also request simple charts (e.g.: bar, line, and pie charts) to be included at the end of the report.
Problems Encountered
Throughout our journey, we encountered some challenges:
- Question Clarity: Questions must be clear and mirror the exact way values appear in the dataset. Precision is key to receiving accurate responses.
- Data Format: Standardizing date formats (e.g., YYYY/MM/DD) was crucial to ensuring consistency in the analysis.
- Chart Generation: To avoid potential errors, we recommend providing default parameter values for chart generation.
In conclusion, generative AI holds immense potential for gaining insights from data and automating report generation. By addressing issues such as data format and ensuring clarity in questions, we have achieved high accuracy in our results. However, it is essential to remember that AI is a tool, and results should be interpreted with caution. While generative AI can enhance our workflow, it is crucial to use it wisely and validate the generated insights when making critical decisions.
We want to emphasize that this is just one of the many approaches to tackle this problem; imagination sets the limits. Our journey has shown us the power of this technology, and we look forward to its continued evolution in the world of data analysis and reporting.
Disclaimer: A portion of this blog was created with the assistance of AI!
