Natural language AI has proliferated into many of today’s applications and platforms. One of the high in demand use cases is the ability to find quick answers to questions about what’s hidden within organizational data, such as operational, financial, or other enterprise type data. Therefore leveraging the latest advancements in the GenAI space together with […]
Posts Tagged ‘sql’
SQL: DML, DDL and DCL
In the realm of databases, SQL (Structured Query Language) serves as the lingua franca, enabling users to interact with data stored in various systems effectively. While SQL encompasses a wide array of commands, understanding the distinctions between Data Manipulation Language (DML), Data Definition Language (DDL), and Data Control Language (DCL) is fundamental for wielding this […]
Introduction to Star and Snowflake schema
In the world of data warehousing and business intelligence, two key concepts are fundamental: Snowflake and Star Schema. These concepts play a pivotal role in designing effective data models for analyzing large volumes of data efficiently. Let’s delve into what Snowflake and Star Schema are and how they are used in the realm of data […]
Spark DataFrame: Writing into Files
This blog post explores how to write Spark DataFrame into various file formats for saving data to external storage for further analysis or sharing. Before diving into this blog have a look at my other blog posts discussing about creating the DataFrame and manipulating the DataFrame along with writing a DataFrame into tables and views. […]
Spark SQL Properties
The spark.sql.* properties are a set of configuration options specific to Spark SQL, a module within Apache Spark designed for processing structured data using SQL queries, DataFrame API, and Datasets. These properties allow users to customize various aspects of Spark SQL’s behavior, optimization strategies, and execution environment. Here’s a brief introduction to some common spark.sql.* […]
Date and Timestamp in Spark SQL
Spark SQL offers a set of built-in standard functions for handling dates and timestamps within the DataFrame API. These functions are valuable for performing operations involving date and time data. They accept inputs in various formats, including Date type, Timestamp type, or String. If the input is provided as a String, it must be in […]
Spark Scala: Approaches toward creating Dataframe
In Spark with Scala, creating DataFrames is fundamental for data manipulation and analysis. There are several approaches for creating DataFrames, each offering its unique advantages. You can create DataFrames from various data sources like CSV, JSON, or even from existing RDDs (Resilient Distributed Datasets). In this blog we will see some approaches towards creating dataframe […]
SQL Tuning
In D & A Projects, building efficient SQL Queries is critical to achieving the Extraction and Load Batch cycles to complete faster and to meet the desired SLAs. The below observations are towards following the approaches to ensure writing SQL queries that meet the Best Practices to facilitate performance improvements. Tuning Approach Pre-Requisite Checks Before […]
SQL Best Practices and Performance Tuning
The goal of performance tuning in SQL is to minimize the execution time of the query and reduce the number of resources while processing the query. Whenever we run the query, performance depends on the amount of data and the complexity of the calculations we are working on. So, by reducing the no of calculations […]
Sneak Peak of the SQL Order Of Execution
After writing SQL codes for a few years now, I noticed I used to make mistakes in the order of execution of SQL Queries. So here I am with my new blog on the same topic. Let’s jump in to understand the SQL order of execution & learn practical, correct ways to write SQL Codes. […]
How to Improve SQL Performance – Antipatterns
Ever come close to writing a perfectly looking and valid SQL just to see it take hours to run? SQL optimization might be the thing you need! For business and customers, time is limited and precious and as Software Developers, QA Engineers, DB Developers, etc we should strive to improve the efficiency of our queries […]
Creating an EBS Target application in Data Management from a SQL query
When integrating an on-premise system like an E-Business Suite application with an Oracle Enterprise Performance Management Cloud application, SQL queries can be used as means to extract the data from the source database. You can extract the data and then load the data directly to the EPM Cloud applications using the EPM Integration Agent. The EPM Integration Agent […]