Introduction In this blog, we will discuss Snowpipe, a very important feature of Snowflake. SnowPipe allows you to automatically load data as soon as files appear in a particular bucket. When new files are added to the bucket, they are automatically loaded into the specified table. This is especially important when it comes to transactional […]
Posts Tagged ‘#Snowflake’
Snowflake Best Practices for Data Engineering
We often end up creating a problem while working on data. So, here are few best practices for data engineering using Snowflake: 1. Transform your data incrementally: A common mistake novice data engineers make is writing huge SQL statements that join, aggregate, and process many tables, misunderstanding that this is an efficient way to work. […]
Dynamic Merge in Snowflake using Stored Procedure and Python – For SCD Type 1
Creating Snowflake Account We can create a 30-day free trial account using below link. https://signup.snowflake.com/ SCD Type 1 In short, keeps the latest the latest data only and old data is overwritten. SCD Type 1 in Snowflake Let us consider we have a scenario to load the data into Target table from the Source table. […]
EXPLORE TIME TRAVEL IN SNOWFLAKE
Have you ever wondered if it would ever be possible to time travel like in the old movies with a time machine? If we could go back in time and see the world, would we? If you asked me, I would have said yes! But not in the way that Hollywood portrays science fiction films. […]
How to implement incremental loading in Snowflake using Stream and Merge
Snowflake is Cloud hosted relational database used to create Datawarehouse on demand. Data in the data warehouse can be loaded as full load or incremental load. The full load is a process of deleting whole existing data and reloading it again. Full loads are time and resource-consuming tasks compared to incremental loads that only load […]
Load Data From Amazon RDS to Snowflake Using Matillion ETL Tool
Amazon Relational Database Service, a service provided by Amazon Web Services, is a fully managed SQL database cloud service that allows you to create and operate relational databases. With Amazon RDS, one can access all the files and database anywhere in a cost-effective and highly scalable way. Snowflake is a cloud-based platform that helps data […]
Getting Started with Matillion ETL Tool
In this article, I will show you how you can set up your own Matillion instance, but before we get started let’s talk about the Matillion ETL tool. What is Matillion, and why should you care? Matillion is an ETL tool that is available on the Cloud Marketplace. It is completely cloud-based, billed at an […]
Best practices for securing Snowflake
Understanding best practices for securing Snowflake and having a concrete implementation plan is a critical Day Zero deliverable. Snowflake is a secure, cloud-based data warehouse. There are no hardware or software components to select, install, configure, or maintain. There is virtually no software to install, configure, or manage. Snowflake takes care of ongoing maintenance and […]
HIPAA compliance with Snowflake
At Perficient, our Data Solutions team has worked closely with our Healthcare division to implement Snowflake for HIPAA and HITECH compliance. Snowflake offers healthcare organizations a secure data warehouse environment with many HIPAA compliance features. Perficient’s implementation team includes Snowflake and health industry subject matter experts. We’ll take a look at Snowflake’s benefits for healthcare providers […]
Time Travel with Snowflake and BigQuery
Hello friends, I am back with another interesting topic called “time travel.” I believe many of you wish you could go back in time and fix something. The idea of time travel was made possible in the fictional world of movies and books, including Aditya 369 (released during my teens), recent Marvel movies, and Tenet. […]