Snowpark : Data Engineering With Snowflake.

0

1

2

3

4

5

Posted on 18 Feb 11:44 | by mitsumi | 13 views

Snowpark : Data Engineering With Snowflake.
Published 2/2023
MP4 | Video: h264, 1280x720 | Audio: AAC, 44.1 KHz
Language: English | Size: 1.89 GB | Duration: 4h 18m

Learn fundamentals of Snowflake-snowpark API.

What you'll learn
Connect Snowpark API with snowflake.
Basic read and write operations using Snowpark.
Do's and Don'ts while using Snowpark.
Building data components and basic data pipeline.
Requirements
Basic knowledge on python.
Description
What is Snowpark ?With Snowpark, Snowflake allows developers to write code in their preferred language.Along with Snowflake's original SQL interface now snowflake allows you to write code in, 1. Python 2. Scala 3. JavaSome of the key features of Snowpark are,Your code will be pushed to snowflake leveraging the compute power of snowflake warehouses.You will not end up exporting data to different environment but rather your code is shipped to the data.You can build complex data pipelines or data products using SnowparkSnowpark also address below overheads in conventional data pipelines,Long startup time of node clusters: Systems like Hadoop and Spark requires cluster of nodes to process data. Most of the time it takes 5-10 minutes to just start the cluster. In case of Snowpark we will be using snowflake warehouse to process our data.Problem of small files , Problem of using right joins to shuffle data across nodes, Problem of garbage collection. uncertainty when the compute nodes goes down.All the above problems are well handled with Snowpark.What you will learn ?You will learn the basics of SNOWPARK API. > Basic read and write operations. > Read data from s3 and load that to snowflake table. > We will do deep analysis of how SNOWPARK API works. > Do's and Don'ts of SNOWPARK. > Build data components to process data. > Build data pipeline to process data.
Overview
Section 1: Introduction
Lecture 1 Snowpark introduction
Lecture 2 Test connection
Lecture 3 Snowpark-demo part 1
Lecture 4 Snowpark-demo part2
Section 2: Snowpark-- Read operations
Lecture 5 Create dataframe--part1
Lecture 6 Create dataframe--part2
Lecture 7 Apply schema
Lecture 8 Create dataframe--part3
Lecture 9 Create dataframe--part4
Lecture 10 Read from table--part1
Lecture 11 Read from s3 csv
Lecture 12 Read from s3 json
Section 3: Assignments-- Read operation
Section 4: Snowpark---Write operations
Lecture 13 Basic write operation
Lecture 14 Write from s3 to table--json
Lecture 15 Write from s3 to table--csv
Section 5: Assignments -- write operations
Section 6: Assignments-- Read and write semistructured data.
Section 7: Snowpark--Copy commands
Lecture 16 Write using copy
Section 8: Transformations and query
Lecture 17 Aggregation in snowpark
Lecture 18 Group by --part 1
Lecture 19 Group by --part 2
Lecture 20 Window function
Lecture 21 Joins
Lecture 22 Using in clause
Section 9: Assignments---Transformation and query
Lecture 23 Prepare data
Section 10: Building generic components
Lecture 24 Download resources
Lecture 25 Create snow connection component-- part1
Lecture 26 Create snow connection component--part2
Lecture 27 Copy to snowflake table
Lecture 28 Creating configuration file
Lecture 29 Collect rejects
Lecture 30 Copy semi structured data part 1
Lecture 31 Copy semi structured data part 2
Lecture 32 Map columns
Lecture 33 Map columns solution
Lecture 34 Summary
Developers who want's to lean Snowflake-Snowpark.