Spark SQL Kickstart: Your first Spark SQL application
Get hands on with Spark SQL to build your first data pipeline. In this video I walk you through how to read, transform, and write the NYC Taxi dataset which can be found on Databricks, Azure Synapse, or downloaded from the web to wherever you run Apache Spark.
Once you have watched and followed along with this tutorial, go find a free dataset and try to write your own Spark application.
All thoughts and opinions are my own *
For links to the code and more information on this course, you can visit my website: https://dustinvannoy.com/2023/05/18/f...
More from Dustin:
Website: https://dustinvannoy.com
LinkedIn: / dustinvannoy
Github: https://github.com/datakickstart
CHAPTERS
00:00 Intro
1:11 Spark SQL data pipeline (notebook)
11:56 Spark SQL locally
18:30 Outro
