CanadianDataGuy’s No Fluff Newsletter

CanadianDataGuy’s No Fluff Newsletter

Home
Notes
TL;DR
Deep Dive
Blogs on Medium
Youtube
Whatsapp Community
About

May 2025

April 2025

When Data Engineering Met AI
Teaching AI to Play Nice in Data Engineering
 • 
Canadian Data Guy
5:03
Why Your PySpark UDF Is Slowing Everything Down
An in-depth exploration of architecture, execution flow, bottlenecks, and optimization strategies for PySpark UDFs
 • 
Canadian Data Guy
What a Netflix Senior Data Engineer Taught Us About Winning in Tech—And It’s Not What You Think
Spoiler: Tech is easy. Business is hard. And your ability to communicate might just be your biggest flex
 • 
Canadian Data Guy
How Do I Think About Setting Spark Shuffle Partitions in 2025?
TLDR: A Quick Guide to setting Spark.Shuffle.Partitions, No Deep Dive Required
 • 
Canadian Data Guy
Spark Join Strategies Explained: Broadcast Hash Join
Everything You Need to Know About Broadcast Hash Join
 • 
Canadian Data Guy
One Spark Function to Rule All Your Custom Streaming Needs
Understand the power of the foreachBatch function, incremental processing, and S3 integration in this hands-on intro to Spark Streaming.
 • 
Canadian Data Guy
3:38
Spark Join Strategies Explained: Shuffle Hash
Everything You Need to Know About Shuffle Hash Join
 • 
Canadian Data Guy
Spark Join Strategies Explained: Sort Merge Join
Slow and Steady always wins the race
 • 
Canadian Data Guy

March 2025

© 2025 Canadian Data Guy
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture