Canadian Data Guy Unfiltered

Inside Delta Lake’s Idempotency Magic: The Secret to Exactly-Once Spark

Learn how txnAppId and epochId work together to create a bulletproof distributed two-phase commit. Achieve true exactly-once semantics for your…

Jan 27 • Canadian Data Guy

13

6:18

How to Choose Between Liquid Clustering and Partitioning with Z-Order in Databricks

The views expressed in this blog are my own and do not represent official guidance from Databricks

Jan 15 • Canadian Data Guy and Geethu

13

2

1

Unlocking Sub-Second Latency with Databricks

How Spark Real Time Mode Achieving Millisecond Latency with a Simple Trigger Switch

Jan 14 • Canadian Data Guy

3

17:55

I Knew the Answer. I Just Couldn’t Remember It.

How you can turn your notes into a personal Knowledge Agent — no code required

Jan 10 • Canadian Data Guy

11

1

13:41

December 2025

4 Surprising Truths That Will Change How You Think About Spark Streaming

Spark gives you Real-Time without the complexity and pain

Dec 15, 2025 • Canadian Data Guy

2

November 2025

Why I Materialize Delta History for Debugging

Just a Quick Tip

Nov 27, 2025 • Canadian Data Guy

1

Stop Waiting for Connectors: Stream ANYTHING into Spark (It's 4 Functions)

How to ingest data from any source into Apache Spark — demystified with real-world example of BlockChain Ingestion

Nov 3, 2025 • Canadian Data Guy and Yogita Nesargi

6

1

2

26:05

October 2025

How to write your first Spark application with Stream-Stream Joins with working code

A Practical, Hands-On Guide to Joining Real-Time Data Streams in Spark Structured Streaming

Oct 15, 2025 • Canadian Data Guy

6

September 2025

Build an Ethereum ETL Pipeline for Free Using Databricks Free Edition

Build a zero-infrastructure streaming pipeline: Step-by-step Ethereum data ingestion, schema evolution, and Delta storage

Sep 23, 2025 • Yogita Nesargi

4

1

June 2025

How to ace and structure your Data Modelling Interview

Prescriptive guidance for conducting your Data Modelling Interview

Jun 18, 2025 • Canadian Data Guy

10

2

1

A Deep Dive into Skewed Joins, GroupBy Bottlenecks, and Smart Strategies to Keep Your Spark Jobs Flying

How to Diagnose, Prevent, and Fix Performance Bottlenecks from Skewed Data in Your Spark Workloads

Jun 6, 2025 • Canadian Data Guy

10

5

May 2025

Decode the Join: A Spark Data Engineer’s Visual Handbook

Understand when and why to use Broadcast, Shuffle, or Sort-Merge Joins in Spark— with clear visuals, real-world use cases, and strategy tips tailored…

May 9, 2025 • Canadian Data Guy and Harathi Pasam

18

4

1