Apache Spark with Scala - Hands On with Big Data!

Course Overview

This course walks you through Apache Spark using Scala as the primary language, built around the idea that you learn big data processing by actually writing and running Spark jobs rather than just watching slides. The instructor — described as a former engineer and senior manager at Amazon and IMDb — structures the course around more than 20 progressively harder exercises, starting with simple RDD operations on movie ratings and text data, then moving into SparkSQL, DataFrames, Datasets, machine learning with MLlib, Spark Streaming, and graph analysis with GraphX. A recurring thread throughout the course is a social graph of comic book superheroes, used to teach concepts like breadth-first search, degrees of separation, and Pregel-based graph traversal — a genuinely clever way to make abstract distributed-computing concepts concrete.

The course has been rebuilt for Spark 3 and IntelliJ, with heavier emphasis on the Dataset API and Structured Streaming compared to older versions. It also pushes beyond your laptop: later sections cover packaging jobs with SBT, running them via spark-submit, and deploying to a real Hadoop cluster using Amazon's Elastic MapReduce, so you get exposure to how Spark jobs actually run in production-like environments, not just in a local notebook.

What You Will Learn

• Writing distributed data processing code in Scala
• Working with Resilient Distributed Datasets (RDDs), key-value pairs, and transformations like map/filter/flatMap
• Querying and transforming structured data with SparkSQL, DataFrames, and Datasets
• Optimizing Spark jobs through partitioning, caching, and persistence
• Packaging and deploying Spark applications with SBT and spark-submit
• Running Spark jobs on a Hadoop cluster via Amazon EMR
• Processing real-time data streams with the legacy DStream API and Structured Streaming
• Analyzing graph-structured data and running BFS-style algorithms with GraphX and Pregel
• Building recommendation systems and regression models using MLlib

Course Structure

The course is split into 10 sections covering: Spark basics and a Scala crash course, RDD fundamentals and transformations, SparkSQL/DataFrames/Datasets, graph and social network analysis (the superhero examples), running Spark on a cluster (SBT, spark-submit, EMR, partitioning), MLlib for recommendations and regression, Spark Streaming (both legacy and structured), and GraphX. Each major topic ends with a short quiz, and several sections include guided exercises followed by a recorded solution walkthrough, which is a useful structure if you want to actually test yourself before being handed the answer.

Who Is This Course For?

This fits software engineers or data practitioners who already have some programming or scripting background and want a practical, example-driven entry into distributed data processing — not a theory-heavy computer science treatment of distributed systems. If you've never written any code before, the built-in Scala primer won't be enough on its own; the course assumes you can already think like a programmer. It's also a reasonable fit for people deciding between Spark's Scala and Python APIs, since a parallel Python version of the same course exists if Scala isn't your preference.

Format & Time Commitment

It's fully self-paced with lifetime access, so there are no deadlines or cohort schedules. At roughly 9 hours of video spread across 69 short lectures, it's realistic to work through at a few lectures per sitting, especially since many sections involve pausing to write and run code alongside the instructor rather than passive watching.

Pros and Cons

Pros

• Large number of genuinely hands-on, runnable examples rather than slide-only theory
• Covers the full Spark ecosystem — SQL, streaming, ML, and graph processing — in one course
• Includes real cluster deployment via EMR, not just local execution
• Exercises with separate solution walkthroughs encourage active practice

Cons

• Requires existing programming experience; absolute beginners will struggle despite the Scala primer
• At under 9 hours total, individual topics like MLlib or GraphX get fairly brief treatment compared to dedicated courses on those subjects
• Course quality and pricing on Udemy can vary significantly depending on which promotional discount is active
• A completion certificate from Udemy carries limited formal recognition compared to university or vendor-issued credentials

FAQ

Does this course require prior Scala experience? No, but it does require some general programming or scripting background — the included Scala section is a crash course, not a from-scratch programming class.

Is the certificate useful for job applications? It documents completion but isn't a widely recognized industry credential; it's more useful as a personal record than as a resume-defining qualification.

Should I take this version or the Python version? If your work environment or team leans toward Python, the instructor's separate Python-based Spark course covers similar ground using PySpark instead of Scala.

Do I need Hadoop installed to take this course? No — the course starts with Spark running locally on your desktop and later shows how to move the same jobs to a Hadoop cluster via Amazon EMR.

If you want a code-first way to actually build and deploy Spark jobs rather than just read about them, it's worth checking the current price and syllabus on Udemy's official course page.