[D&K] IoT Big Data Processing 2016-2017

Course taught in the Data and Knowledge 2nd year Master Program of Université Paris Saclay


This module will present concepts, architectures and algorithms for IoT big data processing and analytics, at a very large scale, in distributed settings. The following topics will be covered:

  • Apache Hadoop
  • Apache Spark
  • Apache Flink
  • Apache Beam/Google Cloud DataFlow
  • Apache Storm
  • Lambda and Kappa Architectures

A strong focus will be given to labs in this class, so that students can gather enough experience with different existing systems, and understand their respective advantages. The architecture of all distributed computing systems will be discussed in detail during lectures.


  • 1/3 Lab Assignments
  • 2/3 Final Test

Lecture Slides

    • 1. Introduction to MapReduce Slides
    • 2. MapReduce Lab Slides (due 4th October 2016)
    • 3. Apache Flink Slides
    • 4. Apache Spark Slides
    • 5. Apache Spark Lab Slides (due 25th October 2016)
    • 6. Apache Kafka, Storm and Samza Slides
    • 7. Google Cloud Data Flow and Apache Beam Slides
    • 8. Apache Spark Lab Slides (due 8th November 2016)