Distributed Data Management (ST 2021) - tele-TASK

The free lunch is over! Computer systems up until the turn of the century became constantly faster without any particular effort simply because the hardware they were running on increased its clock speed with every new release. This trend has changed and today's CPUs stall at around 3 GHz. The size of modern computer systems in terms of contained transistors (cores in CPUs/GPUs, CPUs/GPUs in compute nodes, compute nodes in clusters), however, still increases constantly. This caused a paradigm shift in writing software: instead of optimizing code for a single thread, applications now need to solve their given tasks in parallel in order to expect noticeable performance gains. Distributed computing, i.e., the distribution of work on (potentially) physically isolated compute nodes is the most extreme method of parallelization.Big data analytics and management are a multi-million dollar markets that grow constantly! The ability to control and utilize large amounts of data is the most valuable ability of today's computer systems. Because data volumes grow so rapidly and with them the complexity of questions they should answer, data analytics, i.e., the ability of extracting any kind of information from the data becomes increasingly difficult. As data analytics systems cannot hope for their hardware getting any faster to cope with performance problems, they need to embrace new software trends that let their performance scale with the still increasing number of processing elements.In this lecture, we take a look at various technologies involved in building distributed, data-intensive systems. We start by discussing fundamental concepts in distributed computing, such das data models, encoding formats, messaging, data replication and partitioning, fault tollerance, and batch- and stream processing. In between, we consider different practical systems from the Big Data Landscape, such as Akka and Spark. In the end, we concentrate on data management aspects, such as distributed database management system architectures and distributed query optimization.

Recent Episodes
  • Lecture Summary
    Jul 21, 2021 – 01:57:16
  • Federated DBMSS
    Jul 19, 2021 – 01:16:07
  • Stream Processing - Databases and Streams
    Jul 12, 2021 – 01:29:24
  • Stream Processing
    Jul 7, 2021 – 01:20:16
  • Exercise 1 Evaluation
    Jul 5, 2021 – 01:29:27
  • Spark Batch Processing (2)
    Jun 30, 2021 – 01:37:54
  • Spark Batch Processing
    Jun 28, 2021 – 01:29:30
  • Batch Processing 2 - Distributed File Systems and MapReduce
    Jun 21, 2021 – 01:30:38
  • Batch Processing
    Jun 16, 2021 – 01:31:15
  • Transactions
    Jun 14, 2021 – 01:27:16
  • Consistency and Consensus
    Jun 9, 2021 – 01:41:06
  • Distributed Systems
    Jun 7, 2021 – 01:21:07
  • Partitioning & Distributed Systems
    Jun 2, 2021 – 01:29:17
  • Replication & Partitioning
    May 31, 2021 – 01:22:22
  • Replication
    May 26, 2021 – 01:35:19
  • Storage and Retrieval
    May 19, 2021 – 01:34:40
  • Data Models and Query Languages
    May 17, 2021 – 01:34:25
  • Akka Actor Programming 3 - Patterns
    May 12, 2021 – 00:42:13
  • Akka Actor Programming 2
    May 5, 2021 – 01:33:25
  • Akka Actor Programming
    May 3, 2021 – 01:32:32
  • Communication: Service-oriented & Database-oriented Middleware
    Apr 28, 2021 – 01:31:18
  • Communication: Message-oriented Middleware
    Apr 26, 2021 – 01:31:39
  • Communication
    Apr 21, 2021 – 01:23:58
  • Encoding
    Apr 19, 2021 – 01:33:49
  • Foundations
    Apr 14, 2021 – 01:22:24
  • Introduction
    Apr 12, 2021 – 01:34:22
Recent Reviews
Similar Podcasts
Disclaimer: The podcast and artwork on this page are property of the podcast owner, and not endorsed by UP.audio.