IEEE International Geoscience and Remote Sensing Symposium (IGARSS)
Scalable Machine Learning with High Performance and Cloud Computing
26-27/09/2020
Abstract
Deep Learning is emerging as the leading AI technique owing to the current convergence of scalable computing capability (i.e., HCP and Cloud computing), easy access to large volumes of data, and the emergence of new algorithms enabling robust training of large-scale deep neural networks. The tutorial aims at providing a complete overview for an audience that is not familiar with these topics.
Lecture 1: Introduction
🏛Jülich Supercomputing Centre - Forschungszentrum Jülich
🌐Machine learning and Deep Learning in remote sensing
🎛Deep learning and Supercomputing
Lecture 2: Levels of Parallelism and High Performance Computing
🍴The Free Lunch is Over
⛓Hardware Levels of Parallelism
📱High Performance Computing (HPC)
⚙️Jupyter-JSC
Lecture 3: Distributed Deep Learning
🏄Distributed training
🪓Horovod
🪐DeepSpeed
Lecture 4: Hands-on Distributed Deep Learning
🍽🍛🍹
Become familiar with Horovod, a data distributed training framework
Understand how to modify existing code to enable parallelism
Understand the importance of distributing data beforehand
Understand what Horovod does looking at the lines of code to be added
Create a job script to execute Python code on the GPUs
Play around with model architecture, optimizer, learning rate
Get notebook
Lecture 5: Big Data Analytics using Apache Spark
💥Apache Spark Basics
🔧Developing on Spark and Clouds
☁️Machine Learning on Spark
Previous teaching
