In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

Youcef Barigou

will defend his dissertation

Optimizing Communication-computation Overlap in High Performance Computing

Abstract

The number of compute nodes and cores per node have increased many fold on high end computer systems over the last decade. For a parallel application to scale to tens or even hundreds of thousands of processes, all non-computing related operations have to be kept at an absolute minimum, including communication operations.

Non-blocking collective operations extend the concept of collective operations by offering the additional benefit of being able to overlap communication and computation. However, it has been demonstrated that collective operations have to be carefully tuned for a given platform and application scenario to maximize their performance. Also, using non-blocking collective operations in real-world applications is non-trivial. Application codes often have to be restructured significantly in order to maximize the communication-computation overlap.

The goal of this dissertation is to optimize non-blocking collective communication operations and facilitate their utilization at the end-user level. This is achieved, first, by an automatic run-time tuning of non-blocking collective communication operations, which allows the communication library to maximize communication-computation overlap, and choose the best performing implementation for a non-blocking collective operation on a case by case basis.

Second, by an approach to maximize the communication-computation overlap for hybrid OpenMP/MPI applications. It leverages automatic parallelization by extending existing concepts to utilize non-blocking collective operations. It integrates the run-time auto-tuning techniques of non-blocking collective operations, optimizing both, the algorithms used for the non-blocking collective operations as well as location and frequency of accompanying progress function calls.

Date: Monday, April 25, 2016
Time: 10:00 AM
Place: PGH 501D
Advisor: Dr. Edgar Gabriel

Faculty, students, and the general public are invited.