Job migration in HPC clusters by means of checkpoint/ restart

Abstract

Until now, jobs running on HPC clusters were tied to the node where their execution

started. We have removed that limitation by integrating a user-level checkpoint/

restart library into a resource manager, fully transparent to both the user and running

application. This opens the door to a whole new set of tools and scheduling possibilities

based on the fact that jobs can be migrated, checkpointed, and restarted

on a different place or in a different moment, while providing fault tolerance for

every job running on the cluster. This is of utmost importance in the future generation

of exascale HPC clusters, where an increasing degree and complexities of

efficient scheduling make it challenging to obtain the required degree of parallelism

demanded by the applications.