LOGOS: Enabling Local Resource Managers for the Efficient Support of Data-Intensive Workflows within Grid Sites

Authors

  • David A. Monge ITIC Research Institute, Universidad Nacional de Cuyo, ECT, Padre Jorge Contreras 1300, M5502JMA Mendoza
  • Carlos García Garino ITIC Research Institute & Facultad de Ingeniería, Universidad Nacional de Cuyo, Centro Universitario, M5502JMA Mendoza

Keywords:

Grid sites, data-intensive workflows, scheduling, HTCondor, DAGMan

Abstract

In this study we discuss how to enable grid sites for the support of data-intensive workflows. Usually, within grid sites, tasks and resources are administrated by local resource managers (LRMs). Many of LRMs have been designed for managing compute-intensive applications. Therefore, data-intensive workflow applications might not perform well on such environments due to the number and size of data transfers between tasks. To improve the performance of such kind of applications it is necessary to redefine the scheduling policies integrated on LRMs. This paper proposes a novel scheme for efficiently supporting data-intensive workflows in LRMs within grid sites. Such scheme is partially implemented in our grid middleware LOGOS and used to improve the performance of a well known LRM: HTCondor. The core of LOGOS is a novel communication-aware scheduling algorithm (PPSA) capable of finding near-optimal solutions. Experiments conducted in this study showed that our approach leads to performance improvements up to 52 % in the management of data-intensive workflow applications.

Downloads

Download data is not yet available.

Downloads

Published

2014-06-02

How to Cite

Monge, D. A., & García Garino, C. (2014). LOGOS: Enabling Local Resource Managers for the Efficient Support of Data-Intensive Workflows within Grid Sites. COMPUTING AND INFORMATICS, 33(1), 109–130. Retrieved from https://www.cai.sk/ojs/index.php/cai/article/view/877