Merging On-Demand HPC Resources from Amazon EC2 with the Grid: A Case Study of a Xmipp Application

Authors

  • Alejandro Lorca
  • Javier Martín-Caro
  • Rafael Núnez-Ramírez
  • Javier Martínez-Salazar

Keywords:

Grid, cloud, HPC, resource provisioning, virtualization, , parallelization, image processing

Abstract

We present an infrastructure in which HPC resources from the Amazon Web Services public cloud are combined with specific grid resources at Ibergrid. The integration is done transparently for the GridWay users through a daemon which permanently monitors the pool of available resources and submitted jobs, managing virtual instances for satisfying the demand under budget restrictions. The study has been proved with a specific application from the Xmipp package, which performs image processing from electron microscopy data and requires heavy high-throughput computing offering parallelization capabilities. The application was ported successfully for such a hybrid framework with the help of the MPI-Start package. Some preliminary results from test runs are presented for a controlled sample of thousand input images. Some inconveniences and troublesome aspects of the deployment are also reported.

Downloads

Download data is not yet available.

Author Biographies

Alejandro Lorca

Instituto de Física de Cantabria, CSIC, Avenida de los Castros s/n, E-39005 Santander, Spain

Javier Martín-Caro

Secretaría General Adjunta de Informática, CSIC

Calle del Pinar 19, E-28006 Madrid, Spain

Rafael Núnez-Ramírez

Departmento de Física Macromolecular, Instituto de Estructura de la Materia, CSIC

Calle Serrano 113 bis, E-28006 Madrid, Spain

Javier Martínez-Salazar

Departmento de Física Macromolecular, Instituto de Estructura de la Materia, CSIC

Calle Serrano 113 bis, E-28006 Madrid, Spain

Downloads

Published

2012-06-20

How to Cite

Lorca, A., Martín-Caro, J., Núnez-Ramírez, R., & Martínez-Salazar, J. (2012). Merging On-Demand HPC Resources from Amazon EC2 with the Grid: A Case Study of a Xmipp Application. COMPUTING AND INFORMATICS, 31(1), 17–30. Retrieved from https://www.cai.sk/ojs/index.php/cai/article/view/884

Issue

Section

Special Section Articles