TU Delft
 
GrenchMark
Research
Parallel and Distributed Systems
EWI PDS GrenchMarkResearch
 
 
 
 
 
 
 
 
Quick links
rationale intro results research publications references


Rationale
why and how is this work relevant?
Grid computing is becoming the natural way to aggregate and share large sets of heterogeneous resources. With the infrastructure becoming ready for the challenge, current grid development and acceptance hinge on proving that grids reliably support real applications, and on creating adequate benchmarks to quantify this support. However, grid applications are just beginning to emerge, and traditional benchmarks have yet to prove representative in grid environments. To address this chicken-and-egg problem, we propose a middle-way approach: create and run synthetic grid workloads comprising applications representative for today's grids. For this purpose, we have designed and implemented GrenchMark, a framework for synthetic workload generation and submission. The framework greatly facilitates synthetic workload modeling, comes with over 35 synthetic and real applications, and is extensible and flexible. We show how the framework can be used for grid system analysis, functionality testing in grid environments, and for comparing different grid settings, and present the results obtained with GrenchMark in our multi-cluster grid, the DAS.




Introduction
why do we need MultiProbe and such measurements

In the long term, Grid computing systems (Grids) aim at becoming the standard way of sharing heterogeneous resources, and of aggregating them into virtual platforms, to be used by multiple organizations and independent users alike. With the grid infrastructure starting to meet the requirements of such an ambitious goal [1], the current evolution of grids hinges on proving that it can run real applications, from traditional sequential and parallel applications to new, grid-only, applications. As a consequence, there is a clear need for generating and running workloads comprising grid applications for demonstration and testing purposes. In this paper we present GrenchMark, a framework for synthetic workload generation and submission. With GrenchMark, we try to find a common ground for grid performance analysis and, in the flavor of the Parallel Workloads Archive. The Parallel Workload Archives makes various workload traces from real parallel production environments available at http://www.cs.huji.ac.il/labs/parallel/workload/, we offer the performance-oriented Grid community a tool that can help to bring together Grid performance evaluation approaches, towards the goal of building standard Grid benchmarks. Our main contributions are:

  • A systematic approach to and a set of tools for generating synthetic grid workloads for analyzing, testing, and comparing common grid settings;
  • Modeling and selecting a set of representative real and synthetic grid applications, including applications that require co-allocation ;
  • An experimental validation of our approach. In our setting, we use a multi-cluster environment, the DAS, the Koala co-allocating grid scheduler [2], and the real applications included in the Ibis Java-based Grid programming environment [3]. Koala is developed at TU Delft, NL; more information about Koala is available at http://www.st.ewi.tudelft.nl/koala/. Ibis is developed at VU Amsterdam, NL, and is freely available from http://www.cs.vu.nl/ibis/.

 


top

Results
brief description

For our results please read the related publications.



Research
what did we do, after all?

We have introduced GrenchMark, a framework for synthetic grid workloads generation and submission. The framework supports various workload modeling primitives, comes with over 35 synthetic and real applications, and is portable, flexible, and extensible. We have designed and implemented GrenchMark, and have deployed it in the DAS, our 400-processors grid environment.

We have shown evidence that GrenchMark can be successfully used for grid system analysis, functionality testing in grid environments, and comparing different grid settings. We have presented various examples of how GrenchMark can be used for performance tests, what-if analysis, system functionality tests, periodic system tests, a single vs. co-allocated jobs comparison, and a unitary vs. composite jobs comparison. We have shown how these tests can be extended for different goals than those of this paper, e.g., other performance metrics and other types of tests. Last, but certainly not least, GrenchMark was instrumental in making the Koala grid scheduler reliable on the DAS, and releasing it for the DAS user community.






Publications, conferences, talks
validating our work...
A. Iosup, D.H.J. Epema, C. Franke, A. Papaspyrou, L. Schley, B. Song, R. Yahyapour, On Grid Performance Evaluation using Synthetic Workloads, In The 12th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), held in conjunction with SIGMETRICS'06, Jun 26, 2006, Saint Malo, FR. (accepted).
article to be added
info performance evaluation, synthetic grid workloads, GrenchMark.
 
A. Iosup, C. Dumitrescu, D.H.J. Epema, H. Li, L. Wolters, How are Real Grids Used? The Analysis of Four Grid Traces and Its Implications, The 7th IEEE/ACM International Conference on Grid Computing (Grid), Barcelona, September 28-29, 2006 (accepted, 18%). An extended version, titled An Analysis of Four Long-Term Grid Traces, can be found as Technical Report TU Delft/PDS/2006-003, ISBN 1387-2109).
info Grid computing, trace analysis, real workloads.
 
A. Iosup, D.H.J.Epema, GrenchMark: A Framework for Analyzing, Testing, and Comparing Grids, In the 6th IEEE/ACM Int'l Symposium on Cluster Computing and the Grid (CCGrid'06) (accepted, 25%). An extended version can be found as Technical Report TU Delft/PDS/2005-007, ISBN 1387-2109).
article Article, PDF [510KB] | GrenchMark presentation, CCGrid'06 [PPT, 3MB]
info Using GrenchMark: simple and composite Grid jobs, replaying traces from the Parallel Workloads Archive, 10 use cases for analyzing, testing, and comparing common grid settings.
 
A. Iosup, J.Maassen, R.V.van Nieuwpoort, D.H.J.Epema, Synthetic Grid Workloads with Ibis, KOALA, and GrenchMark, in Proceedings of the CoreGRID Workshop on Integrated Research in Grid Computing, S. Gorlatch and M. Danelutto (Eds.), 27-30 November 2005, Pisa, Italy.
article Article, PDF [250KB]
info Integrating Koala, Ibis, and GrenchMark to generate synthetic grid workloads based on simple Grid jobs.
workshop Program Committee | Workshop Programme.
 


Presentations, talks
presenting our work...
On Grid Performance Evaluation using Synthetic Workloads, In the 12th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), held in conjunction with SIGMETRICS'06, Jun 26, 2006.
presentation Talk
 [PPT, 2.5MB]
 
GrenchMark: A Framework for Analyzing, Testing, and Comparing Grids, In the 6th IEEE/ACM Int'l Symposium on Cluster Computing and the Grid (CCGrid'06), May 19, 2006.
presentation Talk
 [PPT, 2.5MB]
 
GrenchMark: A Synthetic Workload Generator for the DAS, In Koala Days, Delft, NL, Feb 23, 2006.
presentation Talk
 [PPT, 2.5MB]
 
Synthetic Grid Workloads with Ibis, Koala, and GrenchMark, In CoreGRID Integrated Research in Grid Computing Workshop, Pisa, IT, Nov 24, 2005.
presentation Talk [PPT, 2MB]
 
GrenchMark's Second Demo, In CoreGRID WP6.1 Meeting, Delft, NL, Oct 20, 2005.
presentation Talk [PPT, 1.5MB] | demo Demo [PPT, 1.5MB]  
GrenchMark's First Demo, In TU Delft/PDS Group Meeting, Delft, NL, May 19, 2005.
demo Demo I [PPT, 1MB]
 

References
these studies have enabled us to work on this project
  1. F. Berman, A. Hey, and G. Fox. Grid Computing: Making The Global Infrastructure a Reality. Wiley Publishing House, 2003.
  2. H. Mohamed and D. Epema. Experiences with the koala co-allocating scheduler in multiclusters. In Proc. of the 5th IEEE/ACM Int'l Symp. on Cluster Computing and the GRID (CCGrid2005), May 2005.
  3. R. V. van Nieuwpoort, J. Maassen, G. Wrzesinska, R. Hofman, C. Jacobs, T. Kielmann, and H. E. Bal. Ibis: a flexible and effcient java-based grid programming environment. Concurrency & Computation: Practice & Experience., 17(7-8):1079--1107, June-July 2005.

 
     

Last modified: Tue, 29 November, 2005 7:46 AM
The newest version of this page can be found at: http://grenchmark.st.ewi.tudelft.nl/research.html
Copyright © 1998-2005 Alexandru Iosup. All Rights Reserved.
And the famous NedStat counter: