A Pilot Study to Evaluate Development Effort for High Performance Computing

View/ Open
Date
2004-04Author
Basili, Victor R.
Asgari, Sima
Hochstein, Lorin
Hollingsworth, Jeffrey K.
Shull, Forrest
Zelkowitz, Marvin V.
Citation
V. Basili, S. Asgari, J. Carver, L. Hochstein, J. Hollingsworth, F. Shull, and M. Zelkowitz, “A Pilot Study to Evaluate Development Effort for High Performance Computing,” University of Maryland, CS-TR-4588, April 2004.
Metadata
Show full item recordAbstract
The ability to write programs that execute efficiently on modern parallel computers has not been
fully studied. In a DARPA-sponsored project, we are looking at measuring the development time
for programs written for high performance computers (HPC). To attack this relatively novel
measurement problem, our goal is to initially measure such development time in student
programming to evaluate our own experimental protocols. Based on these results, we will
generate a set of feasible experimental methods that can then be applied with more confidence to
professional expert programmers.
This paper describes a first pilot study addressing those goals. We ran an observational study
with 15 students in a graduate level High Performance Computing class at the University of
Maryland. We collected data concerning development effort, developer activities and
chronology, and resulting code performance, for two programming assignments using different
HPC development approaches. While we did not find strong correlations between the expected
factors, the primary outputs of this study are a set of experimental lessons learned and 12 wellformed
hypotheses that will guard future study.