Skip to main content

A survey of controlled experiments in software engineering

TitleA survey of controlled experiments in software engineering
Author(s)D.I.K. Sjoberg and J.E. Hannay and O. Hansen and V.B. Kampenes and A. Karahasanovic and N.K. Liborg and A.C. Rekdal
DetailsArticle: 2005
AbstractThe classical method for identifying cause-effect relationships is to conduct controlled experiments. This paper reports upon the present state of how controlled experiments in software engineering are conducted and the extent to which relevant information is reported. Among the 5,453 scientific articles published in 12 leading software engineering journals and conferences in the decade from 1993 to 2002, 103 articles (1.9 percent) reported controlled experiments in which individuals or teams performed one or more software engineering tasks. This survey quantitatively characterizes the topics of the experiments and their subjects (number of subjects, students versus professionals, recruitment, and rewards for participation), tasks (type of task, duration, and type and size of application) and environments (location, development tools). Furthermore, the survey reports on how internal and external validity is addressed and the extent to which experiments are replicated. The gathered data reflects the relevance of software engineering experiments to industrial practice and the scientific maturity of software engineering research.
author={Sjoeberg, D.I.K. and Hannay, J.E. and Hansen, O. and Kampenes, V.B. and Karahasanovic, A. and Liborg, N.-K. and Rekdal, A.C.},
journal={IEEE Transactions on Software Engineering},
title={A survey of controlled experiments in software engineering},
TopicsApplication, Controlled Experiments, Mapping Study, Secondary Study, Software Engineering