Skip to main content

A systematic review of effect size in software engineering experiments

TitleA systematic review of effect size in software engineering experiments
Author(s)V.B. Kampenes and T. Dyba and J.E. Hannay and D.I.K. Sjoberg
DetailsInformation & Software Technology 49(11-12): 2007
AbstractAn effect size quantifies the effects of an experimental treatment. Conclusions drawn from hypothesis testing results might be erroneous if effect sizes are not judged in addition to statistical significance. This paper reports a systematic review of 92 controlled experiments published in twelve major software engineering journals and conference proceedings in the decade 1993-2002. The review investigates the practice of effect size reporting, summarizes standardized effect sizes detected in the experiments, discusses the results and gives advice for improvements. Standardized and/or unstandardized effect sizes were reported in 29% of the experiments. Interpretations of the effect sizes in terms of practical importance were not discussed beyond references to standard conventions. The standardized effect sizes computed from the reviewed experiments were equal to observations in psychology studies and slightly larger than standard conventions in behavioural science.
DOIhttp://dx.doi.org/10.1016/j.infsof.2007.02.015
BibTex@article{kampenes-2007,
address = {Newton, MA, USA},
author = {Vigdis By Kampenes and Tore Dyb{\aa} and Jo E. Hannay and Dag I.~K. Sj{\o}berg},
doi = {10.1016/j.infsof.2007.02.015},
issn = {0950-5849},
journal = {Information and Software Technology},
keywords = {systematic review, effect size},
number = {11-12},
owner = {lma},
pages = {1073–1086},
publisher = {Butterworth-Heinemann},
timestamp = {2012.11.03},
title = {Systematic review: A systematic review of effect size in software engineering experiments},
url = {http://dx.doi.org/10.1016/j.infsof.2007.02.015},
volume = {49},
year = {2007},
bdsk-file-1 = {YnBsaXN0MDDSAQIDBFxyZWxhdGl2ZVBhdGhZYWxpYXNEYXRhXxB9Li4vLi4vLi4vRG9jdW1lbnRzL0JBS0xpYnJhcnkvQnlLYW1wZW5lcyAyMDA3IElTVCBBIHN5c3RlbWF0aWMgcmV2aWV3IG9mIGVmZmVjdCBzaXplIGluIHNvZnR3YXJlIGVuZ2luZWVyaW5nIGV4cGVyaW1lbnRzIC5wZGZPEQK0AAAAAAK0AAIAAAxNYWNpbnRvc2ggSEQAAAAAAAAAAAAAAAAAAADOl7nTSCsAAAAklm8fQnlLYW1wZW5lcyAyMDA3IElTVCAjMTNFNkM3LnBkZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABPmx8VMkhIAAAAAAAAAAAADAAMAAAkgAAAAAAAAAAAAAAAAAAAACkJBS0xpYnJhcnkAEAAIAADOl6vDAAAAEQAIAADFTIQCAAAAAQAQACSWbwAI+i4ACPotAAJkjgACAF1NYWNpbnRvc2ggSEQ6VXNlcnM6AGJhcmJhcmFraXRjaGVuaGFtOgBEb2N1bWVudHM6AEJBS0xpYnJhcnk6AEJ5S2FtcGVuZXMgMjAwNyBJU1QgIzEzRTZDNy5wZGYAAA4AwABfAEIAeQBLAGEAbQBwAGUAbgBlAHMAIAAyADAAMAA3ACAASQBTAFQAIABBACAAcwB5AHMAdABlAG0AYQB0AGkAYwAgAHIAZQB2AGkAZQB3ACAAbwBmACAAZQBmAGYAZQBjAHQAIABzAGkAegBlACAAaQBuACAAcwBvAGYAdAB3AGEAcgBlACAAZQBuAGcAaQBuAGUAZQByAGkAbgBnACAAZQB4AHAAZQByAGkAbQBlAG4AdABzACAALgBwAGQAZgAPABoADABNAGEAYwBpAG4AdABvAHMAaAAgAEgARAASAIxVc2Vycy9iYXJiYXJha2l0Y2hlbmhhbS9Eb2N1bWVudHMvQkFLTGlicmFyeS9CeUthbXBlbmVzIDIwMDcgSVNUIEEgc3lzdGVtYXRpYyByZXZpZXcgb2YgZWZmZWN0IHNpemUgaW4gc29mdHdhcmUgZW5naW5lZXJpbmcgZXhwZXJpbWVudHMgLnBkZgATAAEvAAAVAAIAGP//AAAACAANABoAJACkAAAAAAAAAgEAAAAAAAAABQAAAAAAAAAAAAAAAAAAA1w=}}
TopicsApplication, Effect Size, Mapping Study, Secondary Study, Software Engineering