A systematic review of effect size in software engineering experiments
Title | A systematic review of effect size in software engineering experiments |
---|---|
Author(s) | V.B. Kampenes and T. Dyba and J.E. Hannay and D.I.K. Sjoberg |
Details | Information & Software Technology 49(11-12): 2007 |
Abstract | An effect size quantifies the effects of an experimental treatment. Conclusions drawn from hypothesis testing results might be erroneous if effect sizes are not judged in addition to statistical significance. This paper reports a systematic review of 92 controlled experiments published in twelve major software engineering journals and conference proceedings in the decade 1993-2002. The review investigates the practice of effect size reporting, summarizes standardized effect sizes detected in the experiments, discusses the results and gives advice for improvements. Standardized and/or unstandardized effect sizes were reported in 29% of the experiments. Interpretations of the effect sizes in terms of practical importance were not discussed beyond references to standard conventions. The standardized effect sizes computed from the reviewed experiments were equal to observations in psychology studies and slightly larger than standard conventions in behavioural science. |
DOI | http://dx.doi.org/10.1016/j.infsof.2007.02.015 |
BibTex | @article{kampenes-2007, address = {Newton, MA, USA}, author = {Vigdis By Kampenes and Tore Dyb{\aa} and Jo E. Hannay and Dag I.~K. Sj{\o}berg}, doi = {10.1016/j.infsof.2007.02.015}, issn = {0950-5849}, journal = {Information and Software Technology}, keywords = {systematic review, effect size}, number = {11-12}, owner = {lma}, pages = {1073–1086}, publisher = {Butterworth-Heinemann}, timestamp = {2012.11.03}, title = {Systematic review: A systematic review of effect size in software engineering experiments}, url = {http://dx.doi.org/10.1016/j.infsof.2007.02.015}, volume = {49}, year = {2007}, bdsk-file-1 = {YnBsaXN0MDDSAQIDBFxyZWxhdGl2ZVBhdGhZYWxpYXNEYXRhXxB9Li4vLi4vLi4vRG9jdW1lbnRzL0JBS0xpYnJhcnkvQnlLYW1wZW5lcyAyMDA3IElTVCBBIHN5c3RlbWF0aWMgcmV2aWV3IG9mIGVmZmVjdCBzaXplIGluIHNvZnR3YXJlIGVuZ2luZWVyaW5nIGV4cGVyaW1lbnRzIC5wZGZPEQK0AAAAAAK0AAIAAAxNYWNpbnRvc2ggSEQAAAAAAAAAAAAAAAAAAADOl7nTSCsAAAAklm8fQnlLYW1wZW5lcyAyMDA3IElTVCAjMTNFNkM3LnBkZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABPmx8VMkhIAAAAAAAAAAAADAAMAAAkgAAAAAAAAAAAAAAAAAAAACkJBS0xpYnJhcnkAEAAIAADOl6vDAAAAEQAIAADFTIQCAAAAAQAQACSWbwAI+i4ACPotAAJkjgACAF1NYWNpbnRvc2ggSEQ6VXNlcnM6AGJhcmJhcmFraXRjaGVuaGFtOgBEb2N1bWVudHM6AEJBS0xpYnJhcnk6AEJ5S2FtcGVuZXMgMjAwNyBJU1QgIzEzRTZDNy5wZGYAAA4AwABfAEIAeQBLAGEAbQBwAGUAbgBlAHMAIAAyADAAMAA3ACAASQBTAFQAIABBACAAcwB5AHMAdABlAG0AYQB0AGkAYwAgAHIAZQB2AGkAZQB3ACAAbwBmACAAZQBmAGYAZQBjAHQAIABzAGkAegBlACAAaQBuACAAcwBvAGYAdAB3AGEAcgBlACAAZQBuAGcAaQBuAGUAZQByAGkAbgBnACAAZQB4AHAAZQByAGkAbQBlAG4AdABzACAALgBwAGQAZgAPABoADABNAGEAYwBpAG4AdABvAHMAaAAgAEgARAASAIxVc2Vycy9iYXJiYXJha2l0Y2hlbmhhbS9Eb2N1bWVudHMvQkFLTGlicmFyeS9CeUthbXBlbmVzIDIwMDcgSVNUIEEgc3lzdGVtYXRpYyByZXZpZXcgb2YgZWZmZWN0IHNpemUgaW4gc29mdHdhcmUgZW5naW5lZXJpbmcgZXhwZXJpbWVudHMgLnBkZgATAAEvAAAVAAIAGP//AAAACAANABoAJACkAAAAAAAAAgEAAAAAAAAABQAAAAAAAAAAAAAAAAAAA1w=}} |
Topics | Application, Effect Size, Mapping Study, Secondary Study, Software Engineering |