What accuracy statistics really measure

Simple item page
aut.researcherMacDonell, Stephen Gerard
dc.contributor.authorKitchenham, B
dc.contributor.authorPickard, L
dc.contributor.authorMacDonell, SG
dc.contributor.authorShepperd, MJ
dc.date.accessioned2011-10-01T06:45:42Z
dc.date.available2011-10-01T06:45:42Z
dc.date.copyright2001-06-01
dc.date.issued2001-06-01
dc.description.abstractProvides the software estimation research community with a better understanding of the meaning of, and relationship between, two statistics that are often used to assess the accuracy of predictive models: the mean magnitude relative error (MMRE) and the number of predictions within 25% of the actual, pred(25). It is demonstrated that MMRE and pred(25) are, respectively, measures of the spread and the kurtosis of the variable z, where z=estimate/actual. Thus, z is considered to be a measure of accuracy, and statistics such as MMRE and pred(25) to be measures of properties of the distribution of z. It is suggested that measures of the central location and skewness of z, as well as measures of spread and kurtosis, are necessary. Furthermore, since the distribution of z is non-normal, non-parametric measures of these properties may be needed. For this reason, box-plots of z are useful alternatives to simple summary metrics. It is also noted that the simple residuals are better behaved than the z variable, and could also be used as the basis for comparing prediction systems
dc.identifier.citationIEE Proceedings: Software, vol.148(3), pp.81-85
dc.identifier.doi10.1049/ip-sen:20010506
dc.identifier.issn1462-5970
dc.identifier.urihttps://hdl.handle.net/10292/2179
dc.publisherIEEE
dc.relation.urihttp://dx.doi.org/10.1049/ip-sen:20010506
dc.rightsCopyright © 2001 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
dc.rights.accessrightsOpenAccess
dc.subjectAccuracy measures
dc.subjectAccuracy statistics
dc.subjectBox-plots
dc.subjectCentral location measure
dc.subjectKurtosis measure
dc.subjectMean magnitude relative error
dc.subjectNonnormal distribution
dc.subjectNonparametric measures
dc.subjectPrediction number
dc.subjectPrediction systems comparison
dc.subjectPredictive models
dc.subjectResiduals
dc.subjectSkewness measure
dc.subjectSoftware estimation
dc.subjectSpread measure
dc.subjectStatistical distribution properties
dc.subjectSummary metrics
dc.titleWhat accuracy statistics really measure
dc.typeJournal Article
pubs.organisational-data/AUT
pubs.organisational-data/AUT/Design & Creative Technologies
pubs.organisational-data/AUT/PBRF Researchers
pubs.organisational-data/AUT/PBRF Researchers/Design & Creative Technologies PBRF Researchers
pubs.organisational-data/AUT/PBRF Researchers/Design & Creative Technologies PBRF Researchers/DCT C & M Computing
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
Kitchenham, Pickard, MacDonell and Shepperd (2001) IEEProc-S.pdf
Size:
86.04 KB
Format:
Adobe Portable Document Format
Description:
Download
License bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
licence.htm
Size:
29.98 KB
Format:
Unknown data format
Description:
Download
Collections
SERL - Software Engineering Research Laboratory