Skip to content

Maintaining software quality with defect detection

I’ve been reading Steve McConnell’s excellent “Code Complete 2” and wanted to share this section on defect detection rates.

Removal Step Lowest Rate Modal Rate Highest Rate
Informal design reviews 25% 35% 40%
Formal design inspections 45% 55% 65%
Informal code reviews 20% 25% 35%
Formal code inspections 45% 60% 70%
Modeling or prototyping 35% 65% 80%
Personal desk-checking of code 20% 40% 60%
Unit test 15% 30% 50%
New function (component) test 20% 30% 35%
Integration test 25% 35% 40%
Regression test 15% 25% 30%
System test 25% 40% 55%
Low-volume beta test (< 10 sites) 25% 35% 40%
High-volume beta test (> 1000 sites) 60% 75% 85%

The most interesting facts that this data reveals is that the modal rates don’t rise above 75 percent for any single technique. Moreover, for the most common kinds of defect detection – unit testing and integration testing – the modal rates are only 30-35 percent.

Whilst important, unit tests aren’t the single ultimate aim, and can only ensure software quality when used in conjunction with other methods of defect detection. It seems obvious that more scrutiny and testing will result in better software quality, but it is very interesting to understand the effectiveness. The difficulty comes in knowing how much to invest and where to get the best return.

Chris Bell

Chris Bell is a software engineer based in Leeds, UK. Enjoy his personal accounts and insights into the weird and wacky world of software development.

Twitter LinkedIn Google+ 

Published inSoftware qualityTesting

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *