One of the goals of the software engineering industry is to produce high-quality defect-free software. Software defects get introduced if a programmer makes mistakes in their assumptions. To reduce the possibility of a software defect, programmers should follow best practices such as code reviews, manual testing, automated testing, and canary testing.
Code Reviews
Programmers conduct code reviews to ensure best practices are followed. Code reviews are useful primarily to ensure good coding patterns are followed. Good coding patterns include defensive code to handle invalid input, separation of logic, meaningful variable names, and using proper libraries. Most importantly, code reviews allows a programmer to check assumptions made by other programmers. Contrary to popular belief, code reviews do not reliably find software defects.
Testing
Most software defects are found by testing. Testing runs the software in a controlled environment. Different inputs are provided to the software and checked against expected output. The software should also be checked against invalid input to ensure it gracefully fails. Testing these input and outputs can be performed by manual or automated testing.
Manual Testing
Manual testing only verifies that the software works at a specific instance in time; after the next code change is made, the tests will have to be performed again. Given that software is changed several times a month, if not every day, significant time and effort is required for manual testing.
Automated Testing
Automated testing ensures that the software continues to work after any code change. Automated testing can be performed by unit tests or integration tests. Unit tests check the smallest chunks of code and integrations tests check entire end-to-end systems. Although there is a significant time to write automated tests initially, it saves time in the long run by making new feature development more stable without breaking existing code.
Canary Testing
A specific type of test is a canary test, which has a higher degree of confidence than automated testing. The software is tested against a copy of real-world inputs. An engineer’s set of test inputs might have missed some assumptions of real-world inputs, which will be caught in this type of testing. For example, canary testing can test algorithm speed and memory usage. The limitations to canary testing are computational resources and company approval to access customer data.