NOT – Not Only Testing

How do you create a quality assurance strategy? 

Is quality assurance the same as testing?

Companies nowadays do not have a separate quality assurance(QA) team. People coming from the old world find this difficult to digest. In the last decade, the way of developing, deploying, and monitoring applications has changed.

Gone are the days of the waterfall model of development. You no longer take months to develop a feature, followed by a long testing cycle and finally hand it over to the production team for deployment. Release cycles have shrunk from months to hours. People who build the application are responsible for deploying, running, and monitoring it.

In this brave new world, base your quality assurance strategy on:

1. What is the cost of a bug in production?
2. How quickly can one detect a bug in production?
3. How quickly can one fix a production bug?
4. If a bug manifests in production, how does one limit the damage?

The cost of bugs is not uniform; bugs in different areas of the application cost differently. The line of business also dictates the price of bugs.

For an aerospace or medical devices company, a bug means the difference between life and death literally, not so much for a social network or an e-commerce website. If an e-commerce website does not load, it significantly hits the company’s bottom line. For a social network, it might make an insignificant dent in advertising revenue. For an e-commerce company, the cost of a bug in the recently purchased items section on the home page is not the same as checkout from the shopping cart not working. While your shopping cart checkout has to be bulletproof, you can be lenient with the recently purchased items section on the home page.

Vary quality control based on the criticality of the functionality. Be tight where it needs to be and loose where you have room to wiggle. You need not have a uniform quality assurance strategy for the entire application.

Testing dents your speed and time to production irrespective of whether you do it manually or automate it; this is the cost you pay for quality – you need to make peace with this. The stricter you get, the costlier it becomes. Automated testing is not free; it too has a cost – more things to maintain and manage. In some cases, you might end up writing twice the code due to automated testing. I am not arguing against automated testing but asking you to – factor in and mentally accept the cost – before you take the plunge.

There is a tendency to equate testing with quality assurance. Testing is a subset of quality assurance; testing is NOT quality assurance. Quality assurance is much more than JUST testing and comprises of a variety of things.

Today, experimentation, incremental development, and speed are the essence. Try a small idea, see whether it sticks, and then rapidly iterate and expand. In such an environment, following a design philosophy that gives your room for error and factors in bugs and things not working is key.

Quick detection of production bugs rests on the observability built into the application. Speedy recovery from production bugs is a function of deployment practices. Reducing the impact of production bugs follows how you roll out features. All these are a result of tooling, development practices, and engineering culture. Today, these are as important as vanilla testing – manual or automated.

The ease with which you can set up a development environment for the application has a direct bearing on the quality of the product. In a world of micro-services and external dependencies, setting up a development environment can get complicated with a lot of moving parts. If you make it difficult for developers to create a robust development environment, the quality of the product takes a hit.

Static code analysis and enforcing best practices through tooling like pre-commit hooks improve the quality of the code, which directly improves the quality of the application. It is paramount when you use languages that are promiscuous with typing; this is one of those things where the cost is low, resistance is nil, and the rewards are high. Be always on the lookout for tools and processes where you get a better quality product with zero resistance and impediment to speed.

Adopting the asynchronous and event-based architecture and design patterns gives room for error and recovery. In today’s fast-paced environment where you do not have the luxury of time to test every minute aspect of the product, this is a boon.

Do not have a tunnel vision when it comes to quality – do not restrict it to only testing; adopt the “NOT – Not Only Testing” strategy.

Comic from XKCD.

Photo by Austin Neill on Unsplash