In my humble opinion, the main purpose of testing can be summed up as…
“Help your team make an informed opinion about the perceived quality of the products and features that they own.”
That’s it. That’s all.
I like to call it: “Perceived Quality Assessment”. I will be briefly touching on ideas and techniques that can allow us to help our teams do this better. Before I continue further, I want to state that the notion of the ‘perception of quality’ is not new, nor is it my idea. There are many thought leaders who have written about this before, like Gerald Weinberg for example. This post is not to take credit for the concept of perceived quality, but my own take on it, and how we can help.
Before I get to my list, let’s first talk about perceived quality. A team can never really know the quality of their products or how they will be perceived in the marketplace. They can know things like: the number of open defects, the missed requirements, the poorly written / thought out designs, or early feedback from demos. Our testing should be helping to provide our teams with as much information as we can, to make an informed opinion on their own perceived view of quality. This perception should help teams decide things like: when to release / not release, when to enhance, when to bug fix, when to scale back, when to move on, etc.
The following, is a brief list of items that can help us provide this information to our teams. I am only including 5 items in this list for the sake of brevity. There are many more, but I like these 5 as starters🙂
- Test Coverage – specifically what is not being tested
- Non functional testing like ‘quality attributes’
- Testing how customers and various personas use your features
Let’s talk unknowns. Testing should always be trying to uncover unknowns. This does not necessarily mean bugs. This can be anything we never thought of, may have overlooked, did not know about in the first place, or that may have dire consequences. This is all just information that your team needs to know about. Look hard, explore and find it. Your team needs you for this!
Let’s talk risk. Risk can manifest itself in many ways. As the aforementioned – unknowns for example. It can also be lurking in new bug fixes or enhancements that were added, other teams work that may impact yours, problems with integrating with other products or features, or really anything that has changed. Testing should always be monitoring these, investigating and exploring. This is something automation can do to a certain extent (by checking known conditions), but a good skilled tester can uncover those risky areas that automation does not know exist.
Let’s talk test coverage. One of my favorite topics that not many really understand that well. Without knowing what your testing (and automation) is covering and more specifically is not covering, is to me incredibly important information that your team needs to know. It’s great to have lots of testing in place, but your team needs to understand what that testing is all about and what is not being tested. This can roll up to the previous two topics I mentioned: unknowns and risk. Without understanding this, can a team really have an educated perceived quality assessment? I think not.
Let’s talk non-functional testing. Not that I am a fan of this terminology necessarily, but since it widely used I am using it here. A tester should always be testing to uncover potential problems with quality attributes that matter. These are also sometimes referred to as the ‘ilities’ of testing. These are critical in many customers minds when basing their perception of a software’s quality. Customers typically do not care that the requirements were met, that the acceptance criteria passed or that test cases have all been executed successfully. The perception of the quality of software can seriously be impacted by things like security, performance, accessibility, usability, and the like. Test for these. Your teams need to know this information.
Let’s talk how customers use software. Since software can be highly configurable, we need testing to flush out potential problems that may occur when using it in differing or in some cases, unintended ways. Our testing should be factoring these in. Seek out information on how your customers use your products and the various personas involved. Test accordingly. Your teams need this information too.
Take a moment and think of the value you are providing with your testing. Are you really helping your team make informed decisions? Are you testing the right things, at the right time? Are you a lighthouse, shining light so your teams boat can avoid obstacles and problems on its way to its destination? Let’s hope you are!
Thanks for reading. I hope something resonated with you. If anyone agrees or disagrees with anything I wrote, feedback is always welcomed.
Test well my friends!