Category: Testing

What Does a Good Tester Do?

Posted on by

Janet Gregory’s and Lisa Crispin’s book “Agile Testing”, which challenged the traditional view on testing sparked a discussion in the testing community that seemed to have lasted ever since the book was published. Before this discussion, we had the issue of the sequential approach to testing, which often resulted in the dreaded “testing crunch” (the development phase taking longer than planned and the testing phase taking the hit by having all testing being crammed into a very narrow time slot). And I bet that quite a few people have heard about testing zombies. If not, here’s a good intro. So, this is a good question, and I’ll try to answer it by listing good behaviors that I have witnessed throughout the years.

A Good Tester Learns the Domain

If anyone on the team should know the domain and business rules, it’s the tester. Not only does a tester need to make sure that the team’s product implements the business rules and does what it’s supposed to, the tester also has the most exposure to the domain and time to get to know it. Figuring out interesting test cases in complex domains requires both testing skills and deep domain knowledge, which is best acquired by studying the domain through books, blogs, or other media, and talking to the product people and customers. One of my best team onboarding experiences was that of being brought up to speed by the team’s tester. She knew everything about the intricacies of realtime trading from the domain point of view.

A Good Tester Keeps Track of Tools

In my mind a good tester knows quite a lot about tools. An average tester doesn’t need to, but a good one does. The good tester knows what testing frameworks the team employs, what they do, and what they don’t cover. Let’s take a simple example. If the team has its unit testing in order by using a unit testing framework with some test double library and has a number of end-to-end tests using WebDriver, the good tester will look into performance testing frameworks or suggests that test data be managed better, or that the team should look more into integration testing. The good tester is by no means an automation person, but wants to understand the tools used in the development process actually do and don’t. 

A Good Tester Leads the Team’s Exploratory Testing

Exploratory testing isn’t a random walk of the system’s features. It’s a rather systematic approach, especially if it involves the entire team. Therefore it needs to be planned and coordinated. If you’re familiar with James Whittaker’s book “Exploratory Testing”, you know that there are many approaches to testing your application. A good tester should know about those and take on the responsibility of ensuring that the team makes exploratory testing happen: What tours do we run? Which do we skip, or do later? How do we document the session? What do we do with the results?

A Good Tester Uses and Teaches Testing Techniques

I have yet to read a captivating and intriguing book on software testing. Still, these things actually do contain descriptions of testing techniques. At the end of the day, these constitute the core of the testing craft. Developer testing incorporates some of them: boundary value analysis and equivalence partitioning, truth tables, state diagrams, and pairwise testing. There’s more, though, and even the average testers should know them in their sleep, not to mention the good testers.

A Good Tester Acts as the Default Spokesperson for Quality

“Default spokesperson.” What does that even mean? It refers to the person who always takes the quality stance; in planning meetings, in architecture workshops, near the coffee machine. It’s the one who asks questions like:

  • How do we know that the users will like it?
  • How do we test this?
  • Have we thought of everything?
  • Will this new feature slow down the system?
  • Doesn’t this new feature clash with existing business rules?
  • Can this be automated, and how?
  • Won’t this change add to out technical debt?

A Good Tester Ensures that Everybody is a Quality Champion

I have never seen laying the burden of ensuring overall quality on one or a few persons work. In day to day development work having dedicated testers or test automation engineers, who are responsible for “testing” or “quality” (whatever those are, by the way), usually results in developers throwing work over the wall: the test automation engineer is supposed to “automate tests” and the tester is supposed to “test.” Nobody benefits from this in the long run. Knowing this, a good tester encourages and invites everybody to take responsibility for quality.

The Quality Champion

Posted on by

As developers take increasingly larger explicit responsibility for quality, many in the testing community have been wrestling with what the tester role should look like. With very few exceptions, it’s generally agreed upon that the tester who only executes scripted tests on a finished product is a threatened species. What many teams need is someone who is a spokesperson for quality from a broader perspective. Enter the tester 2.0—the Quality Champion.

Given the average agile team doing its devops, the Quality Champion must be a jack of many trades. I expect such a person to have knowledge in three major areas: testing, development (including devops), and product (ownership). And since every Quality Champion will be coming from a different background, we must forget about T-shapes and π-shapes. Now we’re talking amoeba shape!

The Quality Champion competency amoeba.

Please keep in mind that it’s really hard to be strong in aspects of quality-related work, hence the amoeba shape. The point is rather than there’s a lot of work for a Quality Champion, and that you can shoulder the role coming from different backgrounds. Here’s a list of some competencies that I think are vital for a Quality Champion. I don’t expect this list to be complete, but it should provide a solid starting point.

Testing

Quality Champions must have testing skills. They should be the team’s go-to-people when it comes to selecting testing techniques, deciding on a test strategy, and handling defects (ideally, all of which should be fixed immediately during the ongoing iteration, but what about the ones that won’t be or need more investigation or external decisions?)

Quality Champions should know what tools there are out there, especially when it comes to more specific uses. Obviously, the developers will choose the unit testing and mocking frameworks (nothing stops the Quality Champion from having knowledge and opinions here), but what about load testing tools? Which vendor do you pay if you want mobile testing in the cloud? Should the team explore testing based on image recognition or model-based testing?

While test data management is a programmatic activity in teams with heavy automation, how data is selected, and what data is relevant, is something the Quality Champion should be concerned with.

Speaking of automation, it’s not bad if the Quality Champion has an understanding of what kind of automation strategy the team should aim for.

Development

The Quality Champion can have a deep expertise in the tools and frameworks the developers are using: What are their capabilities, limitations, and what will the quality of the developers’ automated checking be?

On the must side, is the ability and willingness to coach developers in testing techniques: “How many unit tests does this state diagram translate into? Did you remember this boundary value and this heuristic? This business rule may require two bat wings, one snake tail, and a pinch of generative testing.”

Where we are tight now as an industry—many are running containerized micro services—requires the Quality Champion to understand, and fight for, light-weight virtualization and infrastructure as code as a means to achieve testability. Now that everybody can run the entire system on their machine, with whatever data they like, what expectations on testing and quality do we have? If the team isn’t there yet, what is the first step?

Product

It’s not uncommon for the Quality Champion to be the team member who has the most time to spend with the product owner and other stakeholders. This results in deep understanding of the domain and its business rules. Domain knowledge is therefore a reasonable expectation.

A team’s planning meeting, be it called Sprint planning or something else, is where the quality work begins. It’s imperative that the team thinks about acceptance criteria, test data, test infrastructure, and the overall approach to testing in the upcoming iteration in this very meeting. If the team lacks the habit to tackle these issues, it’s a must that the Quality Champion steps up here. On a related topic, it’s not unreasonable to expect that the Quality Champion is the one who pushes practices such as BDD or ATDD in the team.

Needless to say, the Quality Champion has the most experience in exploratory testing, but may also be quite familiar with usability testing, and general strategies for developing and validating a product.

* * *

This is a mouthful, but building potentially shippable product increments iteratively in a consistent manner is a non-trivial task, and so is ensuring that quality remains prevalent throughout the process.

In the testing parlor, there’s a term called “bug advocacy.” The Quality Champion should strive for quality advocacy.

By the way, did you notice that I capitalized the Qs and Cs in “quality champion” to get QC, the name of a beloved tool? I couldn’t resist 🙂