Observation: How it has helped me as a tester?
In one of the chapters of her book, "Explore IT," Elizabeth Hendrickson discusses observation and how it is crucial for exploratory testing. Checking logs for surprises is one technique to observe during testing. I was brought back to my days as a mid-level test engineer while reading it.
On the user's side, the product appeared to be relatively straightforward, but it actually had a sophisticated architecture and a lot of moving components. For this product, there were very few user interactions; everything took place in the background, and logs were essential for troubleshooting.
When I first discovered a bug, I would gather proof—reproducing steps, error logs—most often grep 'error' log files—and screenshots—and be prepared to report the issue. My manager would ping me with questions like, "Did you try to reproduce the bug in another test combination?" and "Did you check that?" and “Did you try changing data?” etc. I was like 🙄, “I already have all the data for the bug, why can't the developer find the rest?”. I would return to my seat and attempt to reproduce it according to my manager's instructions. It would have taken at least an hour, but I have a solid case of many combinations of where it worked and in which context. This time, the manager would approve my bug report. Here, when an issue occurred, I learned what would go wrong, where to explore further, how to isolate, and where to search for more problems that might be developing.
After some time, as my reports improved, my manager now requests that I investigate other logs on related systems (tail -f xxx.log) . He demonstrated for me what to look for by starting a user activity and monitoring all logs. I was able to better understand the underlying process as a result, and I also began honing my observational skills.
The logs contain a wealth of data, including the timestamp, the start of each event, its progression, its conclusion, any calculations, transfers, receiving, handing off to the next system, etc. What I used to do—and still do—was running a simple test case and store all the logs as individual files on my own PC. When a test later fails, I will compare the logs to those in my stored files and provide more details to the issue report.
Another thing I would do is pick any fixed problems for verification when I finish my testing. I always chose the bugs that my manager had filed since they would contain a lot of information and help me enhance my observational abilities.
I eventually developed the ability to follow logs, knew the next event that would occur, and so forth. By looking at the logs, I found bugs for missing events, skipped processes, events that happened in the wrong sequence, etc.
This was a huge time saver for developers while they fixed the bugs.
According to Cem Kaner in,Testing Computer Software, “The best tester isn’t the one who finds the most bugs or who embarrasses the most programmers.The best tester is the one who gets the most bugs fixed”.
Comments
Post a Comment