I haven’t added a new post here recently. One of the things that has been keeping me busy is putting together some interviews on software testing. Here are excerpts from 3 of the interviews:
Testing Smarter with Alan Page
Hexawise: During your 20 years at Microsoft you have been involved in hiring many software testers. What do you look for when choosing software testers? What suggestions do you have for those looking to advance in their in software testing career?
Alan: The biggest thing I look for in testers is a passion and ability to learn. I’ve interviewed hundreds of testers, including many who came from top universities with advanced degrees who just weren’t excited about learning. For them, maybe learning was a means to an end, but not something they were passionate about.
The testers who really impress me are those who love to learn – not just about testing, but about many different things. Critical thinking and problem solving are also quite important.
As far as suggestions go, keep building your tool box. As long as you’re willing to try new things, you’ll always be able to find challenging, fun work. As soon as you think you know it all, you will be stuck in your career.
…
Hexawise: It seems to me that testing games would have significant challenges not found in testing fairly straightforward business applications. Could you share some strategies for coping with those challenges?Alan: Combinatorial testing is actually pretty useful in game testing. For example, consider a role-playing game with six races, ten character classes, four different factions, plus a choice for gender. That’s 480 unique combinations to test! Fortunately, this has been proven to be an area where isolating pairs (or triples) of variations makes testing possible, while still finding critical bugs.
Beyond that, testing games requires a lot of human eyeballs and critical thinking to ensure gameplay makes sense, objects are in the right places, etc. I’ve never seen a case where automating gameplay, for example, has been successful. I have, however, seen some really innovative tools written by testers to help make game testing much easier, and much more effective.
Testing Smarter with Dorothy Graham
Hexawise: When looking to automate tests one thing people sometimes overlook is that given the new process it may well be wise to add more tests than existed before. If all test cases were manually completed that list of cases would naturally have been limited by the cost of repeatedly manually checking so many test cases in regression testing. If you automate the tests it may well be wise to expand the breadth of variations in order to catch bugs caused by the interaction of various parameter values. What are your thoughts on this idea?
Dot: Nice analogy – I like the term “grapefruit juice bugs”. Using some of the combinatorial techniques is a good way to cover more combinations in a reasonably rigorous way. Automation can help to run more tests (provided that expected results are available for them) and may be a good way to implement the combination tests, using pair-wise and/or orthogonal arrays.
Smarter Software Testing with James Bach
Hexawise: How do/would you test very complex systems such as genetic algorithm systems and evolutionary systems? How do you test systems when we don’t understand how they work? It seems kind of like medical differential diagnosis: poke, observe, learn, hypothesize, poke again. Or is there a better way?
James: I test them using social science methods. That, after all, is how scientists attempt to test their theories about social life. That means an emphasis on qualitative analysis, but bringing in statistical methods whenever applicable.
I agree that the medical world is a good example of where statistical methods and heuristic approaches are also needed. In testing complex things, some of what you need to do includes:
- You must use time to your advantage– observing systems over time the way primatologists observe chimps in the wild.
- You must use Grounded Theory, beginning with immersion and observation, until patterns begin to reveal themselves.
- You must focus on testability. To create an environment where you can control and observe more of what is there.
- You must pay attention to clues. Many, many clues. Stop looking for simplistic “test cases” that will “prove” that the software works.
- You must become expert at data wrangling, since these systems usually involve huge amounts of data.
- Let other people help you.
- Forge partnerships with users.
If you want to follow my software testing interviews you can subscribe to the Hexawise software testing blog RSS feed.
Related: Software Testers Are Test Pilots – Software Testing and the Impact on Quality – Combinatorial Software Testing – The Quadrant of Massive Efficiency Gains – Improving Software Development with Automated Tests
Alan Page’s interview had another quote that I think many people (in any industry) would be wise to learn from:
Hexawise: How do you stay current on improvements in software testing practices; or how would you suggest testers stay current?
Alan: I read a lot of testing blog posts (I use feedly for aggregation). I subscribe to at least fifty software development and test related blogs. I skim and discard liberally, but I usually find an idea or two every week that encourages me to dig deeper.
Biggest tip I have, however, is to know how essential learning is to your success. Learning is as important to the success of a knowledge worker as food is to human life. Anyone who thinks they’re an “expert” and that learning isn’t important anymore is someone on a career death spiral.
RSS feed readers are an essential tool for knowledge workers. Subscribe to our RSS feed (Curious Cat Management Improvement blog). I also offer an RSS feed from my JohnHunter.com site: this feed will be updated with content from the various sources I publish to (I am a bit behind in getting this updated but it is something I will be working on the next few months).
