Tag Archives: inspection

Software Testing and the Impact on Quality

My response to a question on Reddit.

“Software quality does not come from testing”
Does anybody have any thoughts on the validity of the above statement?

That statement is similar to the idea you can’t inspect in quality. Basically “Inspection is too late. The quality, good or bad, is already in the product.” W. Edwards Deming

I agree with those ideas. Software testing is a bit different (at least some of it is) from the inspection mentioned above. You are testing while the product is being developed and adjustments are being made before the product is released to customers. Also with internet based software you have the ability to update the software and now all users have that update. Where for physical devices they already have the product and the only option is a recall which is very expensive and often ignored.

Software testing however should pay attention to those points in the 2 links above (defects should be understood as evidence of a process that needs to be improved so defects are not built in the first place). What you want is not just to fix the bugs software testers catch but figure out the reasons those bugs were created and improve you process so you create fewer bugs in the future.

No matter what the software quality is based on the code that is written. At the best software testing can tell people about the bugs but unless the code is fixed the software quality didn’t change. But to say that software testing doesn’t have a big influence of software quality when testing is well done and the software development process is good (listens to feedback and improves) is not very accurate.

Baking in Quality to Software Development

One of the reasons my organizations switched to Ruby on Rails for software development was the great integration with automated testing. We always wanted to have good test coverage on our software applications (which are web applications – some used only inside our organization) but didn’t actually find the time to do so. Since we adopted Ruby we have been doing much better in this regard. It isn’t just the switch to Ruby, of course, but the switch to Ruby coincided with the beginning of many improvements to our software development practices that have continually improved over the last couple of years.

Here is a post on How to build quality software by an agile, Ruby, lean software developer

I’ve mentioned previously about baking-in quality and not having developers throw code over a wall to testers.
…
Everyone on the team is concerned with not only assuring quality in what we deliver, but making it visible to ourselves and the business.

We work in an agile manner, iterating through development with extreme programming practices and Behaviour Driven Development. Facilitating our relationship with the business is Scrum and we utilise kanban principles and systems thinking to maintain a speedy throughput of high-quality work. This mixture allows us to communicate effectively, develop the correct features properly and continuously deploy our work when it is complete, thus maximising business value. I should also mention that we are fortunate enough to have our business people/customer sat across from us.
…
Without testers or a QA team there is no wall over which work can be thrown and the responsibility for quality absolved.
…
The inspection typically carried out end-of-cycle only yields bugs that were low severity and of no real impact to the end user.
…
An agile testing must-have, we use TeamCity to continuously run our unit tests on each check-in. We also execute our Cucumber acceptance tests on scheduled runs. The status of the builds are visible on dedicated monitors around the office as well as a nice 6”² projected screen.

via: @benjaminm

Customers Get Dissed and Tell

There are those rare companies where interacting with them is not a dreaded experience: Trader Joe’s, Southwest Airlines, Ritz Carlton, Crutchfield, Cannon, Groovix. There are not many. And even just providing something that just works is seen as a treat. The all too common dis-service, combined with the internet, leads to Consumer Vigilantes:

a growing disconnect between the experience companies promise and customers’ perceptions of what they actually get.
…
A swell of corporate distrust – exacerbated by high executive pay, accounting lapses, and the offshoring of jobs – has people feeling more at odds with companies than ever before.
…
Years of dialing the call center for a technician yielded at least eight missed appointments by Comcast, he says, but a post on ComcastMustDie brought a phone call the next morning and, later, a lead technician who showed up on time. Now, Salup says: “Anytime I have a problem, I also post it on the blog.”

Pretty lousy systems thinking (or really failures to think systemically). Pay executives obscenely and cut service until customers literally can’t stand you so much they don’t just want to avoid you they want you out of business.

And then instead of fixing the system, just burn the toast (follow the link for an explanation). Then wait for those that get the burnt toast to tell everyone that you sold them burnt toast. Then, after they do that, go scrape it for them. This is not what Dr. Deming meant when he encouraged companies to eliminate the need to inspect for quality. Of course you know that (you are reading this blog after all). Maybe the business schools decided to cut down Deming’s ideas to just eliminating inspection and a couple other sound bites. And then tell the MBA’s not to bother reading all the rest of that… we have to get on to the cost reduction strategies that will make sure you move into the c-level and get the real money.

Most customers, of course, don’t have the time or energy to go that far in their service insurgencies. They want an apology, a human being who answers the phone, or simply some bottled water after a few hours sitting on the airport tarmac

But some companies just push people so far they have to let people know about how poorly they have been treated. Some past posts highlight the frustrating experiences bloggers, including me, share about how badly we have been treated: Ritz Carlton (good) and Home Depot (bad) – Incredibly Bad Customer Service from Discover Card – More Bad Customer Service Examples – Poor Service, an Industry Standard? (HP) – Comcast HD DVR Is Simply, Terribly Awful

Consumerist, is a great site, doing what it can to counter some of the horrible service.

Cease Mass Inspection for Quality

Comment in response to, Re-Discovering W. Edwards Deming, a partial quote from that post:

Not all of the Deming approach is part of core TPS thinking. In particular, Deming advocated a statistical sampling approach to quality inspection, while Toyota focuses on 100% inspection or eliminating the need for inspection through via the concepts of Poka Yoke and Jidoka. As much as I admire Deming and his philosophy, I agree with the Toyota innovation that it is better to prevent defects from occurring, or at least preventing defects from reaching the customer.

Thanks for your continually interesting blog. I think some might read this post and be confused about what Deming thought about sampling and inspection.

Deming’s point 3 is “Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.” (Out of the Crisis, 1982). I think Toyota’s improvement of the system to build quality into the product is exactly what Deming had it mind.

Deming believed in improving the process, and doing so using process measures (which often may involve sampling) to guide improvement efforts. He did not believe in using inspection to select out the bad products, which is what inspection largely was before Deming.

He also talked about inspection of incoming material from suppliers – see Chapter 15 of Out of the Crisis.

He also did a great deal of work with sampling to improve population estimates for the US Census Bureau and others as well as on surveys and the sampling involved in surveys.

More on Deming’s thought on Inspection