Tag Archives: Public Sector

Combinatorial Testing for Software

Combinatorial testing of software is very similar to the design of experiments work my father was involved in, and which I have a special interest in. Combinatorial testing looks at binary interaction effects (success or failure), since it is seeking to find bugs in software, while design of experiments captures the magnitude of interaction effects on performance. In the last several years my brother, Justin Hunter, has been working on using combinatorial testing to improve software development practices. He visited me this week and we discussed the potential value of increasing the adoption of combinatorial testing, which is similar to the value of increasing the adoption of the use of design of experiments: both offer great opportunities for large improvements in current practices.

Automated Combinatorial Testing for Software

Software developers frequently encounter failures that occur only as the result of an interaction between two components. Testers often use pairwise testing – all pairs of parameter values – to detect such interactions. Combinatorial testing beyond pairwise is rarely used because good algorithms for higher strength combinations (e.g., 4-way or more) have not been available, but empirical evidence shows that some errors are triggered only by the interaction of three, four, or more parameters

Practical Combinatorial Testing: Beyond Pairwise by Rick Kuhn, US National Institute of Standards and Technology; Yu Lei, University of Texas, Arlington; and Raghu Kacker, US National Institute of Standards and Technology.

the detection rate increased rapidly with interaction strength. Within the NASA database application, for example, 67 percent of the failures were triggered by only a single parameter value, 93 percent by two-way combinations, and 98 percent by three-way combinations.2 The detection-rate curves for the other applications studied are similar, reaching 100 percent detection with four- to six-way interactions.
These results are not conclusive, but they suggest that the degree of interaction involved in faults is relatively low, even though pairwise testing is insufficient. Testing all four- to six-way combinations might therefore provide reasonably high assurance.

Related: Future Directions for Agile ManagementThe Defect Black MarketMetrics and Software DevelopmentFull and Fractional Factorial Test DesignGoogle Website Optimizer

Lean Thinking in Minnesota Government

The Office of Governor Pawlenty issued a press release on Minnesota’s Drive to Excellence effort:

Increased Quality via Lean Continuous Improvement Reform – The State has adopted
“Lean Thinking” as its preferred process improvement tool. So far, 18 agencies are actively
involved, with 302 staff members participating in 38 Kaizen improvement events. These
events have yielded significant improvements in the delivery of state government services,
from issuing duplicate birth certificates to processing State Soldiers Assistance requests.

The Drive to Excellence includes 15 specific projects for reforming state government, ranging
from strategic procurement to a Lean continuous improvement effort and the reform of state grants
management. The projects focus on improving quality and customer service and reducing costs in
the delivery of government services to citizens.

Read about more public sector management improvement efforts on my Public Sector Continuous Improvement Site.

Related: Six Sigma In New York Local GovernmentTransformation and Redesign at the White HouseThe Georgetown Kentucky WayPublic Sector Management

Checklists Save Lives

photo of surgery room

Checklists are a simple quality tool that have been used widely for decades. Pilots use them, without fail, to save lives. Some surgeons have been using them and the evidence is mounting that checklists can save many more lives if more in health care use them. Studies Show Surgeons Could Save Lives, $20B by Using Checklists

Eight hospitals reduced the number of deaths from surgery by more than 40% by using a checklist that helps doctors and nurses avoid errors, according to a report released online today in the New England Journal of Medicine.

If all hospitals used the same checklist, they could save tens of thousands of lives and $20 billion in medical costs each year, says author Atul Gawande, a surgeon and associate professor at the Harvard School of Public Health.

In his study, which was funded by the World Health Organization, hospitals reduced their rate of death after surgery from 1.5% to 0.8%. They also trimmed the number of complications from 11% to 7%.

The study shows that an operation’s success depends far more on teamwork and clear communication than the brilliance of individual doctors, says co-author Alex Haynes, also of Harvard. And that’s good news, he says, because it means hospitals everywhere can improve.

Researchers modeled the checklist, which takes only two minutes to go through, after ones used by the aviation industry, which has dramatically reduced the number of crashes in recent years.

This is more great evidence of the value of applying simple management tools that are already well known. The idea that improvement takes brand new breakthrough ideas is just plain wrong.

Related: Using Books to Ignite ImprovementThe Power of a ChecklistNew, Different, BetterManagement Improvement History and Health CareOpen Source Management TermsFast Company Interview with Jeff Immelt

Federal Government Chief Performance Officer

A Quality Manager for Obama

President-Elect Obama has hired a quality manager, and her name is Nancy Killefer. She is the newly appointed “Chief Performance Officer” whose mandate is to manage budget reforms while eliminating waste in government processes, ultimately making it more effective. An MIT & McKinsey alum, Time calls her the “first official waste watchdog.”

Previous administrations have had exactly the same thing (regardless what Time magazine says), so I don’t think we should get carried away. Eliminating wasteful government spending is a refrain from every new administration. She will be running the Office of Management and Budget (OMB) and have this new title “Chief Performance Officer.” OMB has been the official waste watchdog, for at least decades. They are far from understanding muda. Time will tell if there is any change on that score going forward, I am skeptical.

Here is very typical OMB language from a 1995 memo by Alice M. Rivlin, Director of OMB:

Management controls are the organization, policies, and procedures used to reasonably ensure that (i) programs achieve their intended results; (ii) resources are used consistent with agency mission; (iii) programs and resources are protected from waste, fraud, and mismanagement; (iv) laws and regulations are followed; and (v) reliable and timely information is obtained, maintained, reported and used for decision making.

I worked with improving management in the federal government at the Office of Personnel Management, Office of the Secretary of Defense, Quality Management Office and the White House Military Office. I was one of the founders of the ASQ Public Sector Network (now Government Division) and have managed the Public Sector Continuous Improvement Site since 1995. There have been plenty of great efforts to improve management in government that have made real progress. But there is much more that needs to be done.

There are complications in applying management improvement in government but they are fairly minor comparatively. In general, the difficulty is not the necessary adjustments for a different environment than the private sector, but similar challenges to improving private sector management.

In 1982, The Grace Commission provided a report to the Regan Administration. Radio Address to the Nation on the Management of the Federal Government by Ronald Reagan, October 29, 1988

“We also set up the Private Sector Survey on Cost Control led by Peter Grace — almost 200 top business executives. This Commission spent months looking at every part of the Government, finding out where modern business practices could eliminate waste, fraud, and abuse in the Government. When they were through, they’d come up with 2,478 suggestions. And almost every recommendation we could put into effect without congressional action has been implemented. And we’ve saved close to $80 billion. We’re hoping that the next Congress will pitch in and do its part.”

The Clinton administration had the National Performance Review which was the closest thing to an attempt to move toward my concept of management improvement.

The current administration had their own President’s Management Agenda. Government Accountability: Efforts to Identify and Eliminate Waste and Mismanagement Hearing before the Committee on International Relations, House of Representatives, September 4, 2003.
Continue reading

Lean Management in Policing

photo of Jacksonville Sheriff Office Lean Team

Justice served up Jacksonville–style is all lean by Joe Jancsurak:

The Jacksonville Sheriff Office’s (JSO) Crime Reducing Initiatives Management and Enforcement Strategies (CRIMES) measures, tracks and analyzes crime-fighting statistics, such as the number of arrests and value of stolen and recovered property. Such findings are instrumental in determining the overall effectiveness of the JSO and where improvements need to be made.

Investigations stress uniformity. Lean “changed how we approach investigations,” says Sheriff Rutherford. “We found that three officers investigating three different burglaries might ask three different sets of questions. So we developed a standard form showing the questions that should be asked to ensure consistency.”

Hiring of school crossing guards made more expedient. “This one’s amazing,” Sheriff Rutherford chuckles. “It was taking us 68 days to hire someone from our eligibility list because we were sending candidates all over for different parts of the interview process. Now it takes us just three days to make a decision because we’re practicing ‘one-stop hiring.'”

This reminds me of the first efforts I know of for such efforts in policing (from the 1980s): Quality Improvement and Government: Ten Hard Lessons From the Madison Experience by David C. Couper, Chief of Police, City of Madison, Wisconsin.

Via: Upcoming Podcast: Lean Law Enforcement

Related: Failure to Address Systemic SWAT Raid FailuresLA Jail Saves Time Processing CrimeThe Public Sector and DemingCurious Cat Management Improvement Search Engine

Department of Defense Lean Six Sigma

Gordon England, the Deputy Secretary of Defense, signed a directive establishing policy and assigning responsibilities to institutionalize the effort throughout DoD. See a webcast of his speech [the broken link was removed] on lean six sigma to a DoD conference on continuous process improvement.

Leading Business Transformation the “Lean” Way [the broken link was removed]

Since it began employing LSS, the Department of the Navy (DON) has completed 1,700 Black Belt/Green Belt projects and over 2,000 Kaizen events (i.e., action-oriented events designed to improve existing processes). Initial projects were designed to build confidence and gain momentum for success in high-impact core business value streams. The DON’s total of 3,399 trained LSS Green Belts exceeds the Secretary’s goal of 2,000 by the end of 2006, and of the 935 trained LSS Black Belts in the DON, 93 have attained American Society for Quality (ASQ) Black Belt certification.

Naval Air Systems Command (NAVAIR) joined with Raytheon to complete an LSS project, which ultimately saved $133.5M across the 2006 FYDP and $421M over the life of the Joint Standoff Weapon (JSOW) Block II program. The integrated product team developed a three-tier approach to reducing weapon unit cost over a two-year period. Success of the JSOW program has led to development of a follow-on Block III weapon system.

The Marine Corps is applying LSS concepts, analytic techniques, and tools to improve the process for identifying, evaluating and acquiring critically needed warfighting equipment. Initial analysis focused on the evaluation stage, where improvements reduced the time required for this step by 35% – from 131 days to 85 days – and identified savings valued at $135K per year.

The first LSS initiative for Army aviation scheduled maintenance was deemed a success and signals a more efficient future for maintaining the Fort Rucker helicopter fleet. More than 32 days of scheduled maintenance were saved during the first LSS effort for Aviation Unit Maintenance involving UH-60 Black Hawk helicopter scheduled maintenance. The first helicopter inducted into the newly developed process was returned to flying status in just 18 days, which included a four-day break for the Fourth of July weekend. That is a 67% improvement in phase flow efficiency from the previous average time of more than 50 days of phase cycle maintenance for the UH-60.

See: online six sigma resources and lean manufacturing resources from the Curious Cat management improvement web site.

Related: Government Lean Six SigmaPublic Sector Continuous Improvement Site – Transformation Through Lean Six Sigma [the broken link was removed] – Army Business Transformation [the broken link was removed] – History Of Quality Management OnlineMore Lean GovernmentArmy Lean Six Sigma
Doing More With Less in the Public Sector: A Progress Report from Madison, Wisconsin by William G. Hunter, Jan O’Neill, and Carol Wallen, June 1986.

Jeffrey Pfeffer on Evidence-Based Practices

Jeffrey Pfeffer Testifies to Congress About Evidence-Based Practices [the broken link was removed]:

In this short statement, I want to make five points as succinctly as possible, providing references for background and documentation for my arguments. First, organizations in both the public and private sector ought to base policies not on casual benchmarking, on ideology or belief, on what they have done in the past or what they are comfortable with doing, but instead should implement evidence-based management. Second, the mere prevalence or persistence of some management practice is not evidence that it works — there are numerous examples of widely diffused and quite persistent management practices, strongly advocated by practicing executives and consultants, where the systematic empirical evidence for their ineffectiveness is just overwhelming. Third, the idea that individual pay for performance will enhance organizational operations rests on a set of assumptions. Once those assumptions are spelled out and confronted with the evidence, it is clear that many — maybe all — do not hold in most organizations. Fourth, the evidence for the effectiveness of individual pay for performance is mixed, at best — not because pay systems don’t motivate behavior, but more frequently, because such systems effectively motivate the wrong behavior. And finally, the best way to encourage performance is to build a high performance culture. We know the components of such a system, and we ought to pay attention to this research and implement its findings.

Great stuff. Read the entire document. via: Bob Sutton’s Work Matters

Related: Evidence-based ManagementIllusions – Optical and Other

Books: The Knowing-Doing Gap by Jeffrey Pfeffer and Robert Sutton – Hard Facts, Dangerous Half-Truths And Total Nonsense: Profiting From Evidence-Based Management by Jeffrey Pfeffer and Robert Sutton

Six Sigma City Government

A recent report from the Brookings Institution, Reconnecting Massachusetts Gateway Cities [the broken link was removed], has some good words on the efforts of Fort Wayne, Indiana:

In a short time, the city reduced water main replacement costs by 18 percent, cut pothole response time by 86 percent, and slashed the waiting time for building permits from 51 days to 12 days. And because the Six Sigma process permeates all functions of the city’s government, these productivity enhancements have piled up, generating more than $10 million in cost savings over the last five years.

In this time, Fort Wayne’s first-in-the nation municipal foray into Six Sigma practices has proven that statistical analyses and stringent quality control standards do not lose their power outside the boardroom. Such data-centric attention to detail, in fact, is making all the difference.

Related: Doing More With Less in the Public Sector: A Progress Report from Madison, WisconsinPublic Sector ManagementLean Government – Quality Best Practices in Government [the link that ASQ broke was removed] – Six sigma management resources

The Georgetown Kentucky Way

The Scott County Way [the broken link was removed] by Jillian Ogawa:

It seemed only natural that Toyota’s corporate culture would influence the local schools, said Superintendent Dallas Blankenship. He estimated that one in three students in the school district have one or more parents that work for either Toyota or a Toyota supplier. The school district has had several partnership programs with Toyota in Georgetown. “Simply over time, we learned a lot of practices that have helped us to become a better school system,” he said.

Center for Quality People and Organizations:

The QUEST process consists of teaching students teamwork philosophies to learn current curriculum in all different subject areas. We provide a safe environment (parameters/ground rules) and a process for the students to conduct their groups using problem-solving techniques (PDCA: Plan Do Check Act)

Great. The Education area does require special care but management improvement concepts can work very well in education. David Langford has done some great work in this area as has Alfie Kohn. They are not focused on the Toyota Way but their principles and lean thinking go together well and there expertise in the education area is very important.

via: Scott County Schools Trying Out the Toyota Way [the broken link was removed]

Related: K-12 (kindergarten though high school) improvement resourcesarticles on quality educationposts on Toyota management methodsquality learning books

Transformation and Redesign

photo of the White House

Photo of the White House, see more of my photos of Washington DC.

Here is an excellent article from 1999: Transformation and Redesign at the White House Communications Agency (WHCA) (pdf link) by March Laree Jacques

This article describes an organizational transformation effort undertaken at the White House Communications Agency. It shares the Agency’s efforts through the period of 1992-1998, beginning with a Deming-based approach to continuous quality improvement through implementation of a total organizational redesign using systems thinking precepts. It describes the obstacles to implementing quality concepts in a high visibility, high security organization and examines the influence of Agency’s organizational culture on quality performance and improvement. The discussion examines the applicability of several broadly accepted quality concepts to the “ultimate command-and-control” organization.

The article is informative and interesting, enjoy. A couple years after this article I went to work for Gerald Suarez at the White House Military Office (WHMO). WHCA is one of seven operational units of WHMO, others include: Air Force One, Camp David and the White House Medical Unit.

See more management improvement articles including in the Curious Cat Management Improvement Library.

Related: articles and podcasts by Russel AckoffDeming on ManagementDeming related blog postsPublic Sector Continuous Improvement Site