| Page 75

Do What You Say You Will

Posted on July 28, 2008 by John Hunter

In Keeping Good Employees I talked about asking some simple questions. The biggest mistake I see managers make is to fail to deliver on what they say in such meetings.

There is the saying “It is better to be thought a fool than speak, and prove it.” Well it is better to be thought a pointy haired boss than to ask for feedback, then ignore it, and prove you are a PHB. This behavior is extremely common with a survey of employee satisfaction but can extend to any failure of management follow through. If you are not going to act on what good employees tell you – don’t ask.

If some of what they mention is something you disagree with, then explain that to them. Even bad decision making that is explained is better than no explanation and no action. If you end up explaining why no action can be taken on every suggestion then employees should rightfully (most likely) find you lacking. One aspect of the explanation is to educate them for future suggestions – there may well be factors they don’t think about that you must. But, even in such a case the best practice is normally to adjust the idea a bit to make it workable.

Posted in Management, Process improvement, Respect, Systems thinking | Tagged coaching, curiouscat, management, Process improvement, quote, tips | 3 Comments

Amazon S3 Failure Analysis

Posted on July 26, 2008 by John Hunter

Amazon Simple Storage Service (S3) is a service providing web hosting. The cloud computing solution has been used by many organizations successfully. However the solution has experienced some problems including failing for much of the day on July 20th.

Amazon S3 Availability Event [the broken link was removed]

We’ve now determined that message corruption was the cause of the server-to-server communication problems. More specifically, we found that there were a handful of messages on Sunday morning that had a single bit corrupted such that the message was still intelligible, but the system state information was incorrect. We use MD5 checksums throughout the system, for example, to prevent, detect, and recover from corruption that can occur during receipt, storage, and retrieval of customers’ objects. However, we didn’t have the same protection in place to detect whether this particular internal state information had been corrupted. As a result, when the corruption occurred, we didn’t detect it and it spread throughout the system causing the symptoms described above. We hadn’t encountered server-to-server communication issues of this scale before and, as a result, it took some time during the event to diagnose and recover from it.

During our post-mortem analysis we’ve spent quite a bit of time evaluating what happened, how quickly we were able to respond and recover, and what we could do to prevent other unusual circumstances like this from having system-wide impacts. Here are the actions that we’re taking: (a) we’ve deployed several changes to Amazon S3 that significantly reduce the amount of time required to completely restore system-wide state and restart customer request processing; (b) we’ve deployed a change to how Amazon S3 gossips about failed servers that reduces the amount of gossip and helps prevent the behavior we experienced on Sunday; (c) we’ve added additional monitoring and alarming of gossip rates and failures; and, (d) we’re adding checksums to proactively detect corruption of system state messages so we can log any such messages and then reject them.

Finally, we want you to know that we are passionate about providing the best storage service at the best price so that you can spend more time thinking about your business rather than having to focus on building scalable, reliable infrastructure. Though we’re proud of our operational performance in operating Amazon S3 for almost 2.5 years, we know that any downtime is unacceptable and we won’t be satisfied until performance is statistically indistinguishable from perfect.

The failure was significant but in my view the advantages of Amazon S3 are still very significant. A huge advantage is how quickly you can scale if needed be. If your application is not hosted on Amazon S3 and it grows enormously you have to physically deal with buying servers, installing them, installing software… All this takes time. On Amazon S3 when you need the bandwidth you can get it, when you don’t need it you don’t have it sitting around unused. In that way it is very lean, it seems to me.

And while server infrastructure failures are bad, for most organizations the option is not Amazon S3 or some solution that is 100% reliable. Currently it is difficult to keep IT infrastructures online and operating and coping with shifting demand… For many situations Amazon S3 seems to be a great resource. They need to keep improving; and they seem to be doing so. Being open and honest about the challenges is a good sign. And improving the system, not blaming a person is another good sign.

Posted in Innovation, IT, Management, Software Development | Tagged Amazon, disruptive innovation, Innovation, internet, Software Development | 1 Comment

Keeping Good Employees

Posted on July 24, 2008 by John Hunter

Understanding Why Good Workers Quit [the broken link was removed]

“What do you need to want to stay?” Most managers, she acknowledges, are afraid to ask this question and that is a reason why their companies have to do plenty of exit interviews. When stay interviews are part of the culture—and this is a practice in very few companies—attrition of the people you don’t want to lose plummets.

“Ask them directly: What can we do to keep you?,” urges Kaye. And don’t be shy or dishonest. If the employee asks for things you cannot deliver, be direct in acknowledging it but also indicate what you can do. Know, too, that just by talking to employees in this way you are scoring points because it’s something that just does not happen in most companies.

More concretely, Karen Fink, vice president of human resources for Edmunds.com, said that the glue her company uses to keep top IT workers is as simple as interesting work. “Technical workers tend to remain with an organization where they have the opportunity to contribute to interesting projects that stretch their skill sets and where they have the opportunity to be educated on the latest technologies.”

Good advice. I like direct, simple, questions. What can we do to keep you? What do you enjoy about your job? What do you dislike? What can I do to increase your joy in work? What one thing would you most like to see changed? What do you want to see continue? Would you like help in some aspect of your career development? What can I do better? Am I providing too much oversight, not enough?

Give honest straight forward answers to questions. If someone wants to move ahead and needs to work harder to advance their career tell them that. If they need to be more cooperative, develop certain skills… tell them. The idea is not just to make the person happy in that meeting. If they need to work on certain things to get where they want then help them do that. Give your best advice and say what they can do to improve.

Posted in Career, IT, Management, quote, Respect | Tagged commentary, respect for people, Software Development, tips | 3 Comments

Not Exactly Lean Packaging

Posted on July 23, 2008 by John Hunter

HP shatters excessive packaging world record (17 boxes to protect 32 A4 sheets of paper)

Stephen said: “Imagine our excitement as we opened it, hoping against hope that it might contain a copy of some c-class virtual connect firmware that actually works.”

Sadly not. What the überbox did contain was 16 smaller boxes “which in turn [each] contained (wrapped in foam so they wouldn’t get broken) exactly two sheets of A4 paper”

It is hard to imagine what management system creates such solutions. But it is not hard to image Dilbert’s pointy haired boss fitting right in there.

Posted in Customer focus, Lean thinking, Management, Process improvement | Tagged bad customer service, waste | 1 Comment

Some Good IT Business Ideas

Posted on July 22, 2008 by John Hunter

Paul Graham has some excellent ideas. I have written about some of them previously: Innovation Strategy, What Business Can Learn from Open Source and Google and Paul Graham’s Latest Essay. Y Combinator, which he founded, provides seed funding. Here are some ideas they would like to fund:

Outsourced IT. In most companies the IT department is an expensive bottleneck. Getting them to make you a simple web form could take months. Enter Wufoo. Now if the marketing department wants to put a form on the web, they can do it themselves in 5 minutes. You can take practically anything users still depend on IT departments for and base a startup on it, and you will have the enormous force of their present dissatisfaction pushing you forward.
…
Online learning. US schools are often bad. A lot of parents realize it, and would be interested in ways for their kids to learn more. Till recently, schools, like newspapers, had geographical monopolies. But the web changes that. How can you teach kids now that you can reach them through the web? The possible answers are a lot more interesting than just putting books online.
…
Off the shelf security. Services like ADT charge a fortune. Now that houses and their owners are both connected to networks practically all the time, a startup could stitch together alternatives out of cheap, existing hardware and services.

Posted in Creativity, Innovation, IT, Software Development | Tagged disruptive innovation, Paul Graham | 1 Comment

“Pay for Performance” is a Bad Idea

Posted on July 21, 2008 by John Hunter

Pay for performance is a bad investment [the broken link was removed] by Pete Waters

“Teacher pay set by the results” was the headline of a (Baltimore) Sun article I read the other day which suggested that “performance-based bonuses (were) cropping up across Maryland” in our state education system. Bonuses would be given to teachers and principals that were successful in raising test scores of students.
…
One of the many shortcomings of the program was that job duties were often not well defined, and favoritism was difficult for most supervisors to avoid.
…
Deming specifically considered “performance appraisals, merit ratings and annual reviews” as one category under the heading of Seven Deadly Diseases of Management. He thought that the notion of “teamwork” was destroyed by these evaluations. Deming further believed that the morale of the organization suffered because of these individual evaluations.

As Deming said (page 102 of Out of the Crisis): “The idea of a merit rating is alluring. The sound of the words captivates the imagination: pay for what you get; get what you pay for; motivate people to do their best, for their own good. The effect is exactly the opposite of what the words promise.” Understanding enough about managing organizations to know why it doesn’t work is not easy – which I think is a big reason why people go for the nice sounding, but flawed idea, I think. Read our posts on performance appraisals and the works we reference to learn.

Posted in Deming, Management, Performance Appraisal, Systems thinking | Tagged bonus, Deming, Performance Appraisal | 1 Comment

Rhode Island Manufacturing

Posted on July 19, 2008 by John Hunter

Manufacturing has new look in R.I. [the broken link was removed]

She used the auto industry as an example, pointing out that in the early years of the last century Henry Ford manufactured Model-T’s that were all the same. The consumer today demands a choice in models, colors and a host of other features when buying a car. “Manufacturers must be able to change processes very easily and very quickly,” she said, to meet constantly changing consumer demands.

The closing of an old-fashioned assembly-line, low-wage factory always makes headlines, contributing to the image of the industry as one with a bleak future, Taito noted, while advanced manufacturers who steadily grow and add three or four jobs a year win no notice. “But that’s real growth, sustained growth,” she said of the latter.

Grove said RIMES has promoted the advantages of the lean initiative to Rhode Island manufacturers for about 10 years. “When you adopt lean manufacturing, it becomes the process of the whole shop and, by necessity, employees have to be more of a team than in the past,” he said. On-the-job training is routine at Pilgrim, according to Grove.

Still, the industry’s transition has not been painless. The number of manufacturing jobs in the state has declined steadily. In 2002, there were 64,796 people employed in manufacturing in Rhode Island and, 30 years ago in 1978, there were 134,654, according to figures from the R.I. Department of Labor and Training.

Yet another illustration of what I have been saying for years. USA manufacturing continues to grow and USA manufacturing jobs continue to shrink (as do worldwide manufacturing jobs). And as I have been saying for years, China manufacturing output continues to grow very quickly and China manufacturing jobs continue to shrink (China lost 7 times as many manufacturing jobs as the USA from 1995-2002).

Posted in Career, Economics, Lean thinking, Manufacturing | Comments Off

California Uses More Gas than China

Posted on July 17, 2008 by John Hunter

Amazing Stat: California Uses More Gas than China [the broken link was removed]:

California alone uses more gasoline than any country in the world (except the US as a whole, of course). That means California’s 20 billion gallon gasoline and diesel habit is greater than China’s! (Or Russia’s. Or India’s. Or Brazil’s. Or Germany’s.)

That’s according to the California Energy Commission’s State Alternative Fuels Plan [the broken link was removed], which was posted online last Christmas Eve (pdf). The whole report makes for some fascinating reading because it’s a blueprint for a low-carbon and renewable transportation fuel future. The dominant takeaway: it ain’t going to be easy.

One more choice statistic: gasoline usage in California has increased 50 percent, that’s 10 6.7 billion gallons, since 1988.
…
But China’s oil thirst is growing — to almost 20 billion gallons in 2007 — and perhaps as early as this year, China’s 1.3 billion people will overtake California’s 37 million people in total gasoline and diesel usage.

Interesting data. The Curious Cat Economics Blog recently posted on the top oil consuming countries.

Posted in China, Data, Economics | Tagged China, economy | Comments Off

Management Improvement Carnival #39

Posted on July 15, 2008 by John Hunter

“Certifications” – Buying Credibility? by Mark Rosenthal – “if you are looking for your own professional development, and look at a program for what it is: An academic education, and possibly an opportunity to establish professional network, then go for it. Just don’t go in believing that ‘being certified’ means a whole lot else.”
Toyota Invests In Workers Instead of Laying Them Off by Mark Graban – “You can treat people as expendable costs or an asset to train and invest in. Even as Toyota’s truck sales have plummeted, are they resorting to layoffs? Nope!!”
Quality, Productivity and Competitive Position by John Dowd – “I can still clearly hear his words, “There is no substitute for knowledge.” The knowledge is there in the pages of his book. It needs only to be extracted and acted on.”
Deer Poka Yokes by Mike Gardner – “if the deer would just follow the operation standard and flow with the traffic instead of attempting to flow at right angles to it, all of this could be avoided”
Too Bad, So Sad by Kevin Meyer – “Like most companies that try to implement lean, it appears that the second pillar, respect for people, was forgotten. Therefore most of the potential benefit was lost.”
Projects vs. Process Improvement – “By taking a project as opposed to process improvement approach it is very hard to make performance visible and understand the effect improvement interventions are having or will have.”
We Do Not Make What We Do Not Sell by Jon Miller – “Production control is a comprehensive activity of planning, organizing production and related activities including purchasing, managing inventory and production cost controls”
Age and the Entrepreneur by Paul Kedrosky – “People founding tech companies over the last ten years had an average and median age of 39-years, nowhere near the age that makes for good stories about dorm room entrepreneurs”
Queue Management by Mark – “the measurement of ‘on time’ is ‘pull away from the gate’ not “leave the ground” so in order to get an ‘on time’ departure, they will load the plane as scheduled, then go sit on the tarmac rather than delaying the passenger load. A great example of ‘management by measurement’ not getting exactly the intended results.”
Free Download – Chapter 1 of “Lean Hospitals” by Mark Graban
Better Meetings by John Hunter – Document decisions on a flip chart that everyone can see in the meeting and then email everyone the decisions.

Posted in Carnival, Management | Tagged blogs, Carnival, management | Comments Off

Outcome and In-Process Measures

Posted on July 14, 2008 by John Hunter

An outcome measure is used to measure the success of a system. For example, the outcome measure could be the percentage of people who do not get polio (the result). An output measure, for example, would be the number of people vaccinated with the polio vaccine (the output). Often we measure inputs (amount of money spent) or outputs (number of people vaccinated). They are usually easy to measure but obviously less valuable proxies for what the objective of the system (reducing the incidence of polio).

You should have all these types of measures but outcome measures are most likely to be missing so special care should be taken to make sure you are using them. It is important to define good outcome measures to use in determining the success of systems, and in determining the whether improvement projects actually result in improved outcomes.

In-process measures can be valuable in providing actionable information sooner than the outcome measure would allow action. In the polio example, an in process measure example could be % of vaccination by the time a babies is 18 months old. And looking across a country say it might well make sense to stratify the data to see if certain areas were doing poorly on this measure. If so that might be where to focus improvement. You don’t need to wait until people not vaccinated start contracting polio (which will likely be delayed for years after the system starts to have processes fail, in this example) to then notice the problem and then react.

Waiting for the outcome measure to point to a problem in this case (and in many cases) is far too late for process improvement. So process measures are needed to aid in managing the system and reacting to process results, before those processes create poor results (and can be seen as poor outcome measures). More on outcome measures.

Posted in Data, Deming, Management, Process improvement, quote | Tagged curiouscat, Data, Process improvement | 2 Comments