Tag Archives: experiments

What’s the Value of a Big Bonus?

What’s the Value of a Big Bonus? by Dan Ariely

To look at this question, three colleagues and I conducted an experiment. We presented 87 participants with an array of tasks that demanded attention, memory, concentration and creativity. We asked them, for instance, to fit pieces of metal puzzle into a plastic frame, to play a memory game that required them to reproduce a string of numbers and to throw tennis balls at a target. We promised them payment if they performed the tasks exceptionally well. About a third of the subjects were told they’d be given a small bonus, another third were promised a medium-level bonus, and the last third could earn a high bonus.

So it turns out that social pressure has the same effect that money has. It motivates people, especially when the tasks at hand require only effort and no skill. But it can provide stress, too, and at some point that stress overwhelms the motivating influence.

When I recently presented these results to a group of banking executives, they assured me that their own work and that of their employees would not follow this pattern. (I pointed out that with the right research budget, and their participation, we could examine this assertion. They weren’t that interested.)

This is an interesting look at an effect of bonuses. We all know monetary bonuses can influence behavior. The problem is the type of behaviors that result. Huge bonuses, for example, create huge incentives to risk the future of the company for the chance at a huge bonus for the executive. Extrinsic motivation leads to many problems.

Problems with bonuses: Losses Covered Up to Protect Bonuses“Pay for Performance” is a Bad IdeaProblems with BonusesBook: Punished By Rewards: The Trouble With Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes by Alfie Kohn – posts on executive pay

Systemic Workplace Experiments

Workplace Experiments

At our company-wide get together last December we decided that 2008 was going to be a year of workplace experiments. Among other things, we discussed how we could make 37signals one of the best places in the world to work, learn, and generally be happy.

Last summer we experimented with 4-day work weeks. People should enjoy the weather in the summer. We found that just about the same amount of work gets done in four days vs. five days.

So recently we’ve instituted a four-day work week as standard. We take Fridays off. We’re around for emergencies, and we still do customer service/support on Fridays, and but other than that work is not required on Fridays.

We decided that 37signals would help people pay for their passions, interests, or other curiosities. We want our people to experience new things, discover new hobbies, and generally be interesting people. For example, Mark has recently taken up flight lessons. 37signals is helping him pay for those. If someone wants to take cooking lessons, we’ll help pay for those. If someone wants to take a woodworking class, we’ll help pay for that.

Part of the deal is that if 37signals helps you pay, you have to share what you’ve learned with everyone. Not just everyone at 37signals, but everyone who reads our blog. So expect to see some blog posts about these experiences.

We just ask people to be reasonable with their spending. If there’s a problem, we’ll let the person know. We’d rather trust people to make reasonable spending decisions than assume people will abuse the privilege by default.

Dr. Deming proposed supporting education of any type for employees (point 13 in the 14 points). That is not often done, but 37 signals is not alone in doing this. Great stuff. Create a great environment for people to work in and you can get great things done. Also good old PDSA at work – try things on a small scale and then institute those experiments that succeed on a wider scale.

Related: Google Experiments Quickly and OftenVacation: Systems ThinkingGetting and Keeping Great EmployeesJoy in WorkComplicating SimplicityWorkplace Management

Google Website Optimizer

Google’s Website Optimizer allows for multivariate testing of your website.

Website Optimizer, Google’s free multivariate testing application, helps online marketers increase visitor conversion rates and overall visitor satisfaction by continually testing different combinations of site content (text and images).

Rather than sitting in a room and arguing over what will work better, you can save time and eliminate the guesswork by simply letting your visitors tell you what works best. We’ll guide you through the process of designing and implementing your first experiment. Start optimizing your most important web pages and see detailed reports within hours.

Google provides an online slide show with audio (a good example of one way to share online information sharing in my opinion). This tool seems to have limited experimental options to what is on the page (it does not appear, for example, that one variable could be current customer v. new visitor…). Still it looks like an very easy way to do some simple multi-factorial experiments. Google offers a list of partners [the link that Google broke was removed] for those interested in consulting and more advanced features (and for those experts reading this you can apply to be a partner).

Related: Design of experiments postsarticles on multi-factorial experimentationGoogle: Experiment Quickly and OftenData Based Decision Making at Google

Continue reading

All Models Are Wrong But Some Are Useful

“All Models Are Wrong But Some Are Useful” -George Box

A great quote. Here is the source: George E.P. Box, Robustness in the strategy of scientific model building, page 202 of Robustness in Statistics, R.L. Launer and G.N. Wilkinson, Editors. 1979.

See more quotes by George Box.

Related: Dangers of Forgetting the Proxy Nature of Dataarticles by George BoxQuotes by Dr. W. Edwards Deming

Using Design of Experiments

How to Institute DOE in Your Company (link broken – removed) by Davis Balestracci:

DOE works, but I don’t need to sell that to the readers of this newsletter. But as certain as we all are, no one can deny that design of experiments faces resistance even in environments where it is a proven tool. Every research scientist or engineer who has had a major success from DOE can tell you story after story of how management still wanted problems solved one-factor-at-a-time.

Design of Experiments (DoE) was developed by R.A. Fisher in the 1920s (related terms: factorial design, multivariate expertness). Six Sigma was the first general management approach that specifically highlighted the use of Designed Experiments for improvement. Still the use of factorial designed experiments is much less than it could be.
Continue reading

Google: Experiment Quickly and Often

Google Thinks Small by Quentin Hardy, Forbes:

Brin and Page have created a corporate organism that tackles most big projects in small, tightly focused teams, setting them up in an instant and breaking them down weeks later without remorse. “Their view is that there is much greater progress if you have many small teams going out at once,” Schmidt says. The mission overall: to collect “all the world’s information” and make it accessible to everyone. “It’s a cause.”

Hundreds of projects go on at the same time. Most teams throw out new software in six weeks or less and look at how users respond hours later.

Google has advantages in making this work for them (it is easy to find reasons it won’t work elsewhere). However, this is basically piloting changes on a small scale, analyzing the results and repeating that quickly. Quick, frequent experimentation with iteration is a tactic all organizations would benefit from.

The clear visible mission is also helpful. When an organization has an organizing principle everyone can understand then action can be guided by individual aim toward that purpose. When the understanding is missing organizations often have to rely on top down instruction and having far too many issues passed up the hierarchy for a decision.

And getting a small group of people to make things work quickly is also great. Many organizations get bogged down with byzantine management structures that slow action to a crawl.