Category Archives: Science

Getting Retweeted by Marc Andreessen Generated a Flood of Retweets

On Twitter today I was getting more than 30 times the notifications I normally get. So I took a look to see what is going on. One of my tweets was getting retweeted and liked quite a lot (nearly 100 times each, so far). I figure most likely someone with many more followers than I must have retweeted it.

A bit more investigation and sure enough that is what happened. Marc Andreessen had retweeted it. He has 432,000 followers (a bit more than my 1,600).

This minor internet enabled connection with fame is one of the fun aspects of the internet (to me anyway, I might be a bit odd). I emailed Tim Berners Lee (the creator of the world wide web) a long time ago (probably about 15 years – and I still remember) and received a nice reply. I have written a few posts on my science and engineering blog about his work over the years including a short post on the first web server (Tim’s NeXT computer).

For those that don’t know NeXT is the computer company Steve Jobs headed in between his stints at Apple. In 1999, I was giving a presentation at a conference on Using Quality to Develop an Internet Resource (link to my paper for the talk was based on). I was working for the Office of Secretary of Defense, Quality Management Office at the time. In cutting the time down I eliminated saying that the internet was created by the Department of Defense and giving a few sentences on that history as I figured everyone knew that history. After my presentation, one of the people that came up to talk and somehow I mentioned that history and the 3 people standing there didn’t know it and were surprised. Anyway that NeXT comment reminded me of that story…

The tweet Marc Andreessen retweeted was about research by scientists in London that developed pain-free filling that allows teeth to repair themselves without drilling or injections.

Several people responded that we will never see this in use (based on the idea that announcements of research breakthroughs often fail to deliver). Quite a few people we looking forward to the day when it would be available though. Including some that were sitting in the dentist office while they were reading about it.

Continue reading →

Encouraging the Next Generation of STEM Professionals

In the most recent ASQ Influential Voices post, Bill Troy, ASQ CEO, asks: how should we encourage the next generation of STEM Professionals? I addressed a similar question in: Future Engineers and Scientists, which provides many details on this question.

The advantages of gaining science, technology, engineering and math skills (STEM) are fairly well known. However, even so, that is something to emphasize in order to encourage the next generation. While it is fairly well known it still helps to re-enforce and expand on the existing understanding. Some posts from my science and engineering blog on that topic: Engineering Graduates Earned a Return on Their Investment In Education of 21% (the highest of any discipline, math was next); Earnings by College Major, Engineers and Scientists at the Top; Career Prospect for Engineers Continues to Look Positive.

Photo of Cliff Palace at Mesa Verde National Park, Colorado, USA by John Hunter.

STEM careers often appeal to kids and teenagers (I Always Wanted to be Some Sort of Scientist, Apply to be an Astronaut).

Sadly we often discourage them with unnecessarily challenging education hurdles. It is true the education path for STEM is more challenging than for most careers. That is a reality that won’t change. If people are turned off by hard work, they likely wouldn’t like most STEM careers anyway. So that reality I think is fine. But the design of STEM education could be greatly improved to avoiding turning off many people who would enjoy the education and a career if the education process were better. I have also written about this previously: Improving Engineering Education, Primary School Science Education in China and the USA, Innovative Science and Engineering Higher Education, Infinity Project: Engineering Education for Today’s Classroom (providing middle school, high school, and early college engineering curricula), Engineering Education in the 21st Century, Research findings Contradict Myth of High Engineering Dropout Rate, Fun k-12 Science and Engineering Learning.

Those with STEM degrees have better career options than others (in terms of nearly everything: higher pay, lower unemployment and higher satisfaction with their careers). Some of the career options are more rigid than an average career, but many are actually more flexible and still have all the benefits. They have the opportunity for many rewarding jobs. This is of more importance for a sad reason: our failure to create organizations with a priority placed on respect for people.

Getting a STEM degree requires that students see the appeal of gaining those degrees and many do. Many students are turned off by either the hard work required to get such degrees or the less than optimal STEM education process (which often makes it much harder and also much less inspiring than required due to poor educational systems).

While continuing to promote STEM careers to the young is helpful and wise, we are doing this fairly well. Of course, everything can be done better, and we should keep striving to improve. But the main focus, In my opinion, should be on better education from k-12 all the way through the PhD level for STEM. It would also help if we stopped electing anti-science politicians.

George Box Webcast on Statistical Design in Quality Improvement

George Box lecture on Statistical Design in Quality Improvement at the Second International Tampere Conference in Statistics, University of Tampere, Finland (1987).

Early on he shows a graph showing the problems with American cars steady over a 10 years period. Then he overlays the results for Japanese cars which show a steady and significant decline of the same period.

Those who didn’t get to see presentations before power point also get a chance to see old school, hand drawn, overhead slides.

He discusses how to improve the pace of improvement. To start with informative events (events we can learn from) have to be brought to the attention of informed observers. Otherwise only when those events happen to catch the attention of the right observer will we capture knowledge we can use to improve. This results in slow improvement.

A control chart is an example of highlighting that something worth studying happened. The chart will indicate when to pay attention. And we can then improve the pace of improvement.

Next we want to encourage directed experimentation. We intentionally induce informative events and pay close attention while doing so in order to learn.

Every process generates information that can be used to improve it.

He emphasis the point that this isn’t about only manufacturing but it true of any process (drafting, invoicing, computer service, checking into a hospital, booking an airline ticket etc.).

He then discussed an example from a class my father taught and where the students all when to a TV plant outside Chicago to visit. The plant had been run by Motorola. It was sold to a Japanese company that found there was a 146% defect rate (which meant most TVs were taken off the line to be fixed at least once and many twice) – this is just the defect rate before then even get off the line. After 5 years the same plant, with the same American workers but a Japanese management system had reduced the defect rate to 2%. Everyone, including managers, were from the USA they were just using quality improvement methods. We may forget now, but one of the many objections managers gave for why quality improvement wouldn’t work in their company was due to their bad workers (it might work in Japan but not here).

He references how Deming’s 14 points will get management to allow quality improvement to be done by the workforce. Because without management support quality improvement processes can’t be used.

With experimentation we are looking to find clues for what to experiment with next. Experimentation is an iterative process. This is very much the mindset of fast iteration and minimal viable product (say minimal viable experimentation as voiced in 1987).

There is great value in creating iterative processes with fast feedback to those attempting to design and improve. Box and Deming (with rapid turns of the PDSA cycle) and others promoted this 20, 30 and 40 years ago and now we get the same ideas tweaked for startups. The lean startup stuff is as closely related to Box’s ideas of experimentation as an iterative process as it is to anything else.

He also provided a bit of history that I was not aware of saying the first application of orthogonal arrays (fractional factorial designs) in industry was by Tippett in 1933. And he then mentioned work by Finney in 1945, Plackett and Burman in 1946 and Rao in 1947.

The Art of Discovery

Quality and The Art of Discovery by Professor George Box (1990):

Quotes by George Box in the video:

“I think of statistical methods as the use of science to make sense of numbers”

“The scientific method is how we increase the rate at which we find things out.”

“I think the quality revolution is nothing more, or less, than the dramatic expansion of the of scientific problem solving using informed observation and directed experimentation to find out more about the process, the product and the customer.”

“It really amounts to this, if you know more about what it is you are doing then you can do it better and you can do it cheaper.”

“We are talking about involving the whole workforce in the use of the scientific method and retraining our engineers and scientists in a more efficient way to run experiments.”

“Tapping into resources:

Every operating system generates information that can be used to improve it.
Everyone has creativity.
Designed experiments can greatly increase the efficiency of experimentation.

An informed observer and directed experimentation are necessary for the scientific method to be applied. He notes that the control chart is used to notify an informed observer to explain what is special about the conditions when a result falls outside the control limits. When the chart indicates a special cause is likely present (something not part of the normal system) an informed observer should think about what special cause could lead to the result that was measured. And it is important this is done quickly as the ability of the knowledgable observer to determine what is special is much greater the closer in time to the result was created.

The video was posted by Wiley (with the permission of George’s family), Wiley is the publisher of George’s recent autobiography, An Accidental Statistician: The Life and Memories of George E. P. Box, and many of his other books.

Introductory Videos on Using Design of Experiments to Improve Results

The video shows Stu Hunter discussing design of experiments in 1966. It might be a bit slow going at first but the full set of videos really does give you a quick overview of the many important aspects of design of experiments including factorial designed experiments, fractional factorial design, blocking and response surface design. It really is quite good, if you find the start too slow for you skip down to the second video and watch it.

My guess is, for those unfamiliar with even the most cursory understanding of design of experiments, the discussion may start moving faster than you can absorb the information. One of the great things about video is you can just pause and give yourself a chance to catch up or repeat a part that you didn’t quite understand. You can also take a look at articles on design of experiments.

I believe design of experiments is an extremely powerful methodology of improvement that is greatly underutilized. Six sigma is the only management improvement program that emphasizes factorial designed experiments.

Continue reading →

Richard Feynman Explains the PDSA Cycle

Ok, really Richard Feynman Explains the scientific method. But his thoughts make the similarity between the PDSA cycle and the scientific method obvious.

1) Plan, hypothesis.
You make a guess about a theory (in using the PDSA cycle this step is often missed, while in the scientific method this is of the highest priority). You make a prediction based on that theory.

2) Do the experiment

3) Study the results

If the results disprove the theory you were wrong. If they results don’t disprove the theory you may have a useful theory (it can also be that your theory is still wrong, but this experiment happened not to provide results that disprove it).

Step 4, Act, only exists for PDSA. In science the aim is to learn and confirm laws. While the PDSA cycle has an aim to learn and adopt methods that achieve the desired results.

Richard Feynman: “If it disagrees with experiment it is wrong, in that simple statement is the key to science, it doesn’t make any difference how beautiful your guess is, it doesn’t make a difference how smart you are (who made the guess), or what his name is, if it disagrees with experiment it is wrong.”

Actually far to often “PDSA” fails to adopt this understanding. Instead it become PA: no study of the results, just implement and we all already agree it is going to work so don’t bother wasting time testing that it actually does. Some organization do remember to study results of the pilot experiments but then forget to study the results when the new ideas are adopted on a broader scale.

ASQ Influential Voices: Future Engineers and Scientists

As I mentioned previously, I will be posting on a topics raised by Paul Borawski, CEO, ASQ as part of ASQ Influential Voices. This month Paul’s post, New Bloggers, STEM & More, looks at the development of future engineers.

How can we, those who understand, use, and love science and technology, pass it along.

I have discussed this issue often, on one of my other blogs: Curious Cat Science and Engineering blog: Encouraging Curiosity in Kids, Passion for Mechanical Engineering (StoryCorps), Illusion of Explanatory Depth, Teaching Through Tinkering.

They are several critical paths to address in building our pipeline of future scientists and engineers. First we need to encourage kids to explore these areas. In my opinion, we currently do a pretty good job, sadly, of discouraging kids as much as we can. So reducing those barriers is key, then we need to actually build ways that help kids. We actually do have many good efforts in place to encourage kids to explore their natural curiosity (follow that link for tons of great organization: FIRST, Project Lead The Way, Engineering is Elementary, The Infinity Project etc.). This helps balance out the discouraging of students that our normal classrooms do. But the pool of kids we reach with these efforts now is far too small. And many are so turned off by our traditionally science education that no matter how much they enjoy outside science and engineering projects they are not willing to pursue science and engineering in school.

The next big area is undergraduate and graduate education. At this point we do a good job, for those willing to put up with the current model of education, which is not designed to encourage those who are interested. It is basically up to weed out any students not willing to put up with the current painful model of higher education for science and engineering. The system seems designed to wean out those who are not sufficiently willing to put up with the difficulties they are asked to face. If the only people that would benefit from science and engineering education are those that are willing to deal with the current system, then it might be fine. But I believe we have turned away hundreds of thousands of people that would have done great things with what they learned. I believe those that will not put themselves through the current system can offer great value. We will gain great benefits if we create a system that is designed to maximize the benefits to students.

There are good ideas for how to improve. But they are challenging. And we are not doing nearly enough experimenting to find good new models of engineering education. Some of my previous posts on science and engineering education: How the Practice and Instruction of Engineering Must Change – Webcast: Engineering Education in the 21st Century by William Wulf (National Academy of Engineering President), Improving Engineering Education (Olin College of Engineering Experiment), Reforming Engineering Education, Carnegie Foundation Calls for Overhaul of Engineering Education.

Continue reading →

One factor at a time (OFAT) Versus Factorial Designs

Guest post by Bradley Jones

Almost a hundred years ago R. A. Fisher‘s boss published an article espousing OFAT (one factor at a time). Fisher responded with an article of his own laying out his justification for factorial design. I admire the courage it took to contradict his boss in print!

Fisher’s argument was mainly about efficiency – that you could learn as much about many factors as you learned about one in the same number of trials. Saving money and effort is a powerful and positive motivator.

The most common argument I read against OFAT these days has to do with inability to detect interactions and the possibility of finding suboptimal factor settings at the end of the investigation. I admit to using these arguments myself in print.

I don’t think these arguments are as effective as Fisher’s original argument.

To play the devil’s advocate for a moment consider this thought experiment. You have to climb a hill that runs on a line going from southwest to northeast but you are only allowed to make steps that are due north or south or due east or west. Though you will have to make many zig zags you will eventually make it to the top. If you noted your altitude at each step, you would have enough data to fit a response surface.

Obviously this approach is very inefficient but it is not impossible. Don’t mistake my intent here. I am definitely not an advocate of OFAT. Rather I would like to find more convincing arguments to persuade experimenters to move to multi-factor design.

Factorial Designed Experiment Aim

Multivariate experiments are a very powerful management tool to learn and improve performance. Experiments in general, and designed factorial experiments in particular, are dramatically underused by managers. A question on LinkedIn asks?

When doing a DOE we select factors with levels to induce purposely changes in the response variable. Do we want the response variable to move within the specs of the customers? Or it doesn’t matter since we are learning about the process?

The aim needs to consider what you are trying to learn, costs and potential rewards. Weighing the various factors will determine if you want to aim to keep results within specification or can try options that are likely to return results that are outside of specs.

If the effort was looking for breakthrough improvement and costs of running experiments that might produce results outside of spec were low then specs wouldn’t matter much. If the costs of running experiments are very high (compared with expectations of results) then you may well want to try designed experiment values that you anticipate will still produce results within specs.

There are various ways costs come into play. Here I am mainly looking at the costs as (costs – revenue). For example the case where if the results are withing spec and can be used the costs (net costs, including revenue) of the experiment run are substantially lower.
Continue reading →

Problems With Student Evaluations as Measures of Teacher Performance

Dr. Deming was, among other things a professor. He found the evaluation of professors by students an unimportant (and often counterproductive measure) – used in some places for awards and performance appraisal. He said for such a measure to be useful it should survey students 20 years later to see which professors made a difference to the students. Here is an interesting paper that explored some of these ideas. Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors by Scott E. Carrell, University of California, Davis and National Bureau of Economic Research; and James E. West, U.S. Air Force Academy:

our results indicate that professors who excel at promoting contemporaneous student achievement, on average, harm the subsequent performance of their students in more advanced classes. Academic rank, teaching experience, and terminal degree status of professors are negatively correlated with contemporaneous value”added but positively correlated with follow”on course value”added. Hence, students of less experienced instructors who do not possess a doctorate perform significantly better in the contemporaneous course but perform worse in the follow”on related curriculum.

Student evaluations are positively correlated with contemporaneous professor value”added and negatively correlated with follow”on student achievement. That is, students appear to reward higher grades in the introductory course but punish professors who increase deep learning (introductory course professor value”added in follow”on courses). Since many U.S. colleges and universities use student evaluations as a measurement of teaching quality for academic promotion and tenure decisions, this latter finding draws into question the value and accuracy of this practice.

These findings have broad implications for how students should be assessed and teacher quality measured.