The Data Is In

In my days as an engineering student I took a course entitled Engineering Methods. One interesting lecture was on dealing with data. The professor presented us with a vernier caliper and an object of unknown length. This was in a lecture hall containing 150 or so of us. He gave a quick lecture on how to use and read the caliper and then passed the caliper, the object, and a paper upon which we were to record our measurements.

The next day he came in and presented us with copies of the raw data. He then pointed out to us some interesting facts about the data. I can not recall the length of the object but let us suppose it was found to be 16.8 cm. While scanning the raw data we all noted a few measurement recordings such as 170 cm, 1.6 cm, or 8.5 cm; whereas most readings were close to the object's 16.8 cm length. He then launched into a discussion of those fishy recordings. He went on to explain that further analysis would exclude those fishy recordings, explaining we do not need to measure the object to know it is not 8.5 cm and the other two outliers could be justified as someone misplacing a decimal point. Thus he introduced the concept of data outliers to us.

Subsequent lab classes I took (after dropping out as an engineering student and entering the physics program) always discussed the handling of outliers and the raw data. Outliers were not to be discarded from the raw data reporting and that in order to discard an outlier you had to provide justification. That is, proper experiment reporting dictated all data is to be disclosed including data we judged as being suspect.

Now, most college level experimentation replicates historical experiments with well known results. While pride may be lost in not getting the expected result the important part of the process is to learn experimentation and how to properly present the results. I did not lose any sleep when the results I calculated from some laser experimentation did not turn up as expected. However, proper scientific process demands you present the results even if you suspect flaws. Show your data good and bad, justify the included and excluded data, present your calculations and analysis, and then present your conclusions.

I suppose in my earlier example one might object that throwing out the 8.5 cm reading based on the looks of the object is improper, and it should be included in the analysis. That is not a wholly indefensible position and a lot of scientific debate goes on on such points especially in areas of debate where we have a much harder time distinguishing between good and fishy data.