Skip navigation
The Black Hole

What science should look like in practice

It is very hard to practice science well, which is why it is so important to use the scientific method.

BY JONATHAN THON | JAN 21 2020

The following is a five-part series on the core issues facing academic science today that will comprise the foundation of our first book, targeted for publication later this year.

“Nothing is more sad than the death of an illusion.”

– Arthur Koestler

Science is our understanding of the natural world. We review, refine, and grow this understanding using an approach we call the scientific method – a logic system comprised of posing a question, making an educated prediction at the answer, testing that prediction, and reconciling our conclusions to correct our prediction (or sometimes change the way we ask the question) to help arrive at a component of the answer. It works, and easily qualifies among society’s greatest accomplishment to-date. It may even be its greatest! The prediction is called a hypothesis*, and an experiment is the exercise in testing a prediction.

Scientists are practitioners of science, and while many of us receive formal training to be good and eventually great at it, formal training in science does not make one a scientist. A scientist is anyone that takes the step beyond simply posing a question of the natural world and tries to answer it. Not all scientists employ the scientific method, but they should, since it was designed to avoid subjective bias from affecting one’s conclusions. Running counter to the scientific method is beginning with a predetermined answer, then adapting or interpreting the results of an experiment to fit this answer (as opposed to refining the prediction). It is very hard to practice science well, since most of our predictions of how the natural world works are based on (often) very strongly held beliefs that are the bedrock of how we define the world and our place within it. If our understanding of the world changes, so must our narrative of our role within it – and that has historically not been something people are very good at doing. Because the scientific method is so hard to execute without subjective bias, many of us go to school for many years to learn how get better at it. The more capable we become at using the scientific method, the more impactful or nuanced are the questions we can strive to answer.

This is important because scientists are ultimately judged by the research they publish, and pressure to publish can often push scientists to publish splashy results at the expense of quality or meaningful data. Exciting or novel studies are often more publishable than confirmatory or negative results, and this bias can creep into decisions scientists make when considering their scientific design, or even into their interpretation of results (eg. choosing whether or not to randomize participants, include a control group, or account for possible confounding factors). Perverse incentive structures such as dependency on grant funding, publication records being valued by number (vs. quality) of publications and authorship position, and threshold of demonstration of statistical significance (see: “p-hacking”) can push scientists to cut corners in how they analyze their data and tempt scientists to bend and sometimes even break the rules.

This hurts everyone.

We can dissuade this behaviour, and in so doing improve the quality and output of our scientists by identifying and understanding the incentive structures that reinforce undesired outcomes.

 

*Use of term ‘theory’ was corrected to ‘hypothesis’

ABOUT JONATHAN THON
Jonathan Thon
Dr. Thon is the Founder and CEO of STRM.BIO. Before STRM.BIO Dr. Thon Founded Platelet BioGenesis where he served as CEO and Chief Scientific Officer. Before Platelet BioGenesis Dr. Thon was an Assistant Professor at Harvard Medical School.
COMMENTS
Post a comment
University Affairs moderates all comments according to the following guidelines. If approved, comments generally appear within one business day. We may republish particularly insightful remarks in our print edition or elsewhere.

Leave a Reply to Ken Hughes Cancel reply

Your email address will not be published. Required fields are marked *

  1. Ken Hughes / January 22, 2020 at 14:10

    In clinical fields, like the author’s, the scientific method makes sense. In other fields, like mine (oceanography), the scientific method as conventionally described doesn’t really work. Neil Gershenfeld captures the sentiment well (www.edge.org/response-detail/27116) by noting that a good approach to research is “ready, fire, aim” Yes, in that order.

    Put in some groundwork to figure out the general direction (Ready), but take what might otherwise be a shot in the dark (Fire), then spend time making sense of what you hit (Aim). You may well fail and hit nothing, but as Gershenfeld elaborates, you can’t hit anything unexpected by aiming first.

    Paraphrasing myself from https://brushingupscience.com/2019/09/02/ready-fire-aim-the-myth-of-the-scientific-method/

  2. W. Reuben Kaufman / January 23, 2020 at 20:19

    Please do forgive this pedantic comment.

    You say: “The prediction is called a theory, and an experiment is the exercise in testing a prediction.”

    I say: what you call a theory, I call a hypothesis ….. Please Google “theory vs hypothesis” to see what the not trivial difference is between the two terms.

    As a retired professor I like to quip: “You can take the professor out of the university, but it’s much harder to take the pedantry out of the professor!” This comment is just one example of that!

  3. Jonathan Thon / January 23, 2020 at 22:10

    Not pedantic. Thank you for catching the error W. Reuben Kaufman – I’ve updated the post accordingly.

Click to fill out a quick survey