• Patrick Wilson

Reproducibility in Science

There has recently been some discussion on the Society of Vertebrate Paleontology’s stance on the use of private specimens for scientific research. This topic is complicated and wrought with ethical questions. I will attempt to break down some of these ethical problems, discuss why this issue is complicated, and suggest some ways of navigating this issue moving forward.

What Science Is

We need to define what science is before discussing what factors go into using the scientific method. Science is a method for investigating and understanding the universe around us. The process of investigation is the scientific method. We have all had the process of the scientific method pounded into our brains during our formative years in school. We all know that you start with an observation, ask a question based on that observation, form a hypothesis to answer the question, test your hypothesis and observe the results, form conclusions based on your results, and then communicate your results with the rest of the community or your mom and dad. Every researcher and scientist I know still uses this process. As part of our academic training, researchers learn how to ask better questions and how to develop the best hypotheses possible to test. This is on top of learning new and more advanced techniques for testing their hypotheses. However, fundamentally speaking, we “perform” science the same way we did in our elementary school science fair.

When we develop our hypotheses, we are mainly concerned with 3 things. A good hypothesis is testable, falsifiable, and repeatable. To break these down, a testable hypothesis is written in such a way as to allow an experiment to take place. A falsifiable hypothesis is so that we remove or falsify as possible outcome if the hypothesis is supported. SCIENCE. PROVES. NOTHING.

Science is simply a structure of conclusions that are supported by evidence. When we find contradicting evidence, we change our conclusions. This oftens means that we need to retest or repeat our hypotheses to ensure that we are getting reliably accurate and precise results. When all of this works together, our hypotheses hold weight and can be compared to other people hypotheses and conclusions. This makes “good” science.

All of this may seem off topic to the ethics of paleontological science and what specimens can and should be used for research. Rest assured that all of this is the ground work for a complicated discussion.

Science Stewardship

A major part of being a good scientist is being a good steward of the data we collect. We strive to formulate and test hypotheses that avoid introducing biases, whether they are intentional or unintentional, and when biases are unavoidable we attempt to communicate these by listing them as assumptions. One of the biggest assumptions we are forced to deal with is missing data. As scientists, paleontologist, naturalist, biologist, chemists, etc. we must be willing to acknowledge that our data set is never complete and that every data point is valuable. However, while we want to have every single piece to the jigsaw puzzle, it is important for us to recognize that others MUST be able to repeat our experiments, statistics, and descriptions. Remember a repeatable hypothesis is a MAJOR part of science, even if you don't see it much in the media. As a leading organization for vertebrate paleontological sciences, SVP takes a leadership role with regards to stewardship of paleontological data as a whole. This means that they must take a strong stance on using fossils that are within the public trust, because these fossils have the most access for repeatability.

Why Restricted Access is NOT OK

While some individual researchers may be okay with using privately collected specimens, the issue remains that the owners of those specimens may dictate who can look at or study them and therefore they bias the data that is collected. Why might this be a problem? Well, consider this...

Science is constantly refining and expanding itself! New techniques and methods are developed every day, which means we need to be able to test, introduce new techniques and processes, and retest. That we can learn more about the subject matter and ask deeper, more complex questions. Imagine trying to make the best cake. If you don't have the same reliable materials to start with, and a recipe to work from chances are you will have a bland, gross mass or something that isn’t quite as good as it could be. If you change the ingredients or the recipe every time you will never end up with a reliable result. In this instance, specimens are the ingredients, and the recipes are the methods we use to evaluate the specimens. Without using the same specimen and the same methods, you cannot compare your cake to others. This means that when you lose access to a specimen because it is sold, transferred, moved, or simply because the owner doesn’t like your second cousin on your mother’s side, you cannot know if your results are reliable and you cannot compare your results to everyone else’s. You lose the repeatability of your results and any evidence those results show. In essence, you get bad science.

The Other Side of the Coin

Now, some people will point out that “Those specimens with restricted access are unique, special, and deserve attention too. They have just as good of data with them as a professionally collected specimen, so what is wrong with using them to do research?” Well, these people have a point. It is difficult for scientists to determine how unique and special a specimen is if no one looks at it. Are we underrepresenting aspects of these animals because we haven’t included that one specimen? Are we biasing our “side” because we are choosing to look the other way? Herein lies the heart of the ethical problem at hand.

When looking at data, it is important for us to consider the number of samples we currently have available for study. Paleontologists struggle with sample size 95% of the time. This means that a single specimen could be very important, especially with controversial animals, such as the Nanotyrannus. However, paleontology often uses biology concepts, which means that a sample size of 30 to 50 individuals is often enough to get an accurate image of the entire population of that animal. Excluding privately held specimens, we currently have well over 30 mostly complete specimens of Tyrannosaurus rex, which is enough to give us an accurate representation of the animal. So, do we really need one more T. rex specimen to be held in the public trust to study? Not necessarily. But what if that specimen is the best example of the new Nanotyrannus genus? If that is the case, we absolutely need the specimen held in the public trust so that researchers across the globe can study it. It is this balance between the number of specimens available and the significance of the specimen that is constantly debated.

To further complicate the matter, vertebrate fossils are extremely rare. The fossilization process is often very abusive to the specimen and requires just the right conditions to preserve anything at all. This means that each fossil is one-of-a-kind. So we can now ask the question, “if each fossil is one-of-a-kind, do we ever truly meet the magical sample size of 30?”

These are the questions that plague paleontologists who deal with the relationship between private and public specimens. Trying to strike the balance between sample size and uniqueness means that organizations like the Society of Vertebrate Paleontology must take a stance in favor of the data. Ideally, paleontologists in the United States would love the opportunity to be like Canada, where all fossils are the property of the crown and therefore the experts at the province level get what we call “first right of refusal.” This means that if the province already has 100 oreodont skulls, they can say that they don’t want the 101st skull brought in. However, to expect this level of control from the United States is unreasonable because it steps on too many delicate issues and sensibilities.

Where Do We Go From Here?

To wrap this up, I think we must ask ourselves this question. What now? Can we continue to ignore privately held specimens even though they could be unique and new? Can scientists ignore the proper stewardship of their data by using privately held specimens? I personally think that it is important to consider this topic on a case-by-case basis. Maybe we don’t need the 101st specimen of Triceratops. But we do need the 5th specimen of Edmontonia. Maybe there can be a deal struck between private collectors and the public trust where individuals who buy and sell specimens in the US are required to allow any and all access for scientific research regardless of who buys the specimen. Maybe the US can adopt policies and regulations that further protect fossils and include more specimens into the public trust. Or maybe paleontologists should focus on collecting specimens from remote locations that have been relatively unexplored for their paleontological resources. It is difficult to say what solution is the most effective. I do think that no matter what, science must continue to be science and that means that the data we collect and share with peers, mentors, and the world need to be held to the highest standards.

© 2019 The Montana Dinosaur Center  |  120 2nd Ave. S.   |   Bynum, MT   |   (800) 238-6873

Employment          |          Online Store          |          Donate to the Dinosaur Center          |          Sign Up for Dinosaur Center Email          |          Contact Us