Pre registering a claim

Pre registering a claim

In short

I once watched a graduate student sort through a dozen ways to split his survey data until one combination finally showed something interesting, and only then did he decide that particular question had been his real focus all along. Pre registering a claim means you write down your hypothesis, your sample size, and exactly how you plan to analyze the results before you collect a single observation. You lock that plan in a public registry with a time stamp. It stops you from hunting through comparisons after the fact and then pretending you predicted the pattern from the start. Some journals now require it. It does not guarantee you are right, but it does keep the test honest.

The whole of it

What it is

A colleague once showed me a study that tested fourteen different subgroups and celebrated the one significant result as a discovery. Pre registering a claim is the practice of committing your research question, your method, and your analysis plan to a public, time stamped record before you gather or examine your data. You state what you expect to find. You say exactly how you will measure it. That document cannot be edited later without leaving a trail. The idea is simple: if you decide what counts as evidence after you see the pattern, you are no longer testing a hypothesis. You are telling a story. Pre registration separates exploration, which is valuable, from confirmation, which requires discipline.

How it works

You draft a document. It names your question and your variables. You describe the population you will sample and the size of that sample. You write out the statistical test you will use and the threshold you will call significant. If you plan subgroup analyses, you list them now. You then upload that document to a registry such as the Open Science Framework, AsPredicted, or ClinicalTrials.gov. The registry issues a time stamp and a permanent identifier. Some registries make the document public immediately. Others embargo it until you publish or until you choose to release it. Either way, the record proves what you decided before data collection began. When you write up your results, you link to the registration. Readers can compare your plan to your report. They see whether you followed it.

The numbers, and where to find yours

Pre registration itself costs nothing. The Open Science Framework, AsPredicted, and the Registry of Efficacy and Effectiveness Studies are free to use. ClinicalTrials.gov is a legal requirement in the United States for many clinical trials under the Food and Drug Administration Amendments Act of 2007, and it charges nothing. A few journals offer badges or a separate article category for pre registered work. Some grant agencies now expect a pre registration link in your proposal. There is no official number to look up. You simply create an account on the registry, fill in the template, and submit. The platform generates a URL and a date stamp. Done. That is your record.

A worked example

Imagine a researcher named Elliot who wants to know whether a daily ten minute writing exercise improves undergraduate exam scores. Before he recruits a single student, he sits down. He writes a registration. He states that he will randomly assign 100 students, 50 to write each morning for two weeks and 50 to a waiting list. He defines his outcome as the average score on a standardized midterm given the day after the intervention ends. He specifies an independent samples t test, two tailed, with alpha set at point zero five. He lists no subgroups. No secondary outcomes either. He uploads the document to OSF and receives a link and a time stamp showing the current date. Two months later he collects the data. The writing group averages 78 points. The control group averages 75. The t test yields p equals point zero nine. Because his plan said point zero five, he reports no significant effect. He does not split the sample by gender or major to hunt for something smaller. His paper includes the registration link, and any reader can confirm that he tested exactly what he promised. Nothing more.

Where it goes wrong

A friend once told me he registered a study and then realized halfway through data collection that he had forgotten to measure a covariate that mattered. The most common mistake is writing a vague plan that leaves room to adjust later. If you say you will test whether an intervention works but do not name the outcome variable or the analysis, the registration means nothing. Another error is registering after you have already peeked at the data. Sometimes people call that pre registration in bad faith. The time stamp will show the date, but if that date falls after your data collection window, the exercise is theater. Some researchers register and then deviate without acknowledgment. They publish only the analyses that worked. They hope no one checks the link. A smaller problem is over registration: writing such a rigid protocol that any reasonable amendment, like dropping an outlier you discover is a data entry error, feels like cheating. The solution is to register your confirmatory tests and clearly label anything else as exploratory in the paper. Registration does not forbid discovery. It just asks you to be honest about what you planned and what you found along the way.

Questions to answer before you leave this page

What question are you actually trying to confirm, and can you state it as a single sentence with no hedging? Which registry fits your field, and have you created an account? What is your primary outcome variable, and have you defined it so another person could measure it the same way? What sample size will you collect, and did you choose it before looking at any pilot data that includes the same measurement? Which statistical test will you run, and what threshold will you treat as meaningful? Are there subgroups or secondary outcomes you want to explore, and are you willing to label them exploratory in your report? If you deviate from the plan, will you document the change and explain why? Can you imagine a result that would contradict your hypothesis, or have you written a plan so loose that every outcome confirms it? Have you included the registry link in your manuscript, and does the time stamp honestly precede your data collection? Do you understand that pre registration makes your work transparent, not automatically true, and that a null result reported honestly is more valuable than a surprise finding no one can replicate?