Research Design
Designing your research and experiments correctly ahead of time and understanding the core principles of experimental design will increase the reproducibility and usefulness of your research
Familiarise yourself with core principles of study design
-
Learn about different types of study design and consider how this design impacts your data collection and/or analysis.
-
Consider how to develop your research question, and invest time into expressing it clearly and refining it.
-
For quantitative and qualitative research, consider different research methodologies available to you, determine which best suit your research objectives and consider the type of data they will produce. Make sure you are equipped to handle these data and remember to allow for some flexibility in your design in case your data or understanding of the data evolves over time.
-
Familiarise yourself with the concepts and principles around developing, testing, and rejecting/confirming hypotheses, and understand differences between confirmatory vs. exploratory research.
-
Consider design choices like sample size, or when data sampling should end. You should calculate and be able to justify your chosen sample size.
-
Think about which controls or control conditions are most appropriate for your study.
How can you apply these principles to your research project?
- Experimental design and data analysis (UoE School of Biomedical Sciences)
- Simon Schwab, Leonhard Held, Different Worlds Confirmatory Versus Exploratory Research, Significance, Volume 17, Issue 2, April 2020, Pages 8–9, https://doi.org/10.1111/1740-9713.01369
- Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365–376. https://doi.org/10.1038/nrn3475
- Forstmeier, W., Wagenmakers, E.-J., & Parker, T. H. (2017). Detecting and avoiding likely false-positive findings - a practical guide: Avoiding false-positive findings. Biological Reviews, 92(4), 1941–1968 https://doi.org/10.1111/brv.12315
- Halsey, L. G., Curran-Everett, D., Vowler, S. L., & Drummond, G. B. (2015). The fickle P value generates irreproducible results. Nature Methods, 12(3), 179–185. https://doi.org/10.1038/nmeth.3288
- Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie Du Sert, N., Simonsohn, U., Wagenmakers, E. J., Ware, J. J., & Ioannidis, J. P. A. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1(1), 1–9 https://www.nature.com/articles/s41562-016-0021
- Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
- Ioannidis JPA (2005). Why most published research findings are false. PLoS Med 2(8): e124. https://doi.org/10.1371/journal.pmed.0020124
- NC3Rs – Experimental Design Assistant
- National Centre for Research Methods guidance
Consider the importance of objectivity and how to minimise bias
-
Consider the use of randomisation, counterbalancing, and other methods of group allocation, or for observational research, the sampling method you choose.
-
Understand the importance of blinding at different stages of your research work and analysis, and the types of bias they may prevent.
-
Understand the importance of defining a priori inclusion and exclusion criteria and developing rules for excluding data points or subjects from analyses or datasets.
-
For quantitative research, learn about sample size and power calculations (e.g. using G*Power), and learn how to avoid pseudoreplication errors.
-
Learn about different types of statistical assumptions (e.g. normality, homogeneity), and ways of detecting and handling outliers in a dataset.
-
Learn about the negative impacts of poor practice such as flexible analysis, P-hacking, fishing, and HARKing (Hypothesising After Results are Known).
Is your research designed objectively? Can you identify any potential sources of bias within your project?
- Experimental design and data analysis (UoE School of Biomedical Sciences)
- Statistical Design and Analysis of Biological Experiments – Hans-Michael Kaltenbach (Senior Scientist at ETH Zürich)
- Karp, N. A., & Fry, D. (2021). What is the optimum design for my animal experiment? BMJ Open Science, 5(1), e100126. https://doi.org/10.1136/bmjos-2020-100126
- NC3Rs power calculator
- G* power calculator