Understanding Reproducible Research Learn Room

The fruit of scholarship, in computational sciences, is obviously a hybrid of theoretical and experimental research components. However, in our current scientific publication practice, researchers do not include all the necessary components of their research in their academic papers. Indeed, in our traditional publication system, the limitations of a paper medium make it impossible for authors to include their code, data, or any other complementary material in their research papers.

The direct consequence of such a practice is that other researchers who are interested in building their research upon a published paper are left on their own to reproduce and validate the results presented in the paper. This type of situation sounds familiar to many of us. Don’t you agree?

Recent studies go further and show that the lack of reporting experimental details and validating results in computational sciences is leading to a crisis of credibility. These studies argue that the current computational science practice is not reliable!

The term reproducible research was first proposed by Professor Jon Claerbout at Stanford University and is promoted by many other researchers as a good practice and a necessary response to the above issues.

What is Reproducible Research?
Reproducible research refers to the idea that the ultimate product of research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. necessary for reproduction of the results and building upon the research.

In reproducible research practice, researchers provide the product of their research as a Reproducible Research Compendium (RRC). Reproducible Research Compendium is defined as a container for all components of the research that are necessary for others to understand and reproduce the research.

Aside from being a good practice, recent studies show that reproducible research increases the impact of publications. These studies argue that reproducible research compendia are used and cited more often in other papers. This impact enhancement, along with many other advantages of RR, highly motivates researchers to make their research reproducible.

Enhance Your Reproducible Research Compendium


Many different methods are proposed each day by researchers to enhance a reproducible research compendium and to make it easier for others to adapt and extend the reported research. In this spirit, Literate Programming techniques can be greatly used in the enhancement of a reproducible research compendium.

Literate programming, first proposed by Professor Donald E. Knuth at Stanford University, refers to the combination of the text and code within the same document in a manner that is human readable.

Many interesting literate programming tools exist that allow us to mix a narrative description of the analysis together with the appropriate code segments and generate strong dynamic and interactive documents.

Some efforts have been made to create new software packages upon literate programming tools with a focus on reproducibility. A list of such software packages is presented below.

However, these RR adapted software packages are usually developed through the efforts of a small team or dedicated researchers in specialized areas, and there are no widely accepted platforms to create dynamic and interactive reproducible research documents.

Therefore, we think there is an emerging need to develop new software packages, that adopt the ideas of literate programming in reproducible research, in order to generate dynamic, and interactive reproducible research compendia.