Author Archive: Philip Hicks

How to Share Your Reproducible Research Compendium

Reproducibility means publication over the internet. Now that you have created your research compendium, you can share it with other researchers in many different ways. For example, you can simply put it on your personal or institutional website. However, if your institution supports an OAI-compliant repository, it is much more advantageous to put your compendium on such a repository.

Studies show that free on-line availability of research increases the impact of the research articles by providing better chances for publications to be read and cited. In this section we describe how a researcher can provide open access to his research compendia and enjoy the advantages.

1. Self-Archiving– Providing Open Access to the Research Paper

To provide open access to research papers, all that is needed is for authors to exercise self-archiving, i.e. put copies of their “ articles ” on an open access website (ex. a personal or institutional website). The “ articles ” may be in preprint (pre-peer review or pre-refereeing) or postprint (after peer review or refereed) form.

According to Directory of Journal Self-Archiving Policies, of the nearly 10,100 journals surveyed, over 95% have already given their official permission to author self-archiving (63% for postprints, 32% for preprints), and many of the remaining 5% journals will agree if the author asks. For example IEEE allows authors to self-archive as long as an IEEE copyright notice is displayed on the same page.

2. Open Licensing– Providing Open Access to the Code and Data

When a work is shared on the Web, the original expression of the idea falls automatically under copyright. However, authors can remove the copyright from the published code or data by using a suitable license of their choice. An author can choose a suitable license among many different existing licenses ensuring the specific legal requirements of each research project. A listing of more than sixty existing open-source licenses can be found on the Open Source webpage.

Some of popular and widely used licenses are GNU General Public License (GPL), Apache 2.0 license, Creative Commons licenses, and Berkeley Software Distribution (BSD) license. Each license sets specific rights and obligations for other researchers to use the research compendium and build their works upon the performed work.

Moreover, a new licensing structure, adapted to reproducible research and called Reproducible Research Standard (RRS), has recently been proposed. This new standard is going to be an official mark of Science Commons providing researchers with a better legal framework for publishing their research compendia publicly.

How to Enhance the Visibility and Accessibility of the Shared Reproducible Research Compendia

Studies show that free on-line availability of research increases the impact of our research articles by providing better chances for our publications to be read, and cited. This has encouraged many researchers to provide their research compendia on their personal or institutional websites all around the world. This method of providing access to research follows a distributed scheme and brings up some issues about the worldwide visibility of the research compendia.

In such a distributed system, good visibility and retrieval of information are essential for the successful delivery of services. Fortunately, many different systems have been established to improve the information retrieval, notably search engines. While search engines are vital for the retrieval of information on the Web, they do not index websites equally and may not index new pages for months. This usually leads to a delay in the information retrieval whereas delayed indexing of scientific research is not desirable.

Reproducible Research Librum is especially designed to increase the visibility of your reproducible research compendia over Internet. Reproducible Research Librum is an open directory for reproducible research websites and compendia where you can find many RRC and simply add yours.

Our RR open directory is an in-depth approach providing a comprehensive human-edited and quality-controlled list of RR compendia and websites. This approach brings many advantages to both authors and readers some of which are listed here.

We believe that a multiplicity of open access reproducible research sites (i.e. personal or institutional websites and archives) will grow up around the world in a near future. That is why we provide a comprehensive and authoritative list for reproducible research.

Understanding Reproducible Research Learn Room

The fruit of scholarship, in computational sciences, is obviously a hybrid of theoretical and experimental research components. However, in our current scientific publication practice, researchers do not include all the necessary components of their research in their academic papers. Indeed, in our traditional publication system, the limitations of a paper medium make it impossible for authors to include their code, data, or any other complementary material in their research papers.

The direct consequence of such a practice is that other researchers who are interested in building their research upon a published paper are left on their own to reproduce and validate the results presented in the paper. This type of situation sounds familiar to many of us. Don’t you agree?

Recent studies go further and show that the lack of reporting experimental details and validating results in computational sciences is leading to a crisis of credibility. These studies argue that the current computational science practice is not reliable!

The term reproducible research was first proposed by Professor Jon Claerbout at Stanford University and is promoted by many other researchers as a good practice and a necessary response to the above issues.

What is Reproducible Research?
Reproducible research refers to the idea that the ultimate product of research is the paper along with the full computational environment used to produce the results in the paper such as the code, data, etc. necessary for reproduction of the results and building upon the research.

In reproducible research practice, researchers provide the product of their research as a Reproducible Research Compendium (RRC). Reproducible Research Compendium is defined as a container for all components of the research that are necessary for others to understand and reproduce the research.

Aside from being a good practice, recent studies show that reproducible research increases the impact of publications. These studies argue that reproducible research compendia are used and cited more often in other papers. This impact enhancement, along with many other advantages of RR, highly motivates researchers to make their research reproducible.

Enhance Your Reproducible Research Compendium


Many different methods are proposed each day by researchers to enhance a reproducible research compendium and to make it easier for others to adapt and extend the reported research. In this spirit, Literate Programming techniques can be greatly used in the enhancement of a reproducible research compendium.

Literate programming, first proposed by Professor Donald E. Knuth at Stanford University, refers to the combination of the text and code within the same document in a manner that is human readable.

Many interesting literate programming tools exist that allow us to mix a narrative description of the analysis together with the appropriate code segments and generate strong dynamic and interactive documents.

Some efforts have been made to create new software packages upon literate programming tools with a focus on reproducibility. A list of such software packages is presented below.

However, these RR adapted software packages are usually developed through the efforts of a small team or dedicated researchers in specialized areas, and there are no widely accepted platforms to create dynamic and interactive reproducible research documents.

Therefore, we think there is an emerging need to develop new software packages, that adopt the ideas of literate programming in reproducible research, in order to generate dynamic, and interactive reproducible research compendia.