It has been said many times by many authors in the literature that reproducibility is the cornerstone of Science and we, as a scientific community, should aim at such reproducibility. However, good intention are not sufficient and a given computational results can be declared reproducible if and only if it has been actually replicated in a the sense of a brand new open-source and documented implementation. As explained before, writing such replication might be a daunting task that is not really rewarded and we may thus wondered what could be the motivation for doing so ? Such motivations are indeed diverse and range from the simple student who want to train oneself in order to get familiar with a specific scientific domain up to the senior researcher that has a critical need of a specific piece of code:
@stachelek:2016: I was motivated to replicate the results of the original paper because I feel that working through code supplements to blog posts has really helped me learn how to science. I could have published my replication as a blog post but I wanted the exposure and permanency that goes along with journal articles. This was my first experience with formal replication. I think the review was useful because it forced me to consider how the replication would be used by people other than myself. I have not yet experienced any new interactions following publication. However, I did notify the author of the original implementation about the replication's publication. I think this may lead to future correspondence. The original author suggested that he would consider submitting his own replications to ReScience in the future.
@topalidou:2015:b: Our initial motivation and the main reason for replicating the model is that we needed it in order to collaborate with our neurobiologist colleagues. When we arrived in our new lab, the model has just been published (2013) but the original author left the lab a few months before our arrival. And of course, there were no public repository, no version control and the paper describing the model was incomplete and partly inaccurate. In the end, we get our hands on the original sources (6,000 lines of Delphi) only to realize we could not compile them. It took us three months to replicate it using 250 lines of Python. But at this time, there were no place to publish this kind of replication and share the new code with colleagues. Since then, we have refined the model and made new predictions that have been confirmed. Our initial replication effort really gave the model a second life.
@viejo:2016: Replicating previous work is a relatively routine task every time we want to build a new model: either because we want to build on it, or because we want to compare to it. We also give replication tasks to M.Sc. students every year, as projects. In all these cases, we are confronted to incomplete or inaccurate model descriptions, as well as to the impossibility to obtain the original results. Contacting the original authors sometimes solves the problem, but not so often (because of the "dog ate my hard drive" syndrom). We thus accumulate knowledge, internal to the lab, about which model work, which doesn't, and how a given model has to be parameterized to really work. Without any place to publish it, this knowledge is wasted. Publishing it in ReScience, opening the discussion publicly, will be a progress for all of us.
ReScience already published 4 articles and as shown above, the original motivations of these authors are all different and this might become even more obvious and diverse with future publications. But, beyond these motivations, publishing in ReScience may be especially important for students since this represent a unique opportunity to show the community a given student is able to read a scientific article, to have a deep understanding of it, to write a new implementation and to eventually write a scientific article describing his/her work. Although the ReScience publishing model is a bit different from other academic journals, it can give students a first experience at peer-reviewed scholarly publishing, including meeting standards of scientific rigor and addressing reviewers' comments. Furthermore, publishing in ReScience is a way to actively contribute to open science while adding to one's publication record.