Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 47 additions & 7 deletions docs/supplements/OpenScience.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Open Science
The practice of maximizing the accessibility and transparency of science


## Application
The open science supplement applies to all research.

Expand All @@ -14,6 +15,15 @@ Artifacts related to a study and the paper itself should, in principle, be made

provided that there are no ethical, legal, technical, economical, or practical barriers preventing their disclosure.


Artifacts accompanying software engineering submissions serve several purposes:
- **Reproducibility**: by providing the exact code, data, and scripts needed to rerun experiments;
- **Validation**: by allowing reviewers and readers to verify empirical claims;
- **Reuse and Extension**: by offering well‐documented components that others can build upon;
- **Transparency and Trust**: by open and standardized dissemination of research outputs.

By adhering to established artifact‐review and badging processes (e.g., ACM, IEEE) or checklist requirements (e.g., NeurIPS reproducibility checklist), authors demonstrate that their work meets community standards for **usability**, **documentation**, and **accessibility**, ultimately accelerating the advancement of the field.

## Specific Attributes

### Desirable Attributes
Expand Down Expand Up @@ -45,17 +55,47 @@ Researchers should not complain that a study involves artifacts which— for goo

## Suggested Readings

Noemi Betancort Cabrera, Elke C Bongartz, Nora Dörrenbächer, Jan Goebel, Harald Kaluza, & Pascal Siegers. 2020. White Paper on implementing the FAIR principles for data in the Social, Behavioural, and Economic Sciences (No. 274). RatSWD Working Paper. [https://www.econstor.eu/handle/10419/229719](https://www.econstor.eu/handle/10419/229719)
### Open Science Policy Frameworks

- Noemi Betancort Cabrera, Elke C Bongartz, Nora Dörrenbächer, Jan Goebel, Harald Kaluza, & Pascal Siegers. 2020. White Paper on implementing the FAIR principles for data in the Social, Behavioural, and Economic Sciences (No. 274). RatSWD Working Paper. [https://www.econstor.eu/handle/10419/229719](https://www.econstor.eu/handle/10419/229719)

Carlos Diego Nascimento Damasceno. 2022. Guidelines for Quality Management of Research Artifacts in Model-Driven Engineering. _MOdeling LAnguages (blog)_. Retrieved July 17, 2022 from [https://modeling-languages.com/guidelines-for-quality-management-of-research-artifacts-in-model-driven-engineering/#](https://modeling-languages.com/guidelines-for-quality-management-of-research-artifacts-in-model-driven-engineering/#)
- Carlos Diego Nascimento Damasceno. 2022. Guidelines for Quality Management of Research Artifacts in Model-Driven Engineering. _MOdeling LAnguages (blog)_. Retrieved July 17, 2022 from [https://modeling-languages.com/guidelines-for-quality-management-of-research-artifacts-in-model-driven-engineering/#](https://modeling-languages.com/guidelines-for-quality-management-of-research-artifacts-in-model-driven-engineering/#)

Daniel Graziotin. 2020. SIGSOFT open science policies. Retrieved July 12, 2020 from [https://github.com/acmsigsoft/open-science-policies/blob/master/sigsoft-open-science-policies.md](https://github.com/acmsigsoft/open-science-policies/blob/master/sigsoft-open-science-policies.md)
- Daniel Graziotin. 2020. SIGSOFT open science policies. Retrieved July 12, 2020 from [https://github.com/acmsigsoft/open-science-policies/blob/master/sigsoft-open-science-policies.md](https://github.com/acmsigsoft/open-science-policies/blob/master/sigsoft-open-science-policies.md)

Daniel Graziotin. 2018. How to disclose data for double-blind review and make it archived open data upon acceptance
- Daniel Graziotin. 2018. How to disclose data for double-blind review and make it archived open data upon acceptance
Retrieved Feb 24, 2024 from [https://github.com/dgraziotin/disclose-data-dbr-first-then-opendata](https://github.com/dgraziotin/disclose-data-dbr-first-then-opendata)

Daniel Méndez, Daniel Graziotin, Stefan Wagner, and Heidi Seibold. 2019. Open science in software engineering. _arXiv_. [https://arxiv.org/abs/1904.06499](https://arxiv.org/abs/1904.06499)
- Daniel Méndez, Daniel Graziotin, Stefan Wagner, and Heidi Seibold. 2019. Open science in software engineering. _arXiv_. [https://arxiv.org/abs/1904.06499](https://arxiv.org/abs/1904.06499)


### Platforms for Artifact Sharing

- GitHub. 2016. Making Your Code Citable. Retrieved July 12, 2020 from [https://guides.github.com/activities/citable-code/](https://guides.github.com/activities/citable-code/). (How to automatically archive a GitHub repository to Zenodo)

- Figshare. How to connect Figshare with your GitHub account. Retrieved July 12, 2020 from [https://knowledge.figshare.com/articles/item/how-to-connect-figshare-with-your-github-account](https://knowledge.figshare.com/articles/item/how-to-connect-figshare-with-your-github-account) (How to automatically archive a GitHub repository to Figshare)


### Artifact Evaluation and Badging Policies

- IEEE. 2015. Artifact evaluation. 2015 IEEE 3rd Working Conference on Software Visualization (VISSOFT), Bremen, Germany, pp. vii-viii. [https://ieeexplore.ieee.org/document/7332408](https://ieeexplore.ieee.org/document/7332408) doi: 10.1109/VISSOFT.2015.7332408

- ACM. 2022. Artifact Review and Badging Version 1.1. August 24, 2020 from [https://www.acm.org/publications/policies/artifact-review-and-badging-current](https://www.acm.org/publications/policies/artifact-review-and-badging-current)

- NeurIPS. 2025. Reproducibility Checklist. Retrieved May 13, 2025 from [https://neurips.cc/public/guides/PaperChecklist](https://neurips.cc/public/guides/PaperChecklist)

### Best Practices

- Systems Research Artifacts. 2024. Artifact Packaging Guide. Retrieved May 13, 2025 from [https://sysartifacts.github.io/packaging-guide.html](https://sysartifacts.github.io/packaging-guide.html)

- Artifact Evaluation Committees (AEC). HOWTO for AEC Submitters. Retrieved May 13, 2025 from [https://docs.google.com/document/d/1pqzPtLVIvwLwJsZwCb2r7yzWMaifudHe1Xvn42T4CcA/edit?tab=t.0](https://docs.google.com/document/d/1pqzPtLVIvwLwJsZwCb2r7yzWMaifudHe1Xvn42T4CcA/edit?tab=t.0)

- Tianyin Xu. 2021. How Are Award‑winning Systems Research Artifacts Prepared – Part 1. Retrieved May 13, 2025 from [https://www.sigops.org/2021/how-are-award-winning-systems-research-artifacts-prepared-part-1/](https://www.sigops.org/2021/how-are-award-winning-systems-research-artifacts-prepared-part-1/)

- Tianyin Xu. 2021. How Are Award‑winning Systems Research Artifacts Prepared – Part 2. Retrieved May 13, 2025 from [https://www.sigops.org/2021/how-are-award-winning-systems-research-artifacts-prepared-part-2/](https://www.sigops.org/2021/how-are-award-winning-systems-research-artifacts-prepared-part-2/)

- Rohan Padhye. 2019. Artifact Evaluation: Tips for Authors. Retrieved May 13, 2025 from [https://blog.padhye.org/Artifact-Evaluation-Tips-for-Authors/](https://blog.padhye.org/Artifact-Evaluation-Tips-for-Authors/)



GitHub. 2016. Making Your Code Citable. Retrieved July 12, 2020 from [https://guides.github.com/activities/citable-code/](https://guides.github.com/activities/citable-code/). (How to automatically archive a GitHub repository to Zenodo)

Figshare. How to connect Figshare with your GitHub account. Retrieved July 12, 2020 from [https://knowledge.figshare.com/articles/item/how-to-connect-figshare-with-your-github-account](https://knowledge.figshare.com/articles/item/how-to-connect-figshare-with-your-github-account) (How to automatically archive a GitHub repository to Figshare)