Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
2
datasets available to search
ShareScore release 0.9.0
Dataset results
2 results for “archival package”
Source Code Archiving to the Rescue of Reproducible Deployment — Replication Package
<p>Replication package for the paper:</p> <p>Ludovic Courtès, Timothy Sample, Simon Tournier, Stefano Zacchiroli.<br><em>Source Code Archiving to the Rescue of Reproducible Deployment</em><br><a href="https://acm-rep.github.io/2024/">ACM REP'24</a>, June 18-20, 2024, Rennes, France<br><a href="https://doi.org/10.1145/3641525.3663622">https://doi.org/10.1145/3641525.3663622</a></p> <h2>Generating the paper</h2> <p>The paper can be generated using the following command:</p> <pre><code>guix time-machine -C channels.scm \ -- shell -C -m manifest.scm \ -- make </code></pre> <p>This uses GNU Guix to run <code>make</code> in the exact same computational environment used when preparing the paper. The computational environment is described by two files. The <code>channels.scm</code> file specifies the exact version of the Guix package collection to use. The <code>manifest.scm</code> file selects a subset of those packages to include in the environment.</p> <p>It may be possible to generate the paper without Guix. To do so, you will need the following software (on top of a Unix-like environment):</p> <ul> <li>GNU Make</li> <li>SQLite 3</li> <li>GNU AWK</li> <li>Rubber</li> <li>Graphviz</li> <li>TeXLive</li> </ul> <h2>Structure</h2> <ul> <li><code>data/</code> contains the data examined in the paper</li> <li><code>scripts/</code> contains dedicated code for the paper</li> <li><code>logs/</code> contains logs generated during certain computations</li> </ul> <h2>Preservation of Guix</h2> <p>Some of the claims in the paper come from analyzing the Preservation of Guix (PoG) database as published on January 26, 2024. This database is the result of years of monitoring the extent to which the source code referenced by Guix packages is archived. This monitoring has been carried out by Timothy Sample who occasionally publishes reports on his personal website: <a href="https://ngyro.com/pog-reports/latest/">https://ngyro.com/pog-reports/latest/</a>. The database included in this package (<code>data/pog.sql</code>) was downloaded from <a href="https://ngyro.com/pog-reports/2024-01-26/pog.db">https://ngyro.com/pog-reports/2024-01-26/pog.db</a> and then exported to SQL format. In addition to the SQL file, the database schema is also included in this package as <code>data/schema.sql</code>.</p> <p>The database itself is largely the result of scripts, but also of manual adjustments (where necessary or convenient). The scripts are available at <a href="https://git.ngyro.com/preservation-of-guix/">https://git.ngyro.com/preservation-of-guix/</a>, which is preserved in the Software Heritage archive as well: <a href="https://archive.softwareheritage.org/swh:1:snp:efba3456a4aff0bc25b271e128aa8340ae2bc816;origin=https://git.ngyro.com/preservation-of-guix">https://archive.softwareheritage.org/swh:1:snp:efba3456a4aff0bc25b271e128aa8340ae2bc816;origin=https://git.ngyro.com/preservation-of-guix</a>. These scripts rely on the availability of source code in certain locations on the Internet, and therefore will not yield exactly the same result when run again.</p> <h3>Analysis</h3> <p>Here is an overview of how we use the PoG database in the paper. The exact way it is queried to produce graphs and tables for the paper is laid out in the Makefile.</p> <p>The <code>pog-types.sql</code> query gives the counts of each source type (e.g. “git” or “tar-gz”) for each commit covered by the database.</p> <p>The <code>pog-status.sql</code> query gives the archival status of the sources by commit. For each commit, it produces a count of how many sources are <em>stored</em> in the Software Heritage archive, <em>missing</em> from it, or <em>unknown</em> if stored or missing. The <code>pog-status-total.sql</code> query does the same thing but over all sources without sorting them into individual commits.</p> <p>The <code>disarchive-ratio.sql</code> query estimates the success rate of Disarchive disassembly.</p> <p>Finally, the <code>swhid-ratio.sql</code> query gives the proportion of sources for which the PoG database has an SWHID.</p> <h3>Estimating missing sources</h3> <p>The Preservation of Guix database only covers sources from a sample of commits to the Guix repository. This greatly simplifies the process of collecting the sources at the risk of missing a few. We estimate how many are missed by searching Guix’s Git history for Nix-style base-32 hashes. The result of this search is compared to the hashes in the PoG database.</p> <p>A naïve search of Git history results in an over estimate due to Guix’s branch development model. We find hashes that were never exposed to users of ‘guix pull’. To work around this, we also approximate the history of commits available to ‘guix pull’. We do this by scraping push events from the guix-commits mailing list archives (<code>data/guix-commits.mbox</code>). Unfortunately, those archives are not quite complete. Missing history is reconstructed in the <code>data/missing-links.txt</code> file.</p> <p>This estimate requires a copy of the Guix Git repository (not included in this package). The repository can be obtained from GNU at <a href="https://git.savannah.gnu.org/git/guix.git">https://git.savannah.gnu.org/git/guix.git</a> or from the Software Heritage archive: <a href="https://archive.softwareheritage.org/swh:1:snp:9d7b8dcf5625c17e42d51357848baa226b70e4bb;origin=https://git.savannah.gnu.org/git/guix.git">https://archive.softwareheritage.org/swh:1:snp:9d7b8dcf5625c17e42d51357848baa226b70e4bb;origin=https://git.savannah.gnu.org/git/guix.git</a>. Once obtained, its location must be specified in the Makefile.</p> <p>To generate the estimate, use:</p> <pre><code>guix time-machine -C channels.scm \ -- shell -C -m manifest.scm \ -- make data/missing-sources.txt </code></pre> <p>If not using Guix, you will need additional software beyond what is used to generate the paper:</p> <ul> <li>GNU Guile</li> <li>GNU Bash</li> <li>GNU Mailutils</li> <li>GNU Parallel</li> </ul> <h2>Measuring link rot</h2> <p>In order to measure link rot, we ran Guix Scheme scripts, i.e., scripts that exploit Guix as a Scheme library. The scripts depend on the state of world at the very specific moment when they ran. Hence, it is not possible to reproduce the exact same outputs. However, their tendency over the passing of time should be very similar. For running them, you need an installation of <a href="https://guix.gnu.org/manual/deve/en/html_node/Installation.html">Guix</a>. For instance,</p> <pre><code>guix repl -q scripts/table-per-origin.scm </code></pre> <p>When running these scripts for the paper, we tracked their output and saved it inside the <code>logs</code> directory.</p>
DataCrate: a method of packaging, distributing, displaying and archiving Research Objects
<p>In characterizing the term <i>Research Object</i> the call for proposals for Research Object 2018 uses the phrase "multi-part research outcomes with their context". The <a href="https://github.com/UTS-eResearch/DataCrate/">DataCrate specification</a><a href="https://data.research.uts.edu.au/examples/v1.0/datacrate-RO-2018/data/paper.html#fn1">1</a> is a research data packaging and dissemination specification designed to capture exactly that; outcomes (also inputs) and context.</p><p>DataCrate specifies how to gather together data in such a way that it can (a) be packaged via zip, tar, a disc image, a multi-part package or (b) be hosted on a web server or file share for inspection by potential users and/or used directly on High Performance Computing systems or otherwise accessed and analyzed.</p><p>DataCrates can contain any kind of data, and the contextual information may include, but is not limited to, data about the people, software and equipment used in the research as well as supporting documents such as publications, funding agreements or README files.</p><p> </p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.