Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
134
datasets available to search
ShareScore release 0.9.0
Dataset results
134 results for “Notebook”
Data for "Interactive maps in the Jupyter notebook"
<p>Dataset used for the lesson "<a href="https://annefou.github.io/jupyter_maps/index.html">Interactive maps in the Jupyter notebook</a>" </p> <p> </p> <p>Taught at CarpentryConnect, Manchester 2019. </p>
Sample data for "Classification Modeling for Hazardous Rip Current Prediction" Notebook
<p>This sample dataset is used in the notebook "Classification Modeling for Hazardous Rip Current Prediction" to demonstrate the application of using machine learning to identify hazardous rip current. The notebook is available in the NOAA Center for Artificial Intelligence GitHub Learning Journey repository (https://github.com/noaa-ncai/learning-journey). The full dataset is available via NOAA.</p>
Linked collectors and determiners for: Lajitietokeskus/FinBIF - Notebook, general observations.
Natural history specimen data linked to collectors and determiners held within, "Lajitietokeskus/FinBIF - Notebook, general observations". Claims or attributions were made on Bionomia by volunteer Scribes, <a href="https://bionomia.net/dataset/df12ca07-f133-4550-ab3b-fde13f0e76ba">https://bionomia.net/dataset/df12ca07-f133-4550-ab3b-fde13f0e76ba</a> using specimen data from the dataset aggregated by the Global Biodiversity Information Facility, <a href="https://gbif.org/dataset/df12ca07-f133-4550-ab3b-fde13f0e76ba">https://gbif.org/dataset/df12ca07-f133-4550-ab3b-fde13f0e76ba</a>. Formatted as a Frictionless Data package.
Evaluation notebook and files for FAIR Workbench user evaluation
<p>This archive contains the Jupyter notebook and associated (image) files used in the June 2021 evaluation of the FAIR Workbench.</p>
Small-angle Scattering Data Analysis Round Robin: anonymized results, figures and Jupyter notebook
<p>The intent of this round robin was to find out how comparable results from different researchers are, who analyse exactly the same processed, corrected dataset.</p> <p>This zip file contains the anonymized results and the jupyter notebook used to do the data processing, analysis and visualisation. Additionally, TEM images of the samples are included. </p>
Don't mention it: challenges to using software mentions to investigate citation and discoverability - Data and Notebooks
<p>This deposit contains the data and Jupyter notebooks used for sampling and annotation analysis of our submission to the PeerJ Computer Science special issue <em>Software Citation, Indexing, and Discoverability </em>(<a href="https://peerj.com/collections/84-software">https://peerj.com/collections/84-software</a>):</p> <blockquote> <p>Stephan Druskat, Neil P. Chue Hong, Sammie Buzzard, Olexandr Konovalov, and Patrick Kornek. Don’t mention it: challenges to using software mentions to investigate citation and discoverability.</p> </blockquote> <p>See the README in this deposit.</p> <p>The contents of this deposit can also be browsed at <a href="https://github.com/softwaresaved/habeas-corpus/tree/main/replication-package">https://github.com/softwaresaved/habeas-corpus/tree/main/replication-package</a>.</p>
Dataset of a Study of Computational reproducibility of Jupyter notebooks from biomedical publications
<p>This repository contains the dataset for the study of <a href="https://doi.org/10.1093/gigascience/giad113">computational reproducibility of Jupyter notebooks from biomedical publications</a>. Our focus lies in evaluating the extent of reproducibility of Jupyter notebooks derived from GitHub repositories linked to publications present in the biomedical literature repository, PubMed Central. We analyzed the reproducibility of Jupyter notebooks from GitHub repositories associated with publications indexed in the biomedical literature repository PubMed Central. The dataset includes the metadata information of the journals, publications, the Github repositories mentioned in the publications and the notebooks present in the Github repositories.</p> <p><strong>Data Collection and Analysis</strong></p> <p>We use the code for reproducibility of Jupyter notebooks from the study done by <a href="../record/2592524">Pimentel et al., 2019</a> and adapted the code from <a href="https://github.com/fusion-jena/ReproduceMeGit">ReproduceMeGit</a>. We provide code for collecting the publication metadata from PubMed Central using <a href="https://biopython.org/docs/1.76/api/Bio.Entrez.html">NCBI Entrez utilities via Biopython</a>.</p> <p>Our approach involves searching PMC using the esearch function for Jupyter notebooks using the query: ``(ipynb OR jupyter OR ipython) AND github''. We meticulously retrieve data in XML format, capturing essential details about journals and articles. By systematically scanning the entire article, encompassing the abstract, body, data availability statement, and supplementary materials, we extract GitHub links. Additionally, we mine repositories for key information such as dependency declarations found in files like requirements.txt, setup.py, and pipfile. Leveraging the GitHub API, we enrich our data by incorporating repository creation dates, update histories, pushes, and programming languages.</p> <p>All the extracted information is stored in a SQLite database. After collecting and creating the database tables, we ran a pipeline to collect the Jupyter notebooks contained in the GitHub repositories based on the code from Pimentel et al., 2019.</p> <p>Our reproducibility pipeline was started on 27 March 2023.</p> <p><strong>Repository Structure</strong></p> <p>Our repository is organized into two main folders:</p> <ul> <li><strong>archaeology</strong>: This directory hosts scripts designed to download, parse, and extract metadata from PubMed Central publications and associated repositories. There are 24 database tables created which store the information on articles, journals, authors, repositories, notebooks, cells, modules, executions, etc. in the db.sqlite database file.</li> <li><strong>analyses</strong>: Here, you will find notebooks instrumental in the in-depth analysis of data related to our study. The db.sqlite file generated by running the archaelogy folder is stored in the analyses folder for further analysis. The path can however be configured in the config.py file. There are two sets of notebooks: one set (naming pattern N[0-9]*.ipynb) is focused on examining data pertaining to repositories and notebooks, while the other set (PMC[0-9]*.ipynb) is for analyzing data associated with publications in PubMed Central, i.e.\ for plots involving data about articles, journals, publication dates or research fields. The resultant figures from the these notebooks are stored in the 'outputs' folder.</li> <li><strong>MethodsWorkflow</strong>: The MethodsWorkflow file provides a conceptual overview of the workflow used in this study.</li> </ul> <p><strong>Accessing Data and Resources:</strong></p> <ul> <li>All the data generated during the initial study can be accessed at https://doi.org/10.5281/zenodo.6802158</li> <li>For the latest results and re-run data, refer to this link.</li> <li>The comprehensive SQLite database that encapsulates all the study's extracted data is stored in the db.sqlite file.</li> <li>The metadata in xml format extracted from PubMed Central which contains the information about the articles and journal can be accessed in pmc.xml file.</li> </ul> <p><strong>System Requirements:</strong></p> <ul> <li>Centos 7 (Documentation: https://www.centos.org/)</li> <li>Conda 4.9.4 (Installation Guide: https://docs.anaconda.com/anaconda/install/linux/)</li> <li>Python 3.7.6 (Download Link: https://www.python.org/downloads/)</li> <li>GitHub account (Get Started: https://github.com/, Requires GitHub Username and Token)</li> <li>gcc 7.3.0 (Installation Guide: https://gcc.gnu.org/install/)</li> <li>lbzip2 (Command: `conda install -c conda-forge lbzip2')</li> </ul> <p><strong>Running the pipeline:</strong></p> <ul> <li>Clone the computational-reproducibility-pmc repository using Git:<br>git clone https://github.com/fusion-jena/computational-reproducibility-pmc.git<br> </li> <li>Navigate to the computational-reproducibility-pmc directory:<br>cd computational-reproducibility-pmc/computational-reproducibility-pmc</li> <li>Configure environment variables in the config.py file:<br>GITHUB_USERNAME = os.environ.get("JUP_GITHUB_USERNAME", "add your github username here")<br>GITHUB_TOKEN = os.environ.get("JUP_GITHUB_PASSWORD", "add your github token here")</li> <li>Other environment variables can also be set in the config.py file.<br>BASE_DIR = Path(os.environ.get("JUP_BASE_DIR", "./")).expanduser() # Add the path of directory where the GitHub repositories will be saved<br>DB_CONNECTION = os.environ.get("JUP_DB_CONNECTION", "sqlite:///db.sqlite") # Add the path where the database is stored.</li> <li>To set up conda environments for each python versions, upgrade pip, install pipenv, and install the archaeology package in each environment, execute:<br>source conda-setup.sh</li> <li>Change to the archaeology directory<br>cd archaeology</li> <li>Activate conda environment. We used py36 to run the pipeline.<br>conda activate py36</li> <li>Execute the main pipeline script (r0_main.py):<br>python r0_main.py</li> </ul> <p><strong>Running the analysis:</strong></p> <ul> <li>Navigate to the analysis directory.<br>cd analyses</li> <li>Activate conda environment. We use raw38 for the analysis of the metadata collected in the study.<br>conda activate raw38</li> <li>Install the required packages using the requirements.txt file.<br>pip install -r requirements.txt</li> <li>Launch Jupyterlab<br>jupyter lab</li> <li>Refer to the Index.ipynb notebook for the execution order and guidance.</li> </ul> <p><strong>References:</strong></p> <ul> <li>Sheeba Samuel, Daniel Mietchen. (2024). Computational reproducibility of Jupyter notebooks from biomedical publications, https://doi.org/10.1093/gigascience/giad113, GigaScience</li> <li>Sheeba Samuel, Daniel Mietchen. (2022). Computational reproducibility of Jupyter notebooks from biomedical publications, https://arxiv.org/pdf/2209.04308.pdf, CoRR abs/2209.04308</li> <li>Sheeba Samuel, & Daniel Mietchen. (2022). Dataset of a Study of Computational reproducibility of Jupyter notebooks from biomedical publications [Data set]. Zenodo. https://doi.org/10.5281/zenodo.6802158</li> </ul> <p> </p>
Outputs of the Jupyter Notebook - Deep learning and variational inversion to quantify and attribute climate change (CIRC23)
<p>The dataset contains the outputs of the notebook "Deep learning and variational inversion to quantify and attribute climate change (CIRC23)" published in The Environmental Data Science Book.</p>
Anisotropic Gold Nanomaterial Synthesis Using Peptide Facet Specificity and Timed Intervention: Experiment and simulation data, figures, and notebooks for recreating figures
<p><strong>/Data</strong></p> <p>Contains all experimental and simulation data used to generate figures in the manuscript. Each folder is labelled with the relevant technique: atomic force microscopy (AFM), dynamic light scattering (DLS), molecular dynamics (MD), small-angle X-ray scattering (SAXS), scanning electron microscopy (SEM), transmission electron microscopy and selected area electron diffraction (TEM-SAED), and ultraviolet-visible light spectroscopy (UV-Vis). Folders may contain subfolders from different experiments and should be unchanged for the notebooks to point to the correct path when loading data. Any file which contains SI in the filename indicates sample information, which includes sample concentration, delay associated with creating the sample (Delay4 is the relevant column name), unique identification (UID), etc.</p> <p><strong>/Data/AFM</strong></p> <p>Each filename is labelled with the number of delay associated with the sample or No Z2 Control in the case of the control sample. Each sample contains 3 files, the raw AFM data, a processed image, and an excel file with line profile measurements.</p> <p><strong>/Data/DLS</strong></p> <p>Contains exported data from Malvern Nano ZS DLS instrument.</p> <p><strong>/Data/Images</strong></p> <p>Images from the colloidal stability experiment.</p> <p><strong>/Data/MD</strong></p> <p>Molecular dynamics data and some notebooks for processing peptide and facet combinations.</p> <p><strong>/Data/SAXS</strong></p> <p>Reduced SAXS data from time-resolved study of nanoplatelet growth.</p> <p><strong>/Data/SEM</strong></p> <p>SEM images of samples prepared with and without dynamic intervention (control).</p> <p><strong>/Data/TEM_SAED</strong></p> <p>TEM images of samples prepared with and without dynamic intervention (control), and SAED image of a nanoplatelet.</p> <p><strong>/Data/UV-Vis</strong></p> <p>Spectroscopy data from different Z2 variants, the microplate based stability assay, cuvette based stability assay, and the time resolved (kinetics) measurements of particle growth as a result of manual dynamic intervention.</p> <p><strong>/Notebooks</strong></p> <p>This folder contains all of the Jupyter notebooks used to plot data and fit SAXS scattering profiles.</p> <p>See <strong>/Notebooks/environment.yml</strong> for packages necessary to execute the notebooks here and in <strong>/Synthesis_Protocol</strong>. We recommend installing this environment by using:</p> <p> </p> <p>Refer to <a href="https://github.com/SasView/sasmodels">https://github.com/SasView/sasmodels</a> and the first cell of <strong>/Notebooks/SAXS_Fitting.ipynb</strong> for specific instructions on how to complete installation of the sasmodels module (sasmodels will be installed by Pip if you correctly use the shared environment.yml file).</p> <p><strong>/Figures</strong></p> <p>Figures presented in the publication</p>
Outputs of the Jupyter Notebook - Variational data assimilation with deep prior (CIRC23)
<p>The repository contains the outputs of the notebook "Variational data assimilation with deep prior (CIRC23)" published in The Environmental Data Science Book.</p>
Constava example notebook dataset
<p>Data set utilised in the example notebook for the Constava package (<a href="https://pypi.org/project/constava/">https://pypi.org/project/constava/</a>)</p>
Assessing Computational Notebook Understandability through Code Metrics Analysis
<p>Computational notebooks have become the primary coding environment for data scientists. Despite their popularity, research on the code quality of these notebooks is still in its infancy, and the code shared in these notebooks is often of poor quality. Considering the importance of maintenance and reusability, it is crucial to pay attention to the comprehension of the notebook code and identify the notebook metrics that play a significant role in their comprehension. The level of code comprehension is a qualitative variable closely associated with the user's opinion about the code. Previous studies have typically employed two approaches to measure it. One approach involves using limited questionnaire methods to review a small number of code pieces. Another approach relies solely on metadata, such as the number of likes and user votes for a project in the software repository. In our approach, we enhanced the measurement of the understandability level of notebook code by leveraging user comments within a software repository. As a case study, we started with 248,761 Kaggle Jupyter notebooks introduced in previous studies and their relevant metadata. To identify user comments associated with code comprehension within the notebooks, we utilized a fine-tuned DistillBERT transformer. We established a \emph{user comment based criterion} for measuring code understandability by considering the number of code understandability-related comments, the upvotes on those comments, the total views of the notebook, and the total upvotes received by the notebook. This criterion has proven to be more effective than alternative methods, making it the ground truth for evaluating the code comprehension of our notebook set. In addition, we collected a total of 34 metrics for 10,857 notebooks, categorized as script-based and notebook-based metrics. These metrics were utilized as features in our dataset. Using the Random Forest classifier, our predictive model achieved 85% accuracy in predicting code comprehension levels in computational notebooks, identifying developer expertise and markdown-based metrics as key factors.</p> <p> </p>
Initial release of open-science notebook
<p>Initial release of open-science notebook of research on comparative phylogeographical model choice.</p>
Jupyter Notebook and comprising data for GRL2023GL106264R: Understanding the Cascade: Removing GCM biases improves dynamically downscaled climate projections
<p>This notebook and attendant files allows users to interface with a small subset of the data used to create the data in GRL2023GL106264R. Also feel free to check out the overall description of the non-bias corrected dynamically downscaled GCMs in WUS-D3 here: https://zenodo.org/records/10635867. This DOI also contains version of WRF 4.1.3 allowing for yearly CH4, CO2, and N2O updates, as well as a 360-day calendar version.</p>
Datasets for the paper "ReSplit: Improving the Structure of Jupyter Notebooks by Re-Splitting Their Cells"
<p>In this archive, you can find all the data used in the paper "ReSplit: Improving the Structure of Jupyter Notebooks by Re-Splitting Their Cells".</p> <p><strong>sklearn_full_cells.csv</strong> is the dataset from the paper of Pimentel et al. filtered with only Data Science notebooks.<br> <strong>complete.csv</strong> is the dataset obtained after the full run of ReSplit on the dataset: both merging and splitting.<br> <strong>split.csv</strong> is the dataset obtained after running only the splitting part of our dataset.<br> <strong>merged.csv</strong> is the dataset obtained after running only the merging part of our dataset.<br> <strong>duplicates_id.csv</strong> contains the IDs of the duplicate notebooks for deduplication.<br> <strong>changes.csv</strong> contains the IDs of the datasets, as well as their length before and after running ReSplit.<br> <strong>survey.csv</strong> is the table with the results of the survey.</p> <p>In the dataset CSVs, each line is a cell that has a unique identifier and an identifier of the corresonding notebook.</p>
Jupyter Notebooks for "Evaluating CephFS Performance vs. Cost on High-Density Commodity Disk Servers" 10.1007/s41781-021-00071-1
<p>Jupyter notebooks used to create plots in article DOI 10.1007/s41781-021-00071-1</p> <p>Title "Evaluating CephFS Performance vs. Cost on High-Density Commodity Disk Servers"</p> <p>Journal "Computing and Software for Big Science"</p>
Biotechnology data analysis training with Jupyter Notebooks
<p>Biotechnology has experienced innovations in analytics and data processing. As the volume of data and its complexity grows, new computational procedures for extracting information are developed. However, the rate of change outpaces the adaptation of biotechnology curricula, necessitating new teaching methodologies to equip biotechnologists with data analysis abilities. To simulate experimental data, we created a virtual organism simulator (<em>silvio</em>) by combining diverse cellular and sub-cellular microbial models. With the <em>silvio </em>Python package, we constructed a computer-based instructional workflow to teach growth curve data analysis, promoter sequence design, and expression rate measurement. The instructional workflow is a Jupyter Notebook with background explanations and Python-based experiment simulations combined. The data analysis is either conducted within the Notebook in Python or externally with Excel. This instructional workflow was separately implemented in two distance courses for Master's students in biology and biotechnology with assessment of the pedagogic efficiency. The concept of using virtual organism simulations that generate coherent results across different experiments can be used to construct consistent and motivating case studies for biotechnological data literacy.</p> <p>Here, the supplementary material is provided.</p> <table> <tbody> <tr> <td>2207_BLS-RecExpSim.mbz</td> <td>Moodle backup file for import as new moodle function.</td> </tr> <tr> <td>BLS_RecExpSim_PerformanceEvaluation Rubric.docx</td> <td>Expected learning outcomes with associated performance levels.</td> </tr> <tr> <td>BLS_SurveryQuestions.docx</td> <td>Survey questions to evaluate the educational approach.</td> </tr> <tr> <td>RecExpSim.html</td> <td>Html-Export of the Jupyter Notebook to teach biotechnology data analysis. This only serves as visual impression of the course because the dynamic Python-evaluations are not functioning.</td> </tr> <tr> <td>RecExpSim_Lecture.pdf</td> <td>Static pdf of preparatory lecture to cover the theoretical aspects in the simulations and to get student on comparable level.</td> </tr> <tr> <td>RecExpSim_Lecture.pptx</td> <td>Adjustable pptx of preparatory lecture to cover the theoretical aspects in the simulations and to get student on comparable level.</td> </tr> </tbody> </table> <p> </p> <p> </p>
Outputs of the Jupyter Notebook - Sea ice forecasting using the IceNet Library
<p>The dataset contains the outputs of the notebook "Sea ice forecasting using the IceNet library" published in The Environmental Data Science Book.</p> <p><strong>Contributions</strong></p> <p><em>Notebook</em></p> <ul> <li>James Byrne (author), British Antarctic Survey, <a href="https://github.com/JimCircadian">@JimCircadian</a></li> <li>Bryn Noel Ubald (author), British Antarctic Survey, <a href="https://github.com/tom-andersson">@tom-andersson</a></li> <li>Wei Ji (reviewer), Development Seed, <a href="https://github.com/weiji14">@weiji14</a></li> <li>William Gregory (reviewer), Princeton University, <a href="https://github.com/William-gregory">@William-gregory</a></li> <li>Anne Fouilloux (editor), Simula Research Laboratory, <a href="https://github.com/annefou">@annefou</a></li> </ul> <p><em>Modelling codebase</em></p> <ul> <li>James Byrne (Code author)</li> <li>Tom Andersson (Science author)</li> <li>Bryn Noel Ubald (Code maintainer and contributor)</li> </ul>
FAIRmat Tutorial 8: Using NOMAD as an Electronic lab notebook (ELN) for FAIR data
<p>Approaching the era of big data-driven materials science, one crucial step to collecting, describing, and sharing experimental data is the adoption of electronic laboratory notebooks (ELN). At present, most synthesis data are not structured comprehensively or not even stored digitally but in handwritten lab books. The <a href="https://www.fairmat-nfdi.eu/fairmat" target="_blank" rel="noopener">FAIRmat project</a> is offering a solution by developing and operating the open-source software <a href="https://nomad-lab.eu/" target="_blank" rel="noopener">NOMAD</a>. NOMAD provides ELN functionalities that aim for a secure environment to protect the integrity of both data and metadata, whilst also affording the flexibility to adopt new synthetic processes or changes to existing ones without recourse to further software development.</p> <p>In this FAIRmat tutorial, we focus on the usage of NOMAD as an ELN which enables the users to generate data following the FAIR principles. We will show how we adopted NOMAD to capture data from synthesis and experiment and make use of an automated data workflow. The key point here is writing a data schema and its implementation in NOMAD. After defining the used terms, we will start explaining this process by writing a simple schema and then go on to more advanced usage of NOMAD, e.g. using the build in csv/xlsx-file parser, automatized data visualization, adding extra functionality by usage of base classes, referencing to other data entries in NOMAD, and searching your ELN data. The tutorial is aimed at both scientists new to NOMAD and structured data as well as data stewards. Each lecture of the tutorial will be followed directly by a Q&A session and a hands-on tutorial.</p> <p><strong>Disclaimer:</strong> NOMAD is being continuously developed based on input and feedback from the scientific community. Hence the features, services or interface may have changed since the time of recording of this video. For up-to-date information please consult our latest tutorials and the NOMAD documentation <a href="https://nomad-lab.eu/prod/v1/docs/">https://nomad-lab.eu/prod/v1/docs/</a></p>
Notebook Archaeology Data
<p>Two zip files containing datasets related to the publication "Notebook Archaeology: Inferring Provenance from Computational Notebooks", IPAW 2021. sampled-nbs.zip contains the 100,000 Jupyter notebooks randomly sampled from GitHub notebooks published through February 2021. The notebook outputs have been stripped. session-histories.zip contains the IPython session histories downloaded from GitHub that were published through February 2021.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.