Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
166
datasets available to search
ShareScore release 0.7.1
Dataset results
166 results for “open-source”
Ecosystem-Level Factors Affecting the Survival of Open-Source Projects: A Case Study of the PyPI Ecosystem - the dataset
<pre><em>Replication pack, FSE2018 submission #164: </em><em>------------------------------------------ </em></pre> <pre><strong>**</strong>Working title:<strong>** </strong>Ecosystem-Level Factors Affecting the Survival of Open-Source Projects: A Case Study of the PyPI Ecosystem <strong>**</strong>Note:<strong>** </strong>link to data artifacts is already included in the paper. Link to the code will be included in the Camera Ready version as well. <em>Content description </em><em>=================== </em> <strong>- **</strong>ghd-0.1.0.zip<strong>** </strong>- the code archive. This code produces the dataset files described below <strong>- **</strong>settings.py<strong>** </strong>- settings template for the code archive. <strong>- **</strong>dataset_minimal_Jan_2018.zip<strong>** </strong>- the minimally sufficient version of the dataset. This dataset only includes stats aggregated by the ecosystem (PyPI) <strong>- **</strong>dataset_full_Jan_2018.tgz<strong>** </strong>- full version of the dataset, including project-level statistics. It is ~34Gb unpacked. This dataset still doesn't include PyPI packages themselves, which take around 2TB. <strong>- **</strong>build_model.r, helpers.r<strong>** </strong>- R files to process the survival data (`survival_data.csv` in <strong>**</strong>dataset_minimal_Jan_2018.zip<strong>**</strong>, `common.cache/survival_data.pypi_2008_2017-12_6.csv` in <strong>**</strong>dataset_full_Jan_2018.tgz<strong>**</strong>) <strong>- **</strong>Interview protocol.pdf<strong>** </strong>- approximate protocol used for semistructured interviews. <strong>- </strong>LICENSE - text of GPL v3, under which this dataset is published <strong>- </strong>INSTALL.md - replication guide (~2 pages)</pre> <pre><em>Replication guide </em><em>================= </em> <em>Step 0 - prerequisites </em><em>---------------------- </em> <strong>- </strong>Unix-compatible OS (Linux or OS X) <strong>- </strong>Python interpreter (2.7 was used; Python 3 compatibility is highly likely) <strong>- </strong>R 3.4 or higher (3.4.4 was used, 3.2 is known to be incompatible) Depending on detalization level (see Step 2 for more details): <strong>- </strong>up to 2Tb of disk space (see Step 2 detalization levels) <strong>- </strong>at least 16Gb of RAM (64 preferable) <strong>- </strong>few hours to few month of processing time <em>Step 1 - software </em><em>---------------- </em> <strong>- </strong>unpack <strong>**</strong>ghd-0.1.0.zip<strong>**</strong>, or clone from gitlab: git clone https://gitlab.com/user2589/ghd.git git checkout 0.1.0 `cd` into the extracted folder. All commands below assume it as a current directory. <strong>- </strong>copy `settings.py` into the extracted folder. Edit the file: <strong> * </strong>set `DATASET_PATH` to some newly created folder path <strong> * </strong>add at least one GitHub API token to `SCRAPER_GITHUB_API_TOKENS` <strong>- </strong>install docker. For Ubuntu Linux, the command is `sudo apt-get install docker-compose` <strong>- </strong>install libarchive and headers: `sudo apt-get install libarchive-dev` <strong>- </strong>(optional) to replicate on NPM, install yajl: `sudo apt-get install yajl-tools` Without this dependency, you might get an error on the next step, but it's safe to ignore. <strong>- </strong>install Python libraries: `pip install --user -r requirements.txt` . <strong>- </strong>disable all APIs except GitHub (Bitbucket and Gitlab support were not yet implemented when this study was in progress): edit `scraper/init.py`, comment out everything except GitHub support in `PROVIDERS`. <em>Step 2 - obtaining the dataset </em><em>----------------------------- </em> The ultimate goal of this step is to get output of the Python function `common.utils.survival_data()` and save it into a CSV file: # copy and paste into a Python console from common import utils survival_data = utils.survival_data('pypi', '2008', smoothing=6) survival_data.to_csv('survival_data.csv') Since full replication will take several months, here are some ways to speedup the process: <em>####Option 2.a, difficulty level: easiest </em> Just use the precomputed data. Step 1 is not necessary under this scenario. <strong>- </strong>extract <strong>**</strong>dataset_minimal_Jan_2018.zip<strong>** </strong><strong>- </strong>get `survival_data.csv`, go to the next step <em>####Option 2.b, difficulty level: easy </em> Use precomputed longitudinal feature values to build the final table. The whole process will take 15..30 minutes. <strong>- </strong>create a folder `<DATASET_PATH>/common.cache`, where `<DATASET_PATH>` is the value of the variable `DATASET_PATH` in `settings.py` <strong>- </strong>extract <strong>**</strong>dataset_minimal_Jan_2018<strong>** </strong>to the newly created folder <strong>- </strong>rename files: mv backporting.csv monthly_data.pypi_backporting.csv mv cc_degree.csv monthly_data.pypi_cc_degree.csv mv commercial.csv monthly_data.pypi_commercial.csv mv commits.csv monthly_data.pypi_commits.csv mv contributors.csv monthly_data.pypi_contributors.csv mv dc_katz.csv monthly_data.pypi_dc_katz.csv mv downstreams.csv monthly_data.pypi_downstreams.csv mv d_upstreams.csv monthly_data.pypi_d_upstreams.csv mv github_user_info.csv user_info.pypi.csv mv issues.csv monthly_data.pypi_issues.csv mv non_dev_issues.csv monthly_data.pypi_non_dev_issues.csv mv non_dev_submitters.csv monthly_data.pypi_non_dev_submitters mv package_urls.csv package_urls.pypi.csv mv q90.csv monthly_data.pypi_q90.csv # raw_dependencies.csv is not required # raw_packages_info.csv is not required # Feel free to read README.md for more details about the data mv submitters.csv monthly_data.pypi_submitters.csv # In this scenario we'll generate a new survival_data.csv mv university.csv monthly_data.pypi_university.csv mv upstreams.csv monthly_data.pypi_upstreams.csv <strong>- </strong>edit `common/decorators.py`, set `DEFAULT_EXPIRY` to some higher value, e.g. `DEFAULT_EXPIRY = float('inf') # cache never expires` Then, use the Python code above to obtain `survival_data.csv`. <em>####Option 2.c, difficulty level: medium </em> Use predownloaded raw data to build longitudinal feature values, and then the dataset. Despite most of the data is cached, some functions will pull up updates which might take anywhere from days to couple weeks to run. <strong>- </strong>Download <strong>**</strong>dataset_full_Jan_2018.tgz<strong>** </strong>from http://k.soberi.us/dataset_full_Jan_2018.tgz . This file is not included in this archive because of its size (5.4Gb compressed, 34Gb unpacked). <strong>- </strong>edit `common/decorators.py`, set `DEFAULT_EXPIRY` to some higher value, e.g. `DEFAULT_EXPIRY = float('inf') # cache never expires` <strong>- </strong>extract the content of this archive into `<DATASET_PATH>`. <strong>- </strong>clean up `<DATASET_PATH>/common.cache` (otherwise you'll get Step 2.a. You can reproduce Step 2.b by deleting only `survival_data.pypi_2008_2017-12_6.csv`) Run the Python code above to obtain `survival_data.csv`. <em>####Option 2.d, difficulty level: hard </em> Build the dataset from scratch. Although most of the processing is parallelized, it will take at least couple months on a reasonably powerful server (32 cores, 512G of RAM, 2Tb+ of HDD space in our setup). <strong>- </strong>ensure the `<DATASET_PATH>` is empty <strong>- </strong>add more GitHub tokens (borrow from your coworkers) to `settings.py`. Run the Python code above to obtain `survival_data.csv`. <em>Step 3 - run the regression </em><em>--------------------------- </em> install R libraries: install.packages(c("htmlTable", "OIsurv", "survival", "car", "survminer", "ggplot2", "sqldf", "pscl", "texreg", "xtable")) Use `build_model.r` (e.g. in RStudio) and produced `survival_data.csv` to build the regressions used in the paper. This process takes at least 16Gb of RAM and takes few hours to run due to the gigantic size of the dataset. </pre>
Data from: An inexpensive and open-source method to study large terrestrial animal diet and behavior using time-lapse video and GPS
1. The behavior of free-ranging animals is difficult to study, especially on the large spatial and temporal scales relevant to long-lived large species. Animal-borne video and environmental data collection systems (AVEDs) record behavior and other data in real time as animals conduct daily activities. However, few studies have combined systematically collected, long term AVED foraging data with environmental and movement data to test hypotheses on animal foraging. Additionally, AVEDs are often either prohibitively expensive, or require extensive fabrication and programming knowledge. 2. The video and coordinate animal-mounted system (VACAMS) is an animal-mounted data collection system based on a modified GoPro® action camera platform that records short, first "person" perspective videos of animal behavior on an automated time-lapse schedule. As most videos are georeferenced, researchers can return to the locations of specific behaviors and collect accurate, fine-grained data on non-woody vegetation and other habitat characteristics that may influence animal behavior. Moreover, VACAMS are inexpensive and easy to use. 3. This study describes VACAMS preliminary data on cattle foraging and a hypothesis exploring free-ranging cattle browsing habits throughout the rainy season in the tropical dry forest of Sonora, Mexico. I generated a database of vegetation types consumed by cows each month (Annual, Woody, and Leaf litter) and compared actual vegetation type frequencies to a priori assumptions based on seasonal patterns of forage availability. During the monsoons, when palatable vegetation was abundant, frequencies of annual and woody perennial vegetation in cattle diets did not differ from month to month. When the rains ceased and palatable vegetation became scarce, cows switched to leaf litter, dead annual vegetation, twigs, and dried leguminous fruits. 4. Open source software and commercially available hardware make VACAMS financially attainable for many researchers, land managers, students, and other user groups. VACAMS could be used on a range of domestic and semi-domestic free-ranging animals, particularly in dense forests where conventional observations are impossible. With improvements to GPS battery life and durability, the weakest points of the system, VACAMS could also potentially apply to studies of other large terrestrial animals.
Open-Source Software Product Line Extraction Processes: the ArgoUML-SPL and Phaser Cases
<p>Collection of datasets and analysis scripts supporting the information provided in the text.</p> <p>There are two compressed files, one for the ArgoUML data and one for the Phaser data. Each compressed file contains a README describing important information.</p>
OCTAVA: an open-source toolbox for quantitative analysis of optical coherence tomography angiography images
<p>This is a dataset of OCTA images used in the development of the manuscript <em>OCTAVA: an open-source toolbox for quantitative analysis of optical coherence tomography angiography images</em></p>
Dependency Management Bots in Open-Source Systems - Prevalence and Adoption
<p>This is a re-analysis package for the paper: Dependency Management Bots in Open-Source Systems - Prevalence and Adoption. The package includes scripts and processed data collected from GitHub about five investigated dependency management bots: Dependabot, Depfu, Greenkeeper, Pyup and Renovate.</p>
A New Open-source Geomagnetosphere Propagation Tool (OTSO) and its Applications - Data
<p>Data files for the computations done with OTSO for the asymptotic cones and effective cut-off rigidities for several neutron monitor stations during three ground-level enhancement events (GLE 66, 70, 71). The computations for GLE 66 and 71 were done using three external geomagnetic field models (TSY89, TSY96, TSY01). Data for the global map of effective cut-off rigidities during GLE70 is also included. Data is in CSV format.</p>
Open-source Software Governance Documentation Dataset on GitHub
<p>This dataset contains 710 GitHub-hosted OSS projects, which contain a governance file in the root directory of the project. It also contains commits, issues, and comments on each project.</p>
MALDI-MS dataset for use with open-source untargeted metabolomic workflow for complex biological samples
<p class="MsoNormal">Untargeted metabolomics is a powerful tool for measuring and understanding complex biological chemistries. However, employment, bioinformatics and downstream analysis of mass spectrometry (MS) data can be daunting for inexperienced users. Numerous open-source and free to-use data processing and analysis tools exist for various untargeted MS approaches, but choosing the 'correct' pipeline isn't straight-forward. This data set can be used in conjunction with a user-friendly online guide which presents a workflow for connecting these tools to process, analyse and annotate various untargeted MS datasets. The workflow is intended to guide exploratory analysis in order to inform decision-making regarding costly and time-consuming downstream targeted MS approaches. The workflow provides practical advice concerning experimental design, organisation of data and downstream analysis, and offers details on sharing and storing valuable MS data for posterity. The workflow is editable and modular, allowing flexibility for updated/ changing methodologies and increased clarity and detail as user participation becomes more common allowing contributions and improvements to the workflow via the online repository. </p>
Supporting material for PyESDv1.0.1 An open-source Python framework for empirical-statistical downscaling of climate information
<p>The nature and severity of climate change impacts varies significantly from region to region. Consequently, high-resolution climate information is needed for meaningful impact assessments and the design of mitigation strategies. This demand has led to an increase in the coupling of Empirical Statistical Downscaling (ESD) models to General Circulation Model (GCM) simulations of future climate. Here, we present a new open-source Python package (<em>pyESD; </em>github.com/Dan-Boat/PyESD) that implements several Perfect Prognosis ESD (PP-ESD) methods and the whole downscaling cycle. The latter includes routines for data preparation, predictor selection and construction, model selection and training, evaluation, utility tools for relevant statistical tests, visualization, and more. The package includes a collection of well-established Machine Learning algorithms and allows the user to choose a variety of estimators, cross-validation schemes, objective function measures, hyperparameter optimization, etc., in relatively few lines of code. The package is highly modular and flexible and allows quick and reproducible downscaling of any climate information, such as precipitation, temperature, wind speed, or even glacial retreat. The dataset presented here serves as supporting material for the package description and evaluation manuscript</p>
CCG: Beyond the Dams: Combatting Hydropower Over-reliance & Securing Pathways for a Low-carbon Future for Laos' Electricity Sector using OSeMOSYS (Open-Source Energy Modelling System)
<p>Seven clicSAND scenario files for <strong>Beyond the Dams: Combatting Hydropower Over-reliance & Securing Pathways for a Low-carbon Future for Laos' Electricity Sector using OSeMOSYS (Open-Source Energy Modelling System).</strong> </p> <p><strong>How to Visualise Results Online and Offline</strong> outline the steps required to re-run the scenarios on OSeMOSYS Cloud</p> <p><strong>Scenario Short Note</strong> outlines the steps to replicate the analysis and rebuild the scenarios</p> <p><strong>Annex - Input Data and Assumptions</strong> listing the data sources and assumptions in the scenarios</p>
Over and Under Sampled Data-sets of Code Issues in Java Open-Source Projects
<p>The dataset comprises code changes made to 15 Java Open-Source projects, classified with sentiment values (0 for negative and 1 for positive) based on developer reviews during various revision submissions. The dataset is available in 8 versions, each containing a sampled dataset using an over or under-sampling technique.</p>
[Tool demo] Prospector: a Tool to Find Fixes to Known Vulnerabilities of Open-Source Projects
<p> </p> <p><strong>>>>> TOOL DEMO SCREENCAST</strong>: please download file <strong><a href="https://zenodo.org/record/7974442/files/tool_demo_final.mp4?download=1">tool_demo_final.mp4</a></strong>. <strong><<<<</strong></p> <p> </p> <p><strong>ABSTRACT</strong>: Though vulnerability databases are key for monitoring known vulnerabilities in open-source projects, they rarely contain information about the code changes that fix the flaws they describe. Finding them is time-consuming and error-prone as it involves the analysis of multiple, unstructured resources. </p> <p>In this paper we present \prospector, a tool that supports mapping<br> vulnerability advisories from vulnerability databases onto the corresponding fix in the source code. \prospector employs a set of heuristics that mimics and automates the<br> strategies that would be employed by human security experts.</p> <p>Given an advisory expressed in natural language, \prospector processes the commits found in the target source code repository, ranks them based on a set of predefined rules, and produces a report that the user can inspect to<br> determine which commits to retain as the actual fix.<br> The tool is publicly available and is released under the Apache 2.0 license.</p>
Supplementary material for "A Rapid, Open-Source CCT Predictor for Low Alloy Steels, and its Application to Compositionally Heterogeneous Material"
<p>The complete collection of measured, modelled and analysed data associated with the work "A Rapid, Open-Source CCT Predictor for Low Alloy Steels, and its Application to Compositionally Heterogeneous Material" and includes:</p> <ol> <li>Measured and analysed dilatometry data.</li> <li>Optical micrographs of as-cooled microstructures.</li> <li>Microhardness measurements of the as-cooled samples.</li> <li>PAG size analysis.</li> <li>Modelled CCT data.</li> <li>SA-540 EPMA data.</li> <li>Modelled results from adapting the model to consider SA-540 chemical heterogeneity.</li> <li>Full chemical analysis for each alloy examined.</li> <li>Modelled Thermo-Calc CCT data.</li> <li>Modelled JMatPro CCT data.</li> </ol>
An analysis of the impact of domain debt in open-source projects
<p>Datasets related to the thesis paper:</p> <ul> <li><em>original_boa.txt</em> contains the intial output given by <a href="https://boa.cs.iastate.edu/boa/index.php">BOA</a>, while <em>filtered_boa.txt</em> comprises the set of repositories with at most five domain entity packages</li> <li><em>original_seart.txt</em> included data as returned by <a href="https://seart-ghs.si.usi.ch/">SEART</a></li> <li><em>database_structure.sql</em> is the SQL file that may be exploited to perform further analysis on the two datasets</li> <li><em>repos.json</em> contains the final list of repositories, as chosen in the study</li> </ul>
EvoSL: A Large Open-Source Corpus of Changes in Simulink Models & Projects
<p>EvoSL is a corpus of 924 Simulink repositories useful to perform model evolutionary studies. <a href="https://zenodo.org/api/files/cbb76da1-1077-4020-8d7c-1c6f4f7b3d36/dataset_er_derived_full_schema.pdf?versionId=a9270932-acca-40d8-b5aa-88330525ade8">dataset_er_derived_full_schema</a> shows the full schema design of the EvoSL_v1 SQLite database. Refer to the paper for full details of the dataset.</p> <p>EvoSL.sqlite (Table: Model_element_changes) contains over 2+ million element-level change raw data extracted from 14k+ Simulink model snapshots. Refer to the tool (https://github.com/50417/EvoSL-Tool) to discard duplicates using its cleaning module.</p> <p>"EvoSL: A Large Open-Source Corpus of Changes in Simulink Models & Projects" is accepted in <a href="https://conf.researchr.org/track/models-2023/models-2023-technical-track">MODELS 2023</a> (CORE A, acceptance rate: 24.6%)</p> <p> </p>
Data from: Scoring thermal limits in small insects using open-source, computer assisted motion detection
<p>Scoring large amounts of thermal tolerance traits live or with recorded video can be time consuming and susceptible to investigator bias, and as with many physiological measurements, there can be trade-offs between accuracy and throughput. Recent studies show that particle tracking is a viable alternative to manually scoring videos, although it may not detect subtle movements, and many of the software options are proprietary and costly. In this study, we present a novel strategy for automated scoring of thermal tolerance videos by inferring motor activity with motion detection using an open-source Python command line application called DIME (Detector of Insect Motion Endpoint). We apply our strategy to both dynamic and static thermal tolerance assays, and our results indicate that DIME can accurately measure thermal acclimation responses, generally agrees with visual estimates of thermal limits, and can significantly increase the throughput over manual methods.</p>
Data from: An inexpensive and open-source method to study large terrestrial animal diet and behavior using time-lapse video and GPS
Open the record for dataset details and reuse information.
Data from: A look inside a flexible open-source scanning electrochemical probe microscope
Open the record for dataset details and reuse information.
Data from: Scoring thermal limits in small insects using open-source, computer assisted motion detection
Open the record for dataset details and reuse information.
MALDI-MS dataset for use with open-source untargeted metabolomic workflow for complex biological samples
Open the record for dataset details and reuse information.
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.