Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
207
datasets available to search
ShareScore release 0.7.1
Dataset results
207 results for “github”
Extra data to accompany code in GitHub burntfields_punjab, both used in Walker et. al. 2022, Detecting crop burning in India using satellite data
<p>Supplementary data files to accompany GitHub code 'burntfields_punjab' supporting Walker et. al. (2022) Detecting crop burning in India using satellite data [<a href="https://arxiv.org/abs/2209.10148">available here</a>] and Jack et. al. (2024) Money (not) to burn: Payments for ecosystem services to reduce crop residue burning).</p> <p>Includes custom Sentinel-2 cloud masks and data from Sentinel-2 Spectral Mixture Analysis to highlight Char (burning) based on general concept and methods from Daldegan et. al (2019). Spectral mixture analysis in Google Earth Engine to model and delineate fire scars over a large extent and a long time-series in a rainforest-savanna transition zone. Remote Sensing of Environment 232, 111340. </p> <p>Note: Bands in weekly BASMA layers are: 0 = green vegetation, 1 = Non-productive vegetation and bare soil, 2 = Char (burned).</p> <p>further details are provided at: <a href="https://github.com/klwalker-sb/burntfields_punjab">https://github.com/klwalker-sb/burntfields_punjab</a> (archived at: <a href="https://doi.org/10.5281/zenodo.11225292" target="_blank" rel="noopener">DOI: 10.5281/zenodo.11225292</a>)</p>
BotHawk: GitHub Bot Account Detection Dataset
<div> <p>The <strong>"BotHawk: GitHub Bot Account Detection Dataset"</strong> is a specialized resource developed for the research and development community focused on identifying and understanding bot accounts within the GitHub ecosystem. This comprehensive dataset offers an in-depth look into the behaviors, contributions, and patterns of bot accounts, distinguishing them from human user activities on GitHub.</p> <p>Designed to enhance the accuracy and efficiency of bot detection algorithms, the BotHawk Dataset encompasses a variety of features, including commit history, issue participation, pull request activities, and other interaction metrics specific to GitHub. It provides an invaluable foundation for developing machine learning models capable of distinguishing between bot-generated and human-generated activities, contributing to more secure and trustworthy interactions within the GitHub platform.</p> <p>This dataset not only serves as a tool for academic research in cybersecurity, machine learning, and software engineering but also offers practical applications for developers and platform administrators seeking to improve the health of their projects by filtering out bot-based noise and manipulation. The BotHawk dataset is instrumental in advancing our comprehension of bot dynamics within the GitHub ecosystem, supporting the broader objective of fostering a secure, transparent, and collaborative open-source community.</p> </div>
(No) Influence of Continuous Integration on the Development Activity in GitHub Projects — Dataset
<p>This dataset is based on the TravisTorrent dataset released 2017-01-11 (https://travistorrent.testroots.org), the Google BigQuery GHTorrent dataset accessed 2017-07-03, and the Git log history of all projects in the dataset, retrieved 2017-07-16 and 2017-07-17.</p> <p>We selected projects hosted on GitHub that employ the Continuous Integration (CI) system Travis CI. We identified the projects using the TravisTorrent data set and considered projects that:</p> <ol> <li>used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent),</li> <li>were active for at least one year (365 days) before the first build with Travis CI (before_ci),</li> <li>used Travis CI at least for one year (during_ci),</li> <li>had commit or merge activity on the default branch in both of these phases, and</li> <li>used the default branch to trigger builds.</li> </ol> <p>To derive the time frames, we employed the GHTorrent Big Query data set. The resulting sample contains 113 projects. Of these projects, 89 are Ruby projects and 24 are Java projects. For our analysis, we only consider the activity one year before and after the first build.</p> <p>We cloned the selected project repositories and extracted the version history for all branches (see https://github.com/sbaltes/git-log-parser). For each repo and branch, we created one log file with all regular commits and one log file with all merges. We only considered commits changing non-binary files and applied a file extension filter to only consider changes to Java or Ruby source code files. From the log files, we then extracted metadata about the commits and stored this data in CSV files (see https://github.com/sbaltes/git-log-parser).</p> <p>We also retrieved a random sample of GitHub project to validate the effects we observed in the CI project sample. We only considered projects that:</p> <ol> <li>have Java or Ruby as their project language</li> <li>used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent)</li> <li>have commit activity for at least two years (730 days)</li> <li>are engineered software projects (at least 10 watchers)</li> <li>were not in the TravisTorrent dataset</li> </ol> <p>In total, 8,046 projects satisfied those constraints. We drew a random sample of 800 projects from this sampling frame and retrieved the commit and merge data in the same way as for the CI sample. We then split the development activity at the median development date, removed projects without commits or merges in either of the two resulting time spans, and then manually checked the remaining projects to remove the ones with CI configuration files. The final comparision sample contained 60 non-CI projects.</p> <p>This dataset contains the following files:</p> <p><strong>tr_projects_sample_filtered_2.csv</strong><br> A CSV file with information about the 113 selected projects.</p> <p><strong>tr_sample_commits_default_branch_before_ci.csv<br> tr_sample_commits_default_branch_during_ci.csv</strong><br> One CSV file with information about all commits to the default branch before and after the first CI build. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns:</p> <p>project: GitHub project name ("/" replaced by "_").<br> branch: The branch to which the commit was made.<br> hash_value: The SHA1 hash value of the commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit.</p> <p><strong>tr_sample_merges_default_branch_before_ci.csv<br> tr_sample_merges_default_branch_during_ci.csv</strong><br> One CSV file with information about all merges into the default branch before and after the first CI build. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns:</p> <p>project: GitHub project name ("/" replaced by "_").<br> branch: The destination branch of the merge.<br> hash_value: The SHA1 hash value of the merge commit.<br> merged_commits: Unique hash value prefixes of the commits merged with this commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit.<br> pull_request_id: ID of the GitHub pull request that has been merged with this commit (extracted from log message).<br> source_user: GitHub login name of the user who initiated the pull request (extracted from log message).<br> source_branch : Source branch of the pull request (extracted from log message).</p> <p><strong>comparison_project_sample_800.csv</strong><br> A CSV file with information about the 800 projects in the comparison sample.</p> <p><strong>commits_default_branch_before_mid.csv<br> commits_default_branch_after_mid.csv</strong><br> One CSV file with information about all commits to the default branch before and after the medium date of the commit history. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the commits tables described above.</p> <p><strong>merges_default_branch_before_mid.csv<br> merges_default_branch_after_mid.csv</strong><br> One CSV file with information about all merges into the default branch before and after the medium date of the commit history. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the merge tables described above.</p>
Beyond Textual Issues: Understanding the Usage and Impact of GitHub Reactions
<p>Recently, GitHub introduced a new social feature, named reactions, which are pictorial characters similar to the emoji symbols widely used nowadays in text-based communications. Particularly, GitHub users can use a set of such symbols to react to issues and pull requests. However, little is known about the real usage and benefits of GitHub reactions. In this paper, we analyze the reactions provided by developers to more than 2.5 million issues and 9.7 million issue comments, in order to answer an extensive list of ten research questions about the usage and adoption of reactions. We show that reactions are being increasingly used by open-source developers. Moreover, we also found that issues with reactions usually take more time to be closed and have longer discussions.</p> <p>This dataset contains the data used in the paper "Beyond Textual Issues: Understanding the Usage and Impact of GitHub Reactions", accepted for SBES 2019.</p>
A Panel Data Set of Cryptocurrency Development Activity on GitHub
<p>Contents:</p> <ul> <li><strong>all-sorted-recovered-normalized-2018-01-21-to-2019-02-04.csv</strong>: CSV format of all data, sorted by date. This file contains some imputed values for missing data, and all fields across all repositories and normalized to "null". This is the most convenient form to use.</li> <li><strong>all-sorted-2018-01-21-to-2019-02-04.csv</strong>: CSV format of all, sorted by date. It is the raw data after processing the raw format.</li> <li><strong>raw-data-2018-01-21-to-2019-02-04.tar.gz</strong>: The raw format of data collected (S-expressions). Contains additional contributor data and CoinMarketCap data not currently in the CSV datasets.</li> <li><strong>recovered.patch</strong>: The modification on <strong>all-sorted-2018-01-21-to-2019-02-04.csv</strong> after recovering (imputing) data<strong>, </strong>showing what was recovered.</li> <li><strong>recovered-normalized.patch</strong>: The modification of <strong>all-sorted-2018-01-21-to-2019-02-04.csv </strong>after normalizing the recovered data set. Thus, patching <strong>all-sorted-2018-01-21-to-2019-02-04.csv </strong>with<strong> recovered.patch</strong>, then <strong>recovered-normalized.patch </strong>gives <strong>all-sorted-recovered-normalized-2018-01-21-to-2019-02-04.csv</strong></li> <li><strong>missing-dates.txt</strong>: Days for which we missed GitHub data collection (partial or completely).</li> </ul> <p>Related publications:</p> <pre><code>@inproceedings{van-tonder-crypto-oss-2019, title = {{A Panel Data Set of Cryptocurrency Development Activity on GitHub}}, booktitle = "International Conference on Mining Software Repositories", author = "{van~Tonder}, Rijnard and Trockman, Asher and {Le~Goues}, Claire", series = {MSR '19}, year = 2019 } @inproceedings{trockman-striking-gold-2019, title = {{Striking Gold in Software Repositories? An Econometric Study of Cryptocurrencies on GitHub}}, booktitle = "International Conference on Mining Software Repositories", author = "Trockman, Asher and {van~Tonder}, Rijnard and Vasilescu, Bogdan", series = {MSR '19}, year = 2019 }</code></pre> <p>Related code: <a href="https://github.com/rvantonder/CryptOSS">https://github.com/rvantonder/CryptOSS</a></p>
A dataset of GitHub Actions workflow histories
<p>This replication package accompagnies the dataset and exploratory empirical analysis reported in the paper "A dataset of GitHub Actions workflow histories" published in the IEEE MSR 2024 conference. (The Jupyter notebook can be found in previous version of this dataset).</p> <p><em><strong>Important notice :</strong> It looks like Zenodo is compressing gzipped files two times without notice, they are "double compressed". So, when you download them they should be named : <code>x.gz.gz</code> instead of <code>x.gz</code>. Notice that the provided MD5 refers to the original file. </em></p> <p><em><strong>2025-10-09 update: update repositories list and observation period</strong>. We now have 3M+ workflows from 49.2K+ repositories. We consider repositories with at least one commit after August 25th, 2024, and they were pulled on August 25th-26th, 2025.</em></p> <p>2025-04-15 update: fix missing metadata and minor notation bug. (unchanged observation period)</p> <p><strong>2024-10-25 update: update repositories list and observation period</strong>. <em>We now have 2.3M+ workflows from 43.3K+ repositories. We consider repositories with at least one commit after January 1st, 2024, and they were pulled on October 7th, 2024.</em></p> <p>2024-07-09 update: fix sometimes invalid <code>valid_yaml</code> flag.</p> <p>2024-04-30: initial version</p> <p>The dataset was created as follow : </p> <ol> <li>First, we used GitHub SEART (on August 25th, 2025) to get a list of every non-fork repositories created at least one year before. having at least 300 commits and at least 100 stars where at least one commit was made in the last year. (The goal of these filter is to exclude experimental and personnal repositories).</li> <li>We checked if a folder <code>.github/workflows</code> existed. We filtered out those that did not contained this folder and pulled the others (on August 25th-26th, 2025).</li> <li>We applied the tool <code>gigawork</code> (version 1.4.2) to extract every files from this folder. The exact command used is <code>python batch.py -d /ourDataFolder/repositories -e /ourDataFolder/errors -o /ourDataFolder/output -r /ourDataFolder/repositories_everything.csv.gz -- -w /ourDataFolder/workflows_auxiliaries</code>. (The script <code>batch.py</code> can be found <a href="https://github.com/cardoeng/gigawork/blob/master/scripts/batch.py" target="_blank" rel="noopener">on GitHub</a>).</li> <li>We concatenated every files in <code>/ourDataFolder/output</code> into a csv (using <code>cat headers.csv output/*.csv > workflows_auxiliaries.csv</code> in <code>/ourDataFolder</code>) and compressed it.</li> <li>We added the column <code>uid</code> via a script available <a href="https://github.com/cardoeng/gigawork/blob/master/scripts/uid.py">on GitHub.</a></li> <li>Finally, we archived the folder with pigz <code>/ourDataFolder/workflows</code> (<code>tar -c --use-compress-program=pigz -f workflows_auxiliaries.tar.gz /ourDataFolder/workflows</code>)</li> </ol> <p>Using the extracted data, the following files were created :</p> <ol> <li><code>workflows.tar.gz</code> contains the dataset of GitHub Actions workflow file histories.</li> <li><code>workflows_auxiliaries.tar.gz</code> is a similar file containing also auxiliary files.</li> <li><code>workflows.csv.gz</code> contains the metadata for the extracted workflow files.</li> <li><code>workflows_auxiliaries.csv.gz</code> is a similar file containing also metadata for auxiliary files.</li> <li><code>repositories.csv.gz</code> contains metadata about the GitHub repositories containing the workflow files. These metadata were extracted using the SEART Search tool. </li> </ol> <p>The metadata is separated in different columns:</p> <ol> <li><code>repository</code>: The repository (author and repository name) from which the workflow was extracted. The separator "/" allows to distinguish between the author and the repository name</li> <li><code>commit_hash</code>: The commit hash returned by git</li> <li><code>author_name</code>: The name of the author that changed this file</li> <li><code>author_email</code>: The email of the author that changed this file</li> <li><code>committer_name</code>: The name of the committer</li> <li><code>committer_email</code>: The email of the committer</li> <li><code>committed_date</code>: The committed date of the commit</li> <li><code>authored_date</code>: The authored date of the commit</li> <li><code>file_path</code>: The path to this file in the repository</li> <li><code>previous_file_path</code>: The path to this file before it has been touched</li> <li><code>file_hash</code>: The name of the related workflow file in the dataset</li> <li><code>previous_file_hash</code>: The name of the related workflow file in the dataset, before it has been touched</li> <li><code>git_change_type</code>: A single letter (A,D, M or R) representing the type of change made to the workflow (Added, Deleted, Modified or Renamed). This letter is given by <code>gitpython</code> and provided as is. </li> <li><code>valid_yaml</code>: A boolean indicating if the file is a valid YAML file.</li> <li><code>probably_workflow</code>: A boolean representing if the file contains the YAML key <code>on</code> and <code>jobs</code>. (Note that it can still be an invalid YAML file).</li> <li><code>valid_workflow</code>: A boolean indicating if the file respect the syntax of GitHub Actions workflow. A freely available JSON Schema (used by gigawork) was used in this goal.</li> <li><code>uid</code>: Unique identifier for a given file surviving modifications and renames. It is generated on the addition of the file and stays the same until the file is deleted. Renamings does not change the identifier.</li> </ol> <p>Both <code>workflows.csv.gz</code> and <code>workflows_auxiliaries.csv.gz</code> are following this format.</p>
The Unofficial Guide on applying NCN Open Access rules to GitHub repositories.
<p><b>The Unofficial Guide on applying NCN Open Access rules to GitHub repositories.</b> <i>Some</i> HTML code can be used here.</p>
GIRT-Data: Sampling GitHub Issue Report Templates
<p><strong>GIRT-Data</strong> is the first and largest dataset of <strong>issue report templates (IRTs)</strong> in both YAML and Markdown format. This dataset and its corresponding open-source crawler tool are intended to support research in this area and to encourage more developers to use IRTs in their repositories. The stable version of the dataset, containing <code>1_084_300</code> repositories, that <code>50_032</code> of them support IRTs.</p> <p>For more details see the GitHub page of the dataset: <a href="https://github.com/kargaranamir/girt-data">https://github.com/kargaranamir/girt-data</a></p> <p><br> The dataset is accepted for <a href="https://conf.researchr.org/track/msr-2023/msr-2023-data-showcase">MSR 2023</a> conference, under the title of "GIRT-Data: Sampling GitHub Issue Report Templates" <a href="https://scholar.google.com/scholar?q=GIRT-Data:+Sampling+GitHub+Issue+Report+Templates">Search in Google Scholar</a>.</p>
GitHub Top 25 Software Project Analysis
<p>Companion dataset for the paper "For a More Transparent Governance of Open Source" published in the Communications of the ACM, 66, 8, 28-30, 2023.</p> <p>Data collected on May, 13th, 2022.</p> <p><strong>Note:</strong> cell annotations are only visible in the Excel version of the dataset.</p>
A Dataset of Bot and Human Activities in GitHub
<p><strong>A Dataset of Bot and Human Activities in GitHub</strong></p> <p>This repository provides an updated version of a dataset of GitHub contributor activities that is accompanied by a paper published at MSR 2023 in the <strong>Data and Tool Showcase Track</strong>. The paper is entitled <strong>A Dataset of Bot and Human Activities in GitHub</strong> and is co-authored by Natarajan Chidambaram, Alexandre Decan and Tom Mens (Software Engineering Lab, University of Mons, Belgium). DOI: <a href="https://www.doi.org/10.1109/MSR59073.2023.00070">https://www.doi.org/10.1109/MSR59073.2023.00070</a>. This work is done as a part of Natarajan Chdiambaram's PhD research in the context of <a href="https://www.digitalwallonia.be/ia/">DigitalWallonia4.AI research project ARIAC (grant number 2010235)</a> and <a href="https://trail.ac/en/">TRAIL</a>.</p> <p>The dataset contains 1,015,422 high-level activities made by 350 bots and 620 human contributors on GitHub between 25 November 2022 and 15 April 2023. The activities were generated from 1,221,907 low-level events obtained from the GitHub's Event API and cover 24 distinct activity types. This dataset facilitates the characterisation of bot and human behaviour in GitHub repositories, by enabling the analysis of activity sequences and activity patterns of bot and human contributors. This dataset could lead to better bot identification tools and empirical studies on how bots play a role in collaborative software development.</p> <p><strong>Files description</strong></p> <p>The following files are provided as part of the archive:</p> <ul> <li>bot_activities.json - A JSON file containing 754,165 activities made by 350 bot contributors;</li> <li>human_activities.json - A JSON file containing 261,258 activities made by 620 human contributors (anonymized);</li> <li>JsonSchema.json - A JSON schema that validates the above datasets;</li> <li>bots.txt - A TEXT file containing login names of all the 350 bots</li> </ul> <p><strong>Example</strong></p> <p>Below is an example of a <em>Closing pull request</em> activity:</p> <pre><code>{ "date": "2022-11-25T18:49:09+00:00", "activity": "Closing pull request", "contributor": "typescript-bot", "repository": "DefinitelyTyped/DefinitelyTyped", "comment": { "length": 249, "GH_node": "IC_kwDOAFz6BM5PJG7l" }, "pull_request": { "id": 62328, "title": "[qunit] Add `test.each()`", "created_at": "2022-09-19T17:34:28+00:00", "status": "closed", "closed_at": "2022-11-25T18:49:08+00:00", "merged": false, "GH_node": "PR_kwDOAFz6BM4_N5ib" }, "conversation": { "comments": 19 }, "payload": { "pr_commits": 1, "pr_changed_files": 5 } }</code></pre> <p><strong>List of activity types</strong></p> <p>In total, we have identified 24 different high-level activity types from 15 different low-level event types. They are <em>Creating repository</em>, <em>Creating branch</em>, <em>Creating tag</em>, <em>Deleting tag</em>, <em>Deleting repository</em>, <em>Publishing a release</em>, <em>Making repository public</em>, <em>Adding collaborator to repository</em>, <em>Forking repository</em>, <em>Starring repository</em>, <em>Editing wiki page</em>, <em>Opening issue</em>, <em>Closing issue</em>, <em>Reopening issue</em>, <em>Transferring issue</em>, <em>Commenting issue</em>, <em>Opening pull request</em>, <em>Closing pull request</em>, <em>Reopening pull request</em>, <em>Commenting pull request</em>, <em>Commenting pull request changes</em>, <em>Reviewing code</em>, <em>Commenting commits</em>, <em>Pushing commits</em>.</p> <p><strong>List of fields</strong></p> <p>Not only does the dataset contain a list of activities made by bot and human contributors, but it also contains some details about these activities. For example, <em>commenting issue</em> activities provide details about the author of the comment, the repository and issue in which the comment was created, and so on.</p> <p>For all activity types, we provide the <strong>date</strong> of the activity, the <strong>contributor</strong> that made the activity, and the <strong>repository</strong> in which the activity took place. Depending on the activity type, additional fields are provided. In this section, we describe for each activity type the different fields that are provided in the JSON file. It is worth to mention that we also provide the corresponding JSON schema alongside with the datasets.</p> <p><em><strong>Properties</strong></em></p> <ul> <li><strong>date</strong> <ul> <li>Date on which the activity is performed</li> <li>Type: <code>string</code></li> <li>e.g., "2022-11-25T09:55:19+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>activity</strong> <ul> <li>The activity performed by the contributor</li> <li>Type: <code>string</code></li> <li>e.g., "Commenting pull request"</li> </ul> </li> <li><strong>contributor</strong> <ul> <li>The login name of the contributor who performed this activity</li> <li>Type: <code>string</code></li> <li>e.g., "analysis-bot", "anonymised" in the case of a human contributor</li> </ul> </li> <li><strong>repository</strong> <ul> <li>The repository in which the activity is performed</li> <li>Type: <code>string</code></li> <li>e.g., "apache/spark", "anonymised" in the case of a human contributor</li> </ul> </li> <li><strong>issue</strong> <ul> <li>Issue information - provided for <em>Opening issue</em>, <em>Closing issue, Reopening issue</em>, <em>Transferring issue</em> and <em>Commenting issue</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>id</strong> <ul> <li>Issue number</li> <li>Type: <code>integer</code></li> <li>e.g., 35471</li> </ul> </li> <li><strong>title</strong> <ul> <li>Issue title</li> <li>Type: <code>string</code></li> <li>e.g., "error building handtracking gpu example with bazel", "anonymised" in the case of a human contributor</li> </ul> </li> <li><strong>created_at</strong> <ul> <li>The date on which this issue is created</li> <li>Type: <code>string</code></li> <li>e.g., "2022-11-10T13:07:23+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>status</strong> <ul> <li>Current state of the issue</li> <li>Type: <code>string</code></li> <li>"open" or "closed"</li> </ul> </li> <li><strong>closed_at</strong> <ul> <li>The date on which this issue is closed. "null" will be provided if the issue is open</li> <li>Types: <code>string</code>, <code>null</code></li> <li>e.g., "2022-11-25T10:42:39+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>resolved</strong> <ul> <li>The issue is resolved or not_planned/still open</li> <li>Type: <code>boolean</code></li> <li>true or false</li> </ul> </li> <li><strong>GH_node</strong> <ul> <li>The GitHub node of this issue</li> <li>Type: <code>string</code></li> <li>e.g., "IC_kwDOC27xRM5PHTBU", "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>pull_request</strong> <ul> <li>Pull request information - provided for <em>Opening pull request</em>, <em>Closing pull request</em>, <em>Reopening pull request</em>, <em>Commenting pull request changes</em> and <em>Reviewing code</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>id</strong> <ul> <li>Pull request number</li> <li>Type: <code>integer</code></li> <li>e.g., 35471</li> </ul> </li> <li><strong>title</strong> <ul> <li>Pull request title</li> <li>Type: <code>string</code></li> <li>e.g., "error building handtracking gpu example with bazel", "anonymised" in the case of a human contributor</li> </ul> </li> <li><strong>created_at</strong> <ul> <li>The date on which this pull request is created</li> <li>Type: <code>string</code></li> <li>e.g., "2022-11-10T13:07:23+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>status</strong> <ul> <li>Current state of the pull request</li> <li>Type: <code>string</code></li> <li>"open" or "closed"</li> </ul> </li> <li><strong>closed_at</strong> <ul> <li>The date on which this pull request is closed. "null" will be provided if the pull request is open</li> <li>Types: <code>string</code>, <code>null</code></li> <li>e.g., "2022-11-25T10:42:39+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>merged</strong> <ul> <li>The PR is merged or rejected/still open</li> <li>Type: <code>boolean</code></li> <li>true or false</li> </ul> </li> <li><strong>GH_node</strong> <ul> <li>The GitHub node of this pull request</li> <li>Type: <code>string</code></li> <li>e.g., "PR_kwDOC7Q2kM5Dsu3-", "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>review</strong> <ul> <li>Pull request review information - provided for <em>Reviewing code</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>status</strong> <ul> <li>Status of the review</li> <li>Type: <code>string</code></li> <li>"changes_requested" or "approved" or "dismissed"</li> </ul> </li> <li><strong>GH_node</strong> <ul> <li>The GitHub node of this review</li> <li>Type: <code>string</code></li> <li>e.g., "PRR_kwDOEBHXU85HLfIn", "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>conversation</strong> <ul> <li>Comments information in issue or pull request - Provided for <em>Opening issue</em>, <em>Closing issue</em>, <em>Reopening issue</em>, <em>Transferring issue</em>, <em>Commenting issue</em>, <em>Opening pull request</em>, <em>Closing pull request</em>, <em>Reopening pull request</em> and <em>Commenting pull request</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>comments</strong> <ul> <li>Number of comments present in the corresponding issue or pull request</li> <li>Type: <code>integer</code></li> <li>e.g., 5</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>comment</strong> <ul> <li>Comment information - Provided for all the activities for which the field issue or pull_request is reported and additionally for commit comment</li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>length</strong> <ul> <li>Length of the comment text (or description text if comment is not provided)</li> <li>Type: <code>integer</code></li> <li>e.g., 25</li> </ul> </li> <li><strong>GH_node</strong> <ul> <li>The GitHub node of this comment or description. "null" will be provided if there is no comment expected</li> <li>Types: <code>string</code>, <code>null</code></li> <li>e.g., "IC_kwDOEj6V8c5PHT78", "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>gitref</strong> <ul> <li>Tag information - provided for <em>Creating branch</em>, <em>Creating tag</em>, <em>Deleting branch</em>, <em>Deleting tag</em>, <em>Editing wiki page</em> and <em>Publishing a release</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>type</strong> <ul> <li>Type of the gitref</li> <li>Type: <code>string</code></li> <li>"tag" or "branch" or "commit"</li> </ul> </li> <li><strong>name</strong> <ul> <li>Name of the gitref</li> <li>Type: <code>string</code></li> <li>e.g., "cherry-pick-11-to-release-4.10"</li> </ul> </li> <li><strong>description_length</strong> <ul> <li>Length of the description text provided while creating the gitref. "null" be provided if the type is "branch" or "commit" as they do not have any description</li> <li>Type: <code>integer</code>, <code>null</code></li> <li>e.g., 23</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>release</strong> <ul> <li>Release information - provided for <em>Publishing a release</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>name</strong> <ul> <li>The name of the release that is created. "null" will be provided if the name is not provided</li> <li>Type: <code>string</code>, <code>null</code></li> <li>e.g., "v0.65.9"</li> </ul> </li> <li><strong>description_length</strong> <ul> <li>Length of the description of the release that is created</li> <li>Type: <code>integer</code></li> <li>e.g., 888</li> </ul> </li> <li><strong>created_at</strong> <ul> <li>The date at which the release is created (activity date is the release published date)</li> <li>Type: <code>string</code></li> <li>e.g., "2022-11-25T11:34:48+00:00"</li> <li>String format must be a "date-time"</li> </ul> </li> <li><strong>prerelease</strong> <ul> <li>If the release that is created is a prerelease or not</li> <li>Type: <code>boolean</code></li> <li>true or false</li> </ul> </li> <li><strong>new_tag</strong> <ul> <li>If a new tag is created for this release or another tag is re-used</li> <li>Type: <code>boolean</code></li> <li>true or false</li> </ul> </li> <li><strong>GH_node</strong> <ul> <li>The corresponding release node ID</li> <li>Type: <code>string</code></li> <li>e.g., "RE_kwDOCm6M2s4FBGxT", "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>page</strong> <ul> <li>Page information - provided for <em>Editing wiki page</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>name</strong> <ul> <li>Name of the page</li> <li>Type: <code>string</code></li> <li>e.g., "Workflow-status"</li> </ul> </li> <li><strong>title</strong> <ul> <li>Title of the page</li> <li>Type: <code>string</code></li> <li>e.g., "Workflow status"</li> </ul> </li> <li><strong>new</strong> <ul> <li>If the page is created new or existing page is edited</li> <li>Type: <code>boolean</code></li> <li>true or false</li> </ul> </li> </ul> </li> </ul> </li> <li><strong>payload</strong> <ul> <li>Other additional details - Provided for <em>Opening pull request</em>, <em>Closing pull request</em>, <em>Reopening pull request</em> and <em>pushing commits</em></li> <li>Type: <code>object</code></li> <li><em><strong>Properties</strong></em> <ul> <li><strong>pr_commits</strong> <ul> <li>The number of commits in this pull request</li> <li>Type: <code>integer</code></li> <li>e.g., 3</li> </ul> </li> <li><strong>pr_changed_files</strong> <ul> <li>The number of files that are changed in this pull request</li> <li>Type: <code>integer</code></li> <li>e.g., 2</li> </ul> </li> <li><strong>pushed_commits</strong> <ul> <li>The number of commits present in this push</li> <li>Type: <code>integer</code></li> <li>e.g., 4</li> </ul> </li> <li><strong>distinct_pushed_commits</strong> <ul> <li>The distinct number of commits present in this push</li> <li>Type: <code>integer</code></li> <li>e.g., 1</li> </ul> </li> <li><strong>github_push_id</strong> <ul> <li>The corresponding GitHub push ID</li> <li>Type: <code>integer</code></li> <li>e.g., 11790446870, "anonymised" in the case of a human contributor</li> </ul> </li> </ul> </li> </ul> </li> </ul> <p><strong>Mapping between activities and events</strong></p> <p>For many activity types, the corresponding activity can be observed by the occurrence of a single event type. For example, the activity types <em>Forking repository</em> and <em>Starring repository</em> would require the occurrence of a single event type for each as given below.</p> <table> <tbody><tr> <th>Activity type</th> <th>Event type</th> <th>Payload</th> </tr> </tbody><tbody> <tr> <td><em>Forking repository</em></td> <td><code>ForkEvent</code></td> <td>-</td> </tr> <tr> <td><em>Starring repository</em></td> <td><code>WatchEvent</code></td> <td>action = "started"</td> </tr> </tbody> </table> <p>However, in some cases, the same event type yields different activity types depending on the value present in the payload. For example, three different activity types can be generated from the same low-level event type CreateEvent, depending on the value of its ref_type (either "repository", "branch", or "tag") present in the payload.</p> <table> <tbody><tr> <th>Activity type</th> <th>Event type</th> <th>Payload</th> </tr> </tbody><tbody> <tr> <td><em>Creating repository</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "repository"</td> </tr> <tr> <td><em>Creating branch</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "branch"</td> </tr> <tr> <td><em>Creating tag</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "tag"</td> </tr> </tbody> </table> <p>In some cases, there is no one-to-one mapping between events and activities. This is because some actions on GitHub may generate more than a single event and lead to a sequence of one mandatory event and a second optional event (marked with <em>?</em>). For example, for the activity type <em>Publishing a release</em>, event type ReleaseEvent is mandatory with payload's action value = "published", while event type CreateEvent is optional as it is required only when a new tag is created along with the published release.</p> <table> <tbody><tr> <th>Activity type</th> <th>Event type</th> <th>Payload</th> </tr> </tbody><tbody> <tr> <td><em>Publishing a release</em></td> <td><code>ReleaseEvent</code></td> <td>action = "published"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>CreateEvent</code></td> <td>ref_type = "tag"</td> </tr> </tbody> </table> <p>All the identified activities along with their events type(s) and payload information is given in the following table.</p> <table> <tbody><tr> <th>Activity type</th> <th>Event type</th> <th>Payload</th> </tr> </tbody><tbody> <tr> <td><em>Creating repository</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "repository"</td> </tr> <tr> <td><em>Creating branch</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "branch"</td> </tr> <tr> <td><em>Creating tag</em></td> <td><code>CreateEvent</code></td> <td>ref_type = "tag"</td> </tr> <tr> <td><em>Deleting tag</em></td> <td><code>DeleteEvent</code></td> <td>ref_type = "tag"</td> </tr> <tr> <td><em>Deleting repository</em></td> <td><code>DeleteEvent</code></td> <td>ref_type = "branch"</td> </tr> <tr> <td><em>Publishing a release</em></td> <td><code>ReleaseEvent</code></td> <td>action = "published"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>CreateEvent</code></td> <td>ref_type = "tag"</td> </tr> <tr> <td><em>Making repository public</em></td> <td><code>PublicEvent</code></td> <td>-</td> </tr> <tr> <td><em>Adding collaborator to repository</em></td> <td><code>MemberEvent</code></td> <td>action = "added"</td> </tr> <tr> <td><em>Forking repository</em></td> <td><code>ForkEvent</code></td> <td>-</td> </tr> <tr> <td><em>Starring repository</em></td> <td><code>WatchEvent</code></td> <td>action = "started"</td> </tr> <tr> <td><em>Editing wiki page</em></td> <td><code>GollumEvent</code></td> <td>pages-->action = "created" or "edited"</td> </tr> <tr> <td><em>Opening issue</em></td> <td><code>IssuesEvent</code></td> <td>action = "opened"</td> </tr> <tr> <td><em>Closing issue</em></td> <td><code>IssuesEvent</code></td> <td>action = "closed"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Reopening issue</em></td> <td><code>IssuesEvent</code></td> <td>action = "reopened"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Transferring issue</em></td> <td><code>IssuesEvent</code></td> <td>action = "opened"</td> </tr> <tr> <td><em>Commenting issue</em></td> <td><code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Opening pull request</em></td> <td><code>PullRequestEvent</code></td> <td>action = "opened"</td> </tr> <tr> <td><em>Closing pull request</em></td> <td><code>PullRequestEvent</code></td> <td>action = "closed"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Reopening pull request</em></td> <td><code>PullRequestEvent</code></td> <td>action = "opened"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Commenting pull request</em></td> <td><code>IssueCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Commenting pull request changes</em></td> <td><code>PullrequestReviewCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td> </td> <td><em>?</em> <code>PullRequestReviewEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Reviewing code</em></td> <td><code>PullRequestReviewEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Commenting commits</em></td> <td><code>CommitCommentEvent</code></td> <td>action = "created"</td> </tr> <tr> <td><em>Pushing commits</em></td> <td><code>PushEvent</code></td> <td>-</td> </tr> </tbody> </table> <p> </p>
Dataset for the study of Potential Code Borrowing and License Violations in Java Projects on GitHub
<p>This is the dataset for the study of Potential Code Borrowing and License Violations in Java Projects on GitHub. The dataset is based on the Public Git Archive and consists of projects on GitHub that have at least 50 stars and have at least one line in Java. A total of 23,378 projects are listed here that we downloaded for analysis on June 1st, 2019.</p>
List of licenses discovered in the study of Potential Code Borrowing and License Violations in Java Projects on GitHub
<p>This is the list of licenses discovered in the study of Potential Code Borrowing and License Violations in Java Projects on GitHub. The licenses are ranged by the amount of files that they cover, there are a total of 94 different licenses. Where possible, the names are presented as identifiers at https://spdx.org/licenses/. "GitHub" stands for no license in the file or the project.</p>
Dataset of merge conflicts collected from GitHub repositories
<p>Within each nested folder of the archive you will find files A,O,B and M. They each represent a conflict where file O was altered in two different ways, resulting in A and B. Finally, a developer solved the merge conflict committing M as the solution.</p> <p>We have selected these by manually searching for a programming language on GitHub and selecting those repositories that had a large number of forks, commits and contributors.</p>
A ground-truth dataset to identify bots in GitHub
<p>This dataset is a ground truth dataset we used to identify bots. Each account in this dataset is rated by at least 3 raters with high interrater agreement.</p> <p>===</p> <p>This dataset is outdated (it was created in 2020) and therefore no longer recommended for use. Many of the classified GitHub bot accounts are no longer active or even available today, and some may even have changed their status from bot to human (or conversely) since. If you want to use a ground-truth dataset of bot accounts for academic (or other) purposes, we therefore recommend to use a more recent and more complete dataset of GitHub bot accounts. Such a dataset can be found here:</p> <p><a href="https://doi.org/10.5281/zenodo.7740520">https://doi.org/10.5281/zenodo.7740520</a></p> <p>===</p>
GitHub issues dataset
<p>Datasets for the paper: <em>Predicting the Objective and Priority of Issue Reports of Software Repositories</em></p> <p>- <strong>objective issues.rar</strong>: is used for the objective detection task</p> <p>- <strong>ranking issues - inter project.rar & ranking issues - intra project.rar</strong>: are used for the priority detection task in two context</p> <p> </p> <p><strong>Arxiv link</strong>: https://arxiv.org/abs/2012.10951</p> <p> </p> <p><strong>Citation</strong>: If you use our datasets, please cite as below:</p> <pre>@article{izadi2020predicting, title={Predicting the Objective and Priority of Issue Reports in a Cross project Context}, author={Izadi, Maliheh and Akbari, Kiana and Heydarnoori, Abbas}, journal={arXiv preprint arXiv:2012.10951}, year={2020} }</pre>
GitSED: GitHub Socially Enhanced Dataset
<p>Software Engineering has evolved as a field to study not only the many ways software is created but also how it evolves, becomes successful, is effective and efficient in its objectives, satisfies its quality attributes, and much more. Nonetheless, there are still many open issues during its conception, development, and maintenance phases. Especially, understanding how developers collaborate may help in all such phases, but it is also challenging. Luckily, we may now explore a novel angle to deal with such a challenge: studying the social aspects of software development over social networks.</p> <p>With <em>GitHub</em> becoming the main representative of collaborative software development online tools, there are approaches to assess the follow-network, stargazer-network, and contributors-network. Moreover, having such networks built from real software projects offers support for relevant applications, such as detection of key developers, recommendation of collaboration among developers, detection of developer communities, and analyses of collaboration patterns in agile development.</p> <p><strong>GitSED</strong> is a dataset based on GitHub that is curated (cleaned and reduced), augmented with external data, and enriched with social information on developers’ interactions. The original data is extracted from <a href="https://ghtorrent.org/">GHTorrent</a> (an offline repository of data collected through the GitHub REST API). Our final dataset contains data from up to June 2019. It comprises:</p> <ul> <li><strong>8,556,778</strong> repositories</li> <li><strong>32,411,674 </strong>developers</li> <li><strong>6</strong> programming languages (Assembly, JavaScript, Pascal, Python, Ruby, Visual Basic)</li> <li><strong>13</strong> collaboration metrics</li> </ul> <p>There are two previous versions of GitSED, which were originally built for the following conference papers:</p> <p><a href="https://dl.acm.org/doi/10.1145/3243082.3243101">v2 (May 2017)</a>: <em>Gabriel P. Oliveira, Natércia A. Batista, Michele A. Brandão, and Mirella M. Moro. Tie Strength in GitHub Heterogeneous Networks. In Proceedings of the 24th Brazilian Symposium on Multimedia and the Web (WebMedia'18), 2018.</em></p> <p><a href="https://dl.acm.org/doi/10.1145/3106426.3106480">v1 (Sep 2015)</a>: <em>Natércia A. Batista, Michele A. Brandão, Gabriela B. Alves, Ana Paula Couto da Silva, and Mirella M. Moro. Collaboration strength metrics and analyses on GitHub. In Proceedings of the International Conference on Web Intelligence (WI'17), 2017.</em></p>
Application Domain of 5,000 GitHub Repositories
<p>We provide a manual classification of the application domain of 5,000 GitHub repositories (the most popular ones, by number of stars, on January, 2017).<br> <br> We classified each system in one of the following application domains:</p> <ul> <li><strong>Application software:</strong> systems that provide functionalities to end-users, like browsers and text editors (e.g., WordPress/WordPress and adobe/brackets).</li> <li><strong>System software:</strong> systems that provide services and infrastructure to other systems, like operating systems, middleware, and databases (e.g., torvalds/linux and mongodb/mongo).</li> <li><strong>Web libraries and frameworks</strong> (e.g., twbs/bootstrap and angular/angular.js).</li> <li><strong>Non-web libraries and frameworks</strong> (e.g., google/guava and facebook/fresco).</li> <li><strong>Software tools:</strong> systems that support development tasks, like IDEs, package managers, and compilers (e.g., Homebrew/homebrew and git/git).</li> <li><strong>Documentation:</strong> repositories with documentation, tutorials, source code examples, etc. (e.g., iluwatar/java-design-patterns).</li> </ul> <p>To cite the dataset, please use the following paper (which proposes and uses a first dataset version):</p> <p>Hudson Borges, Andre Hora, Marco Tulio Valente. <em>Understanding the Factors that Impact the Popularity of GitHub Repositories</em>. In 32nd IEEE International Conference on Software Maintenance and Evolution (ICSME), pages 334-344, 2016.</p>
An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects [Replication Package]
<h2>Artefact Summary</h2> <p>This repository contains the replication package for the paper 'An Exploratory Study of Documentation Strategies for Product Features in Popular GitHub Projects,' presented at the <em><a href="https://cyprusconferences.org/icsme2022/" target="_blank" rel="noopener">38th IEEE International Conference on Software Maintenance and Evolution (ICSME'22)</a></em>.</p> <p>The purpose of the package is to facilitate the verification and reproduction of the study results.<br>It provides all computational notebooks used to collect and analyse data, as well as the slides of the conference presentation.</p> <h2>Paper Abstract</h2> <p>[Background] In large open-source software projects, development knowledge is often fragmented across multiple artefacts and contributors such that individual stakeholders are generally unaware of the full breadth of the product features. However, users want to know what the software is capable of, while contributors need to know where to fix, update, and add features. [Objective] This work aims at understanding how feature knowledge is documented in GitHub projects and how it is linked (if at all) to the source code. [Method] We conducted an in-depth qualitative exploratory content analysis of 25 popular GitHub repositories that provided the documentation artefacts recommended by GitHub’s Community Standards indicator. We extracted strategies used to document software features in textual artefacts and which strategies were used to link the feature documentation with source code. [Results] We observed feature documentation in all studied projects in artefacts such as READMEs, wikis, and website resource files. However, the features were often described in an unstructured way. Additionally, tracing techniques to connect feature documentation and source code were rarely used. [Conclusions] Our results suggest a lacking (or a low-prioritised) feature documentation in open-source projects, little use of normalised structures, and a rare explicit referencing to source code. As a result, product feature traceability is likely to be very limited, and maintainability to suffer over time.</p> <h2>References</h2> <p>The published paper is available on <a href="https://doi.org/10.1109/ICSME55016.2022.00043" target="_blank" rel="noopener">IEEE Xplore</a> and the preprint on <a href="https://doi.org/10.48550/arXiv.2208.01317" target="_blank" rel="noopener">arXiv</a>.</p>
BotHawk: GitHub Bot Account Detection Dataset
<p>The <strong>"BotHawk: GitHub Bot Account Detection Dataset"</strong> is a specialized resource developed for the research and development community focused on identifying and understanding bot accounts within the GitHub ecosystem. This comprehensive dataset offers an in-depth look into the behaviors, contributions, and patterns of bot accounts, distinguishing them from human user activities on GitHub.</p> <p>Designed to enhance the accuracy and efficiency of bot detection algorithms, the BotHawk Dataset encompasses a variety of features, including commit history, issue participation, pull request activities, and other interaction metrics specific to GitHub. It provides an invaluable foundation for developing machine learning models capable of distinguishing between bot-generated and human-generated activities, contributing to more secure and trustworthy interactions within the GitHub platform.</p> <p>This dataset not only serves as a tool for academic research in cybersecurity, machine learning, and software engineering but also offers practical applications for developers and platform administrators seeking to improve the health of their projects by filtering out bot-based noise and manipulation. The BotHawk dataset is instrumental in advancing our comprehension of bot dynamics within the GitHub ecosystem, supporting the broader objective of fostering a secure, transparent, and collaborative open-source community.</p>
Github Repository for: European green crab predation in a Washington State estuary revealed with DNA metabarcoding
<p><strong>Fisher, MC, Grason, EW, Stote, A, Kelly, RP, Litle, K, & PS McDonald. (2024).<em> </em>Invasive European green crab (<em>Carcinus maenas</em>) predation in a Washington State estuary revealed with DNA metabarcoding. DOI:10.1371/journal.pone.0302518<em><br></em></strong></p> <p>Github release v1.1 of the repository for Fisher et al. 2024, "European green crab predation in a Washington State estuary revealed with DNA metabarcoding." For the most updated repository, see: <a href="https://github.com/mfisher5/Green-crab-dDNA/tree/main/doc">github.com/mfisher5/Green-crab-dDNA</a></p> <p>Contains the code and minimum dataset necessary to replicate study findings.</p> <p> </p> <p>---</p> <p>Abstract: Predation by invasive species can threaten local ecosystems and economies. The European green crab (<em>Carcinus maenas</em>), one of the most widespread marine invasive species, is an effective predator associated with clam and crab population declines outside of its native range. In the U.S. Pacific Northwest, green crab has recently increased in abundance and expanded its distribution, generating concern for estuarine ecosystems and associated aquaculture production. However, regionally-specific information on the trophic impacts of invasive green crab is very limited. We compared the stomach contents of green crabs collected on shellfish aquaculture beds versus natural intertidal sloughs in Willapa Bay, Washington, to provide the first in-depth description of European green crab diet at a particularly crucial time for regional management. We first identified putative prey items using DNA metabarcoding of stomach content samples. We compared diet composition across sites using prey presence/absence and an index of species-specific relative abundance. For eight prey species, we also calibrated metabarcoding data to quantitatively compare DNA abundance between prey items, and to describe an ‘average’ green crab diet at an intertidal slough and an actively cultivated Manila clam bed. From the stomach contents of 61 green crabs, we identified 54 unique taxa belonging to nine phyla. The stomach contents of crabs collected from cultivated Manila clam beds were significantly different from the stomach contents of crabs collected at natural intertidal sloughs. Across all sites, arthropods were the most frequently detected prey, with the native hairy shore crab (<em>Hemigrapsus oregonensis</em>) the single most common prey item. Of the eight species included in the quantitative model, two ecologically-important native species – the sand shrimp (<em>Crangon franciscorum</em>) and the Pacific staghorn sculpin (<em>Leptocottus armatus</em>) – were the most abundant in crab stomach contents, when present. In addition to providing timely information on green crab diet, our research demonstrates the novel application of a recently developed model for more quantitative DNA metabarcoding. This represents another step in the ongoing evolution of DNA-based diet analysis towards producing the quantitative data necessary for modeling invasive species impacts.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.