Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
207
datasets available to search
ShareScore release 0.7.1
Dataset results
207 results for “github”
Software Developer Expertise GitHub and Stack Overflow data sets
<p>Cross-Platform Software Developer Expertise Learning by Norbert Eke</p> <p>This data set is part of my Master's thesis project on developer expertise learning by mining Stack Overflow (SOTorrent) and Github (GHTorrent) data. Check out my portfolio website at norberte.github.io</p>
Malware Repositories and Their Authors on GitHub
<p>This dataset is rooted in a study aimed at unveiling the origins and motivations behind the creation of malware repositories on GitHub. Our research embarks on an innovative journey to dissect the profiles and intentions of GitHub users who have been involved in this dubious activity. </p> <p>Employing a robust methodology, we meticulously identified 14,000 GitHub users linked to malware repositories. By leveraging advanced large language model (LLM) analytics, we classified these individuals into distinct categories based on their perceived intent: 3,339 were deemed Malicious, 3,354 Likely Malicious, and 7,574 Benign, offering a nuanced perspective on the community behind these repositories. </p> <p>Our analysis penetrates the veil of anonymity and obscurity often associated with these GitHub profiles, revealing stark contrasts in their characteristics. Malicious authors were found to typically possess sparse profiles focused on nefarious activities, while Benign authors presented well-rounded profiles, actively contributing to cybersecurity education and research. Those labeled as Likely Malicious exhibited a spectrum of engagement levels, underlining the complexity and diversity within this digital ecosystem.</p> <p> </p> <p>We are offering two datasets in this paper. First, a list of malware repositories - we have collected and extended the malware repositories on the GitHub in 2022 following the original papers. Second, a csv file with the github users information with their maliciousness classfication label. </p> <ol> <li> <p><strong>malware_repos.txt</strong></p> <ul> <li><strong>Purpose</strong>: This file contains a curated list of GitHub repositories identified as containing malware. These repositories were identified following the methodology outlined in the research paper <a href="https://www.usenix.org/conference/raid2020/presentation/omar">"SourceFinder: Finding Malware Source-Code from Publicly Available Repositories in GitHub."</a></li> <li><strong>Contents</strong>: The file is structured as a simple text file, with each line representing a unique repository in the format <code>username/reponame</code>. This format allows for easy identification and access to each repository on GitHub for further analysis or review.</li> <li><strong>Usage</strong>: The list serves as a critical resource for researchers and cybersecurity professionals interested in studying malware, understanding its distribution on platforms like GitHub, or developing defense mechanisms against such malicious content.</li> </ul> </li> <li> <p><strong>obfuscated_github_user_dataset.csv</strong></p> <ul> <li><strong>Purpose</strong>: Accompanying the list of malware repositories, this CSV file contains detailed, albeit obfuscated, profile information of the GitHub users who authored these repositories. The obfuscation process has been applied to protect user privacy and comply with ethical standards, especially given the sensitive nature of associating individuals with potentially malicious activities.</li> <li><strong>Contents</strong>: The dataset includes several columns representing different aspects of user profiles, such as obfuscated identifiers (e.g., ID, login, name), contact information (e.g., email, blog), and GitHub-specific metrics (e.g., followers count, number of public repositories). Notably, sensitive information has been masked or replaced with generic placeholders to prevent user identification.</li> <li><strong>Usage</strong>: This dataset can be instrumental for researchers analyzing behaviors, patterns, or characteristics of users involved in creating malware repositories on GitHub. It provides a basis for statistical analysis, trend identification, or the development of predictive models, all while upholding the necessary ethical considerations.</li> </ul> </li> </ol>
GitHub Profiles (users/organisations) and Repositories (research/non-research) of Potsdam Researchers and Research Organisations: An annotated dataset of with howfairis and software quality variables.
<p>This dataset accompanies the paper <em>"Software FAIRness, Documentation and Development Practices in Potsdam Researchers' GitHub Repositories"</em> It includes 3 CSV files that contain data related to github profiles of users/organisations, their repositories annotated as research/non-research repositories and followed by FAIRness and other software qualtiy variables. The data were collected using <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP">SWORDS-template-UP</a> (v1.0.0) methods (collect_users, collect_repositories, collect_variables) which is extended version of <a href="https://github.com/UtrechtUniversity/SWORDS-template">SWORS-template</a> adopted according our needs and detailed in the paper.</p> <p><strong>GitHub (research) user/organisation profiles. ( <em>github_profiles.csv )</em></strong></p> <table> <tbody> <tr> <td><strong>Column name</strong></td> <td><strong>Description </strong></td> </tr> <tr> <td>user_id</td> <td>GitHub username </td> </tr> <tr> <td>html_url </td> <td>URL of the GitHub profile </td> </tr> <tr> <td>type </td> <td>Type of profile (user or organization)</td> </tr> <tr> <td>organisation</td> <td>Acronym or name of the organization </td> </tr> </tbody> </table> <p><strong>GitHub repositories <em>(github_repositories.csv)</em></strong></p> <p>This file contains the repositories scraped from the GitHub profiles of research users and organizations.</p> <table> <tbody> <tr> <td><strong>Column name </strong></td> <td><strong>Description </strong></td> </tr> <tr> <td>html_url </td> <td>URL link to the repository </td> </tr> <tr> <td>description</td> <td>GitHub project description </td> </tr> <tr> <td>project</td> <td>Specifies if the project is research or non-research</td> </tr> <tr> <td>language</td> <td>Programming language used in the project </td> </tr> <tr> <td>organisation</td> <td>Acronym or name of the university, institution, or research organization</td> </tr> <tr> <td>research_group</td> <td>Acronym or name of the research group the repository belongs to</td> </tr> </tbody> </table> <p><strong>Research repositories filtered and annotated <em>(github_research_repositories_filtered_annotated.csv)</em></strong></p> <p>This file contains filtered and annotated information about research repositories.</p> <table> <tbody> <tr> <td><strong>Column Name </strong></td> <td><strong>Description </strong></td> <td><strong>Collection Method </strong></td> </tr> <tr> <td>html_url </td> <td>Repository URL </td> <td> </td> </tr> <tr> <td>howfairis_repository</td> <td>Indicates if the repository is public or private (True/False) </td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/tree/main/collect_variables#usage">howfairis_variable.py</a>) is a wrapper for <a href="https://pypi.org/project/howfairis/">howfairis</a> pypi library that checks the 5 recommendations of <a href="https://fair-software.nl">FAIR</a></td> </tr> <tr> <td>howfairis_license </td> <td>Indicates if the repository has a license (True/False)</td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/tree/main/collect_variables#usage">howfairis_variable.py</a>) is a wrapper for <a href="https://pypi.org/project/howfairis/">howfairis</a> pypi library that checks the 5 recommendations of <a href="https://fair-software.nl">FAIR</a></td> </tr> <tr> <td>howfairis_registry</td> <td>Indicates if the repository has implemented community registry (True/False)</td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/tree/main/collect_variables#usage">howfairis_variable.py</a>) is a wrapper for <a href="https://pypi.org/project/howfairis/">howfairis</a> pypi library that checks the 5 recommendations of <a href="https://fair-software.nl">FAIR</a></td> </tr> <tr> <td>howfairis_citation</td> <td>Indicates if the repository has a .cff file (True/False) </td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/tree/main/collect_variables#usage">howfairis_variable.py</a>) is a wrapper for <a href="https://pypi.org/project/howfairis/">howfairis</a> pypi library that checks the 5 recommendations of <a href="https://fair-software.nl">FAIR</a></td> </tr> <tr> <td>howfairis_checklist</td> <td>Indicates if the repository has implemented OpenSSF best practices badge (True/False)</td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/tree/main/collect_variables#usage">howfairis_variable.py</a>) is a wrapper for <a href="https://pypi.org/project/howfairis/">howfairis</a> pypi library that checks the 5 recommendations of <a href="https://fair-software.nl">FAIR</a></td> </tr> <tr> <td>fair_score</td> <td>Score based on howfairis variables (0-5) </td> <td> </td> </tr> <tr> <td>dlr_soft_class</td> <td>Name of the university, company, research institute, or research organization</td> <td>(Manual) Annotated the repository based on <a href="https://core.ac.uk/reader/211557820">DLR software engineering guideline.</a> There are no specific definitions on metrics how to categorise them (github repositories) into application classes. Which were needed to do a comparitive analysis. </td> </tr> <tr> <td>installation_instruction</td> <td>Presence of installation instruction (True/False) </td> <td>(Manual) Checked the presense of Installation Instruction in the readme or in the project wiki pages. </td> </tr> <tr> <td>project_information </td> <td>Presence of basic project information in README (True/False) </td> <td>(Manual) Checked if the readme have basic information about the project. </td> </tr> <tr> <td>usage_guide</td> <td>Presence of folder named test/tests in the root directory (True/False)</td> <td>(Manual) Checked the presense of Usage Guide in the readme or in the project wiki pages. For command line tools checked if they have help command which guides how to use the tool. </td> </tr> <tr> <td>test_folder</td> <td>Presence of folder named test/tests in the root directory (True/False)</td> <td> <p>(Script - <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/docs/collect_variables/scripts/soft_dev_pract/test_folder.py">test_folder.py</a>) Checks the folder names test/tests in the root directory of the repository.</p> </td> </tr> <tr> <td>requirements_explicit </td> <td>Explicit requirements for Python, R, C++ repositories (True/False)</td> <td>(Script - <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/requirement_explicit.py">requirement_explicit.py</a>) Checks the files (requirements.txt, DESCRIPTION, CMakeLists.txt) in the root directory. </td> </tr> <tr> <td>continuous_integration</td> <td>Indicates if the repository uses continuous integration (True/False)</td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/continious_integration.py">continious_integration.py</a>) Checks the presence of folder .github (github actions) same for other continious integration (travisCI, CircleCI, Jekins, azure pipeline)</td> </tr> <tr> <td>ci_tool </td> <td>Name of the continuous integration tool used</td> <td>(Script- <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/continious_integration.py">continious_integration.py</a>) Checks the presence of folder .github (github actions) same for other continious integration (travisCI, CircleCI, Jekins, azure pipeline)</td> </tr> <tr> <td>add_lint_rule </td> <td>Indicates if additional linting rules are present (True/False)</td> <td>(Script - <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/add_ci_rules.py">add_ci_rules.py</a>) - it scans the YAML files in the <br>.github/workflows directory to detect the presence of (linters) Python, R, and C++.</td> </tr> <tr> <td>add_test_rule</td> <td>Indicates if additional testing rules are present (True/False) </td> <td>(Script - <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/add_ci_rules.py">add_ci_rules.py</a>) - it scans the YAML files in the <br>.github/workflows directory to detect the presence of (testing libraries) Python, R, and C++.</td> </tr> <tr> <td>comment_at_start</td> <td>Indicates the level of comments at the start of the program (most, more, some, less)</td> <td>(Script - <a href="https://github.com/Software-Engineering-Group-UP/SWORDS-template-UP/blob/main/collect_variables/scripts/soft_dev_pract/comment_at_start.py">comment_at_start.py</a>) Checks the presence of brief comments at the start at source code files in GitHub repositories.</td> </tr> <tr> <td>language </td> <td>Programming language used in the repository </td> <td> </td> </tr> <tr> <td>type </td> <td>Specifies if the profile is a user or organization </td> <td>Github organisation or user profiles.</td> </tr> <tr> <td>organisation </td> <td>Name of the university, company, research institute, or research organization</td> <td>Oraganisation name (from where the user was found)</td> </tr> <tr> <td>research_group</td> <td>Name or acronym of the research group </td> <td> </td> </tr> </tbody> </table> <p> </p> <p>Data for publication - https://github.com/Software-Engineering-Group-UP/potsdam-research-repos</p>
Github commit data for the article "Beyond Zipf's law: Exploring the discrete generalized beta distribution in open-source repositories"
<p><span>This dataframe corresponds to the data used in the Nowak's et al. 2024 article "Beyond Zipf’s law: Exploring the discrete generalized beta distribution in open-source repositories" (see reference below).</span></p> <p><span>It consists of the distirbutions of number of commits per user across a number of GitHub repositories. <br><br>There are three columns:</span></p> <ul> <li><span>repository: the repository name</span></li> <li><span># of commits: the number of commits of a given individual</span></li> <li><span>rank: the user rank in the repository (by decreasing number of commits)<br><br></span></li> </ul> <p><strong><span>Reference:</span></strong></p> <p><span>Nowak, P., Santolini, M., Singh, C., Siudem, G., & Tupikina, L. (2024). Beyond Zipf’s law: Exploring the discrete generalized beta distribution in open-source repositories. <em>Physica A: Statistical Mechanics and Its Applications</em>, <em>649</em>, 129927. <a href="https://doi.org/10.1016/j.physa.2024.129927">https://doi.org/10.1016/j.physa.2024.129927</a></span></p>
A Dataset for GitHub Repository Deduplication
<p>GitHub projects can be easily replicated through the site's fork process or through a Git clone-push sequence. This is a problem for empirical software engineering, because it can lead to skewed results or mistrained machine learning models. We provide a dataset of 10.6 million GitHub projects that are copies of others, and link each record with the project's ultimate parent. The ultimate parents were derived from a ranking along six metrics. The related projects were calculated as the connected components of an 18.2 million node and 12 million edge denoised graph created by directing edges to ultimate parents. The graph was created by filtering out more than 30 hand-picked and 2.3 million pattern-matched clumping projects. Projects that introduced unwanted clumping were identified by repeatedly visualizing shortest path distances between unrelated important projects. Our dataset identified 30 thousand duplicate projects in an existing popular reference dataset of 1.8 million projects. An evaluation of our dataset against another created independently with different methods found a significant overlap, but also differences attributed to the operational definition of what projects are considered as related. </p> <p>The dataset is provided as two files identifying GitHub repositories using the <em>login-name/project-name</em> convention. The file <em>deduplicate_names</em> contains 10,649,348 tab-separated records mapping a duplicated <em>source project</em> to a definitive <em>target project</em>.</p> <p>The file <em>forks_clones_noise_names</em> is a 50,324,363 member superset of the source projects, containing also projects that were excluded from the mapping as noise.</p>
GitRec - Github Project Recommender Systems
<p>This dataset contains the data collected using the Google API for the GHTorrent project and which were applied in the doctoral thesis directed to recommending projects on the GitHub platform</p>
GitHub Java Corpus - Function Identifiers
<p>This dataset contains function identifiers extracted from the GitHub Java Corpus (<a href="http://groups.inf.ed.ac.uk/cup/javaGithub/">http://groups.inf.ed.ac.uk/cup/javaGithub/</a>).</p> <p>Each line corresponds to a method declaration. A line contains the name of the method declaration followed by the function identifiers (i.e., function calls) contained within the method body. </p> <p>The file embeddings_train.json can be used to train a word/sentence embedding model using the code in the Github repository (link below).</p> <p>The corpus was used for the experiments in the paper <strong>Combining Code Embedding with Static Analysis for Function-Call Completion</strong>.</p> <p>Github repository to replicate the experiments: https://github.com/mweyssow/cse-saner</p>
Github data for static site generators (SSG) popularity
<p>Number of Github stars, forks, open issues, create and last modified dates for 30 open source static site generators (SSG), including Hugo, Jekyll and Gatsby.</p>
1,237 Annotated Developer Apologies from GitHub
<p><em><strong>Software Developer Apologies</strong></em></p> <p>This dataset contains 1,237 GitHub comments with apology annotations (apology vs not apology), released as part of the following publication:</p> <ul> <li>Benjamin S. Meyers. <a href="https://scholarworks.rit.edu/theses/11609/">Human Error Assessment in Software Engineering.</a> Rochester Institute of Technology. 2023. </li> </ul> <p><em><strong>Included Files</strong></em></p> <p>The "github_apologies.csv" file contains the full dataset of 1,237 GitHub comments with apology annotations. In total, there are 365 comments containing an apology (872 non apologies). The comments themselves are a subset of those included in <a href="../records/5603093">88.6 Million Developer Comments from GitHub</a>.</p> <p><em><strong>Annotation Details</strong></em></p> <p>Full details are provided in the above publication. We implemented a naive classifier (Precision: 41.7%, Recall: 99.7%, F1: 86.9%, Accuracy: 91.1%) using counts of apology lemmas. 91% of developer comments containing at least one apology lemma matched our manual annotations. Agreement between raters was almost perfect (Cohen's Kappa = 0.94).</p> <p><em><strong>CSV Fields</strong></em></p> <ul> <li><strong>ID</strong>: Unique identifier for the comment.</li> <li><strong>SOURCE</strong>: Whether this comment originates from a commit, issue, or pull request.</li> <li><strong>COMMENT_URL</strong>: The URL linking to the comment.</li> <li><strong>COMMENT_TEXT</strong>: The raw comment text.</li> <li><strong>NUM_APOLOGY_LEMMAS</strong>: The count of apology lemmas present in the comment.</li> <li><strong>CLASSIFIER_LABEL</strong>: The automatically assigned label ("Apology" or "Not Apology").</li> <li><strong>RATER_1_LABEL</strong>: The manually assigned label ("Apology" or "Not Apology") from Rater 1.</li> <li><strong>RATER_2_LABEL</strong>: The manually assigned label ("Apology" or "Not Apology") from Rater 2.</li> <li><strong>AGREED_LABEL</strong>: The agreed upon label ("Apology" or "Not Apology") after Rater 1 and Rater 12 resolved disagreements.</li> </ul> <p><em><strong>Contact</strong></em></p> <p>Please contact Benjamin S. Meyers (<a href="mailto:bsm9339@rit.edu">email</a>) with questions about this data and its collection.</p> <p><em><strong>Acknowledgments</strong></em></p> <p>Collection of this data has been sponsored in part by the National Science Foundation (grant 1922169), by the NSA Science of Security Lablet program (grant H98230-17-D-0080/2018-0438-02), and by a Department of Defense DARPA SBIR program (grant 140D63-19-C-0018).</p>
200 Annotated Developer Human Errors from GitHub
<p><em><strong>Software Engineers' Human Errors</strong></em></p> <p>This dataset contains 200 GitHub comments with manual human error annotations, released as part of the following publication:</p> <ul> <li>Benjamin S. Meyers. <a href="https://scholarworks.rit.edu/theses/11609/">Human Error Assessment in Software Engineering.</a> Rochester Institute of Technology. 2023.</li> </ul> <p><em><strong>Included Files</strong></em></p> <p>The "developer_human_errors.csv" file contains the full dataset of 200 software defect descriptions annotated with human error types (slips, lapses, mistakes) and T.H.E.S.E. categories.</p> <p><em><strong>CSV Fields</strong></em></p> <ul> <li><strong>ID</strong>: Unique identifier for the comment.</li> <li><strong>SOURCE</strong>: Whether this comment originates from a commit, issue, or pull request.</li> <li><strong>COMMENT_URL</strong>: The URL linking to the comment.</li> <li><strong>COMMENT_TEXT</strong>: The raw comment text.</li> <li><strong>HUMAN_ERROR_TYPE</strong>: Whether the software defect described is a slip, lapse, or mistake.</li> <li><strong>THESE_V4_ID</strong>: Manually assigned T.H.E.S.E. category with labels corresponding to Version 4 of T.H.E.S.E.</li> <li><strong>THESE_NAME</strong>: Name corresponding to manually assigned T.H.E.S.E. category.</li> </ul> <p><em><strong>Annotation Details</strong></em></p> <p>Human error types span slips, lapses, and mistakes from James Reason's Generic Error Modelling System (GEMS):</p> <ul> <li><strong>Slips</strong>: Failures of attention.</li> <li><strong>Lapses</strong>: Failures of memory.</li> <li><strong>Mistakes</strong>: Failures of planning.</li> </ul> <p>T.H.E.S.E. categories are summarized below:</p> <ul> <li>S01: Typos & Misspellings</li> <li>S02: Syntax Errors</li> <li>S03: Overlooking documented Information</li> <li>S04: Multitasking Errors</li> <li>S05: Hardware Interaction Errors</li> <li>S06: Overlooking Proposed Code Changes</li> <li>S07: Overlooking Existing Functionality</li> <li>S08: General Attentional Failure</li> <li>L01: Forgetting to Finish a Development Task</li> <li>L02: Forgetting to Fix a Defect</li> <li>L03: Forgetting to Remove Development Artifacts</li> <li>L04: Working with Outdated Source Code</li> <li>L05: Forgetting an Import Statement</li> <li>L06: Forgetting to Save Work</li> <li>L07: Forgetting Previous Development Discussion</li> <li>L08: General Memory Failure</li> <li>M01: Code Logic Errors</li> <li>M02: Incomplete Domain Knowledge</li> <li>M03: Wrong Assumption Errors</li> <li>M04: Internal Communication Errors</li> <li>M05: External Communication Errors</li> <li>M06: Solution Choice Errors</li> <li>M07: Time Management Errors</li> <li>M08: Inadequate Testing</li> <li>M09: Incorrect/Insufficient Configuration</li> <li>M10: Code Complexity Errors</li> <li>M11: Internationalization/String Encoding Errors</li> <li>M12: Inadequate Experience Errors</li> <li>M13: Insufficient Tooling Access Errors</li> <li>M14: Workflow Order Errors</li> <li>M15: General Planning Failure</li> </ul> <p><em><strong>Contact</strong></em></p> <p>Please contact Benjamin S. Meyers (<a href="mailto:bsm9339@rit.edu">email</a>) with questions about this data and its collection.</p> <p><em><strong>Acknowledgments</strong></em></p> <p>Collection of this data has been sponsored in part by the National Science Foundation (grant 1922169), by the NSA Science of Security Lablet program (grant H98230-17-D-0080/2018-0438-02), and by a Department of Defense DARPA SBIR program (grant 140D63-19-C-0018).</p>
Open Research Skills Workshops - GitHub basics
<p>This is the third workshop on GitHub basics<strong> </strong>in a series of workshop about Open Research Skills.</p><p>This workshop covers:</p><p><strong>-</strong> Introduction to Github and its uses</p><p>- Demonstration on using GitHub <strong> </strong></p><p>- Basic repo set up and editing</p><p><strong>List of training workshops in Open Research Skills:</strong></p><ul><li>24th February 2023 - Open access publishing</li><li>24th March 2023 - Using repositories</li><li><strong>21st April 2023 - GitHub basics</strong></li><li>28th April 2023 - GitHub collaborative workflows</li><li>26th May 2023 - Standard vocabularies and ontologies</li><li>30th June 2023 - FAIR data</li></ul><p><strong>Project overview:</strong></p><p>Our project aims to upskill participants in open research skills to increase the quality and reusability of phytolith research and related disciplines such as archaeology, palaeosciences and plant sciences. We will run six hands-on training workshops on open access publishing and research outputs, using repositories, ontologies and standard vocabularies, implementation of FAIR Guidelines for phytolith research, and two workshops on Github basic and advanced skills. The materials from all workshops will be archived as self-study courses on our website (<a href="https://open-phytoliths.netlify.app/">https://open-phytoliths.netlify.app/</a>). We will also provide translation during workshops and training materials into multiple languages. </p><p>This video is a basic course in Github. Github is a tool that is used for research project management and history tracking of your work during projects. It can be used to store and collaborate during projects with data, code and documentation. It covers the basic web interface of Github and how to make repositories, add files and folders. It will also include some examples of uses of Github.</p>
Open Research Skills Workshops - GitHub collaborative workflows
<p>This is the fourth workshop on GitHub collaborative workflows in a series of workshop about Open Research Skills.</p><p>This workshop covers:</p><p>- Introduction to version control</p><p>- How to fork a repository</p><p>- Forking exercises</p><p>- How to work in a team and create and merge branches</p><p>- Branching exercises</p><p><strong>List of training workshops in Open Research Skills:</strong></p><ul><li>24th February 2023 - Open access publishing</li><li>24th March 2023 - Using repositories</li><li>21st April 2023 - GitHub basics</li><li><strong>28th April 2023 - GitHub collaborative workflows</strong></li><li>26th May 2023 - Standard vocabularies and ontologies</li><li>30th June 2023 - FAIR data</li></ul><p><strong>Project overview:</strong></p><p>Our project aims to upskill participants in open research skills to increase the quality and reusability of phytolith research and related disciplines such as archaeology, palaeosciences and plant sciences. We will run six hands-on training workshops on open access publishing and research outputs, using repositories, ontologies and standard vocabularies, implementation of FAIR Guidelines for phytolith research, and two workshops on Github basic and advanced skills. The materials from all workshops will be archived as self-study courses on our website (<a href="https://open-phytoliths.netlify.app/">https://open-phytoliths.netlify.app/</a>). We will also provide translation during workshops and training materials into multiple languages. </p><p>Github is a collaborative, project management tool used to run reproducible research projects with version control. In these videos, you will learn how to use version control, how to branch and fork a repository, how to pull a request and how to collaborate as part of a team on GitHub.</p>
Github Directory Listings Dataset 2021
<p>Directory listings for the HEAD revisions of all publically-accessible Github repositories in the <a href="https://ghtorrent.org/">GHTorrent</a> database dump from 2021-03-06. See the Readme.md file for additional details.</p>
Github BPMN Artifacts Dataset 2021
<p>Information about 327,436 potential BPMN artifacts identified in all public Github repositories referenced in the <a href="https://ghtorrent.org/">GHTorrent</a> dump from <a href="https://ghtorrent-downloads.ewi.tudelft.nl/mysql/mysql-2021-03-06.tar.gz">March 2021</a>. </p> <p>The data file is in line-delimited JSON format, with each row containing an array with the following six elements:</p> <ol> <li>GHTorrent project ID</li> <li>GitHub user name</li> <li>GitHub repository name</li> <li>GitHub branch name</li> <li>Path to file inside repository</li> <li>SHA1 hash of the file's contents</li> </ol> <p>To get a list of retrievable URLs, use e.g. the following Python one-liner:</p> <pre><code class="language-bash">python3 -c 'import json; import sys; print(*[f"https://raw.githubusercontent.com/{u}/{r}/{b}/{f}" for _, u, r, b, f, _ in map(json.loads, sys.stdin)], sep="\n")' < bpmn-artifacts.jsonl > urls.txt</code></pre> <p>(using the hashes to filter out duplicates first is recommended though)</p>
QuerTCI: A Tool Integrating GitHub Issue Querying with Comment Classification
<p>Issue tracking systems enable users and developers to comment on problems plaguing a software system. Empirical Software Engineering (ESE) researchers study (open-source) project issues and the comments and threads within to discover---among others---challenges developers face when, e.g., incorporating new technologies, platforms, and programming language constructs. However, issue discussion threads accumulate over time and thus can become unwieldy, hindering any insight that researchers may gain. While existing approaches alleviate this burden by classifying issue thread comments, there is a gap between searching popular open-source software repositories (e.g., those on GitHub) for issues containing particular keywords and feeding the results into a classification model. In this paper, we demonstrate a research infrastructure tool called QuerTCI that bridges this gap by integrating the GitHub issue comment search API with the classification models found in existing approaches. Using queries, ESE researchers can retrieve GitHub issues containing particular keywords, e.g., those related to a certain programming language construct, and subsequently classify the kinds of discussions occurring in those issues. Using our tool, our hope is that ESE researchers can uncover challenges related to particular technologies using certain keywords through popular open-source repositories more seamlessly than previously possible. A tool demonstration video may be found at: https://youtu.be/fADKSxn0QUk.</p>
amel-github/sars-ani: Releasing new data fields in SARS-ANI dataset
<p>2022-06-20 - Release v1.1</p> <p>The original SARS-ANI dataset displayed common and scientific names of the animal host as found in the information source and/or inferred from the literature or expert knowledge.<br> Misspelled animal names and errors in taxonomy can lead to incorrect scientific conclusions and poor policy design. Moreover, harmonized host names can aid integrating other datasets (e.g. data on host biological traits, geographic distribution, or association with other pathogens).<br> Therefore, for each event, we programmatically performed taxonomic validation of the animal host name, using the R package taxize (Chamberlain et al. 2013). For more information on our validation process, see the R script <strong>sars_ani_validation.R.</strong></p> <p>Version 1.1. contains seven fields related to the identification of the animal host:</p> <ul> <li> <p>host_com_orig: Most specific designation of the animal host provided by the source(s), in English.</p> </li> <li> <p>host_sci_orig: Scientific name of the animal host as mentioned in the source(s) (scientific names are harmonized so that only the first letter of the genus is capitalized).</p> </li> <li> <p>host_com_res: Common name of the animal host, harmonized against the National Center for Biotechnology Information (<a href="https://www.ncbi.nlm.nih.gov/">NCBI</a>) taxonomic backbone.</p> </li> <li> <p>host_sci_res: Scientific name of the animal host (resolved to species or subspecies level), harmonized against the National Center for Biotechnology Information (<a href="https://www.ncbi.nlm.nih.gov/">NCBI</a>) taxonomic backbone.</p> </li> <li> <p>host_colloq: The colloquial name of the host, i.e. the name commonly used to identify the animal in non-specialist language (e.g. "tiger" for "Sumatran tiger").</p> </li> <li> <p>host_sci_spec_res: The scientific name of the host resolved to the species level.</p> </li> <li> <p>family: Animal family of the animal host.</p> </li> </ul>
How Developers Review Tests in GitHub?
<p>This dataset contains 330 code reviews with 40 tips, 16 request categories, 8 response categories, and 13 Pull Request and comment features. This dataset's column titles are:</p> <p>Project Owner Project Repo Project URI Project Language Pull Request URI Pull Request ID Pull Request Author Pull Request Merge Commit Hash Review comment URI Review comment ID Review comment Text Solving Commit Hash Solving Commit URI Validation Request Category Tips Response Category Test Case Unspecified Test Method Fix SUT Optional Test Improve Test Refactor Test Test Class Fix Test Test Branch Test Statement Achieve Specific Coverage Goal ML Model Test Prevent Exploit Disagreement Revert Test Removal Unit Test Code Snippet End-to-End Test Edge Case Expected Exception Filepath Set Up Parametric Negative Test Test Double Type Support Based On External Resource Integration Test Positive Test Avoid Wrong API Usage Increase Testability Rename Test Dependency Reproduce Issue More Specific Test Event Test Assert Message Fix Based On Test Boundary Test Regression Test Async Thread Consistency Test Move Test Compilation Check Extract Member Cache Modifier Empty Test File Merge Test Readability Sleep Invalid Test Remove Reflection</p>
Dataset of Automatically Orchestrable GitHub Projects
<p>This dataset accompanies the submission "Generating representative, live network traffic out of millions of code repositories" at HotNets'22: The 21st ACM Workshop on Hot Topics in Networks.</p> <p>Please see the files:<br> - `list_of_github_repositories.txt` for a list of GitHub repositories that we found containing a `docker-compose*.yml` file<br> - `list_of_executed_repositories.csv` for more detailed information on the success of capturing traffic with specific orchestration files found in ~67% of the repositories<br> <br> If you use our dataset, please cite our work as follows:</p> <blockquote> <p>Tobias Bühler, Roland Schmid, Sandro Lutz, and Laurent Vanbever.<br> 2022. Generating representative, live network traffic out of millions<br> of code repositories. In The 21st ACM Workshop on Hot Topics<br> in Networks (HotNets ’22), November 14–15, 2022, Austin, TX,<br> USA. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/<br> 3563766.3564084</p> </blockquote>
Extracted MSR GitHub Repository URLs
<p>This dataset contains text files of <a href="https://github.com">GitHub</a> URLs pointing to hosted git repositories.</p> <p>These URLs come from mining software repository (MSR) datasets. URLs are built by taking the repository owner's name (OWNER) and it's name (REPO) and appending them to https://github.com/. There is one URL per line. <em>URLs have not been tested for their current availibility</em>. An example URL format is provided below:</p> <pre><code>https://github.com/OWNER/REPO</code></pre> <p> Current URLs are from the following datasets:</p> <ul> <li>libraies.io January 12th, 2020 dataset <ul> <li>Jeremy Katz, "Libraries.io Open Source Repository and Dependency Metadata". Zenodo, Jan. 12, 2020. doi: 10.5281/zenodo.3626071.</li> </ul> </li> <li>RepoReapers/reaper dataset <ul> <li>Munaiah, N., Kroh, S., Cabrey, C. et al. Curating GitHub for engineered software projects. Empir Software Eng 22, 3219–3253 (2017). https://doi.org/10.1007/s10664-017-9512-6</li> </ul> </li> <li>GH Torrent dataset <ul> <li>G. Gousios, “The GHTorent dataset and tool suite,” in <em>Proceedings of the 10th Working Conference on Mining Software Repositories</em>, San Francisco, CA, USA, May 2013, pp. 233–236.</li> </ul> </li> </ul>
Usage and Attribution of Stack Overflow Code Snippets in GitHub Projects — Supplementary Material
<p><em>Background:</em> Stack Overflow (SO) is the largest Q&A website for software developers, providing a huge amount of copyable code snippets. Using those snippets raises various maintenance and legal issues. SO’s license (CC BY-SA 3.0) requires attribution, i.e., referencing the original question or answer, and requires derived work to adopt a compatible license. While there is a heated debate on SO’s license model for code snippets and the required attribution, little is known about the extent to which snippets are copied from SO without proper attribution.</p> <p><em>Aim:</em> Our main goal was to analyze how often code from SO posts is used in public GitHub projects, but not attributed as required by the license. Further, we wanted to investigate if developers are aware of SO’s license and its implications, and to what degree they adhere to the attribution requirements defined in SO’s terms of service.</p> <p><em>Method:</em> We present results of a large-scale empirical study analyzing the usage and attribution of non-trivial Java code snippets from SO answers in public GitHub projects. We followed three different approaches to triangulate an estimate for the ratio of unattributed usages and conducted two online surveys with software developers to complement our results.</p> <p><em>Results:</em> For the different sets of projects that we analyzed, the amount of projects containing files with a reference to SO varied between 3.3% and 11.9%. We found that at most 1.8% of all analyzed repositories containing code from SO used the code in a way compatible with CC BY-SA 3.0. Moreover, we estimate that at most a quarter of the copied code snippets from SO are attributed as required, i.e., using a link in a source code comment. About half of the surveyed developers admitted copying code from SO without attribution. Furthermore, about two thirds of them were not aware of the license of SO code snippets and its implications.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.