Find research datasets worth reusing
Search datasets from major research repositories and use ShareScore to quickly assess how well each record supports discovery, access, and reuse.
49
datasets available to search
ShareScore release 0.9.0
Dataset results
49 results for “Continuous Integration”
Continuous depth-integrated fluorometric phytoplankton measurements in Lake Bonney, McMurdo Dry Valleys, Antarctica (2013-2017)
Phytoplankton are key primary producers in the photic zones of permanently ice-covered lakes in Antarctica’s McMurdo Dry Valleys (MDV), playing a crucial role in regional carbon cycling. However, their seasonal dynamics remain poorly understood, particularly during winter, when the region is inaccessible to researchers. To address this gap, the Autonomous Lake Profiler and Samplers (ALPS) project, part of the McMurdo Dry Valleys Long Term Ecological Research (MCM LTER) program, deployed sensors and samplers to collect year-round data on phytoplankton dynamics. This data package provides continuous, depth-integrated fluorometric measurements of phytoplankton communities in Lake Bonney from 2013 to 2017. A submersible spectrofluorometer (bbe Moldaenke FluoroProbe) was used to quantify the vertical distribution of key algal classes – brown/mixed algae, green algae, and cryptophytes – within the deep photic zone of the lake’s eastern (21–24 m) and western (16–24 m) lobes. Lake Bonney serves as a year-round refugium for life in extreme environments, but rapid lake level rise over the past three decades has introduced new uncertainties regarding phytoplankton community dynamics. These data contribute to ongoing efforts to understand how environmental change influences microbial ecology in polar aquatic ecosystems.
Leveraging Serverless Computing for Continuous Integration and Delivery
<p>This work has been a first implementation of the usage of AWS Lambda for running CI/CD tasks. This has been implemented in TeamCity, a CI/CD tool by JetBrains. We investigated the possible solutions for using AWS Lambda for running CI/CD tasks, but also did a depth analysis on how it compares to Amazon ECS for running CI/CD tasks.</p> <p>We were able to understand that there are CI/CD tasks that can be optimised by making use of FaaS. These tasks must be able to be executed in under 15 minutes (or be splittable into 15 minutes tasks), and would benefit more if they're either a high throughput of tasks in a small period of time, or very little executions over time. These tasks will benefit from both a higher performance and cost efficiency.</p> <p>This package includes: </p> <ul> <li>TeamCity Backup - backup with all of the CI/CD configurations used for these experiments. It is important to mention that the replication of these experiments does require a license for TeamCity and to apply a new connection to AWS;</li> <li>Tasks Performance tests - all of the results obtained during the experiments led to our conclusions.</li> </ul>
(No) Influence of Continuous Integration on the Development Activity in GitHub Projects — Dataset
<p>This dataset is based on the TravisTorrent dataset released 2017-01-11 (https://travistorrent.testroots.org), the Google BigQuery GHTorrent dataset accessed 2017-07-03, and the Git log history of all projects in the dataset, retrieved 2017-07-16 and 2017-07-17.</p> <p>We selected projects hosted on GitHub that employ the Continuous Integration (CI) system Travis CI. We identified the projects using the TravisTorrent data set and considered projects that:</p> <ol> <li>used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent),</li> <li>were active for at least one year (365 days) before the first build with Travis CI (before_ci),</li> <li>used Travis CI at least for one year (during_ci),</li> <li>had commit or merge activity on the default branch in both of these phases, and</li> <li>used the default branch to trigger builds.</li> </ol> <p>To derive the time frames, we employed the GHTorrent Big Query data set. The resulting sample contains 113 projects. Of these projects, 89 are Ruby projects and 24 are Java projects. For our analysis, we only consider the activity one year before and after the first build.</p> <p>We cloned the selected project repositories and extracted the version history for all branches (see https://github.com/sbaltes/git-log-parser). For each repo and branch, we created one log file with all regular commits and one log file with all merges. We only considered commits changing non-binary files and applied a file extension filter to only consider changes to Java or Ruby source code files. From the log files, we then extracted metadata about the commits and stored this data in CSV files (see https://github.com/sbaltes/git-log-parser).</p> <p>We also retrieved a random sample of GitHub project to validate the effects we observed in the CI project sample. We only considered projects that:</p> <ol> <li>have Java or Ruby as their project language</li> <li>used GitHub from the beginning (first commit not more than seven days before project creation date according to GHTorrent)</li> <li>have commit activity for at least two years (730 days)</li> <li>are engineered software projects (at least 10 watchers)</li> <li>were not in the TravisTorrent dataset</li> </ol> <p>In total, 8,046 projects satisfied those constraints. We drew a random sample of 800 projects from this sampling frame and retrieved the commit and merge data in the same way as for the CI sample. We then split the development activity at the median development date, removed projects without commits or merges in either of the two resulting time spans, and then manually checked the remaining projects to remove the ones with CI configuration files. The final comparision sample contained 60 non-CI projects.</p> <p>This dataset contains the following files:</p> <p><strong>tr_projects_sample_filtered_2.csv</strong><br> A CSV file with information about the 113 selected projects.</p> <p><strong>tr_sample_commits_default_branch_before_ci.csv<br> tr_sample_commits_default_branch_during_ci.csv</strong><br> One CSV file with information about all commits to the default branch before and after the first CI build. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns:</p> <p>project: GitHub project name ("/" replaced by "_").<br> branch: The branch to which the commit was made.<br> hash_value: The SHA1 hash value of the commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit.</p> <p><strong>tr_sample_merges_default_branch_before_ci.csv<br> tr_sample_merges_default_branch_during_ci.csv</strong><br> One CSV file with information about all merges into the default branch before and after the first CI build. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the following columns:</p> <p>project: GitHub project name ("/" replaced by "_").<br> branch: The destination branch of the merge.<br> hash_value: The SHA1 hash value of the merge commit.<br> merged_commits: Unique hash value prefixes of the commits merged with this commit.<br> author_name: The author name.<br> author_email: The author email address.<br> author_date: The authoring timestamp.<br> commit_name: The committer name.<br> commit_email: The committer email address.<br> commit_date: The commit timestamp.<br> log_message_length: The length of the git commit messages (in characters).<br> file_count: Files changed with this commit.<br> lines_added: Lines added to all files changed with this commit.<br> lines_deleted: Lines deleted in all files changed with this commit.<br> file_extensions: Distinct file extensions of files changed with this commit.<br> pull_request_id: ID of the GitHub pull request that has been merged with this commit (extracted from log message).<br> source_user: GitHub login name of the user who initiated the pull request (extracted from log message).<br> source_branch : Source branch of the pull request (extracted from log message).</p> <p><strong>comparison_project_sample_800.csv</strong><br> A CSV file with information about the 800 projects in the comparison sample.</p> <p><strong>commits_default_branch_before_mid.csv<br> commits_default_branch_after_mid.csv</strong><br> One CSV file with information about all commits to the default branch before and after the medium date of the commit history. Only commits modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the commits tables described above.</p> <p><strong>merges_default_branch_before_mid.csv<br> merges_default_branch_after_mid.csv</strong><br> One CSV file with information about all merges into the default branch before and after the medium date of the commit history. Only merges modifying, adding, or deleting Java or Ruby source code files were considered. Those CSV files have the same columns as the merge tables described above.</p>
Enhanced Westermo dataset - Transformed and Modified for Test case Selection and Priorotization in the context of Continuous Integration and Reinforcement Learning.
<p><strong>Overview</strong></p> <p>This repository contains a modified version of the existing, recently published dataset, Westermo. The initial dataset was gathered at Westermo Network Technologies AB, located in Västerås, Sweden. It encompasses over <strong>1 Million verdicts</strong> obtained from testing embedded systems, collected over a span of more than <strong>500 consecutive days</strong> of nightly testing. The dataset has been transformed and tailored specifically to cater to the research community, particularly for addressing challenges such as regression test selection, identification of flaky tests, and visualization of test results. The original dataset can be accessed through the reference provided in <strong>[1]</strong>.</p> <p>The Westermo dataset offers valuable historical information regarding the execution of test cases and their corresponding results. It serves as a valuable resource for evaluating and comparing different Test case Selection and Prioritization (TSP) techniques, enabling researchers to identify test cases that are more likely to fail during subsequent executions. Test cases in the dataset are characterized by attributes such as execution duration, previous last execution time, and the results of their recent executions.</p> <p>This dataset offers valuable historical information regarding the execution of test cases and their corresponding results. It serves as a valuable resource for evaluating and comparing different test case prioritization and selection techniques, enabling researchers to identify test cases that are more likely to fail during subsequent executions. Test cases in the dataset are characterized by attributes such as execution duration, previous last execution time, and the results of their recent executions.</p> <table align="left"> <caption><strong>Table 1: Dataset Overview</strong></caption> <tbody> <tr> <td>Test Cases</td> <td>1855</td> </tr> <tr> <td>CI Cycles</td> <td>15,197</td> </tr> <tr> <td>Verdict</td> <td>1,036,818</td> </tr> <tr> <td>Failed</td> <td>5.03%</td> </tr> </tbody> </table> <p> </p> <p> </p> <p> </p> <p> </p> <p> </p> <p> </p> <p> </p> <p>However, the diversity and multitude of the features in the dataset can be irrelevant to some TSP approaches. This led us to perform a dataset conversion, where we customized Westermo to have the same features from Paint Control and IOF/ROL, two widely used datasets in Reinforcement Learning based TSP approaches.</p> <p>This conversion required the combination of multiple variables and generating the target ones. When it comes to generating the “LastResults” and “Cycle” values, further analysis was required and the data handling needed an in-depth understanding of how the nightly testing was conducted. This led us to investigate what a CI cycle is in their context, and we followed their definition of a session, stating that “a session is when we run a suite of tests on one test system with a certain software version and testware version”. When splitting the data according to the 9 different systems used, we were able to generate 9 different sub-sets that fit the CI context.</p> <p> </p> <p><strong>File Format</strong></p> <p>The compressed .zip file contains 9 files, each one corresponding to each of the 9 systems. The datasets are available in CSV format, with the semicolon (;) serving as the delimiter. The columns included are represented in the table below along with their descriptions.</p> <table> <caption><strong>Table 2: Parameters of the dataset</strong></caption> <thead> <tr> <th scope="col">Column Name</th> <th scope="col">Content</th> </tr> </thead> <tbody> <tr> <td>Id</td> <td>Unique numeric identifier of the test execution </td> </tr> <tr> <td>Name</td> <td>Unique numeric identifier of the test case</td> </tr> <tr> <td>Duration</td> <td>Approximated runtime of the test case</td> </tr> <tr> <td>CalcPrio</td> <td>Priority of the test case, calculated by the prioritization algorithm (output column, initially 0)</td> </tr> <tr> <td>LastRun</td> <td>Previous last execution of the test case as date-time-string (Format: <em>YYYY-MM-DD HH:ii </em>)</td> </tr> <tr> <td>LastResults</td> <td>List of previous test results (Failed: 1, Passed: 0), ordered by ascending age. Lists are delimited by [ ].</td> </tr> <tr> <td>Verdict</td> <td> <p>Test verdict of this test execution (Failed: 1, Passed: 0)</p> </td> </tr> <tr> <td>Cycle</td> <td>The number of the CI cycle this test execution belongs to.</td> </tr> </tbody> </table> <p> </p> <p>The implications of this conversion are important as it can help the previous works to re-assess their approaches and have more data for training and testing, as well as opening a broader data spectrum for future researchers in this field to find ready-to-use, rich datasets, on which they could evaluate their approaches and contribute to the TSP community. This also addresses the limitations in the field discussed in the systematic literature review <strong>[2]</strong>, stating that future research on TSP techniques should focus on collecting data from more recent subjects in a CI context with varying failure rates and larger execution times, as reproducible studies with appropriate datasets are needed to develop a usable body of knowledge regarding TSP over time. We believe that this conversion of the Westermo dataset is our contribution to alleviating the gap for the RL-based approaches.</p> <p>The original dataset can be found <a href="https://sites.mdu.se/aidoart/results/open-source/test-results-dataset-westermo">here.</a></p>
An Empirical Study of Activity, Popularity, Size, Testing, and Stability in Continuous Integration
<p>A good understanding of the practices followed by software development projects can positively impact their success --- particularly for attracting talent and on-boarding new members. In this paper, we perform a cluster analysis to classify software projects that follow continuous integration in terms of their activity, popularity, size, testing, and stability. Based on this analysis, we identify and discuss four different groups of repositories that have distinct characteristics that separates them from the other groups. With this new understanding, we encourage open source projects to acknowledge and advertise their preferences according to these defining characteristics, so that they can recruit developers who share similar values.</p>
Contextual Factors Research in Continuous Integration (CI) Projects
<p>These files include process documentation for the research on project contextual factors in Continuous Integration (CI). They cover previous research studies and survey details. </p>
Dataset of the manuscript "Problems and Solutions in Applying Continuous Integration and Delivery to 20 Open-Source Cyber-Physical Systems"
<p>This archive contains the artifacts (datasets) for the manuscript "Problems and Solutions in Applying Continuous Integration and Delivery to 20 Open-Source Cyber-Physical Systems"</p>
Continuous Integration and Delivery Practices for Cyber-Physical Systems: An Interview-Based Study - Online Dataset
<p>This package contains the online dataset of the manuscript:</p> <p>Continuous Integration and Delivery Practices for Cyber-Physical Systems: An Interview-Based Study</p>
Widespread Error Detection in Large Scale Continuous Integration Systems
<p>Dataset of 5000 json documents describing verification process of <a href="https://github.com/facebook/react">React</a> project collected between 2023 and 2024. Errors recorded in this dataset were used in the presentation at CCIW workshop.</p> <p><strong>Abstract: </strong>Continuous Integration systems are widely used in the software industry to validate and integrate code changes into central repositories. Their effectiveness can be impacted by non-deterministic tests which can fail in the absence of any regression. Integration tests which depend on external services are particularly prone to this problem. We present a system which allows us to reduce the impact of non-deterministic failures by detecting widespread errors. The key assumption, which works well in practice, is that developers tend not to make identical mistakes simultaneously. If we observe a widespread error, it strongly suggests there is a problem with upstream services and not with the code change being evaluated. The detection algorithm consists of three main phases. First, the error text gets extracted from logs using predefined heuristics or automated methods. Then, this text gets fuzzy matched against a database of recently observed errors. Finally, statistics get checked to determine if they meet the criteria for a widespread error. When an error meets the criteria it either gets demoted to a warning or it gets enriched with information about an ongoing incident.</p>
Dataset for Understanding and Improving Regression Test Selection in Continuous Integration
<p>This is the dataset for the paper "Understanding and Improving Regression Test Selection in Continuous Integration". The dataset consists of the logs collected from running RTS tools on Travis CI along with a CSV summary of those results.</p>
Lightweight Dynamic Build Batching Algorithms for Continuous Integration
<p>Replication package for Journal publication: "Lightweight Dynamic Build Batching Algorithms for Continuous Integration"</p>
Unveiling the Relationship Between Continuous Integration and Code Coverage
<p>Artifacts from <strong>Unveiling Relationships Between Continuous Integration and Code Coverage</strong></p>
Data from: When discrete characters are wanting: Continuous character integration under the phylospecies concept informs the revision of the Australian land snail <em>Thersites</em> (Eupulmonata, Camaenidae)
Open the record for dataset details and reuse information.
Uncovering the Benefits and Challenges of Continuous Integration Practices - Reproduction Package
<p>This is a reproduction package for our paper on continuous practices in software organizations.</p>
Data for: Including Performance Benchmarks into Continuous Integration to Enable DevOps
<p>The DevOps movement intends to improve communication, collaboration, and integration between software developers (Dev) and IT operations professionals (Ops). Automation of software quality assurance is key to DevOps success. We present how automated performance benchmarks may be included into continuous integration. As an example, we report on regression benchmarks for application monitoring frameworks and illustrate the inclusion of automated benchmarks into continuous integration setups.</p> <p>This dataset supplements the paper and contains the raw experimental data for each experiment.</p>
Monitoring Continuous Integration Practices in Industry: A Case Study
<p><span>In our previous studies </span><span>have demonstrated </span><span>that only executing automatic builds and checking test coverage </span><span>have been considered to measure the level of Continuous Integra</span><span>tion maturity in several projects. In this paper we seek for under</span><span>stand the the benefits and challenges of monitoring CI practices in </span><span>day-to-day software development.</span><span> </span><span>We aim to evaluate the </span><span>impact of monitoring seven CI practices in a real-world scenario on </span><span>three public organizations in Brazil.</span><span> </span><span>We first developed a </span><span>CI practices monitoring suite tool and conducted a multiple-case </span><span>study applying a mixed-methods strategy, combining surveys, in</span><span>terviews, log data, and mining data from CI services.</span><span> </span><span>We </span><span>verified organization’ interest in monitoring CI practices. Moni</span><span>toring provided an overview of the organization’s CI status, not </span><span>covered by other tools, motivated constant improvement in these </span><span>practices, a perception of software quality, improve the communica</span><span>tion and and it is easy to adopt.</span><span> </span><span>Conclusions:</span><span> </span><span>We recommend that </span><span>companies adopt monitoring of CI practices and that CI services </span><span>integrate this monitoring into their dashboards.</span></p>
On the Need to Monitor Continuous Integration Practices: An Empirical Study
<p>This repository contains all the artifacts and data related to the <strong>On the Need to Monitor Continuous Integration Practices: An Empirical Study</strong>. The folder structure and their entailments are as follows:</p> <ul> <li><strong>data</strong>: Contains .sql files with the dataset collected during the study, which includes all project information retrieved from GitHub. A README.txt file inside the folder explains how to restore the dataset.</li> <li><strong>document_analysis</strong>: Includes the analysis files for specific research questions: <ul> <li>RQ1 subdirectory: Contains the pull request (PR) comments dataset and an Excel file with the analysis data for Research Question 1.</li> <li>RQ3 subdirectory: Contains documentation from CI services and third-party tools analyzed in Research Question 3, along with an Excel file summarizing the analysis data.</li> </ul> </li> <li><strong>scripts</strong>: Provides the R scripts used for data analysis and chart generation.</li> <li><strong>survey</strong>: Includes the collected survey responses and a template of the survey sent to developers.</li> </ul> <p><br>The total size of the repository when decompressed is approximately <strong>8.14 GB</strong>.</p>
Optimizing Continuous Integration by Dynamic Test Selection
<p>Continuous integration (CI) is widely used in modern software engineering. However, it is an expensive practice. Some proposed approaches only focus on either intra- or inter-build cost reduction. In this paper, we propose an adaptive technique for dynamic test selection DTS, which combines intra- and inter-build cost reduction techniques. DTS uses build features to construct machine learning models to predict the probability of a specific build failure and transform the probability into the necessary test proportion, with respect to a selected test case prioritization technique. Based on the output of prediction model, it thus selects a prioritized test suite and a variable proportion of test cases with respect to a build. We constructed a large-scale dataset with approximately 115,000 builds, and conducted a controlled experiment using the dataset. The experiment shows that DTS outperforms existing techniques significantly. It detects 19.9% to 32.5% more failed test cases, compared with state-of-the-art techniques evaluated in the experiment. At the same time, DTS performs better than all three existing peer techniques on approximately 47% of projects. Moreover, the experiment also shows that our failure prediction model has an improvement of 0.15 in Area Under Curve (AUC), compared to prior machine learning models.</p>
Photonic integrated circuit based continuous-travelling-wave parametric amplifier
<p>Dataset for the manuscript "Photonic integrated circuit based continuous-travelling-wave parametric amplifier".</p> <p>Contains all raw data and code used to produce the Figures and Extended Data Figures in the manuscript. </p> <p> </p> <p> </p> <p> </p>
Replication Package for "Automated Reporting of Anti-Patterns and Decay in Continuous Integration"
<p>This is the replication package for the paper "Automated Reporting of Anti-Patterns and Decay in Continuous Integration" accepted for publication at <a href="https://2019.icse-conferences.org/track/icse-2019-Technical-Papers#event-overview">ICSE 2019 (Technical Track)</a>.</p> <p>We include all the artifacts necessary to replicate the results obtained in our paper. Specifically, we provide (i) all the scripts used to conduct our statistical tests and to process the data obtained from our surveys, (ii) the queries used to perform the project selection, (iii) and a runnable version of our CI anti-patterns detection tool along with the external source code used to quantify the presence of CI anti-patterns in our dataset.<br> Furthermore, we provide raw and processed data from our surveys and data (build logs and repositories) that can be used as input to our detection pipeline. We also include a Docker container image with a working environment containing the artifacts.</p> <p>Preprint of corresponding paper is available <a href="http://doi.org/10.5281/zenodo.2578271">here</a>.</p>
ScienceDex guides
Understand access before you commit
These curated guides explain access requirements, typical timelines, costs, and reuse considerations for widely used research datasets.
Allen Brain Atlas
Allen Brain Atlas is an Allen Institute collection of brain map atlases, datasets, APIs, and analysis tools covering mouse, human, and non-human primate brain resources.
Annotated Behaviour and Observability Dataset (ABODe)
ABODe is a University of Edinburgh DataShare dataset for behavior classification in group-housed mice using home-cage video, identities, bounding boxes, ground-plate positions, and annotator labels.
DANDI Archive for NWB datasets
DANDI is a BRAIN Initiative archive for publishing and sharing neurophysiology data, including electrophysiology, optophysiology, and behavioral data packaged as NWB and related standards.
International Brain Laboratory public data
The International Brain Laboratory public data releases expose standardized mouse decision-making experiments, including Neuropixels recordings, widefield calcium imaging, behavior, and session metadata accessed through the ONE API.
OpenNeuro
OpenNeuro is a free, open platform for sharing neuroimaging datasets, with public search, dataset pages, and download paths for web, S3, DataLad, and the OpenNeuro CLI.