Find research datasets worth reusing

opencc-by-4.0Jun 2017View details →

How Developers Locate Performance Bugs — Supplementary Material

Abstract: Background: Performance bugs can lead to severe issues regarding computation efficiency, power consumption, and user experience. Locating these bugs is a difficult task because developers have to judge for every costly operation whether runtime is consumed necessarily or unnecessarily. Objective: We wanted to investigate how developers, when locating performance bugs, navigate through the code, understand the program, and communicate the detected issues. Method: We performed a qualitative user study observing twelve developers trying to fix documented performance bugs in two open source projects. The developers worked with a profiling and analysis tool that visually depicts runtime information in a list representation and embedded into the source code view. Results: We identified typical navigation strategies developers used for pinpointing the bug, for instance, following method calls based on runtime consumption. The integration of visualization and code helped developers to understand the bug. Sketches visualizing data structures and algorithms turned out to be valuable for externalizing and communicating the comprehension process for complex bugs. Conclusion: Fixing a performance bug is a code comprehension and navigation problem. Flexible navigation features based on executed methods and a close integration of source code and performance information support the process. Dataset: <ol> <li> Tutorial: We provide the slides (PDF) and the video (MP4) we used in the tutorial phase of our study. </li> <li> Locating Bugs: We also provide supplementary material for each research question. We provide the advices we prepared for each bug in case a team got stuck (PDF); the questions we asked after each bug fixing session can be found on the introduction slides (PDF). <ul> <li> RQ1: Navigating and Understanding <ul> <li> RQ1.1: How was information from the profiling tool or other parts of the IDE used to locate the performance bug? Cross-case analysis (in German) (XLSX+ODS) </li> <li> RQ1.2: Is the in-situ visualization of the profiling data beneficial compared to a traditional list representation? Cross-case analysis (in German) (XLSX+ODS) </li> <li> RQ1.3: What navigation strategies do developers pursue to locate a specific performance bug? Interaction logs (TXT), Navigation visualizations (SVG), Screen recordings for Bug 3 (MP4, without audio because of confidentiality) </li> </ul> </li> <li> RQ2: Understanding and Communicating <ul> <li> RQ2.1: How do developers communicate with each other when locating a performance bug? Coding (XLSX+ODS), Sketches (PDF), Screen recordings for Bug 3 (MP4, without audio because of confidentiality) </li> <li> RQ2.2: Could sketches help to understand and communicate a performance bug? Coding (XLSX+ODS), Sketches (PDF), Cross-case analysis (in German) (XLSX+ODS), Sketching videos for Bug 3 (MP4, without audio because of confidentiality) </li> </ul> </li> </ul> </li> <li> Questionnaire: The questionnaire that the participants filled out at the end of the study can be found here (PDF). </li> </ol>

opencc-zeroJun 2024View details →

Dataset: Global X Cybersecurity ETF (BUG) Stock Performance

This dataset provides historical stock market performance data for specific companies. It enables users to analyze and understand the past trends and fluctuations in stock prices over time. This information can be utilized for various purposes such as investment analysis, financial research, and market trend forecasting.

opencc-by-4.0Oct 2019View details →

On the Performance of Method-Level Bug Prediction: A Negative Result

Bug prediction is aimed at identifying software artifacts that are more likely to be defective in the future. Most approaches defined so far target the prediction of bugs at class/file level. Nevertheless, past research has provided evidence that this granularity is too coarse-grained for its use in practice. As a consequence, researchers have started proposing defect prediction models targeting a finer granularity (particularly method-level granularity), providing promising evidence that it is possible to operate at this level. Particularly, models mixing product and process metrics provided the best results. We present a study in which we first replicate previous research on method-level bug-prediction, by using different systems and timespans. Afterward, based on the limitations of existing research, we (1) re-evaluate method-level bug prediction models more realistically and (2) analyze whether alternative features based on textual aspects, code smells, and developer-related factors can be exploited to improve method-level bug prediction abilities. Key results of our study include that (1) the performance of the previously proposed models, tested using the same strategy but on different systems/timespans, is confirmed; but, (2) when evaluated with a more practical strategy, all the models show a dramatic drop in performance, with results close to that of a random classifier. Finally, we find that (3) the contribution of alternative features within such models is limited and unable to improve the prediction capabilities significantly. As a consequence, our replication and negative results indicate that method-level bug prediction is still an open challenge.

opencc-by-4.0Dec 2018View details →

On the Performance of Method-Level Bug Prediction: A Negative Result. Appendix

Abstract Bug prediction is aimed at identifying software artifacts that are more likely to be defective. Most approaches defined so far target the prediction of bugs at class/file level. Nevertheless, past research has provided evidence that this granularity might be too coarse-grained, thus reducing the usability of bug prediction in practice. As a consequence, researchers have started proposing defect prediction models targeting a finer granularity, particularly targeting methods, providing promising evidence that it is possible to operate at this granularity. Particularly, models based on a mixture of product and process metrics provided the best results.  In this paper, we first replicate previous research on method-level bug- prediction using different systems and timespans. Afterward, based on the limitations of existing research, we (1) re-evaluate method-level bug prediction models more realistically and (2) analyze the whether textual features— previously shown as valuable sources of information for the evaluation of software quality (yet surprisingly unexplored in this research field)—can be exploited to improve method-level bug prediction abilities. Key results of our study include that (1) the performance of the previously proposed models, tested using the same strategy but with different systems/timespans, is con- firmed. However, (2) when evaluated with a more realistic strategy all the models show a dramatic drop in performance exhibiting results close to that of a random classifier. In addition, we find that (3) the contribution of textual features within such models is limited and unable to improve the prediction capabilities significantly. As a consequence, our replication and negative results indicate that method-level bug prediction is still an open challenge.

zenodo36/100

Artifact for "MAAT: A Novel Ensemble Approach to Addressing Fairness and Performance Bugs for Machine Learning Software"

This artifact is for the paper entitled “MAAT: A Novel Ensemble Approach to Addressing Fairness and Performance Bugs for Machine Learning Software”, which is accepted by ESEC/FSE 2022. MAAT is a novel ensemble approach to improving the fairness-performance trade-off for ML software. It outperforms state-of-the-art bias mitigation methods. The artifact has also been placed on GitHub (https://github.com/chenzhenpeng18/FSE22-MAAT) under the Apache License, publicly accessible to other researchers. In this artifact, we provide the source code of MAAT and other existing bias mitigation methods that we use in our study, as well as the intermediate results, the installation instructions, and a replication guideline (included in the README). The replication guideline provides detailed steps to replicate all the results for all the research questions.

opencc-by-4.0Dec 2021View details →

opencc-by-4.0Apr 2022View details →

On Reporting Performance and Accuracy Bugs for Deep Learning Frameworks: An Exploratory Study from GitHub

This repository aims to store the dataset of performance and accuracy bug reports, which belongs to "On Reporting Performance and Accuracy Bugs for Deep Learning Frameworks: An Exploratory Study from GitHub"

Fig. 5 in Pretrained Convolutional Neural Networks Perform Well in a Challenging Test Case: Identification of Plant Bugs (Hemiptera: Miridae) Using a Small Number of Training Images

Fig. 5. Visualization of a feature with high importance and a feature with low importance from a configuration B. The importance of these features for the identification accuracy was determined using permutation tests (see the methods). For each taxon or group (rows) several randomly selected specimens (columns) are shown. For the two selected Global Average Pooling layer features, the corresponding features of the preceding (Max Pooling) layer are visualized as those show specific image parts that had higher activations.Yellow represents the maximal activation strength; dark blue represents the minimal activation strength.

Fig. 2 in Pretrained Convolutional Neural Networks Perform Well in a Challenging Test Case: Identification of Plant Bugs (Hemiptera: Miridae) Using a Small Number of Training Images

Fig. 2. Schematic representation of the networks used. On top are the convolutional layers of VGG16, grouped into five blocks. Output of each Max Pooling layer is fed into Global Average Pooling layer. Numbers near each block name indicate number of features in the Global Average Pooling layer. Height of layers roughly corresponds to resolution (except for Global Average Pooling layer), while width roughly corresponds to the number of feature maps or features produced.Then, in approach A, outputs of five blocks are concatenated and passed to the linear classifier. In approach B, output of only one block (block 3 in the final configuration) is passed to the linear classifier. In approach C CNN outputs are as in approach A, but instead connected to a DNN with two layers of 320 fully connected (FC) neurons followed by a prediction layer (PL), with number of neurons equal to number of species classified. Finally, approach D features CNN as in approach B which is connected to DNN as in approach C.

Fig. 1 in Pretrained Convolutional Neural Networks Perform Well in a Challenging Test Case: Identification of Plant Bugs (Hemiptera: Miridae) Using a Small Number of Training Images

Fig. 1. Dorsal habitus photos of males and females of Tuxedo spp., Pygovepres vaccinicola, and Phallospinophylus setosus generated for and used in this study.

Fig. 4 in Pretrained Convolutional Neural Networks Perform Well in a Challenging Test Case: Identification of Plant Bugs (Hemiptera: Miridae) Using a Small Number of Training Images

Fig. 4. Validation accuracy and accuracy on test data for SVM linear classifier (A, B) and DNN approaches (C, D) for the three datasets.

Fig. 3 in Pretrained Convolutional Neural Networks Perform Well in a Challenging Test Case: Identification of Plant Bugs (Hemiptera: Miridae) Using a Small Number of Training Images

Fig. 3. Identification accuracy for the male (top) and female (bottom) Tuxedo dataset for the five selected resolutions and the individual blocks 1–5 and the concatenated block.